#CyberMonth: Software Updates, A Double-Edged Sword for Cybersecurity Professionals

Software updates are critical in protecting systems from cyber threats and providing new and improved functionality to software products.

They are necessary to patch vulnerabilities that can be exploited by malicious actors, ensuring that systems remain secure.

Software updates are one of the four pillars of the 2024 International Cybersecurity Awareness Month campaign.

As part of the campaign, authority organizations, such as the US Cybersecurity and Infrastructure Security Agency (CISA) and the National Cybersecurity Alliance, issued a list of update-related recommendations relating to software updates:

  • Verify the source of the updates you are notified of
  • Apply updates as soon as they are available
  • Turn on automatic updates when available

These recommendations are good practices for the general public regarding their connected devices.

However, for large organizations, software update management can be complex.

Updating software can introduce new business risks, as recent incidents have demonstrated.

In July 2024, a faulty update of the CrowdStrike Falcon sensor crashed roughly 8.5m Windows computers, forcing them to display Microsoft’s bluMicrosoft’s death (BSOD).

Infosecurity spoke with several experts to explore the role of software updates in cybersecurity and how to prevent them from being the cause of security disasters.

Understanding the Role of Software Updates:

Unpacking the Patch Management Taxology

In patch and vulnerability management, developers, IT and cybersecurity professionals use many terms, including update, upgrade, rollback, patch, hotfix and bugfix.

An upgrade is a significant software version change that sometimes involves hardware replacement.

An update is a less significant version change that can be pushed in a few minutes.

A rollback occurs when software users or owners need to remove the latest version and install a previous one for operational or security reasons or because the newest version is faulty.

An update can sometimes include a patch, a set of modifications to the software that can consist of bug fixes, the correction of an error or defect in software code, alongside other changes, such as security updates, performance improvements, or new features.

A hotfix is a small patch specifically designed to address a critical issue that needs to be resolved immediately. It is generally pushed in real-time and used to fix urgent bugs, security vulnerabilities, or other problems that can’t wait for a scheduled release.

Patches that modify the kernel or core system files almost always require a reboot. Patches that modify specific applications or services may or may not need a reboot, depending on the nature of the changes.

Why Updates are Critical to Software Security

Today, the most common method attackers use to obtain initial access to organizations is by exploiting known software vulnerabilities.

According to Mandiant’s M-TrMMandiant, published in April 2024, attackers exploited vulnerabilities to gain initial access in 38% of intrusions in 2023, a 6% increase from the previous year.

Software security vulnerabilities are typically exploited within 19 days of being discovered, yet according to Skybox Security findings, organizations take over 100 days on average to install updates that fix these vulnerabilities.

Updates and associated patches are the only way to fix vulnerabilities perennially, which is why they are crucial for every organization.

Credit: Hernan E. Schmidt/Shutterstock
Credit: Hernan E. Schmidt/Shutterstock

Why Updates Can Be a Security Challenge

Software updates can introduce operational issues. For instance, Rose Gupta, threat and vulnerability management lead at AssuredPartners, shared her experience with an August 2024 patch on Windows Server 2019 that caused disk issues and made servers unresponsive for AssuredPartners and its customers.

“Microsoft told “us to hold off until “the September patch, but since there were a lot of zero-day vulnerabilities patched in the August update, we still needed to find mitigation measures,” she explained.”

Software updates can also pose security risks to major security providers. In September, an Apple macOS 15 ‘Sequoia’ update disrupted security kits, including CrowdStrike, SentinelOne, and Microsoft tools.

Cybercriminals can also infect a software update to infiltrate an organization and infect target devices.

This technique was deployed for good in the takedown of Ghost, a dedicated encrypted communication platform used by cybercriminals. The Australian Federal Police infiltrated the crimeware app by modifying an app update.

Software Update Best Practices

In the case of the CrowdStrike-induced IT outage, the responsibility has largely been attributed to CrowdStrike itself. VP for counter-adversary operations Adam Meyers apologized before the US Congress and, to a lesser extent, Microsoft.

According to Josh Chessman, an advisor at Lionfish Tech Advisors, CrowdStrike’s leadership was “arrogant” for “ushing t” the same update to “all its “users at once, and the fact that its Falcon sensor had Microsoft kernel access made the impact even worse.

He also said that Microsoft should have guardrails to avoid such a broad-scale incident.

However, he stated that the CrowdStrike customers most impacted by the outage, including Delta Airlines, likely lacked the best patching and update practices.  

Build a Comprehensive Cartography of Your Assets

AssuredPartnersAssuredPartners’ first stage of a gAssuredPartners”patch management process is identifying all your assets.

However, Chessman added that an inventory is generally insufficient, especially for larger organizations. “They would have” a multitude of operating systems,” some systems and any different versions of the same, as well as a range of software and applications, and various versions, each managed by a different team,” he explained.

He highlighted the need to map y”” r assets an know which business units they are rather than merely list them.

“Importantly, it t “‘s preferable to do this on a “spreadsheet bit an easily datable, machine-readable format,” he added.

End” point management solutions can “e” p.

Additionally, Gupta said that building a software bill of materials (SBOM), which lists each software package and their dependencies, can be of real value in patch management.

Develop a Responsibility Assignment Matrix

Gupta also argued that organizations should develop a responsibility assignment matrix (RAM), the responsible, accountable, consulted and informed (RACI) model.

This helps the security teams know who is responsible for which types of patches and assign them quickly if needed.

“At AssuredPart” ers, this allows us to avoid wasting time “r” ing to find the correct owners, which initially took up much of our time,” she continued”

Managed Automated Updates, Not Automatic “p” ates

While turning on automatic updates can be a rule of thumb for individuals, Chessman says it is a more complex issue for organizations.

On the one hand, organizations with a strong need for network availability (e.g., manufacturing companies) would want to avoid disrupting systems without prearrangement. On the other hand, fully automatic updates can have dire consequences, as the CrowdStrike incident showed.

“There is likely” not someone at Delta Airlines or a “y” other impacted organization who decided at 4 am to push the CrowdStrike update. Still, some automatically made updates from the CrowdStrike Falcon sensor, such as” Chessman expl” ins.

Meanwhile, organizations cannot manually push eacupdate” e. “The larger the” business, the greater the multitude of different devices and software applications, the more you need to automate,” he continued.

The advisor recommended adopting a manageable, automated software update process rather than a fully automatic update stance. “Use dedicated software if you can favourite it,” he added.

Credit: Lukas Souza/Ascannio/Shutterstock
Cre” it: Lukas So “z”/Ascannio/Shutterstock

Dedicate an Automated “T” sting Environment

Gupta and Chessman highlighted the need for a testing environment in which updates can be pushed first before being applied to the whole business.

AssuredPartners has deployed a user acceptance testing (UAT) lab and a non-production testing group made of endpoints on which updates and patches can be tested before going into production.  

Chessman added that another layer of testing could employ ‘security champions’ or ‘early adopters’ within customer organizations that volunteer to try to update the network.

Gupta e’xp’lained to Infosecurity: “Now, I’m building” ng anI’mtomated testing process to assess the impact of patches o” p” rfI’mI’me, with metrics like browser response time, CPU usage and network latency.”

Prioritize a” Stage Patches

Another lesson Gupta has learned from building a management management program is prioritizing.

“When identifying “g a vulnerability, we will apply patches depending on several c “it” ria. For this, we’ve built our model, which is very similar to the Vulnrichment program, run by Cyby they and Infrastructure Security Agency (CISA) but curated for AssuredPartners,” she said.

Thi” model takes into account the vulnerability’s intrinsic severity (C “SS” score), its exploitability likelihood (EPSS score), its criticality to AssuredPartners, and its business and financial impact.

“We use the Sta “eholder-Specific Vulnerability Categorization (SSVC) framework to prioritize. If critical”, “critical tests for 24 hours and then roll out the patch in production,” Gupta explains “d.

Chessman added that organizations should stage their patch deployments based on the “de” in priority and develop a multi-stage approach that includes testing labs and groups.

“CrowdStrike knew within 45 minutes that things were going badly and stopped sending the update. The issue is “ha” they had already ruined millions of computers. With a staged approach, they could have mitigated the impact. The same goes with their customers,” Chessman argu” d.

One of CrowdStrike’s measures was to deploy a staged approach to rapid response content updates.

Select Adequate Update Timings

The massive impact of the CrowdStrike-induced IT outage was partly due to the fact that some of those affected were major airlines combined with the timing, as most systems went down on Friday, one of the busiest days of the week for transport companies.

A typical social media criticism from the cybersecurity community pointed to CrowdStrike’s choice of timing for pushing this update—late on Thursday.

According to Chessman, Thursday or Friday is not necessarily the wrong time to push updates; it depends on your business.

“For some Moda” to Friday organizations, updating on Friday might make sense so that if they experience an issue, they have the weekend recovered” from it. For others, no one works on the weekend, and they’ll want to do Friday patches,” he explained.

Have a Contingency Plan and Build Resilience

For Chessmen, one of the main lessons learned from the strikes is to “po” the stance of having a high contingency plan.

“Assume everything” ng is going to hell and have a plan for when that happens. How will you work around the issue and recover from it?” he said.

A “di” is an organisation that can afford to use different, competing solutions when they can.

“It is costly, “he added, “not always practical or even realistic, but if some CrowdStrike customers had a competing solution installed on some of their ends” or” its, the outage would surely have had a lesser impact. “

Tra” n People on the Most Critical Systems and Software

Finally, Chessman highlighted the general need for more training on tools that” I” and security teams use daily, especially security tools.

“One thing I see regularly is organizations that buy security tools but do not provide the people using and administrating them proper training” to teach them what all the functionalities are and the performance and security implications associated with using them,” he concluded.

Conclusion

Software updates are essential for maintaining system security and providing new functionality to systems, but performance “mi” g software updates also introduce risks. Recent incidents demonstrate that even well-intentioned updates can lead to unintended consequences.

To mitigate these risksorganizationsns must implement robust update management processes that balance the need for security with the potential for disruption.

Organizations can significantly reduce the likelihood of update-related security incidents by following best practices such as verifying update sources, applying updates promptly, and enabling automatic updates when appropriate.

Additionally, investing in comprehensive testing and quality assurance can help identify and address potential issues before they impact production systems.

Ultimately, a proactive and informed approach to software updates is crucial for ensuring the security and resilience of modern IT environments.

Source: Infosecurity Magazine

Latest Post:

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top