Ransomware clean up, Aisle 4. The last few weeks have been interesting ones in the cybersecurity space. We received notifications from MGM and Clorox, with MGM being the highlight, based on the visible impact of the breach and the over $100 million it may have cost them.
Clorox experienced a “breach” in August and filed their notification with the Securities and Exchange Commission as part of new reporting requirements a couple of weeks ago:
“The Clorox Company (the “Company” or “Clorox”) has identified unauthorized activity on some of its Information Technology (IT) systems. After becoming aware of the activity, the Company began taking steps to stop and remediate the activity, including taking certain systems offline. The Company is working diligently to respond to and address this issue, and is also coordinating with law enforcement. To the extent possible, and in line with its business continuity plans, Clorox has implemented workarounds for certain offline operations in order to continue servicing its customers. However, the incident has caused, and is expected to continue to cause, disruption to parts of the Company’s business operations.
Clorox has engaged leading third-party cybersecurity experts to support its investigation and recovery efforts. The investigation to assess the nature and scope of the incident remains ongoing and is in its early stages.”
-From Clorox SEC Filing 1-0715, August 14, 2023
What we know about the breach:
At this time we don’t quite know what the attack was, but we do know from the filing, that it caused Clorox to take automated ordering processing offline. Damage to these systems paused production at a number of sites in August, and at this point, production is beginning to restart.
It is important to note that production restarting is not returning to normal production levels. This attack impacted the automated order processing system, and as this is a manufacturing company, we have to take a bit of a dive into manufacturing to really understand the impact of the issue.
Manufacturing generally requires the sourcing of components for warehousing, manufacturing, packaging and then distribution. The number of components, and pieces that have to come together to fill an assembly line, to packaging, and then to distribution centers to move fulfilled orders from sellers is a complex chain that is highly susceptible to disruption, and then slow restarts. In manufacturing today most companies have moved to a “JIT production methodology”. JIT stands for Just-In-Time Manufacturing, which has become the standard production methodology.
The core tenant of JIT is to ensure the right amount of supplies, storage and transport is available for the orders that have been received within a production cycle. JIT is completely understandable and rational, but can be incredibly difficult to get right. Not enough supplies and the company misses out on sales. Too many supplies, and perishable ones go to waste. This causes the company to lose money and waste storage space, along with the additional lost time and money spent to dispose of expired inventory.
Clorox shutting down automated ordering systems then moving to manual order fulfillment has a tremendous impact on the company, and soon to our store shelves. Order processing slows considerably, predictive orders go from data and algorithmic-based decisions to gut and feel. Systems that are this complex also take time to normalize once things are put back into place. Again, too many supplies ordered without the manufacturing, packaging, or distribution to create and move supplies leads to more losses for the company.
Cyber Resilience and Manufacturing Considerations:
Why is a cybersecurity company that specializes in cyber resilience talking about this?
Simply put, it is a great example of understanding the business risks of the systems that could be impacted by a cyber attack. In cybersecurity we think of ransomware impacting financial systems, trying to access the crown jewels of the company for ransom purposes, or to exfiltrate data to sell for profit on the dark web. The Clorox breach is not one of those breaches. This breach hit a connected system that might not have been top of mind. But from a business risk perspective it could have a greater impact than the Caesars or MGM breaches.
Clorox reported they were unsure of when processes would get back to normal.
“Can’t find the right Clorox product? A recent cyberattack is causing some shortages,” Yahoo news, September 18, 2023:
“The company said it could not predict how long it would take to return to normal operations but expects to begin shifting back to automated order processing next week.
The cyberattack is expected to be “material” to the company’s first-quarter financial results. Clorox shares were down 0.2% early Monday afternoon, trading at $145.85 on the New York Stock Exchange.”
And from the New York Times:
“We have resumed production, getting certain shipments out the door, and are remaining in constant contact with customers about their immediate needs,” Clorox said in a written response to questions on Tuesday. But the company added that it could not quantify the amounts because of delays and product outages.
Lasting and Costly Impacts
Losses from this breach are going to impact Clorox business operations for a while as they deal with the repercussions of system restarts including missing, or unfilled orders and brand damage. They’ll have to adapt their thinking and review processes under the new understanding of the reality of their vulnerabilities.
Cyber resilience can’t change the production process, but it could have helped to include order systems as part of the business and system risk considerations, and may have been able to execute a faster response to contain the breach before it impacted as much as it did. Cyber Resilience also requires that we look at the lessons learned from this breach, and consider what detections, risk categorizations, and responses can be taken in the future that would have limited the business impact.
Just in time production is going to remain a critical process. JIT is the more efficient and cost effective way to produce and distribute products. But for manufacturing environments, we need to remember the complexity of everyday operating systems and take the potential loss of these systems into account as part of risk management and our cyber resilience planning.