How on earth can you ensure your supply chain is secured? On average, a large enterprise uses up to 15 security vendors, according to 451Research. In reality, the supply chains of most organizations have grown so large (and changed over time) that an exact figure might even be unknown. According to Okta, there was an average of 190 apps per company in 2020. Additionally, it’s not only about the official count of vendors and apps that a company uses, employees are also accessing other apps and software on their work devices. Supply chain security has always been a problem but 2020 showed the reality of how this can be a massive threat, and more importantly, that nobody really knows how to counter it.
So a megabreach happened last year (and is still happening and will go on for a while I believe). SolarWinds, a software company that does IT management software, was breached. The investigation (including great research from Fireeye [1][2]) showed that the perpetrators spent months (from back in September 2019 according to a post from SolarWinds itself) inside the company’s DevOps before implanting malicious code into updates that were then shipped to thousands of customers. And what has been done to SolarWinds last year could be done to others (and has been done to others for sure). This particular instance could still be unknown if Fireeye hadn’t been breached and the intruders hadn’t been ‘enticed’ by its red team tools.
The backdoors were injected into a tool called Orion, used by many Fortune 500 and a number of US federal government to manage their networks. SolarWinds pushed injected updates to their Orion customers twice; first in late 2019 with test code from the intruders, then in February 2020 with the actual backdoor that ended up being widely propagated to its customers. The attackers took all sorts of safeguards to prevent the injected code from appearing in build logs or causing build errors. They apparently did a great job in properly inserting their code and ensuing it remained undetected, and SolarWinds developers didn’t realize any of that was going on!
Can you even counter this?
First, there is a very limited set of bad guys who can pull this stuff off. Also, think about the massive amount of data that’s generated by such a megabreach. You’d need monstrous capabilities, both in terms of backend infrastructure and human resources to sift through the huge amounts of victims and data you’re collecting. However, this was always going to be a threat that’s coming from somebody with large resources, either someone who controls the technology (such as in the Supermicro story) or a sophisticated attacker (likely a nation-state actor) who’s going after the supply chain to either target specific markets (to scope better and lessen its exposure) or maybe shooting wide open and see how it goes (as it might have been the case with the SolarWinds hack). But in any case, there are no easy answers to this.
You can only assume your partners and vendors are reliable and secure, and trust them! I don’t think many people questioned the reliability of SolarWinds as a partner before this breach. And even going forward, very few will do that. Everyone realizes this could have happened to anyone. So how can you provide risk assurance that your vendors are secure? You simply can’t. It’s not possible to scrutinize a company to a level where you can uncover issues even they couldn’t see! Can network monitoring play a role there? Not really. Certainly not when attackers have such high regard for operational security as in the SolarWinds breach.
At the end of the day, it’s not only a technical issue. Better transparency and accountability could also contribute to both greater protection and better response, but the thing is, supply chain security and risk management (and third-party risk in particular) are hard problems simply because of the sheer scale of what, where, and how these breaches can happen. I take my hat off though to the folks at Fireeye for uncovering this, which gives me a bit of hope going forward.
