Comments by "GH1618" (@GH-oi2jf) on "Continuous Delivery"
channel.
-
This incident should come as no surprise to anyone who has been around computers as long as I have (almost 60 years). Most people just naïvely assume it wasn't tested. You look at it more broadly, discussing the process of rolling out upgrades and the advantage of releasing in restricted domains before general release. But you, like most people, put the entire focus on CrowdStrike. I look at it a little differently.
How did this faulty module get installed around the world in countless platforms before being found to be faulty? It can only be because the practice of automatic updates has been embraced everywhere. We are all used to it on our personal computers. Some things are upgraded without any action on our part, whether we like it or not. The problem here was that the module was kernel mode software, hence had the potential to crash the system. Such changes should only be made by IT personnel at the end users, in restricted test environments. This isn't done only because it requires time and people, i.e. money.
Looking at the problem more generally, the computer industry has always (during my involvement in it) treated security, not exactly as an afterthought, but as a secondary consideration, necessary, but not the first priority. The first priority has always been features, which can be sold, and convenience, which can be thought of as a feature. Convenience reduces the demand for time and people, hence has value. Until something goes wrong, as it did here. Now, a lot of people are spending a lot of time fixing it, and a lot more people have had their plans disrupted.
I have never liked automatic updates, having been one of those people who, long ago, was responsible for installing uogrades. Nowadays, in retirement, I mostly use an iPad. Some things
I use, like YouTube, get updated without my involvement, whether I want it or not. At least my YT app is not likely to do any harm. But periodically I get a notice from Apple that a new release of IOS is available. I never enable automatic update, and I am never in a hurry to update. I ignore the notice for weeks, sometimes months, on the assumption that if something is seriously wrong with it, I will hear about it before I install it. This conservative update policy has, in fact, saved me from installing one bad release.
The only thing that surprised me about this fiasco was its scale.
1