Comments by "" (@grokitall) on "Cloudflare: Pay Me 120k Or We Shut You Down" video.
-
7
-
3
-
2
-
2
-
1
-
it is not business ethics which require the shift your company policy, but the resiliency lessons learned after 9/11 which dictate it.
many businesses with what were thought to be good enough plans had them fail dramatically when faced with the loss of the data centers duplicated between the twin towers, the loss of the main telephone exchange covering a large part of the city, and being locked out of their buildings until the area was safe while their backup diesel generators had their air intake filters clog and thus the generator fail due to the dust.
the recovery times for these businesses for those it did not kill were often on the order of weeks to get access to their equipment, and months to get back to the levels they were at previously, directly leading to the rise of chaos engineering to identify and test systems for single points of failure and graceful degradation and recovery, as seen with the simian army of tools at netflix.
load balancing against multiple suppliers across multiple areas is just a mitigation strategy against single points of failure, and in this case the bad actors at cloudflare were clearly a single point of failure.
with a good domain name registrar, you can not only add new nameservers, which i would have done as part of looking for new providers, but you can shorten the time that other people looking up your domain cache the name server entries to under an hour, which i would have also done as soon as potential new hosting was being explored and trialed.
as long as your domain registrar is trustworthy, and you practice resiliency, the mitigation could have been really fast. changing the name server ordering could have been done as soon as they received the 24 hour ransom demand, giving time for the caches to move and making the move invisible for most people.
not only did they not do that, or have any obvious resiliency policy, but they also built critical infrastructure around products from external suppliers without any plan for what to do if there was a problem.
clearly cloudflare's behaviour was dodgy, but the casino shares some of the blame for being an online business with insufficient plans for how to stay online.
1
-
1
-
1
-
1
-
1
-
it clearly stated that the first email was saying there was a problem affecting the network, and when they turned up it was a meeting with a completely d8fferent department, sales, and that there was no problem. also no mention as to the enterprise offering being mandatory.
at that point i would return to my company and start putting resiliency measures in place with the intent to min8mise exposure to cloudflare with the intent to migrate, but the option to stay if they were not complete dicks.
the second contact was about was about potential issues with multiple national domains, with a clear response that it is due to differing national regulations requiring that.
the only other issue mentioned was a potential tos violation which they refused to name, and an immedia5e attempt to force a contract with a 120k price tag with only 24 hours notice and a threat to kill your websites if you did not comply.
at this point i would then have immediately triggered the move.
on the legal view, they are obviously trying to force a contract, which others have said is illegal in the us where cloudflare has its hardware based. it is thus subject to those laws.
by only giving 24 hours from the time that they were informed it was mandatory, they are clearly guilty of trying to force the contract, and thus likely to win.
if they can win on that, then their threat to pull the plug on their business on short notice in pursuit of an illegal act also probably makes them guilty of tortuous interference, for which they would definitely get actual damages, which would cover loss of business earnings, probably get reputational damages, probably get to include all the costs for having to migrate to new providers, and legal costs.
when i sued them, i would also go after not only cloudflare, but the entire board individually, seeking to make them jointly and severally liable, so that when they tried to delay payment, you could go after them personally.
the lesson is clear, for resiliency, always have a second supplier in the wings which you can move to on short notice, and have that move be a simple yes or no decision that can be acted upon immediately. by virtue of this, don't get overly relient on external tools to allow the business to continue to be able to work to mitigate the disaster if it happens. also keep onsite backups of any business critical information.
m9st importantly, make sure you test the backups. at least one major business i know of did everything right including testing the backup rec9very process, but kept the only copy of the recovery key file on the desktop of one machine in one office, with the only backup of this key being inside the encrypted backups.
th8s killed the business.
1
-
1
-
1