Major IT outage affecting banks, airlines, media outlets across the world - eviltoast

All our servers and company laptops went down at pretty much the same time. Laptops have been bootlooping to blue screen of death. It’s all very exciting, personally, as someone not responsible for fixing it.

Apparently caused by a bad CrowdStrike update.

Edit: now being told we (who almost all generally work from home) need to come into the office Monday as they can only apply the fix in-person. We’ll see if that changes over the weekend…

  • sugar_in_your_tea@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    3
    ·
    4 months ago

    And honestly, anything that can be done Monday is probably better done on Tuesday. Why start off your week by screwing stuff up?

    We have a team policy to never do externally facing updates on Fridays, and we generally avoid Mondays as well unless it’s urgent. Here’s roughly what each day is for:

    • Monday - urgent patches that were ready on Friday; everyone WFH
    • Tuesday - most releases; work in-office
    • Wed - fixing stuff we broke on Tuesday/planning the next release; work in-office
    • Thu - fixing stuff we broke on Tuesday, closing things out for the week; WFH
    • Fri - documentation, reviews, etc; WFH

    If things go sideways, we come in on Thu to straighten it out, but that almost never happens.