Crowdstrike Outage a/k/a Everything's Fucked

[UTexasFight 9,517]

 

siap

[JMFP 1,120]

2 hours ago, Vegas64 said:

This is epic.

How do you parlay this? Do you wait for them to come to you tail between legs? Or do you proactively send an email or eyes emoji their way?

Either way, get your flowers on the next company All Hands

That's not how chiefs act. They will forget all about F250 saying anything in the first place and if he brings it up they'll act oblivious and will let him know that he doesn't need to bring it up again. 

Edited July 19 by JMFP
Conjunction junction, what's your function...

[ztejas 43,526]

36 minutes ago, Newdoc said:

to a guy with a 620 credit score and a monthly income of $3500.

Wow. Shots fired. 

(I'll have you know my credit score is a respectable 690 currently)

[MissingInAction 25,139]

13 minutes ago, elfenix said:

Fwiw, if you link to a tweet it embeds a tweet. If you link to a profile it embeds their entire Twitter timeline. So, it worked as intended, though not by you.

Thanks. My first fuck up around here in that manner.

[Pato del Muerto 52,543]

5 minutes ago, JMFP said:

That's not how chiefs act. They will forget all about F250 saying anything in the first place and if he brings it up they'll act oblivious and will let him know that he doesn't need to bring it up again. 

If they didn’t purchase that solution, those folks will take credit for not purchasing. 

[PvilleStang 2,917]

Well, worst day I've had in ages. And it beats yesterday by a mile.  

Yesterday: son wakes up in excruciating pain, wouldn't move, had to call an ambulance to come out at 6:45am.  Trip to Dell Childrens screws the day.

Today: get hit up by APAC support at 1:30am stating we have systems down all over with BSoD and need help.  My counterpart starts in, looks at watch, 30 minutes.  Make BS excuse as it sounds like our usual mountain out of a mole hill type issue.  I get the sitrep a few hours later after he's pretty much knocked out the low hanging fruit, and it's end users with laptops that are all over the damn planet that can't get to the office.

 

EASIEST FIX: tear laptop bottom cover off, remove drive, mount to USB-C to NVMe Adapter, then mount it up to my machine, enter bitlocker key, remove offending file, then reassemble.  Can do in roughly 5 minutes, 8 minutes if I didn't have the bitlocker key already on hand.

Shitty fix: Most newer Dell machines mount the primary drive in a RAID-type config, regardless of whether it's an array or not.  The WinRE does not see RAID drives, so if you were to find yourself in TROUBLESHOOTING menus in the boot process, you didn't have the option to boot into safe mode if your Dell XPS has been built in the last 2 years or so.  Swapping it to AHCI / NVMe operation mode lets the WinRE recovery wizard see the drive after inputting the BitLocker key, and then allows you to either use CMD (if you know the damn Admin password, which no one does, as it's a rando 20 character p/w cycled every 60 days), or you can reboot to Safe Mode.  This was tested and proven to work at 4pm, and every other user that was left had signed out and decided to drink heavily by the time the procedures were written up.

 

All this said, I've received 1 communication from our CS account exec, and that's after calling him and asking for an update from them.   Dell had at least 40+ tickets opened with the same issues I described in the shitty fix, but hadn't found that work around yet.  I will be in a drunken coma just as soon as the kids get down.  

[boilerhorn 1,397]

5 hours ago, MissingInAction said:

Thanks. My first fuck up around here in that manner.

That's what the CrowdStrike dev said. 

Edited July 20 by boilerhorn

[ABSR 5,320]

What are the odds there is some Surlyite out there who will be able to say in a few years “Remember that computer update that crippled business for a few days….I was the QA test/dev guy for that.”

[Dahobbs 17,627]

1 minute ago, ABSR said:

What are the odds there is some Surlyite out there who will be able to say in a few years “Remember that computer update that crippled business for a few days….I was the QA test/dev guy for that.”

There was most certainly was no QA or test guy involved in this one. 

[rage-a-holic 4,426]

16 minutes ago, PvilleStang said:

Well, worst day I've had in ages. And it beats yesterday by a mile.  

Yesterday: son wakes up in excruciating pain, wouldn't move, had to call an ambulance to come out at 6:45am.  Trip to Dell Childrens screws the day.

Today: get hit up by APAC support at 1:30am stating we have systems down all over with BSoD and need help.  My counterpart starts in, looks at watch, 30 minutes.  Make BS excuse as it sounds like our usual mountain out of a mole hill type issue.  I get the sitrep a few hours later after he's pretty much knocked out the low hanging fruit, and it's end users with laptops that are all over the damn planet that can't get to the office.

 

EASIEST FIX: tear laptop bottom cover off, remove drive, mount to USB-C to NVMe Adapter, then mount it up to my machine, enter bitlocker key, remove offending file, then reassemble.  Can do in roughly 5 minutes, 8 minutes if I didn't have the bitlocker key already on hand.

Shitty fix: Most newer Dell machines mount the primary drive in a RAID-type config, regardless of whether it's an array or not.  The WinRE does not see RAID drives, so if you were to find yourself in TROUBLESHOOTING menus in the boot process, you didn't have the option to boot into safe mode if your Dell XPS has been built in the last 2 years or so.  Swapping it to AHCI / NVMe operation mode lets the WinRE recovery wizard see the drive after inputting the BitLocker key, and then allows you to either use CMD (if you know the damn Admin password, which no one does, as it's a rando 20 character p/w cycled every 60 days), or you can reboot to Safe Mode.  This was tested and proven to work at 4pm, and every other user that was left had signed out and decided to drink heavily by the time the procedures were written up.

 

All this said, I've received 1 communication from our CS account exec, and that's after calling him and asking for an update from them.   Dell had at least 40+ tickets opened with the same issues I described in the shitty fix, but hadn't found that work around yet.  I will be in a drunken coma just as soon as the kids get down.  

Good times. I've been thanking Jebus all day for not being in desktop support. Half our windows servers could be down right now, but I have no idea, since my laptop won't boot and I only work remotely. So not my problem unless my boss drives to my house and asks me to help out....and since he lives in New England, that's unlikely.

[General Specific 3,262]

The CEO of Crowdstrike is exactly who I picture as a sales guy at a Dodge dealership trying to sell an $80K Hellcat to a guy with a 620 credit score and a monthly income of $3500. 120 month note at 12.5%. 

We all can’t be a nurse!

[Viper 12,111]

8 hours ago, crash_davis said:

Well, let's start with these degenerate asshats pushing code on a Friday. Think of all the IT people who are going to have to work all weekend. Those same IT people will be sure to start looking at competitor software this weekend as they are working. Cloudstrike done fuck themselves good.

 

No shit, pushing a production update on a Friday is pretty much a meme on the sysadmin world at this point. As of a few hours ago we had 500 servers still down and 2,500 workstations. I wasn't even able to login until 9am today 

[Buzzrock 14,936]

3 hours ago, atomheartbevo said:

I got to live through Nimda, Code Red, and a few others, but those were easier to fix. I can’t imagine what this is like with all of the remote stuff that rarely sees an admin in person.

SQL Slammer and Love Bug were two of my Waterloos. 

[Pato del Muerto 52,543]

Assuming most devices are scheduled to update at the same time in their time zone, did this start somewhere way east of here and roll west?  

[MissingInAction 25,139]

The worst fuck up I had to deal with was when a patch was pushed out without testing it on Win10. The entire budget/financial dept ran on win10 due to their old AF financial software wouldn't let us upgrade to 11.

So no one was getting paid until that shit got fixed. 

[blacklab 11,941]

Thank god we don't use Crowdstrike. 

Was woken up this morning by a text from our CTO asking if we use it anywhere. Wondered WTF is he asking that for. 

Maybe 10% of our staff is on windows, and the only things at AWS on windows are used as remote desktop for large processing by some worker bees.

Only reason I kinda knew what crowdstrike is due to looking it up when I saw them as a sponsor of Mercedes F1 and wondered what it was.

Less than a month ago I was talking to our dropbox rep and I told him McLaren team was never going to win shit because they have to use dropbox to share files and dropbox is shit. 

 

[ztejas 43,526]

16 minutes ago, MissingInAction said:

The worst fuck up I had to deal with was when a patch was pushed out without testing it on Win10. The entire budget/financial dept ran on win10 due to their old AF financial software wouldn't let us upgrade to 11.

So no one was getting paid until that shit got fixed. 

Shit man... I liked Win 10 better than 11.

Got 11 on this gaming laptop that I just got and it's making me want to switch to Linux. 

Edited July 20 by ztejas

[Rimbo 26,028]

6 hours ago, ztejas said:

Yeah I figured there's no way that was true.

 

 

[Rimbo 26,028]

4 hours ago, Shaggy3.0 said:

actually, it's not really free.  TCOO can skyrocket if you don't know what you're doing.

 

The TCO myth was horseshit when Microsoft first peddled it in the 90s, and it's even more horseshit now.

People still believing this in 2024 is like finding people who still believe the Earth is flat in 2024...

[Slacks 6,834]

Was expecting a 135k purchase order today...  Procurement emails me... No PO because CRWD. FML.

[MissingInAction 25,139]

1 hour ago, ztejas said:

Shit man... I liked Win 10 better than 11.

Got 11 on this gaming laptop that I just got and it's making me want to switch to Linux. 

Same but once shit goes EOL at Enterprise level you are fucked.

[MissingInAction 25,139]

1 hour ago, blacklab said:

Thank god we don't use Crowdstrike. 

Was woken up this morning by a text from our CTO asking if we use it anywhere. Wondered WTF is he asking that for. 

Maybe 10% of our staff is on windows, and the only things at AWS on windows are used as remote desktop for large processing by some worker bees.

Only reason I kinda knew what crowdstrike is due to looking it up when I saw them as a sponsor of Mercedes F1 and wondered what it was.

Less than a month ago I was talking to our dropbox rep and I told him McLaren team was never going to win shit because they have to use dropbox to share files and dropbox is shit. 

 

Shit even the state stopped using Dropbox nearly a decade ago, and they are usually way behind the curve.

[F250 28,805]

2 hours ago, Buzzrock said:

SQL Slammer and Love Bug were two of my Waterloos. 

Love Bug and Blaster for me.

[Dahobbs 17,627]

6 minutes ago, F250 said:

Love Bug and Blaster for me.

Kinky. 

[Hate 20,128]

I was working at the Dell Large Corporate Account helpdesk during the Y2K fiasco.  It turned out to be no big deal so I was let go early and had a great New Year.

[Rimbo 26,028]

In other news, I'm in charge of deciding what "endpoint security" my company uses, and guess who's glad he doesn't use Crowdstrike today? THIS guy!

[C-Man 27,239]

You’ll notice Southwest was not affected.

[Schulz2.0 28,997]

3 minutes ago, C-Man said:

You’ll notice Southwest was not affected.

[0xdeadbeef 1,651]

I'm sure I'm feeling 100x overly pessimistic because I'm short on sleep and because I feel completely helpless. 

I'm on the software side of a big hardware tech company and don't work in IT.  My company has a huge internal datacenter (aka on-prem cloud), that is run "as a Service".  Meaning the development teams don't have direct access to the vCenter console, so we can't fix our own VMs. We have to open a ticket and wait for IT to fix them. Fuck me; there are a _lot_ of VMs.  I assume most/all of the Windows ones have this problem.   My worry is the tools from my team and related teams are all sales related (quoting, sizing, and ordering) tools.  Nobody can sell anything with these tools offline. A few days offline can be absorbed, but if it's 1 or 2 weeks, it seems like it will directly impact the quarter.  This is not good. The double whammy is the companies we sell to are also hit and are probably going to tap the breaks on new purchases. This quarter has the potential to be really really bad.

I probably need to go the fuck to sleep and let IT burn the midnight oil.  

 

 

[WelfareBuysMyWeed 5,515]

10 hours ago, PvilleStang said:

Well, worst day I've had in ages. And it beats yesterday by a mile.  

Yesterday: son wakes up in excruciating pain, wouldn't move, had to call an ambulance to come out at 6:45am.  Trip to Dell Childrens screws the day.

Today: get hit up by APAC support at 1:30am stating we have systems down all over with BSoD and need help.  My counterpart starts in, looks at watch, 30 minutes.  Make BS excuse as it sounds like our usual mountain out of a mole hill type issue.  I get the sitrep a few hours later after he's pretty much knocked out the low hanging fruit, and it's end users with laptops that are all over the damn planet that can't get to the office.

 

EASIEST FIX: tear laptop bottom cover off, remove drive, mount to USB-C to NVMe Adapter, then mount it up to my machine, enter bitlocker key, remove offending file, then reassemble.  Can do in roughly 5 minutes, 8 minutes if I didn't have the bitlocker key already on hand.

Shitty fix: Most newer Dell machines mount the primary drive in a RAID-type config, regardless of whether it's an array or not.  The WinRE does not see RAID drives, so if you were to find yourself in TROUBLESHOOTING menus in the boot process, you didn't have the option to boot into safe mode if your Dell XPS has been built in the last 2 years or so.  Swapping it to AHCI / NVMe operation mode lets the WinRE recovery wizard see the drive after inputting the BitLocker key, and then allows you to either use CMD (if you know the damn Admin password, which no one does, as it's a rando 20 character p/w cycled every 60 days), or you can reboot to Safe Mode.  This was tested and proven to work at 4pm, and every other user that was left had signed out and decided to drink heavily by the time the procedures were written up.

All this said, I've received 1 communication from our CS account exec, and that's after calling him and asking for an update from them.   Dell had at least 40+ tickets opened with the same issues I described in the shitty fix, but hadn't found that work around yet.  I will be in a drunken coma just as soon as the kids get down.  

might I suggest alternate fix 3: use a linux live USB. From there, you can mount your raid configuration directly and make your changes far more easily than disassembling a laptop to get to the drive, and you don't run into admin problems. You could even setup the live usb to run a bash script at boot to automate the fix, so it'd be something you just pop into a computer, boot into, get a prompt for the bitlocker key, then reboot. I have a couple of servers in RAID0 and RAID1 configurations, and I've had to do this before when rebuilding the raid arrays went wonky. A live USB is a life saving tool in IT.

Edited July 20 by WelfareBuysMyWeed

[ABSR 5,320]

4 hours ago, WelfareBuysMyWeed said:

might I suggest alternate fix 3: use a linux live USB. From there, you can mount your raid configuration directly and make your changes far more easily than disassembling a laptop to get to the drive, and you don't run into admin problems. You could even setup the live usb to run a bash script at boot to automate the fix, so it'd be something you just pop into a computer, boot into, get a prompt for the bitlocker key, then reboot. I have a couple of servers in RAID0 and RAID1 configurations, and I've had to do this before when rebuilding the raid arrays went wonky. A live USB is a life saving tool in IT.

Am I the only one who thinks you are just making up words now?

[immamac 42,878]

15 minutes ago, ABSR said:

Am I the only one who thinks you are just making up words now?

I think this highlights the problem with undereducated population relying on technology that they have no fucking idea what it is. It's why people think AI is magic or real. 

This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

[David Dennison 18,941]

1 minute ago, immamac said:

I think this highlights the problem with undereducated population relying on technology that they have no fucking idea what it is. It's why people think AI is magic or real. 

This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

[Captainant 27,706]

4 minutes ago, immamac said:

I think this highlights the problem with undereducated population relying on technology that they have no fucking idea what it is. It's why people think AI is magic or real. 

This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

Tfw 40k grimdark was closer to reality than we think lol. Buncha folks today look at IT staff as if they're tech priests with exclusive access to unknowable information. 

No, dumbass. Just use your fuckin brain and think for a second and Google some shit and then see what works. It's not magic, but sooooo many people exhibit this strange pride in being technically unprofficient

[immamac 42,878]

4 minutes ago, Captainant said:

Tfw 40k grimdark was closer to reality than we think lol. Buncha folks today look at IT staff as if they're tech priests with exclusive access to unknowable information. 

No, dumbass. Just use your fuckin brain and think for a second and Google some shit and then see what works. It's not magic, but sooooo many people exhibit this strange pride in being technically unprofficient

It's seriously equivalent to not know how to change a tire or that cars need gasoline or that doors lock. 

[Brisketexan 105,187]

I think this highlights the problem with undereducated population relying on technology that they have no fucking idea what it is. It's why people think AI is magic or real. 
This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

[Bozo_Casanova 26,516]

15 hours ago, PvilleStang said:

Yesterday: son wakes up in excruciating pain, wouldn't move, had to call an ambulance to come out at 6:45am.  Trip to Dell Childrens screws the day.

Hope your son is OK

[royiv 9,333]

44 minutes ago, immamac said:

I think this highlights the problem with undereducated population relying on technology that they have no fucking idea what it is. It's why people think AI is magic or real. 

This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

This isn’t a dig on you, but it’s totally a dig on you.

[Shaggy3.0 1,695]

13 hours ago, Rimbo said:

The TCO myth was horseshit when Microsoft first peddled it in the 90s, and it's even more horseshit now.

People still believing this in 2024 is like finding people who still believe the Earth is flat in 2024...

chillax.  i remember the linux and OSS FUD in the mid-90s while attending UTexas.

but yes, high TCO is a reality.  for example, imagine  you're asked to partner with/consult data engineering devs and recent college grads.  and now imagine this team needs to stand-up airflow and implement OLAP pipelines in less than 3 months.  big corp X wants this accomplished with low-IQ linux newbs, many not accustomed to a terminal because windows "point and click thingies" makes work easier.  big corp's C-Suite decided to layoff experienced linux talent in 2023 so they're kinda stuck. yeah, not ideal.   mix in Copilot  to further confuse inexperienced juniors.   shit breaks,  nobody really understands linux they way they should.  you're off to the races.  linux training $$$ comes into focus, and that's just the start.  big corp X throws money at the problem because they can't measure their total addressable market.  data inconsistencies all over the place.

nothing's free.  and yes, i enjoy profiting from this nonsense.       

[WelfareBuysMyWeed 5,515]

1 hour ago, ABSR said:

Am I the only one who thinks you are just making up words now?

 

[RPM 26,330]

My first computer class involved one of these. Pretty sure that's all I need to get through life.

[WelfareBuysMyWeed 5,515]

11 minutes ago, RPM said:

My first computer class involved one of these. Pretty sure that's all I need to get through life.

If you weren't phone phreaking were you really ever living?

[Francisco 2.0 32,395]

1 hour ago, immamac said:

 

This isn't a dig on you, but there needs to be basic computer literacy if we are going to be this dependent on computers as a society. 

 

Goddamn, the lulz.

You aren't wrong, but that ship sailed loooooooooooong ago.  To many being great with "tech" means you are great in SEO, social media and other shit.    

 

20 years or so ago we ordered a Cisco PIX and some other hardware from a very large Cisco VAR.  They sent an engineer to the site; I met him (it was a Saturday) and he proceeded to unbox stuff, get it squared away, etc.  At the time, I was a Windows network administrator.  He has his laptop out and is connected to the console port, doing his thing.  I notice he's running Win XP home on his laptop.

He was whatever the highest level Cisco engineer you could be at the time, and knew his Cisco shit.  But in talking, he had absolutely no idea what a Windows AD was, how to disable the nagging "restart for updates" notice he was getting on his laptop, etc.  It was baffling to me at the time.   Yeah, he had a skillset that was invaluable, but knew very little about the OS on the device he used for his day to day job.  

 

 

 

 

Edited July 20 by Francisco 2.0

[Armybrat 40,858]

16 hours ago, PvilleStang said:

Well, worst day I've had in ages. And it beats yesterday by a mile.  

Yesterday: son wakes up in excruciating pain, wouldn't move, had to call an ambulance to come out at 6:45am.  Trip to Dell Childrens screws the day.

Today: get hit up by APAC support at 1:30am stating we have systems down all over with BSoD and need help.  My counterpart starts in, looks at watch, 30 minutes.  Make BS excuse as it sounds like our usual mountain out of a mole hill type issue.  I get the sitrep a few hours later after he's pretty much knocked out the low hanging fruit, and it's end users with laptops that are all over the damn planet that can't get to the office.

 

EASIEST FIX: tear laptop bottom cover off, remove drive, mount to USB-C to NVMe Adapter, then mount it up to my machine, enter bitlocker key, remove offending file, then reassemble.  Can do in roughly 5 minutes, 8 minutes if I didn't have the bitlocker key already on hand.

Shitty fix: Most newer Dell machines mount the primary drive in a RAID-type config, regardless of whether it's an array or not.  The WinRE does not see RAID drives, so if you were to find yourself in TROUBLESHOOTING menus in the boot process, you didn't have the option to boot into safe mode if your Dell XPS has been built in the last 2 years or so.  Swapping it to AHCI / NVMe operation mode lets the WinRE recovery wizard see the drive after inputting the BitLocker key, and then allows you to either use CMD (if you know the damn Admin password, which no one does, as it's a rando 20 character p/w cycled every 60 days), or you can reboot to Safe Mode.  This was tested and proven to work at 4pm, and every other user that was left had signed out and decided to drink heavily by the time the procedures were written up.

 

All this said, I've received 1 communication from our CS account exec, and that's after calling him and asking for an update from them.   Dell had at least 40+ tickets opened with the same issues I described in the shitty fix, but hadn't found that work around yet.  I will be in a drunken coma just as soon as the kids get down.  

 

[Armybrat 40,858]

1 hour ago, Bozo_Casanova said:

Hope your son is OK

Ditto.

[WelfareBuysMyWeed 5,515]

I actually think it's a really good thing mostly that being able to use a computer has gotten so dumbed down that my 71 year old near-illiterate mother can use a laptop now. A lot of modern interface requires people use technology to communicate, like email is an official government form of communication for many years now. I especially think it's generally a good thing that there are such abstracted high level programming languages out there now that there are programmers who don't know how their computers work, because knowledge of the gear head ins and outs of your computer is pretty much useless for the vast majority of people. Yes, very basic maintenance and troubleshooting skills should be necessary, but going back to the comparison about cars: I know jack shit about how my car works. Cars don't interest me in the least. A car is just a tool to lug my ass from point A to point B. I don't care about cars, don't follow car makers or models or any of that shit. I only buy cash beaters and drive them till they disintegrate because I have other shit in my life that I want to consume my time with than cars. I know so much about computers, because I wasn't spending my time working on cars.

I was suggesting to someone a method to automate breaking into harddrives using a 3rd party tool installed on a non-standard medium to an IT guy who needs to quickly do this on likely hundreds of computers over the next several days, so speed counts. That's definitely not the equivalent of knowing your car needs gas. I'd say maybe being able to follow directions telling you to jam F7 as you boot to get into a menu, then specifically what to click and where to go, that would be the basic computer skills most need and actually do possesses if they aren't being whiny bitches.

The benefit of computers and shit becoming so easy to use that laymen can operate them is that you're not having computer experts reach beyond their field. I give an example from when I first started my own business, I got work from a doctor who wanted me to build him this custom productivity software for his clinic. He had a bunch of menial tasks in his day to day operation and he wanted them all automated and put into a database to be called up. The kind of shit he described was incredibly simple to make, so most of my time on the project amounted to me following this dude around, trying to osmosis his routine well enough that I could fart out the app he needed. The dude was already an expert in what he needed, it's just that creating the very simple app he needed was a little too complex for him. Not because he couldn't grasp it -- he was a doctor and thus can follow procedure well. Rather, he didn't have the time to acquire such skill.

The world is better when experts can craft the tools they need, not reliant on magicians who know the right incantations. Designing the app that doctor wanted could have been a drag and drop experience if the right tools existed. I would say this crowdstrike shit is less an example of people not knowing how their computers work, and more like people not being very diligent with obvious security vectors.

Edited July 20 by WelfareBuysMyWeed

[JMFP 1,120]

1 hour ago, WelfareBuysMyWeed said:

 

 

[Chopper 17,242]

13 hours ago, Hate said:

I was working at the Dell Large Corporate Account helpdesk during the Y2K fiasco.  It turned out to be no big deal so I was let go early and had a great New Year.

fwiw the Y2K fiasco didn't happen because as early as 10 years before the year 2000, industry began spending hundreds of millions to prepare for it.

It's my understanding that the crowdstrike update was pushed out by Microsoft. So 2 companies majorly fucked up. It's not one guy. It's the entire system that's completely fucked. People who knew how to do their jobs have been laid off and either not replaced or replaced by someone making next to nothing in India or Mexico.

Crowdstrike's culture is as rotten as Dell's, Google's, Microsoft's and what I've found has become extremely common throughout Silicon Valley.

Quote

This is as much a cultural failing as it is a technological one, the result of management geared toward value extraction — building systems that build monopolies by attaching themselves to other monopolies. Crowdstrike went public in 2019, and immediately popped on its first day of trading thanks to Wall Street's appreciation of Crowdstrike moving away from a focused approach to serving large enterprise clients, building products for small and medium-sized businesses by selling through channel partners — in effect outsourcing both product sales and the relationship with a client that would tailor a business' solution to a particular need.

Crowdstrike's culture also appears to fucking suck. A recent Glassdoor entry referred to Crowdstrike as "great tech [with] terrible culture" with no work life balance, with "leadership that does not care about employee well being." Another from June claimed that Crowdstrike was "changing culture for the street,” with KPIs (as in metrics related to your “success” at the company) “driving behavior more than building relationships” with a serious lack of experience in the public sector in senior management. Others complain of micromanagement, with one claiming that “management is the biggest issue,” with managers “ask[ing] way too much of you…and it doesn’t matter if you do what they ask since they’re not even around to check on you,” and another saying that “management are arrogant” and need to “stop lying to the market on product capability.”

While I can’t say for sure, I’d imagine an organization with such powerful signs of growth-at-all-costs thinking — a place where you “have to get used to the pressure” that’s a “clique that you’re not in”  — likely isn’t giving its quality assurance teams the time and space to make sure that there aren’t any Kaiju-level security threats baked into an update. And that assumes it actually has a significant QA team in-house, and hasn’t just (as with many companies) outsourced the work to a “bodyshop” like Wipro or Infosys or Tata. 

-Ed Zitron

 

 

Edited July 20 by Chopper

[Blotto 23,560]

12 minutes ago, Chopper said:

fwiw the Y2K fiasco didn't happen because as early as 10 years before the year 2000, industry began spending hundreds of millions to prepare for it.

It's my understanding that the crowdstrike update was pushed out by Microsoft. So 2 companies majorly fucked up. It's not one guy. It's the entire system that's completely fucked. People who knew how to do their jobs have been laid off and either not replaced or replaced by someone making next to nothing in India or Mexico.

Crowdstrike's culture is as rotten as Dell's, Google's, Microsoft's and what I've found has become extremely common throughout Silicon Valley.

 

I have not heard this. Source? 

[Rimbo 26,028]

2 hours ago, Shaggy3.0 said:

linux training $$$

dafuq?

See, all the things you need to learn everything about Linux are free. You just go read the fucking manual.

And in my career for the past 15 years or so, every dev is working on a Mac, which is ALSO a variant of UNIX, so they already know almost all the shit they needed to.

The only reason any engineer, data scientist or what have you under the age of 40 even bothered with Windows is so they can play games on Steam.