Jump to content

Crowdstrike Outage a/k/a Everything's Fucked

RPM

By RPM
in Daily Texan

 Share

Recommended Posts

kevwun

kevwun 15795

Posted
Posted
Just now, PvilleStang said:

VMs are the easy ones to repair, it's the end user laptops that are being a PITA right now.  Forcing safe mode when the end user is remote and doesn't understand how to use F keys is almost as fun as trying to show aggy their empty trophy case.

Not to mention that like every fucking laptop seems to have a different key combination to start safe mode.

  • Rage+1 2
Link to comment
Share on other sites
Dahobbs

Dahobbs 17600

Posted
Posted
14 minutes ago, crash_davis said:

You're the reason why your company will soon be held ransom by some Russian hacker gang.

 

 

This is just bad process governance. Not sure how this could have been missed if they have the proper test cases and if they internally test for a period of time before releasing in Prod. Not sure how a 83 BILLION dollar company is this inept?

Or if they had just done a staged rollout of the update, this would have been identified before it shut down the fucking planet. Fucking idiots. 

  • Like 1
Link to comment
Share on other sites
PvilleStang

PvilleStang 2911

Posted
Posted
6 minutes ago, crash_davis said:

Well, let's start with these degenerate asshats pushing code on a Friday. Think of all the IT people who are going to have to work all weekend. Those same IT people will be sure to start looking at competitor software this weekend as they are working. Cloudstrike done fuck themselves good.

 

This was a Thursday evening patch release.  And frankly, it's better to do it then vs Friday at 5pm and walk out the door.

Link to comment
Share on other sites
crash_davis

crash_davis 23113

Posted
Posted
3 minutes ago, PvilleStang said:

This was a Thursday evening patch release.  And frankly, it's better to do it then vs Friday at 5pm and walk out the door.

Same thing. No patches after Thu COB. It's asking a lot to unfuck shit one day before the weekend. Sun nigh thru Wed night/Thu morning was my company's window.

Link to comment
Share on other sites
Gatorubet

Gatorubet 30246

Posted
Posted (edited)

IMG_2957.thumb.jpeg.72572f3429719775f5b31e0b9132e8ea.jpeg

American did not send me a single text about my flight being canceled prior to my arrival at the airport.  I am supposed to be at AAJ convention in Nashville today, but they gave me a 16 hour layover in Charlotte and a flight out to Nashville Saturday morning.  

The good news is, I can swill free alcohol at this Priority Pass lounge. First world problems, I guess

i’m on a few waitlists on other airlines so fingers crossed and another old-fashioned on the way

Edit:  here is the last text from American

IMG_2956.thumb.png.279c007240c34918ed4db6a4fc6b4e1d.png

Edited by Gatorubet
  • Like 1
  • Haha 2
Link to comment
Share on other sites
Captainant

Captainant 27680

Posted
Posted
Just now, Dahobbs said:

Are the systems with bitlocker installed extra turbo fucked? 

Oh we got hit by that! We have to call our central IT, verify our identity, read the serial number sticker off our machine, and then boot into safe mode, enter the bitlocker pin, and then go into command line and delete the bad DLL. Reboot and you're good!

Super fast and easy and scalable lol

  • Haha 1
Link to comment
Share on other sites
Dahobbs

Dahobbs 17600

Posted
Posted
2 minutes ago, Captainant said:

Oh we got hit by that! We have to call our central IT, verify our identity, read the serial number sticker off our machine, and then boot into safe mode, enter the bitlocker pin, and then go into command line and delete the bad DLL. Reboot and you're good!

Super fast and easy and scalable lol

Yeah, I figured it might be an issue. End users are already going to struggle to deploy this fix. Adding bitlocker on top of it seems like a nightmare. And that's assuming central IT can actually access those keys. 

Link to comment
Share on other sites
crash_davis

crash_davis 23113

Posted
Posted (edited)

Good news! My go to pron sites still work. Companies should seek IT admins from pron companies. Those sites never go down. Imagine how much war humanity would wage if we didn't have access to fine bitches.

Edited by crash_davis
  • Hook 'Em 2
  • Fuck Around and Find Out 1
Link to comment
Share on other sites
SubliminalHorn

SubliminalHorn 6806

Posted
Posted
9 minutes ago, Gatorubet said:

IMG_2957.thumb.jpeg.72572f3429719775f5b31e0b9132e8ea.jpeg

American did not send me a single text about my flight being canceled prior to my arrival at the airport.  I am supposed to be at AAJ convention in Nashville today, but they gave me a 16 hour layover in Charlotte and a flight out to Nashville Saturday morning.  

The good news is, I can swill free alcohol at this Priority Pass lounge. First world problems, I guess

i’m on a few waitlists on other airlines so fingers crossed and another old-fashioned on the way

If I had known you were at CLT I would have given you a good shoulder check 

  • Haha 3
Link to comment
Share on other sites
Dahobbs

Dahobbs 17600

Posted
Posted
2 minutes ago, Shaggy3.0 said:

even if he did, would he admit it?  would anyone list CS on their resume after this?

 

Anarchist? And really, anyone from the sales team can point out that they got everyone in the fucking world to buy their shitty product/service. 

  • Haha 1
Link to comment
Share on other sites
MissingInAction

MissingInAction 25104

Posted
Posted
3 minutes ago, Dahobbs said:

Yeah, I figured it might be an issue. End users are already going to struggle to deploy this fix. Adding bitlocker on top of it seems like a nightmare. And that's assuming central IT can actually access those keys. 

The keys should be saved in active directory.

  • Haha 3
  • Fuck Around and Find Out 1
Link to comment
Share on other sites
boilerhorn

boilerhorn 1387

Posted
Posted
10 minutes ago, Dahobbs said:

Are the systems with bitlocker extra turbo fucked? 

They are not.  Just need the keys to get through to the command prompt.  That said, you either had to save your keys from before or access them from your corp portal.  The process is quite easy for those who are tech savvy.  Less so for those who are not.

  • Fuck Around and Find Out 1
Link to comment
Share on other sites
randomhorn

randomhorn 1755

Posted
Posted
18 minutes ago, Deej said:

Frost says they can't accept deposits right now. Something tells me they'll find a way to withdraw from my accounts. 

The withdrawal system is an indestructible AS/400 kept secure at area 51. The deposit system is a fragile leftover box of parts running windows 95.

  • Haha 1
  • Rage+1 1
Link to comment
Share on other sites
Celery Man

Celery Man 20965

Posted
Posted
30 minutes ago, PvilleStang said:

VMs are the easy ones to repair, it's the end user laptops that are being a PITA right now.  Forcing safe mode when the end user is remote and doesn't understand how to use F keys is almost as fun as trying to show aggy their empty trophy case.

i'm in a discord giving shit to some old work buddies and one of them is complaining that the process to get a vm into safe mode in azure is actually complicated. of course he was the person most likely to cause a global outage so I'm not sure about it.

 

if i was responsible for this i think i'd move and change my name

  • Haha 1
Link to comment
Share on other sites
Gatorubet

Gatorubet 30246

Posted
Posted
1 hour ago, boilerhorn said:

True agile has that in tier 7 of the testing, which is customer-testing.  Windows boot, post-deployment, will now be part of the "DOA Testing" adding 5 minutes to the CI, slowing developers. ;)

9 minutes ago, boilerhorn said:

They are not.  Just need the keys to get through to the command prompt.  That said, you either had to save your keys from before or access them from your corp portal.  The process is quite easy for those who are tech savvy.  Less so for those who are not.

 

  • Haha 2
Link to comment
Share on other sites
  • Popular Post
Blotto

Blotto 23538

Posted
  • Popular Post
Posted

Lulz, look at this fucking dildo. He's been getting into his kid's hair gel again. From the looks of it, he might be drinking it. 

image.png.9d8142dc067f4686f869e83c13550ba8.png

Saw this  earlier as well..

Quote

Also the CEO was the CTO of McAfee in 2010 when they released an updated that made the antivirus think svchost.exe (a system file) was a virus. Bricked tens of thousands of computers.

He's got the golden touch. 

  • Haha 11
Link to comment
Share on other sites
Brisketexan

Brisketexan 105154

Posted
Posted
Just now, Blotto said:

Lulz, look at this fucking dildo. He's been getting into his kid's hair gel again. From the looks of it, he might be drinking it. 

image.png.9d8142dc067f4686f869e83c13550ba8.png

Saw this  earlier as well..

He's got the golden touch. 

Hmmm....there's a real chance he'll be forced out as CEO.

After the board grants him a $150 million exit package.

  • Hook 'Em 1
  • Haha 1
  • Rage+1 2
Link to comment
Share on other sites
Celery Man

Celery Man 20965

Posted
Posted
Just now, Blotto said:

Lulz, look at this fucking dildo. He's been getting into his kid's hair gel again. From the looks of it, he might be drinking it. 

image.png.9d8142dc067f4686f869e83c13550ba8.png

Saw this  earlier as well..

He's got the golden touch. 

there's a specific brand of leadership dude I've seen several times at these kinds of companies that is.... gen x guy who traded the puka shell necklace for a print button down but has changed nothing else about his style since 1998

  • Hook 'Em 1
  • Like 3
Link to comment
Share on other sites
Shaggy3.0

Shaggy3.0 1695

Posted
Posted

uncle bob is probably smiling ear to ear.  he deserves it.

i met him at sfo in 2012 during confluent.  the guy is wise and an absolute wizard.

the longer this plays out, his pov becomes more important because he's right.  software engineering, as a discipline, should require higher credentials and licensing.  he took a lot of shit for his takes.  but i agree with him 10000%.  

https://dev.to/bosepchuk/is-uncle-bob-serious-dhi?ref=highscalability.com

  

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing


Football ... Basketball ... Baseball ... Other Sports ... Futbol ... 🤫995🤫 ... Gambling ... Movies & TV ... Music ... Hobbies ... Lulz ... Food & Travel ... Daily Texan ... Business and Markets ... Cloak Room ... Help ... For Sale ... Board Discussion ... Subscribe!... Donate!... Advertise... COOKIE MONSTER!

×
×
  • Create New...