Jump to content

Shit's Borked

RPM

By RPM
in Please do this better?

 Share

Recommended Posts

  • Replies 6.3k
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

immamac
immamac

Tornado hit data center. Sorry for downtime. Going to have more downtime to fix later Please report weird shit here but I'm with my family and mobile only. I just spent about an hour on a bench o

hookemATL
hookemATL

I had to draw porno. Wasn’t great but I finished

immamac
immamac

https://www.surlyhorns.com/board/settings/account-security/#piiDataRequest if you want to know what data could have possibly been leaked request here.  The passwords are encrypted at rest in

Posted Images

thunderlounge

thunderlounge 7,733

Posted
Posted
On 9/20/2022 at 10:55 AM, immamac said:

I was trying something to fix the ios bug that keeps videos from playing. It should be all fixed.

I changed the root URL from www.surlyhorns.com to just surlyhorns.com so your cookies don't carry over. I put it back to www.

 

That explains that. Thanks. Code someone mentioned looked kinda json-ish. Wasn’t much to it, no real meaningful value or some weird exposure. 
 

 

3 hours ago, chainsaw said:

Mainly CR threads

Well there’s your problem. 

Link to comment
Share on other sites
immamac

immamac 42,492

Posted
Posted
6 minutes ago, shadow_operative2.0 said:

Imagine what will happen when the shit really gets fixed?

That will be a happy day on ol’ Surl.

One day. 

I haven't been to the colo since the beginning of covid when they kicked us out of austin and we moved to dallas. Its really tough to do everything through remote hands when I want to do major backend system updates. FYI I'm currently upgrading the storage/media backend to get ready for hosting substantially more video/image content.

Link to comment
Share on other sites
immamac

immamac 42,492

Posted
Posted
2 minutes ago, shadow_operative2.0 said:

Man, here’s my honest question... Why is this a weekly thing? Seriously, every single week the search index, or whatever you want to call it, is fucked up. Every single week. 

I have literally never seen another website have this issue. Shag never did. 

Shaggy didn't even have a real search function, you had to use google. Shaggy didn't have feeds. Shaggy didn't have really any of the features that are on this board. The search index isn't fucked up and it's not every single week - there was a bug in the software (since patched) that was removing the index. 

Link to comment
Share on other sites
Lurch

Lurch 5,919

Posted
Posted
I haven't been to the colo since the beginning of covid when they kicked us out of austin and we moved to dallas. Its really tough to do everything through remote hands when I want to do major backend system updates. FYI I'm currently upgrading the storage/media backend to get ready for hosting substantially more video/image content.

Plans on moving to cloud?
Link to comment
Share on other sites
immamac

immamac 42,492

Posted
Posted

shaggy also crashed every gameday or big commit - Peoples memories are short, but surly doesn't even slow down on gameday anymore and we were serving over 30k requests/hr sustained for the entire day during alabama and it was flawless. We haven't had site hiccups due to being too busy in years - sometimes you have to take the good with the bad. I also don't tell everyone all the stuff I'm doing that comes with no downtime or really any hiccups and how quickly those hiccups are resolved/fixed before anyone even notices. 

  • Hook 'Em 6
  • Like 1
Link to comment
Share on other sites
thunderlounge

thunderlounge 7,733

Posted
Posted

The search index seems to be a sticking point. O get it though. 
 

Just be thankful it doesn’t affect hundreds of high paying client (6figs each, annually) with 100’s of millions of records each. 
 

That shit sucks the fat one. 😂

hope its ass is “elastic”, and can take the length. 

 

 

I feel the pain Imma, really do. And it sucks when you wake up and wonder what fresh hell is headed your way. We deal with it too, and have had to do a lot of backend updates and additions to help it.

Thankfully ours doesn’t have to be real time. A few minutes difference helps a lot.   

Link to comment
Share on other sites
immamac

immamac 42,492

Posted
Posted
2 minutes ago, Lurch said:


Plans on moving to cloud?

no, this isn't an issue with the colocation provider or facility. even though it's a hardware issue, obviously I have enough redundant hardware for it not to matter, I just didn't want to have to failover because it's a pain in the ass from a configuration perspective.

For those curious Surly runs on 2 enterprise grade NAS that are in an active/passive configuration + a backup to a remote offsite + another local backup just in case shit really hits the fan and we just need to go get the data. The active/passive are synced every 15 minutes and snapshotted every 15 minutes as well. When I first introduced surly storage it wasn't this "enterprise grade" and I told people not to put stuff they care about on it. Now it's as robust as most cloud solutions, and the uptime (despite how it feels) actually meets or exceeds any cloud provider. 

Right now we are running on the passive node, which I'll have to sync the changes back to the active node when it recovers from a boot drive failure that cause the simple reboot to fail. Each of these nodes can have 3 entire disk failures before 1 of them loses data. Over the last 4 years I've replaced 4 disks with no downtime between the systems. It's robust and tested working, sometimes shit just happens like boot drive failures after a long time and that's what happened this time. 

  • Hook 'Em 3
Link to comment
Share on other sites
elfenix

elfenix 34,168

Posted
Posted
4 hours ago, immamac said:

no, this isn't an issue with the colocation provider or facility. even though it's a hardware issue, obviously I have enough redundant hardware for it not to matter, I just didn't want to have to failover because it's a pain in the ass from a configuration perspective.

For those curious Surly runs on 2 enterprise grade NAS that are in an active/passive configuration + a backup to a remote offsite + another local backup just in case shit really hits the fan and we just need to go get the data. The active/passive are synced every 15 minutes and snapshotted every 15 minutes as well. When I first introduced surly storage it wasn't this "enterprise grade" and I told people not to put stuff they care about on it. Now it's as robust as most cloud solutions, and the uptime (despite how it feels) actually meets or exceeds any cloud provider. 

Right now we are running on the passive node, which I'll have to sync the changes back to the active node when it recovers from a boot drive failure that cause the simple reboot to fail. Each of these nodes can have 3 entire disk failures before 1 of them loses data. Over the last 4 years I've replaced 4 disks with no downtime between the systems. It's robust and tested working, sometimes shit just happens like boot drive failures after a long time and that's what happened this time. 

i run a similar config for my porn collection.  but with solid state boot drives.  in the process of upgrading to 14TB enterprise grade drives now that drive prices are settling down from chia rush.

  • Haha 1
Link to comment
Share on other sites
Ghost of LL

Ghost of LL 15,348

Posted
Posted
1 minute ago, Vertigo said:

I am getting a ton of ads popping up all over my screen on surly all of the sudden even while I am logged in. I click them away and they pop right back up. Running on Chrome/Mac. I dont get it on any other site. 

Good god--there's now a pop-up on every single post.  It's made the site completely unusable.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing


Football ... Basketball ... Baseball ... Other Sports ... Futbol ... 🤫995🤫 ... Gambling ... Movies & TV ... Music ... Hobbies ... Lulz ... Food & Travel ... Daily Texan ... Business and Markets ... Cloak Room ... Help ... For Sale ... Board Discussion ... Subscribe!... Donate!... Advertise... COOKIE MONSTER!

×
×
  • Create New...