Recovering data from RAID 1 Setup

[dad 557]

My work development workstation won't boot up. It can't find a boot device and the hard drive(s) are no longer recognized. Two 1 TB hard drives were set up in a RAID 1 configuration. Has anyone had any luck recovering this themselves? IT is going to take a look at my machine tomorrow and they've already given me a heads up that they've never successfully recovered data from this configuration. I'm doing a little bit of light research to figure out if there's a way I could take a stab at it if they don't figure it out (fat chance). Not a hardware guy here!

[kevwun 15790]

You can pay someone to retrieve it, but it's not cheap.  RAID arrays are great for the hard drive redundancy, but kind of suck when the array itself fails.  If you have a backup of your data( you still need backups even with a RAID setup), it's not worth the expense trying to get it back.

Edited December 4, 2018 by kevwun

[Chips O'Toole 1324]

It's possible that the RAID controller failed, and if that's the case, I've had success replacing a RAID controller with the same model (I've only used LSI) and then letting the new controller import the RAID configuration metadata from the drives.  I've only done this on servers though, not workstations.  If it's a software RAID or something, I have no idea.

[tokamak 6431]

Even if the RAID controller itself failed and can't be replaced, couldn't he just plug one of the HDs in to a separate computer and pull whatever data he needs? That's the whole purpose of RAID 1, right - all of the data is there on both drives. In order for the OP to have actually LOST everything, both HDs would have had to have failed at the same time and the chances of that are extremely small.

[kevwun 15790]

You can't just attach the drives to another pc and access them.  The data is there, but is not in an easily readable state.  It's like a combination lock that you lost the combination to right now.

Edited December 4, 2018 by kevwun

[Dahobbs 17560]

1 minute ago, kevwun said:

You can't just attach the drives to another pc and access them.  The data is there, but is not in an easily readable state because the array is what lets that happen.

Correct. Without the array, you can't really read the drives. If it is hybrid software driven RAID (would be common on consumer boards), you can sometimes rebuild using Intel Matrix Manager (if that is what was used for the RAID) assuming the drives themselves didn't fail. I'd image the drives rather than working on them directly if you can.  The purpose of RAID 1 is uptime--i.e., to keep everything up and running if one drive fails (you just replace the bad drive once you get notice and the array will keep running on the working drive and then incorporate the new one). If the array controller fails, RAID 1 doesn't help you.  It isn't a backup.  The data is probably there, but it can be a bitch to get back. 

[dad 557]

Several of my coworkers have the same setup since we all purchased our Dell Precision Tower 5810 workstations at the same time. We wiped the HD's and installed Windows Server 2012 R2 on them as soon as we got them. Most of us went with a mirrored RAID setup on our HD's. 

When I was troubleshooting and in the BIOS settings none of the two hard drives were shown in ports 0 or 1 even though they were connected. I think took these HD's out and put them in my coworker's working computer and received the exact same messages and none were shown as being plugged in to ports 0 or 1. One of the hard drives was making a metallic grinding noise. I read that one shouldn't try to access these after they fail b/c one might corrupt more data. Like Dahobbs said they should be imaged first. I don't know what IT has found out yet I haven't heard from them. They just took the machine this morning. 

One of my monitors was blurry so I disconnected the video cable which then caused everything to freeze. It stayed frozen for 20 minutes so I performed a hard shut down. I don't know if possibly unplugging my monitor video cable could have somehow sent a power glitch to the hard drives as ridiculous as that sounds? That would be the only way both fried at the exact same time. OR could both of them have been corrupted because they were writing data while getting hard shut downed.

All of our code is backed up of course. We use TFS as our code repository and that's backed up and so are our databases. It's just the little stuff I lost like all of my old emails saved to outlook .pst files, passwords, old notes, software etc. It'll just be a bitch to build up a dev station from scratch if I have to and i'm thinking I might have to.

[CoTex 1160]

I’ve heard that if you drop them on the ground from way high up, sometimes that fixes the problem.

[kevwun 15790]

Sounds like a hardware problem and pretty much the only way to recover it if the drive has already failed is going to be to send it off to someone who specializes in hard drive recovery.  They will take the drive a part and replace the broken parts in a clean room.  It will be very expensive.

Edited December 5, 2018 by kevwun

[EE2B 1565]

This is why I’m glad all of my engineering workstations are virtual.

[dad 557]

7 minutes ago, kevwun said:

Sounds like a hardware problem and pretty much the only way to recover it if the drive has already failed is going to be to send it off to someone who specializes in hard drive recovery.  They will take the drive a part and replace the broken parts in a clean room.  It will be very expensive.

In that case it's not worth the trouble. I think my company would be willing to pay up to a few grand for the recovery because that makes sense for all the time wasted losing the data but more than that I doubt they would. FUCK.

[kevwun 15790]

If they would be willing to pay a few thousand, you might check in to it.  If in Austin, here's a place.

http://certifieddatarecovery.com/

Never used them, but they have good reviews.

Edited December 5, 2018 by kevwun

[Dahobbs 17560]

53 minutes ago, dad said:

Several of my coworkers have the same setup since we all purchased our Dell Precision Tower 5810 workstations at the same time. We wiped the HD's and installed Windows Server 2012 R2 on them as soon as we got them. Most of us went with a mirrored RAID setup on our HD's. 

When I was troubleshooting and in the BIOS settings none of the two hard drives were shown in ports 0 or 1 even though they were connected. I think took these HD's out and put them in my coworker's working computer and received the exact same messages and none were shown as being plugged in to ports 0 or 1. One of the hard drives was making a metallic grinding noise. I read that one shouldn't try to access these after they fail b/c one might corrupt more data. Like Dahobbs said they should be imaged first. I don't know what IT has found out yet I haven't heard from them. They just took the machine this morning. 

One of my monitors was blurry so I disconnected the video cable which then caused everything to freeze. It stayed frozen for 20 minutes so I performed a hard shut down. I don't know if possibly unplugging my monitor video cable could have somehow sent a power glitch to the hard drives as ridiculous as that sounds? That would be the only way both fried at the exact same time. OR could both of them have been corrupted because they were writing data while getting hard shut downed.

All of our code is backed up of course. We use TFS as our code repository and that's backed up and so are our databases. It's just the little stuff I lost like all of my old emails saved to outlook .pst files, passwords, old notes, software etc. It'll just be a bitch to build up a dev station from scratch if I have to and i'm thinking I might have to.

The monitor and hard-drives going out at the same time point to a common problem source, as I don't think it is possible for the display to kill the drives. Could be that an issue with the motherboard or memory caused both problems. 

What RAID controller were you using?

[kevwun 15790]

Yeah if it was just a failing drive, it should have kept working and something would hopefully have notified you that the array was degraded because only 1 drive was functioning.  When you get it back up, you should have dell open manage installed if you don't have a way to monitor the array now.  It can be configured to let you know if one of your hard drives is predicted to fail and lots of other potential problems.  It is very handy for catching issues before they can cause trouble.

Edited December 5, 2018 by kevwun

[dad 557]

The monitor and hard-drives going out at the same time point to a common problem source, as I don't think it is possible for the display to kill the drives. Could be that an issue with the motherboard or memory caused both problems. 
What RAID controller were you using?

This machine used a software raid controller. Whatever comes with these Dell workstations. Now I’m worried bc IT plans to just replace the drives but if it was caused by another failing part I don’t want it to happen again. I’ll need to discuss backing up our workstations to tape like we do some of our other servers. IT can get me a new machine too since these are just CSC loaners.

[dad 557]

Yeah if it was just a failing drive, it should have kept working and something would hopefully have notified you that the array was degraded because only 1 drive was functioning.  When you get it back up, you should have dell open manage installed if you don't have a way to monitor the array now.  It can be configured to let you know if one of your hard drives is predicted to fail and lots of other potential problems.  It is very handy for catching issues before they can cause trouble.

Is this info that shows the raid status at start up from the motherboard or is it from Dell open manage? It looks like my coworkers raid is degraded. I think mine were always normal.

[George 493]

For cheap backup use veeam endpoint backup (free) and dump to an external hard drive or file share if you have the space. It’s good shit and you can just bare metal restore or restore to a VM if needed. Fuck tape

[kevwun 15790]

Open Manage is inside windows and yeah something is wrong.  The drives should be identical in size.  A RAID doesn't do much good if you can't monitor it's status.  You want to be able to replace a drive as soon as there is a problem.

Edited December 6, 2018 by kevwun

[EE2B 1565]

For cheap backup use veeam endpoint backup (free) and dump to an external hard drive or file share if you have the space. It’s good shit and you can just bare metal restore or restore to a VM if needed. Fuck tape

We use Veeam for our VM backups and it’s awesome. If we just need a single file off the VM, we can “power it up” inside Veeam and transfer the file out.

[dad 557]

Open Manage is inside windows and yeah something is wrong.  The drives should be identical in size.  A RAID doesn't do much good if you can't monitor it's status.  You want to be able to replace a drive as soon as there is a problem.

I think this view that pops up upon boot up will always only show the first drive in port 0 and not the second mirrored drive in port 1 because it shows it as a volume which this two combine to form just one. I really think both fried otherwise if only one had open manage would have warned me to replace and I would have kept going about my business.

[kevwun 15790]

It is possible your motherboard has a problem.  Your IT dept will hopefully find out when they try to reinstall windows.  They either won't be able to do it or may run in to problems when they are setting it up.

[Dahobbs 17560]

2 hours ago, kevwun said:

It is possible your motherboard has a problem.  Your IT dept will hopefully find out when they try to reinstall windows.  They either won't be able to do it or may run in to problems when they are setting it up.

I'm pretty sure the motherboard or memory is bad, causing some bad information to be written to critical sectors of the hard drives making them unreadable. But the noise makes it sound like at least one of the drives had a physical failure as well. 

[kevwun 15790]

Yeah, maybe the motherboard shorted out and got one of the drives in the process.  Hopefully it is completely dead and not an intermittent problem.

[dad 557]

This is what I should see upon start up since I had the same setup as this other coworker.

 

 

I’ll discuss that backup software with our lead after I research how it works. Thanks guys I’ll post an update when I know results.

 

[kevwun 15790]

Thanks for the Veeam recommendation.  I have been looking for something like that and have been playing around with it today.  Can't believe there's a free version that does most everything we need.  They also sell a version that will back up your files to their cloud, correct?

[EE2B 1565]

Yes, I believe it is called Veeam Cloud Connect.

[George 493]

Cloud connect integrates with their enterprise/pay offering. Veeam itself doesn’t have cloud but they have partners who offer it. Don’t think it works with the free version but I could be wrong as they may have changed things. 

[kevwun 15790]

Yeah, cloud connect isn't free.

[elfenix 34266]

If there's random hardware failing mysteriously I'd look at the power supply first.

[Dahobbs 17560]

10 minutes ago, elfenix said:

If there's random hardware failing mysteriously I'd look at the power supply first.

Really good point. Although, normally (for me at least) things work fine once moved into a new system or given a new power supply. These drives didn't apparently. 

[dad 557]

How would one test the power supply? All I know is that it is on but I can’t detect if it’s having spikes or surges.

Would this check that I did on the machine afterwards have indicated a faulty power supply?

[dad 557]

Just wanted to clarify that the hard drive shown as "ok" in the check above is a separate hard drive that I used just for storage. It was not bootable or part of the raid.

[elfenix 34266]

i don't know if there is a good test, aside from running it in a hot box with a load tester ($$$).  there's little things you can buy that clip onto the ATX connector, but all those really do is say "yes, power is on."  easier and cheaper just to get a new one, usually.  problem with dell is that sometimes they use power supplies that meet ATX spec, and some other times they use power supplies that don't, but still use the same connectors so you could destroy your computer if you plug in an ATX supply and feed 12V into a ground. 

Edited December 8, 2018 by elfenix