I recently built an UNRAID server for use as backup storage. The plan was to build a computer and then shuck and use the many USB drives I already had for storage. This mostly went to plan.
TLDR: Network equipment lies about connection speed.
I went with UNRAID because it boots off of a USB drive; saving a SATA port, works well with mixed drive types and sizes, and you can take a drive out of the array and read/write to it from any other PC.
Case: Cooler Master Elite 342 RC-342-KKRJ-GP 400W
CPU: Intel Celeron 2.90 GHz Dual Core FCLGA1151
Motherboard: MSI Pro Series Intel B250
Memory: Corsair Vengeance LPX 16GB (2x8GB) DDR4 DRAM 2400MHz
Boot Drive: SanDisk Cruzer Fit CZ33 16GB
Misc Items likes fans and cables I already had.
I had five 4TB Western Digital drives in USB enclosures that I planned on using in UNRAID. I bought a 4TB SATA drive, getting me a total of six drives. All of the USB drives appeared to work fine in their enclosures, no SMART errors, no problems reading and writing to them. After taking the drives out of their enclosures and putting them into the new server, three of them started reporting SMART errors. I wasn’t happy about this, but figured I would load them up and see what happens, only buying new drives if they actually failed.
Right off my transfer speeds were limited to 10-12MB/s. I was expecting more like 100MB/s, being on a gigabit network. After fiddling around for a bit I did a restart, UNRAID refused to boot. I got a monitor on it and saw in the message buffer that one of the drives was freaking out. I removed the drive and UNRAID booted up. I still had the slow transfer speeds though.
I disconnected the other two SMART error reporting drives. It was still slow. I logged in via the command line and did an rsync between one of the formerly USB drives and the new drive, still slow, albeit I was getting 15MB/s now. I put the two SMART error reporting drives back in.
I was a little lost at this point. If rsync between two sata drives was only getting me 15MB/s, something was seriously up. I figured the former USB drives were crappy and bought another new 4TB SATA drive. A day later I tried another rsync between the new drives. Same problem.
Now I started down the horrible path of googling ‘UNRAID slow.’ There wasn’t much to see here. Lots of posts about parity drives causing slow downs; I wasn’t using a parity drive. Lots of posts about using a cache drive; also not relevant since I’m not using parity.
I kept pounding my head trying everything and anything I could think of until I stumbled onto a problem with rsync. I was using the -z option, this compresses the files before they are copied. This is all good when sending files over the Internet like I normally would, but when going disk to disk the overhead kills the transfer speed. I ditched the -z option from my rsync command and I was getting full speed between all of my disks. 140MB/s+
Then I tried to do a copy from my Windows desktop to UNRAID’s windows share. Slow again. 10-12MB/s. I tried FTP, same deal. I tried rsync (no -z), same deal. Now I had a network problem.
My desktop, the unraid server, and my router all had gigabit ports. My desktop could do 100MB/s (Gigabit speeds, bit vs byte) to other devices in my house. Nothing could do better than 12MB/s to UNRAID though. UNRAID and my router both showed that UNRAID was connected at 1Gbp/s. Something was a liar.
I tried my laptop on the same ethernet cable and wall jack that UNRAID was using. The speed was bad. I tried UNRAID on a different cable and wall jack. The speed was great. Something was wrong with my cabling.
I swapped cables, redid did the wall jacks, no luck. I then tried making a simple network between my desktop and UNRAID, just a switch and two ethernet cables. The speed was great. There was only one thing left that could be the problem, the switch built into my router.
In the end, it ended up being a dodgy port in the router’s switch. Even though it says devices are connected at gigabit speeds, they are not. It works fine as a 100mbp/s port. No latency or dropped packet problems. I plugged the UNRAID server into a different port on the router, problem solved.
That’s my UNRAID saga. Hopefully my experience will help someone else.
Almost as a side note, I’ve been very happy with UNRAID. I have a 24TB array about 1/3 full. I have this 24TB array split into two 12TB user shares. I sync up the shares with rsync now and again as a backup. I’m not using parity. If a drive fails in the array I’ll either replace it with live data or use the backup in the other share to replace it. I still have a good collection of USB drives and I plug those in and rsync to them too, making for a nice third backup.
Here are some of the scripts I use to keep everything synced.
Syncing between the user shares:
rsync -Wa –delete –progress /mnt/user/UNRAID/* /mnt/user/UNRAID-BK/
Syncing between internal disks and USB disks:
dmesg | tail -10
echo -n “Mount which device? ”
echo “Mounting $DEVICE”
mount $DEVICE /root/disk
if [ $? -gt 0 ]
echo “Could not mount $DEVICE”
echo -n “Which disk to sync? ”
echo “SYNCING disk$DISK!”
rsync -aW –delete –progress /mnt/disk$DISK/* /root/disk
echo “UMOUNT $DEVICE”