ReadyNAS PRO - Performance

[immy]

Hi Guys,

I have been struggling to get any decent performance out of the ReadyNAS PRO, I've had this unit for sometime and I've always struggled to get peak performance from this unit.
The current setup is simple:
X-RAID2
3 x Seagate 2TB - ST32000644NS
4GB DDR2 800MHz RAM
RAIDiator 4.2.19

Below are some test results:
ReadyNAS-PRO:/media# dd if=/dev/zero of=zerofile12.txt bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB) copied, 39.1821 seconds, 26.8 MB/s

ReadyNAS-PRO:/media# dd if=/dev/zero of=zerofile12s.txt bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB) copied, 1.32474 seconds, 396 MB/s (I assume this was cached)

I ran the same test as above 10 seconds later and got:
ReadyNAS-PRO:/media# dd if=/dev/zero of=zerofile1s2s.txt bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB) copied, 19.7901 seconds, 26.5 MB/s

ReadyNAS-PRO:/media# hdparm -tT /dev/md0
/dev/md0:
Timing cached reads: 2218 MB in 2.00 seconds = 1108.90 MB/sec
Timing buffered disk reads: 386 MB in 3.01 seconds = 128.29 MB/sec

I think I've noticed the drop in performance since 4.2.19 but I am not 100% certain, anyone got any ideas?

Cheers

[mdgm]

You have an unsupported memory upgrade installed. Whilst it's possible this may help with performance a little, it's not recommended.

Since you have 4GB RAM installed it would be best to do tests transferring a file at least 8GB in size.

What do you get if you do the following?:

Code: Select all

MDGM-NAS:~# df -h


Also note that NetGear's performance numbers are from a fully populated ReadyNAS. Not one with half the bays empty.

What does your initrd.log look like?

Code: Select all

MDGM-NAS:~# cat /var/log/frontview/initrd.log


You may want to check if you have a bad disk. You can do this one of two ways:
1. Run "Disk Test" boot option: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu
2. Power down, remove disks (label order), connect to SATA port on PC and test using vendor tools.

The tests the ReadyNAS runs daily and on boot are short. Whilst they are convenient to run regularly if you have problems running longer tests is a good thing to do.

[immy]

Hi mdgm,

I appreciate you help however...

You have an unsupported memory upgrade installed. Whilst it's possible this may help with performance a little, it's not recommended.

I understand this upgrade is unsupported but it would have no bearing on the dire disk performance.

Since you have 4GB RAM installed it would be best to do tests transferring a file at least 8GB in size.

I can do this but I know it will take an age, see below:

Code: Select all

ReadyNAS-PRO:/media# dd if=/dev/zero of=1gb.txt bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB) copied, 119.334 seconds, 8.8 MB/s
ReadyNAS-PRO:/media# dd if=/dev/zero of=2gb.txt bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB) copied, 126.079 seconds, 8.3 MB/s
ReadyNAS-PRO:/media# dd if=/dev/zero of=8gb.txt bs=1000M count=8
8+0 records in
8+0 records out
8388608000 bytes (8.4 GB) copied, 1027.62 seconds, 8.2 MB/s


What do you get if you do the following?:

df-h shows the following:

Code: Select all

ReadyNAS-PRO:# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              4.0G  2.7G  1.2G  70% /
tmpfs                  16K     0   16K   0% /USB
/dev/c/c              3.6T  2.1T  1.5T  60% /c
/dev/c/c_2011_12_06_00_00
                      3.6T  2.1T  1.5T  60% /c_2011_12_06_00_00


Also note that NetGear's performance numbers are from a fully populated ReadyNAS. Not one with half the bays empty.

hmm, this is interesting. I don't recall any mention of this when purchasing the unit and it seems rather far fetched to expect everyone to fully populate a NAS to get netgear performance numbers.
As you can imagine, spending more money on a unit which is not entirely suitable for business (no encryption or fw) is not on my todo list

What does your initrd.log look like?

Code: Select all

ReadyNAS-PRO:/media# cat /var/log/frontview/initrd.log
[2010/10/02 22:11:53] Factory default initiated due to new disks (no RAID, no partitions)!
[2010/10/02 22:18:02] Updated from RAIDiator  to 4.2.12.
[2010/10/03 11:07:47] Updated from RAIDiator 4.2.12 to 4.2.13.
[2010/10/16 16:32:42] Updated from RAIDiator 4.2.13 to 4.2.15.
[2011/05/07 13:31:41] Updated from RAIDiator 4.2.15 to 4.2.16.
[2011/05/07 15:45:45] Updated from RAIDiator 4.2.16 to 4.2.17.
[2011/09/09 15:16:22] Updated from RAIDiator 4.2.17 to 4.2.19.
[2011/12/08 23:40:15] Updated from RAIDiator 4.2.19 to 4.2.20-T15.


You may want to check if you have a bad disk. You can do this one of two ways:
1. Run "Disk Test" boot option: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu
2. Power down, remove disks (label order), connect to SATA port on PC and test using vendor tools.

The tests the ReadyNAS runs daily and on boot are short. Whilst they are convenient to run regularly if you have problems running longer tests is a good thing to do.

This is going to be tricky but maybe worth a shot, regardless wouldn't SMART errors appear?
Thanks for your assistance thus far.

[immy]

All the disks have been checked and are confirmed working 100%, SMART has also been checked and again confirmed as working with no errors.
The only thing that can possibly be causing this issue is RAIDiator itself.

As a comparison, I have a HP MicroServer with 4 x 3TB drives in RAID10, I can both upload and download via my laptop connected at 1Gbps at a sustained 105MB/sec (via switch).
The same DD test on the HP MicroServer boasts figures of over 220MB/sec, I am inclined to rebuild the ReadyNAS and use RAID10 or RAID5 and see what difference this makes, sadly the ReadyNAS is useless as it currently stands.
This leads to a question, if I do go down the route of RAID5 or RAID10 will RAIDiator allow me to grow/expand the RAID once new drives are added?

If anyone has any other suggestions please do let me know, I do recall reading somewhere about connecting the ReadyNAS via USB to the UPS but I can't seem to find that thread or article any more, can someone point me in the right direction.
Cheers

[mdgm]

I understand this upgrade is unsupported but it would have no bearing on the dire disk performance.

Perhaps. However if there is problems with the memory or compatibility issues there could be any of a large variety of effects.

What do you get if you do the following?:

df-h shows the following:

Code: Select all

ReadyNAS-PRO:# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              4.0G  2.7G  1.2G  70% /
tmpfs                  16K     0   16K   0% /USB
/dev/c/c              3.6T  2.1T  1.5T  60% /c
/dev/c/c_2011_12_06_00_00
                      3.6T  2.1T  1.5T  60% /c_2011_12_06_00_00


That seems to indicate you have a lot on your OS partition. There's plenty of space free, but just for comparison, mine has 798MB in use and has 3.1GB free. What have you installed on the NAS (e.g. community addons or stuff via SSH)? There could be an issue with something you've installed that's effecting performance.

Have you checked the CPU and memory usage of different processes (e.g. using the "top" command) to running to see if something is hogging the CPU or memory.

I think it's probably that you've caused the performance problem.

It also appears you have an active snapshot. Performance will be reduced while you have an active snapshot. Snapshots are intended to be used for periods when you will be backing up data and making minimal writes (If you make too many writes while a snapshot is active it will be invalidated), and then to be removed afterwards.

Also note that NetGear's performance numbers are from a fully populated ReadyNAS. Not one with half the bays empty.

hmm, this is interesting. I don't recall any mention of this when purchasing the unit and it seems rather far fetched to expect everyone to fully populate a NAS to get netgear performance numbers.

You should probably be approaching those numbers with 3 or 4 disks installed, I think. The Pro can easily more than saturate a gigabit connection.

As you can imagine, spending more money on a unit which is not entirely suitable for business (no encryption or fw) is not on my todo list

It is suitable for business use in my view. Encryption is a resource hog and unless you have legal requirements to use it or are paranoid about being hacked it's not worth using it.

What does your initrd.log look like?

Code: Select all

ReadyNAS-PRO:/media# cat /var/log/frontview/initrd.log
[2010/10/02 22:11:53] Factory default initiated due to new disks (no RAID, no partitions)!
[2010/10/02 22:18:02] Updated from RAIDiator  to 4.2.12.
[2010/10/03 11:07:47] Updated from RAIDiator 4.2.12 to 4.2.13.
[2010/10/16 16:32:42] Updated from RAIDiator 4.2.13 to 4.2.15.
[2011/05/07 13:31:41] Updated from RAIDiator 4.2.15 to 4.2.16.
[2011/05/07 15:45:45] Updated from RAIDiator 4.2.16 to 4.2.17.
[2011/09/09 15:16:22] Updated from RAIDiator 4.2.17 to 4.2.19.
[2011/12/08 23:40:15] Updated from RAIDiator 4.2.19 to 4.2.20-T15.


That looks good to me. You last did a factory default on 4.2.12 so you'll have both a native EXT4 volume and 4k sector alignment on all disks.

You may want to check if you have a bad disk. You can do this one of two ways:
1. Run "Disk Test" boot option: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu
2. Power down, remove disks (label order), connect to SATA port on PC and test using vendor tools.

The tests the ReadyNAS runs daily and on boot are short. Whilst they are convenient to run regularly if you have problems running longer tests is a good thing to do.

This is going to be tricky but maybe worth a shot, regardless wouldn't SMART errors appear?
Thanks for your assistance thus far.

Not necessarily. The short tests run daily should pick up issues over time with SMART errors, but sometimes problems can evade detection by the short tests. SMART is good but it isn't perfect.

I am inclined to rebuild the ReadyNAS and use RAID10 or RAID5 and see what difference this makes, sadly the ReadyNAS is useless as it currently stands.
This leads to a question, if I do go down the route of RAID5 or RAID10 will RAIDiator allow me to grow/expand the RAID once new drives are added?

I do think at this point rebuilding the ReadyNAS could be a good way to go, getting a clean setup on the latest firmware. There are some improvements made over time that can unfortunately can only be obtained via a factory default, usually these are only minor benefits but these can be nice to have. Personally I'd stick with X-RAID2. However should you wish to use Flex-RAID (standard RAID levels) here is the info on what volumes are expandable: http://support.netgear.com/app/answers/detail/a_id/19043/~/expanding-the-readynas-volume-when-in-flex-raid-mode.

If you do rebuild the NAS, do performance testing before and after each major change you make particularly if installing community addons or stuff via SSH.

If anyone has any other suggestions please do let me know, I do recall reading somewhere about connecting the ReadyNAS via USB to the UPS but I can't seem to find that thread or article any more, can someone point me in the right direction.

Connecting your ReadyNAS to a compatible UPS (Uninterruptible Power Supply), connecting both the power cable from the ReadyNAS and a USB monitoring cable is a good plan. In the event of a power failure the ReadyNAS would be shutdown safely (of course things like UPS battery failure can still happen but this is typically far less often than power failures). The ReadyNAS is compatible with most APC UPS units that either have a USB port (must not be Serial-to-USB as these are incompatible) or an alternative to USB namely SNMP.

[immy]

Hi mdgm,
Thanks for the advice, I will mull over this in detail and report back any findings.