ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE & CAT6

Please post questions regarding performance of the ReadyNAS here. (How to optimize the ReadyNAS performance)

ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE & CAT6

Postby emory » Tue Feb 21, 2012 8:20 pm

hello,

I have a ReadyNAS NV+ that is giving me fits. I recently got iperf installed in order to do some network testing.

nas01:~# uptime
21:09:45 up 28 days, 23:58, 1 user, load average: 0.40, 0.51, 1.75
nas01:~# iperf -s -i 10
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 256 KByte (default)
------------------------------------------------------------
[ 6] local 192.168.11.6 port 5001 connected with 192.168.11.250 port 59760
[ 6] 0.0-10.0 sec 136 MBytes 114 Mbits/sec
[ 6] 0.0-10.0 sec 136 MBytes 114 Mbits/sec

Other destinations on the same switch don't seem to have this problem:

[ 4] 4.0- 6.0 sec 204 MBytes 855 Mbits/sec
[ 4] 6.0- 8.0 sec 201 MBytes 842 Mbits/sec
[ 4] 8.0-10.0 sec 186 MBytes 780 Mbits/sec
[ 4] 0.0-10.0 sec 1000 MBytes 838 Mbits/sec

All of these systems are Mac OS X.

This is with a Netgear unmanaged gigabit switch, swapped cables, ports, etc. No jumbo frames enabled anywhere (or supported on my switch).

The web admin console says I'm negotiating at gigabit ethernet (I have it on auto). ethtool says:

nas01:~# ethtool eth0
Settings for eth0:

Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: externel
Auto-negotiation: on

Pretty odd that full duplex gigabit ethernet can only manage to move ~100Mbit on the wire.
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby mdgm » Tue Feb 21, 2012 8:24 pm

What version of RAIDiator?

What brand and model disks?

The NV+ (v1) is an evolutionary improvement over a product released back in February 2006 with almost the same hardware. It is faster than a 100Mbit ethernet device but it can go nowhere near saturating gigabit ethernet. The NV+ (v1) has a very slow CPU by today's standards.
User avatar
mdgm
NETGEAR ReadyNAS Technical Expert
 
Posts: 32485
Joined: Tue Feb 17, 2009 9:34 pm
Location: Down Under
ReadyNAS: RN516

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Tue Feb 21, 2012 10:58 pm

Even the old sparcs could use an SBUS gigabit ethernet adapter. I even had an IPX with FDDI Back In The Day™.

114Mbit is a pretty far cry from gigabit ethernet though.

RAIDiator!!version=4.1.8,time=1314924646

3x ST31000528AS
1x ST31000524AS

What bearing would the disks have in raw network performance? I'm not testing reads or writes just wire speed.
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby mdgm » Tue Feb 21, 2012 11:46 pm

emory wrote:114Mbit is a pretty far cry from gigabit ethernet though.

Still faster than 100Mbit. You should be getting faster than 114Mbit though.
emory wrote:RAIDiator!!version=4.1.8,time=1314924646

Try 4.1.9-T2: http://www.readynas.com/forum/viewtopic.php?f=17&t=59222
There is a general performance issue with RAIDiator 4.1.8.
emory wrote:What bearing would the disks have in raw network performance? I'm not testing reads or writes just wire speed.

Good point.
User avatar
mdgm
NETGEAR ReadyNAS Technical Expert
 
Posts: 32485
Joined: Tue Feb 17, 2009 9:34 pm
Location: Down Under
ReadyNAS: RN516

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Thu Mar 08, 2012 2:31 pm

It's very frustrating, there are few things as frustrating as watching an rsync copy from the NAS skip along at 12MB/sec.

If I mount the filesystem using AFP I can read 18MB/sec so about ~144Mbit which is an improvement but still not good. I'm not willing to install a beta firmware on this system to test, will likely just have to replace with a Synology or finally just give up and buy a JBOD again.
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby TeknoJnky » Thu Mar 08, 2012 4:24 pm

rsync is cpu bound, that is about as fast as it is going get on the nv+ v1
nv+ ~ 1gb ram ~ 4x WDC WD20EARS-00S8B1 ~ 5555 GB ~ power supply dead
ultra4 ~ 4 gb ram ~
pro business ~ 4gb ram ~ 2x Seagate ST4000DX000-1C5160 ~ 2x Seagate ST4000DM000-1F2168 ~ 10tb
pro 6 ~ 4gb ram ~ dual redundancy ~ 4x Hitachi HDS724040ALE640 ~ 2x Seagate ST4000DM000-1F2168 ~ 14tb
A/V streaming ---> Subsonic ---> HTC One ~ Note 2 ~ Nexus 7
User avatar
TeknoJnky
ReadyNAS Addict
 
Posts: 2963
Joined: Mon Oct 13, 2008 1:34 pm
Location: MO
ReadyNAS: Pro

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Thu Mar 08, 2012 9:47 pm

not using the rsync daemon on the nv+, but against filesystems exported from it.

thanks for the attempt though.

swing and a miss i'm afraid :(
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby mdgm » Thu Mar 08, 2012 9:56 pm

Whether you're using the NAS as a Rsync client or server the CPU's going to be a bottleneck.

If you download your logs (Status > Logs > Download all logs) what is the "block size" in your volume.log?
User avatar
mdgm
NETGEAR ReadyNAS Technical Expert
 
Posts: 32485
Joined: Tue Feb 17, 2009 9:34 pm
Location: Down Under
ReadyNAS: RN516

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Thu Mar 08, 2012 10:09 pm

well, again, it isnt using rsync on the readynas at all. it isnt using rsync, it is just presenting filesystems.

im just using rsync as a way to copy data since it lets me see throughput when it reads from the readynas.

i will look for that in the logs, but vmstat on the readynas hasnt given me reason to believe im hitting a cpu wall with one client.
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby mdgm » Thu Mar 08, 2012 10:19 pm

Presenting filesystems how? If the ReadyNAS isn't the Rsync client/server then what machines are? If NAS is computer A, then copying to computer C via computer B isn't going to be that quick.

Do look for that entry in the logs as depending on the value it may indicate a way performance can be improved.

The NV+ (v1) is pretty old hardware (evolutionary improvement over the NV which was released in Feb 2006). Considering the age of the hardware the performance isn't that surprising.

Newer models such as the ReadyNAS Pro Series as much faster.
User avatar
mdgm
NETGEAR ReadyNAS Technical Expert
 
Posts: 32485
Joined: Tue Feb 17, 2009 9:34 pm
Location: Down Under
ReadyNAS: RN516

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby TeknoJnky » Thu Mar 08, 2012 10:35 pm

emory wrote:well, again, it isnt using rsync on the readynas at all. it isnt using rsync, it is just presenting filesystems.


Uhm. I'm afraid you are wrong.

rsync works at a client/server level.

an rsync client connects to an rsync server/service.

it is not a simple matter of 'presenting filesystems'.

it works by comparing checksums on both ends of the connection, this is a primary part that requires cpu processing and a limitation for low power systems.

it does not matter whether the rsync initiates the connection to your computer, or vice versa, there is processing going on at both ends.

I would suggest you start @ http://en.wikipedia.org/wiki/Rsync or any number of other rsync info on google
nv+ ~ 1gb ram ~ 4x WDC WD20EARS-00S8B1 ~ 5555 GB ~ power supply dead
ultra4 ~ 4 gb ram ~
pro business ~ 4gb ram ~ 2x Seagate ST4000DX000-1C5160 ~ 2x Seagate ST4000DM000-1F2168 ~ 10tb
pro 6 ~ 4gb ram ~ dual redundancy ~ 4x Hitachi HDS724040ALE640 ~ 2x Seagate ST4000DM000-1F2168 ~ 14tb
A/V streaming ---> Subsonic ---> HTC One ~ Note 2 ~ Nexus 7
User avatar
TeknoJnky
ReadyNAS Addict
 
Posts: 2963
Joined: Mon Oct 13, 2008 1:34 pm
Location: MO
ReadyNAS: Pro

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Fri Mar 09, 2012 6:11 am

I'm not a very good teacher, so I'm probably going to go about this all wrong.

When you read that Wikipedia page or the "any number" of other materials on rsync on Google, you're absolutely right that they are pretty consistent and that methods of using rsync are pretty much common knowledge. I'm a grizzly neck beard and I prefer to do things Ye Olde Fashioned way by reading the man page, so that's what my examples will refer to.

Code: Select all
USAGE
     You use rsync in the same way you use rcp. You must specify a source and a
     destination, one of which may be remote.


One of the interesting things that apparently you're not aware of is that rsync doesn't need to be using another system at all. Rsync doesn't care if a group of files is local, presented by a ReadyNAS, living on an EMC iSCSI monstrosity or a microwave oven.

In some cases such as very high-latency low-bandwidth links between two locations, you will want to have that rsync "client/server" relationship, if not as a daemon or service on the remote end, at least to do the calculations and comparisons to avoid transferring the whole file back and forth over that slow connection. This is what rsync was really built for originally. Even if you don't connect over the rsync service, you can use ssh to connect to a remote host and then use the other end's rsync software and only shuffle over the bits that you need to.

Code: Select all
You  can also use rsync in local-only mode, where both the source and des-
       tination don't have a ':' in the name. In this case  it  behaves  like  an
       improved copy command.


In a situation like mine, where the CPU would be the limiting factor, I can (or rather, should be able to if the ReadyNAS worked as designed) move the contents of filesystems over the wire faster than the SPARC can compute hashes on large files (in my case backed up snapshots of a VMWare VM) and figure out which differences there are in rsync. My workstation will be the system conducting the comparisons of the two files. I have the filesystems presented by my ReadyNAS mounted locally, and as far as rsync would be concerned, they're a "local file" if I don't tell it otherwise.

This is pretty common use, since rsync can smartly copy things from Directory A to Directory B, respecting metadata, permissions, and only copying over the things it needs to. Doing it this way puts all of the computational load of the comparison of the SOURCE and DESTINATION, but my workstation doesn't even notice it since it's ample (i7 2600k @4GHz). By using --progress and --partial, I can recover from an interrupted copy and also see how much throughput I've got. That's how I know how slow it is to write to and read from my ReadyNAS over the network and how I can test multiple protocols (NFS/AFP) so easily.

I am very familiar with the computational limitations of the SPARC family of CPUs. I ported SETI@Home to sun4c and sun4m on two BSD distributions and was the port maintainer of those and m68k for a little while Back In The Day™. That is why I don't put the load of calculating file checksums on the ReadyNAS NV+, it clearly has enough to worry about.

Hopefully this illustrates things better and now you can go forth and use the mighty rsync in new ways, like the developers intended!
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby TeknoJnky » Fri Mar 09, 2012 7:19 am

In any case, the readynas is still involved in all the IO that must take place and is still limited by the processing power of the cpu.

The CPU does not even have to be maxed out at 100% utilization for it to be the limit.

In the 5? some years I have had my nv+, I can tell you that you will not get more than about 12-15 max xfer rate with rsync on it, and often much less.

The generally fastest way to make transfers, is to use frontview to backup initially via NFS, then modify the backup job to use rsync for subsequent transfers.
nv+ ~ 1gb ram ~ 4x WDC WD20EARS-00S8B1 ~ 5555 GB ~ power supply dead
ultra4 ~ 4 gb ram ~
pro business ~ 4gb ram ~ 2x Seagate ST4000DX000-1C5160 ~ 2x Seagate ST4000DM000-1F2168 ~ 10tb
pro 6 ~ 4gb ram ~ dual redundancy ~ 4x Hitachi HDS724040ALE640 ~ 2x Seagate ST4000DM000-1F2168 ~ 14tb
A/V streaming ---> Subsonic ---> HTC One ~ Note 2 ~ Nexus 7
User avatar
TeknoJnky
ReadyNAS Addict
 
Posts: 2963
Joined: Mon Oct 13, 2008 1:34 pm
Location: MO
ReadyNAS: Pro

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby StephenB » Fri Mar 09, 2012 8:23 am

emory wrote:
Code: Select all
USAGE
     You use rsync in the same way you use rcp. You must specify a source and a
     destination, one of which may be remote.


One of the interesting things that apparently you're not aware of is that rsync doesn't need to be using another system at all.
Hey, the posters are only trying to help you. TeknoJnky was perfectly correct in what he said. Rsync is not a networked file system like NFS, CIFS or AFP. Your posts did not describe how you are using rsync very clearly. In your early posts you talked about sparc performance, which led folks to the reasonable conclusion that you were running it on the NAS. They responded accordingly.

I gather you are mounting the NAS folders with NFS, and using a workstation to run rsync - which forces the workstation to compute all the checksums by reading all the file data over the network. It's not immediately obvious to me if that is faster than running rsync on the NAS or not.

What I do know is that you get the best performance on the NAS by running an NFS copy first, and then follow that up with rsync to get incremental updates. Also, that NV+ V1 is not a very high performing NAS, you will get much better speeds with an ultra or pro. So if you choose to replace the NAS you should look at those products also.
User avatar
StephenB
Incurable ReadyNAS Junkie
 
Posts: 9474
Joined: Mon Oct 05, 2009 5:08 pm
ReadyNAS: Pro

Re: ReadyNAS NV+ with slow networking, 114Mbit/sec @ GigE &

Postby emory » Fri Mar 09, 2012 8:47 am

Well you're right about one thing: the IO is taking place on the ReadyNAS even though rsync is not!

No disrespect intended, I genuinely assumed that TeknoJnky didn't realize you could use rsync in this manner since they said:

Uhm. I'm afraid you are wrong.
rsync works at a client/server level.


I said repeatedly I wasn't using rsync on the ReadyNAS. I don't know how I can be more clear than that, but I'd welcome feedback. I would like to be better at teaching and communicating new concepts to technical people.
emory
ReadyNAS Newbie
 
Posts: 14
Joined: Sat May 26, 2007 2:47 pm

Next

Return to Performance



Who is online

Users browsing this forum: No registered users and 4 guests