FAQ/Article Request: Mounting ReadyNAS Disks with Linux

[TeknoJnky]

Thanks for the reply.

I would say that the majority of space will be used by large files (iso, avi and flac), but that may or may not make up the majority of quantity (images, program files and user data).

This is an NV+ so mostly I expect an eventual max of 2tb x4 (6tb raid-x'd, assuming future 2tb drive support).

I expect that by the time I max out a 4+tb array I wll be looking to upgrade to the pro or its successor.

I assume that larger inodes will be faster to rebuild/expand?

I'm further assuming that formatting a drive with 4k blocks will remain expandable (raid-x) to begin with?

[Zappes]

The number of inodes should not affect the expansion time at all, unless I got something fundamentally wrong, and 16K should be a reasonable size for the inodes in your case - as 16K is the default for a ReadyNAS, I guess that it will do for all situations that Netgear expects in normal usage scenarios.

[Jeroen1000]

Hey Zappes,

I went clue finding about the issue. As I still suspect it is not the CPU's fault we cannot mount disks with a blocksize larger than the page size (that's the real issue here)


http://lwn.net/Articles/249169/

Still don't get the details but it's safe to say adjusting the blocksize is a very good idea indeed.

[Zappes]

That post actually confirms that it's the CPU, and the patch does quite exactly what I suggested earlier - it splits up the larger blocks so they are stored on more than one page in memory. The surprising part (at least for me) is that this seems to be less of a problem than I thought.

[Jeroen1000]

Oh well, learning is fun right:)! So...the Windows kernel would handle the split ups then. I'm starting to get it..slowwwwly

[mhe4]

This is a very interesting thread. I have an old 600/X6 which is running Radiator 4.1.3, but I can't recall if I did it as a clean install after a crash or as an upgrade from 3.x. Is there a way to tell what my block size is without SSH, linux knowledge, or removing a drive? In other words, does Frontview (or an add-on) tell you anywhere what your disk block size is?

[chirpa]

In other words, does Frontview (or an add-on) tell you anywhere what your disk block size is?

viewtopic.php?p=91869#p91869

[eisandar]

Hi, all and Zappes in particular.
Well apart from me being a newbie in the NAS and linux business. Is this something you might recommend? Since my readynas duo is new I have no data on it that needs to be saved as yet, which means that this procedure might be less painful to do now instead of later.

How do I get shell access to the readynas, is this via the root ssh addon? stopping samba is that done on your linux computer or do you mean on the duo?

As you might realise I have lots of gaps in my knowledge regarding the use of linux, more or less my experience stretches to recompiling the kernel and mounting usb/hd stuff on already working linux installs.
But I don't want to have this rude awakening when my nas solution ends belly-up on me...anything that might save me when that day happens is worth doing.

Well, it worked.

After the backup was through, I disabled all add-ons and stopped Samba in order to avoid any kind of open files on the volume. The I unmounted the volume and formatted it with the following parameters:

Code: Select all

mkfs.ext3 -b 4096 -i 16384 /dev/c/c

I thought that the assumption of having files of at least 16K was OK for a volume that mostly contains media files, so I decided to stick with 16K per inode even though the block size was just 4K.

After the file system was written, I mounted it with "mount /dev/c/c" and I had my volume back. Then I manually created the share and home directories, set the permissions correctly and tested if the symlinks in the root directory where still working. As everything was OK, I started Samba again and tested share access - no problems there, either.

Unfortunately it is possible to create a backup job that saves the complete RAUD volume without having a normal way to completely restore it again... This is why I restored the home directories manually:

Code: Select all

rsync -av /USB_HDD_2/home /c

This worked as expected, my home shares are up and running.

Then I have created a backup job for every share that restores the share's content from the external disk. The first job is running right now, the second one is queued - if the restore speed equals the excruatingly slow backup process (26 hours for 159GB), I should be ready to restart Twonky in about 25 hours.

Bottom line: If you have basic linux skills, a certain amount of confidence in your abilities, an external disk for backing up your data, shell access and no fear of blowing your warranty to tiny little pieces, it is actually possible to switch the block size of the ReadyNAS to something that enables basic disaster recovery. I still think that it's more or less outrageous that a procedure like this is required in order to turn the device into what I would call a proper NAS. A storage solution that uses a proprietary file system (EXT3 with 16K blocks is exactly that) is something I would never recommend buying - Netgear should act really fast and include the option to change block size into the fatory-reset-functionality for Frontview.

[Zappes]

Hi, all and Zappes in particular.
Well apart from me being a newbie in the NAS and linux business. Is this something you might recommend? Since my readynas duo is new I have no data on it that needs to be saved as yet, which means that this procedure might be less painful to do now instead of later.

How do I get shell access to the readynas, is this via the root ssh addon? stopping samba is that done on your linux computer or do you mean on the duo?

As you might realise I have lots of gaps in my knowledge regarding the use of linux, more or less my experience stretches to recompiling the kernel and mounting usb/hd stuff on already working linux installs.
But I don't want to have this rude awakening when my nas solution ends belly-up on me...anything that might save me when that day happens is worth doing.

It is definitely worth doing as it will save you if the device should ever break. If your ReadyNAS is still empty, you can't do anything wrong - if the procedure shouldn't work, simply do a factory reset and you're golden.

You get shell access using the two addons "enable shell access" and "enable root access" - note that you must install BOTH of them. After that you connect to your ReadyNAS using SSH (search for a tool named "putty" with google...), log in as "root" with the admin password you use in frontview and you are in.

After that you might follow the procedure I've described to the forum user "maddoc" in a PM I wish to quote here:

Thanks a lot for your help... I am a linux newbie...Can you please write me a detailed guide on which commands I should enter?
that means I do not know how to stop addons or samba, unmount or other... I do not need to make any backup or restore since I've already backupped everything.

Thanks a lot for you help...it will be really annoying to downgrade the firmware, install 500gb drives, format, change drives with 1tb and upgrade firmware!

I would save a lot of time!

Hi Maddoc,

well, it's not too difficult, actually. You stop the addons by simply deactivating them with Frontview - that doesn't even involve any Linux skills. To stop Samba, use this command:

Code: Select all

/etc/init.d/samba stop

If your RAID is completely empty, you can make the part where you have to save and restore the home directories really easy... Just use the following commands before doing the actual thing:

Code: Select all

mkdir /oldfiles
cp -a /c/* /oldfiles

This simply copies anything on the RAID array to the system partition. As your NAS is esentially empty, this will be no problem.

Now you have to unmount the RAID array as it can only be formatted when it's not mounted:

Code: Select all

umount /dev/c/c

Now format the bugger - this is the moment we have been waiting for...

Code: Select all

mkfs.ext3 -b 4096 -i 16384 /dev/c/c

Mount the partition:

Code: Select all

mount /dev/c/c

Restore your copied data and delete the backup - be sure to get the second command (rm) EXACTLY as it is writte down here, errors may be quite unpretty:

Code: Select all

cp -a /oldfiles/* /c
rm -rf /oldfiles

Restart Samba:

Code: Select all

/etc/init.d/samba start

At this point you should be able to access your home shares again. Now go to Frontview, re-enable the services you need (uPnP-Sever or whatever) and reboot the device. If everything works: Great. If not: Factory reset, redo from start, try top figure out what went wrong. Google is your friend.

I hope that this helps.

edit 2008-11-09: As Eisandar pointed out, there is a directory "/backup" on the ReadyNAS per default and it's actually a symlink to a dir on the RAID volume... I originally suggested that directory name in the guide above, which would have been a fatally bad idea. I changed "/backup" to "/oldfiles" in the guide so nobody would lose data because of my fault. Thanks for the hint, Eisandar.

[eisandar]

Ok, I followed these instructions almost exactly and everything worked like a charm.

The small differences that I made were:
1. I only installed one add-on, the "EnableRootSSH_v1.0", this gave me root access over ssh so I guess that it made the same thing as the two other add ons that Zappes recommended.
2. On my new Duo there already existed a /backup directory, which in addition also was an symbolic link to /c/backup so I didn't use that name. Instead I created my own directory: /oldfiles to use as an backup during the formatting of c.
This meant that everywhere in the instructions where /backup is mentioned I instead wrote /oldfiles

BTW: I used the old trustworthy "Putty" software as SSH-client.
http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Now I have a filesystem that should be readable in any computer with ext3 support, this feels good as an starting point for my computer security upgrade. (moving on from burning DVD's as backup discs, way to seldom and also I have noticed that some of my old discs arent readable anymore either.) The next step for me is to add a second NAS in a friends home to make the family photos redundant, theft and fire might still do a lot of damage to my digital life.

Thank you Zappes for not only giving me the solution but also for pointing out the existance of this problem. I didn't know about this before accidentally stumbling upon this thread.
/Eisandar

You get shell access using the two addons "enable shell access" and "enable root access" - note that you must install BOTH of them. After that you connect to your ReadyNAS using SSH (search for a tool named "putty" with google...), log in as "root" with the admin password you use in frontview and you are in.

Stop the addons by simply deactivating them with Frontview - that doesn't even involve any Linux skills.
To stop Samba, use this command:

Code: Select all

/etc/init.d/samba stop

If your RAID is completely empty, you can make the part where you have to save and restore the home directories really easy... Just use the following commands before doing the actual thing:

Code: Select all

mkdir /oldfiles
cp -a /c/* /oldfiles

This simply copies anything on the RAID array to the system partition. As your NAS is essentially empty, this will be no problem.

Now you have to unmount the RAID array as it can only be formatted when it's not mounted:

Code: Select all

umount /dev/c/c

Now format the bugger - this is the moment we have been waiting for...

Code: Select all

mkfs.ext3 -b 4096 -i 16384 /dev/c/c

Mount the partition:

Code: Select all

mount /dev/c/c

Restore your copied data and delete the backup - be sure to get the second command (rm) EXACTLY as it is written down here, errors may be quite unpretty:

Code: Select all

cp -a /oldfiles/* /c
rm -rf /oldfiles

Restart Samba:

Code: Select all

/etc/init.d/samba start

At this point you should be able to access your home shares again. Now go to Frontview, re-enable the services you need (uPnP-Sever or whatever) and reboot the device. If everything works: Great. If not: Factory reset, redo from start, try top figure out what went wrong. Google is your friend.

Edit: I have too replaced the /backup with /oldfiles so no one will get confused and make a bad mistake. /Eisandar

[TeknoJnky]

Great, thanks for putting it all together.

I have what might be a silly question, but here goes..

If I start out with a single drive, formatted with 4k blocks as instructed above... when I add additional drives, will they be formatted the same (4k blocks) automatically, or would this require re-building and reformatting the array once all drives are added?

[Zappes]

The mirror copies will also have 4K blocks as the mirroring happens at a lower level than the file system. No problem there.

[Davide-NYC]

Why oh why is this not an option at setup?
Feature request!

[mspen]

The significance of being able to recover my data long after the ReadyNAS NV+ has died without replacement cannot be emphasized enough. The ReadyNAS NV+ is a good concept and a reasonable implementation compared to competitive versions. But at some point it will die perhaps after the device is not supported any longer. At that point I will need to move my data. Of course that point will be predicted imediately after failure.

First of all, Zappes, thank you for both your procedure and noticing that this is a problem. Soon after the purchase of the ReadyNAS, I did test moving a disk to a Linux box to view. This was easy and comforting. I was not paying attention until after I had updated my ReadyNAS NV+ to 4.1.4 that I no longer had a generic way to do this.

Your's and Eisandar's procedures were described well. I have implemented them on a ReadyNAS NV+. All seems well so far. I am curious how it will go after the third drive is added. I saw in an earlier post that it would be correct. I am currently copying substantial amounts of data to the ReadyNAS to solidify the testing.

[bcurry]

Just for clarification for me please...

I just ordered an NV+. Will my warranty be blown if I format to 4k block size?

I'd hate to blow a 5 year warranty for the off chance that the NAS will fail before the main Linux kernel development updates to read 16k block size.

Thanks!

bc