Rsync to USB - swapping USB disks

[2112.snowdog]

Here is my Goal: 1 backup job with file exclusions on the source to Front USB as the destination, swapping USB disks daily.

I have a Backup job with Rsync as the *source* and Front USB as the destination (full backup every time, delete files on destination prior to backup). I use the Rsync source to access the file exclusions from the source - I want I don't want to copy all files from the source.

This works fine except I get the ACL_TYPE_DEFAULT Operation not supported error 95. If I run the job with large files ( larger than 10GB ) I get the ACL error. When I run the job manually with smaller files (< 5gb), I get no errors.

I have since read that the destination should be set to Rsync, with the FAT32 compatibility enabled. How does one specify Front-USB for the destination when Rsync is selected as the destination?

[TeknoJnky]

fat32 is not compatible with large files over 4gb.

http://en.wikipedia.org/wiki/File_Allocation_Table

you will have to format the usb drives with NTFS or EXT3 to support large files

[2112.snowdog]

Thanks for the reply - but my goal, as mentioned in OP, is to get a backup from the NAS to USB, including large files, and rotate the USB disks. The USB disks are formatted NTFS, and need to remain NTFS.

In fact, I'm getting backups now using Rsync as the source and Front USB as the destination - but I get the ACL_TYPE_DEFAULT error. The files are on the USB destination. Can I ignore those errors? Will I be able to read from the USB disk if I need to?

[TeknoJnky]

with the FAT32 compatibility enabled

this is causing the problems with large files, as to be compatible with fat32 files can't be over 4gb

as far as the acl error, there is another thread @ viewtopic.php?f=31&t=53697


I suspect it has something to do with limited support for access list permissions on linux.

[2112.snowdog]

Apparently I am not explaining things well. Please let me know if this clarifies my environment and situation.

My USB disks are NTFS.
I am not allowed to reprovision them as anything but NTFS for DR/BCP reasons.
I need to exclude files from the source share when copied to the USB disk
Files copied to USB are much larger than 4gb
I am using Rsync as the SOURCE
I am using Front USB as the DESTINATION
USB disks are rotated and occasionally replaced (i.e., the share name is "unknown" at backup time)

With Rsync as the SOURCE, there is no fat32 compatibility option on the DESTINATION. I am running RSync as the SOURCE, and Front USB as the DESTINATION. The job backs up files well over 4GB (one file is 230GB). I get the ACL_TYPE_DEFAULT error on this job.

Pros (as I see them)
- Can specify Front USB as the destination
- Can exclude files I don't want copied to the USB disk
- fast - I am getting about 1.25 GB/min
- Can rotate USB disks without changing the job (or other kludgey methods)

Cons
- ACL_TYPE_DEFAULT error
- Unsure if USB data can be read (haven't tested it yet).

As an alternative, I have tested with a job using the local share name (i.e., *NOT* Rsync) as the SOURCE and RSync as the DESTINATION.
Pros (as I see it)
- No errors in the job

Cons
- Cannot specify Front USB as the destination - this is a deal-breaker for me
- slightly slower at about 1 GB/min

[TeknoJnky]

Cons
- Cannot specify Front USB as the destination - this is a deal-breaker for me

rsync is enabled on a per share basis, have you gone into frontview share management and enabled rsync for the usb share ?

you may have to have a drive connected (ie front usb share active) in order to enable it, but it should work fine because the usb share is the same as any other share.

[2112.snowdog]

There is a distinct difference between choosing a share name and a port name. I am talking specifically about selecting "Front USB" in the dropdown list. If the SOURCE is Rsync, then "Front USB" is available as an option, and this works regardless of the device connected to the front USB port - so well, in fact, that it will create the destination folder if it does not already exist.

The issue is that if Rsync is the DESTINATION, you cannot choose "FRONT USB" (note, this is different than choosing the share name). The only option is to create a share name for the USB (e.g., USB_HDD_5) and explicitly choose the share name as the rsync destination.

As I've mentioned, I would like to have one job that sends the large files to *WHATEVER* is connected the Front USB port, and does not give an error.

I am trying to avoid having to pre-define every USB disk, since I have many NAS's and 100's of USB disks - so managing the USB_HDD_## names individually is problematic at best.

[MSD]

I have exactly the same issue. Surely there is a method as this is for a client. I want them to rotate the drives daily and for a single Rsync backup job to run.
I can't setup two jobs Drive 1 Drive 2 as I am unable to guarantee they will indeed swap out the drives. In turn a single Rsync job to multiple drives is essential. Is there not method of either
1 - Naming the two USB devices with the same name (tried but doesn;t work)
2 - a naming convention that represents to Front Port.

Am very interested in the outcome of this.

[2112.snowdog]

Still having the issue - I see that many other people are as well, with no known solution.

The "Front USB" option in the backup job is what we need to work, as the device connected will change daily. I also use this for clients and do not want to have to change their backup job every time they get a new USB disk (or worse, they never *tell* me they put a new disk in the rotation).

In addition to many rsync issues - permissions getting out of whack and this (95) error, I frequently see the front USB port show that nothing is connected. The only workaround to that is to pull the A/C power and plug it back in - a simple reboot (which can conveniently be done remotely) does not work.

I was actually told by Netgear support that the front usb 3.0 port is not designed for devices that draw a lot of power - like a USB disk!

[MSD]

The "Front USB" option in the backup job is what we need to work, as the device connected will change daily. I also use this for clients and do not want to have to change their backup job every time they get a new USB disk (or worse, they never *tell* me they put a new disk in the rotation).

I couldn't agree more and am in a bit of a dilemma with this as I can be "CERTAIN' that the clients won't place the correct disk in. In turn there is just too much that can go wrong. Crazy!
"NETGEAR" can we PLEASE have the "USB Device - Front Port" named as such when using Rsync. We need a backup job to be able to backup to anydisk in Front Port.

I was actually told by Netgear support that the front usb 3.0 port is not designed for devices that draw a lot of power - like a USB disk!

Oh dear, now that indeed is a problem if true, I have being testing with 2 X USB3 2.5" disks over last few days. So far so good. Disks do go to sleep but luckily so far have woken up for a Backup Job. I will monitor this.

In addition to many rsync issues - permissions getting out of whack and this (95) error

I have my 2 USB Disks formatted differently to test this annoying error. It s been around for "YEARS'
USB HDD1 = NTFS - Backup fails with the ACL_TYPE_DEFAULT: Operation not supported (95). error.
USB HDD2 = EXT3. Backup = 100% sucessful.
Annoying as it may be Just format the drives as EXT3, It backs up a lot faster as well. If you need to access the USB Data directly plugged into a PC you could use various utilities for that...... a couple mentioned here http://www.howtoforge.com/access-linux- ... om-windows.

[2112.snowdog]

MSD - Thanks for the tips. I have been hesitant to reformat the disks with anything but NTFS because of the reasons I mentioned (most importantly, these are out in the field at customer locations and if they drop a new disk into the rotation, it'll be NTFS).

I'm really looking to get this working using "off the shelf" USB disks.

The Netgear claim that the front usb port is "not designed for hard drives" (again, I ask...what the #3!! else would we connect to a USB 3.0 port on a NAS!?!?) is in response to the fact that the hard drive occasionally is not recognized as even being connected - "No USB shares exist". I think this happens as a result of swapping the disks.

USB 3.0 is a necessity since we're dropping 300-500+ GB on the NAS and then need to get it to the USB - that's the whole idea here, to have the NAS handle getting the data to the USB and not bother the rest of the infrastructure/bandwidth with what is essentially a second backup job every night.

[mdgm]

Whoever told you that is incorrect. The USB 3 is for use with USB disks however unfortunately the USB ports aren't compatible with as many devices as the USB 2 ports.

Now as for drives going to sleep that's nothing to do with USB3 vs USB2 and a setting that may be alterable using software from the manufacturer of the drive. I think in the case of SeaGate FreeAgent drives the NAS can stop the USB drive from going to sleep on current firmware.

Note you can format the disks to EXT3 using Frontview so if you need to use that you can format new USB disks remotely if need be (or tell the client you need to supply the USB disks used with the NAS and format them before sending them to them).

[2112.snowdog]

MDGM - Thank you!

I was hoping someone could contradict what we were told by Netgear...that just didn't make sense.

I will definitely try EXT3 on a few devices and make sure I can access the disks from other devices (Windows PC's) for recovery. The whole Idea behind the USB is in case the servers and NAS are inaccessible (i.e., building failure), so it is absolutely 100% required to be accessible from a device *other* than the NAS.

[mdgm]

EXT3 is a native Linux filesystem and can be read by any Linux machine. There are 3rd party drivers/software you can get to read the disks using Windows/Mac.

[2112.snowdog]

I did not mean to suggest the EXT3 filesystem on a USB disk is unreadable - my concern is finding a device in a DR situation in which we can access the data. Compared to a tape solution in which finding a matching tape drive would be extremely difficult, having to load an EXT3 utility on a Windows box is easy enough.

It is certainly easier to find a Windows box in a pinch than a 'nix device that'll read USB 3.0....

I am looking, however, to keep the solution as flexible and easy to manage as possible. And by "easy to manage", I'm shooting for email alerts to tell me everything is running, and quarterly test restores to verify the USB viability - with no concern for which disk is connected or if the remote users add a new disk into the rotation.