When I'm running an rsync backup job which involves copying over large files, the machine running the backup (both Mac desktops and Linux servers) grinds to a halt and the load average goes through the roof.
I've tried:
niceing thersyncprocess (doesn't help - the bottleneck is the disk)- On Linux,
reniceing akjournald(helps, but seems like a hack and doesn't work on the Mac) - Using the
--bwlimitrsyncflag (helps, but it means all the transfers are slow - even when they don't need to be)
So, is there any way I can "nice" rsync's IO so the machines are useable while the backup is running?
PS: I'm aware of the dangers of rsync on the Mac… But I've used BackupBouncer to verify my backups, though, and they seem OK.
-
It looks like the
setpriorityAPI on Mac OS X is supposed to be able to alter IO scheduling (see http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/setpriority.2.html). I don't have any MacOS handy so I can't test thatniceactually changes the IO priority.On the Linux side,
ioniceis what you're looking for.David Wolever : Ah, thanks. Here's what the `setpriority` manpage says: "When setting a thread into background state the scheduling priority is set to lowest value, disk and network IO are throttled." -- which seems to indicate that disk/network IO are only throttled when something is niced down to 20 (the "background state").David Wolever : Some quick testing also seems to confirm this - when the backup script was only niced to 10 (ie, `nice backup`), I noticed a slowdown... But when it's niced down to 20 (ie, `nice -n 20 backup`), I don't notice any slowdown.David Wolever : (and by "noticed a slowdown" I mean "noticed that interactive programs running on the computer didn't respond as quickly as they normally do")Evan Anderson : It's a little disappointing that you only get throttling at a priority of 20, though.Avery Payne : +1, Nice answer. Always wondered about twiddling IO.From Evan Anderson -
On linux you can use
ionicehttp://linux.die.net/man/1/ioniceFrom Rory McCann
0 comments:
Post a Comment