When I'm running an rsync
backup job which involves copying over large files, the machine running the backup (both Mac desktops and Linux servers) grinds to a halt and the load average goes through the roof.
I've tried:
nice
ing thersync
process (doesn't help - the bottleneck is the disk)- On Linux,
renice
ing akjournald
(helps, but seems like a hack and doesn't work on the Mac) - Using the
--bwlimit
rsync
flag (helps, but it means all the transfers are slow - even when they don't need to be)
So, is there any way I can "nice
" rsync
's IO so the machines are useable while the backup is running?
PS: I'm aware of the dangers of rsync
on the Mac… But I've used BackupBouncer to verify my backups, though, and they seem OK.
-
It looks like the
setpriority
API on Mac OS X is supposed to be able to alter IO scheduling (see http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/setpriority.2.html). I don't have any MacOS handy so I can't test thatnice
actually changes the IO priority.On the Linux side,
ionice
is what you're looking for.David Wolever : Ah, thanks. Here's what the `setpriority` manpage says: "When setting a thread into background state the scheduling priority is set to lowest value, disk and network IO are throttled." -- which seems to indicate that disk/network IO are only throttled when something is niced down to 20 (the "background state").David Wolever : Some quick testing also seems to confirm this - when the backup script was only niced to 10 (ie, `nice backup`), I noticed a slowdown... But when it's niced down to 20 (ie, `nice -n 20 backup`), I don't notice any slowdown.David Wolever : (and by "noticed a slowdown" I mean "noticed that interactive programs running on the computer didn't respond as quickly as they normally do")Evan Anderson : It's a little disappointing that you only get throttling at a priority of 20, though.Avery Payne : +1, Nice answer. Always wondered about twiddling IO.From Evan Anderson -
On linux you can use
ionice
http://linux.die.net/man/1/ioniceFrom Rory McCann
0 comments:
Post a Comment