Rsync and Cron
I was approached to build a web application that extracts student applications from a database and sends the results over to another server in a .csv file at regular intervals.
I needed to build a light-weight application that functioned in the background, was reliable and required little maintenance or intervention. Since creating a web service was off the table for this client, it was a perfect opportunity to use both
cron in any meaningful way. I’m a fan of both. But first,
Who doesn’t love a good cron job? I do. For me it reinforces that machines are good at doing things I’d rather not do; repetition.
1 0-23/2 * * * /usr/bin/php /var/www/app/cronjob.php
- every minute, every two hours, tell php to hit the script at that location
rsync -azvv --delete --log-file=/var/www/html/app/storage/logs/rsync_log -e ssh /var/www/html/app/storage/payload/ firstname.lastname@example.org:/cygdrive/c/Shared/app/data/
- –delete — any files that are deleted on the server that generated the rsync command, will also get deleted on the other server.
- –log-file — gotta write that down.
- – e — choose your shell, in this case ssh
Love it…better than SCP. Built into the behaviour of
rsync is a fallback measure. Should one or more of the rsync cron jobs fail, whatever .csv wasn’t transferred at the time of failure will be transferred at the next successful attempt.