Rsync and Cron

I was approached to build a web application that extracts student applications from a database and sends the results over to another server in a .csv file at regular intervals.

I needed to build a light-weight application that functioned in the background, was reliable and required little maintenance or intervention. Since creating a web service was off the table for this client, it was a perfect opportunity to use both rsync and cron in any meaningful way. I’m a fan of both. But first, cron.

CRON

Who doesn’t love a good cron job? I do. For me it reinforces that machines are good at doing things I’d rather not do; repetition.

1 0-23/2 * * * /usr/bin/php /var/www/app/cronjob.php
  • every minute, every two hours, tell php to hit the script at that location

RSYNC

rsync -azvv --delete --log-file=/var/www/html/app/storage/logs/rsync_log -e ssh /var/www/html/app/storage/payload/ user@111.11.11.111:/cygdrive/c/Shared/app/data/
  • –delete  — any files that are deleted on the server that generated the rsync command, will also get deleted on the other server.
  • –log-file  — gotta write that down.
  • – e  — choose your shell, in this case ssh

Love it…better than SCP. Built into the behaviour of rsync is a fallback measure. Should one or more of the rsync cron jobs fail, whatever .csv wasn’t transferred at the time of failure will be transferred at the next successful attempt.

Brad

Brad Payne is currently the lead developer for the Open Textbook Project whose work focuses on open source software using PHP (LAMP). When not contributing to other developers’ projects on github, he builds his own. Through exploiting API’s and with a penchant for design patterns, he helps BCcampus implement new technologies for post-secondary institutions. Prior to his current position at BCcampus, Brad worked in IT at Camosun College and the BC Ministry of Education.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

*