Brad Payne

Developer

Rsync and Cron

by · November 23, 2013

I was approached to build a web application that extracts student applications from a database and sends the results over to another server in a .csv file at regular intervals.

I needed to build a light-weight application that functioned in the background, was reliable and required little maintenance or intervention. Since creating a web service was off the table for this client, it was a perfect opportunity to use both rsync and cron in any meaningful way. I’m a fan of both. But first, cron.

CRON

Who doesn’t love a good cron job? I do. For me it reinforces that machines are good at doing things I’d rather not do; repetition.

[bash]1 0-23/2 * * * /usr/bin/php /var/www/app/cronjob.php[/bash]

  • every minute, every two hours, tell php to hit the script at that location

RSYNC

[bash]rsync -azvv –delete –log-file=/var/www/html/app/storage/logs/rsync_log -e ssh /var/www/html/app/storage/payload/ user@111.11.11.111:/cygdrive/c/Shared/app/data/[/bash]

  • –delete  — any files that are deleted on the server that generated the rsync command, will also get deleted on the other server.
  • –log-file  — gotta write that down.
  • – e  — choose your shell, in this case ssh

Love it…better than SCP. Built into the behaviour of rsync is a fallback measure. Should one or more of the rsync cron jobs fail, whatever .csv wasn’t transferred at the time of failure will be transferred at the next successful attempt.

Tags:

Brad

I am a developer with an 18-year career in the Information Technology sector. Over the last half-decade, I’ve dedicated myself to advancing my expertise in the realm of intelligent information systems with a Master of Science in Information Systems (MScIS) degree. Notably, I recently completed a substantial socio-technical study, examining the feasibility of implementing responsible AI (RAI) within the public sector. Prior to my role in the public service, I undertook diverse software development roles as a contractor, a team lead and providing valuable services to post-secondary institutions. My driving passion revolves around the convergence of technology and the law, with a particular focus on the capacity of ethical AI systems to shed light on critical issues.

You may also like...