Download and Upload over an unstable connection in Linux

Posted on Wed 13 April 2016 by Pavlo Khmel

RSYNC has option --partial (keep partially transferred files) or -P same as --partial --progress.
Script below uploads file over ssh and restart rsync if it failed in 1 min.

Script: rsync-retry.sh

#!/bin/bash
source=$1
destination=$2
command="rsync -avP -e ssh $source $destination"
counter=0
$command
while [ $? -ne 0 ]; do
    let counter=counter+1 
    echo "########################################"
    echo "## rsync failed - restart $counter in 1 min. ##"
    echo "########################################"
    sleep 60
    $command
done

Output example:

# /root/rsync-retry.sh ./backup.tar root@192.168.0.1:/storage/backup.tar
sending incremental file list
backup.tar
Write failed: Broken pipekB/s 2118:43:29

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (61131 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(605) [sender=3.0.9]
########################################
## rsync failed - restart 1 in 1 min. ##
########################################
sending incremental file list
backup.tar
Write failed: Broken pipekB/s 2818:22:52

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (68782 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(605) [sender=3.0.9]
########################################
## rsync failed - restart 2 in 1 min. ##
########################################
sending incremental file list
backup.tar
   128759392   0%  184.01kB/s 3436:48:32