Moving files to remote server via rsync

Today I find myself once again having to free up space on a production box. I had to make a choice between resizing a partition to accommodate more storage space and moving/backing up certain files to a remote backup box.

I figured resizing data partition on the server was a toll order. It would result in long down-time which is unacceptable for our clients. There's also the risk of data loss with this procedure. So I opted for a safe way although not scalable or sustainable in a long run.

I wanted to copy files/directories to a remote box over SSH deleting the copied data.

I have been using rsync which is an incredibly powerful tool but for incremental backups. This time I wanted rsync to delete the source files and directories as well.

It turns out you can do that by passing --ignore-existing --remove-source-files options. However, this will remove source files but not their corresponding emptied directories. So I had to use a combo; rsync + find which work incredibly well.

Find comes in to search and delete the empty directories. I pass the safe rmdir which strictly deletes empty directories as opposed to more risky rm -r.

I store the directory names which I want to move in a separate file called files-to-move which I loop through using for loop each time passing the directory name to rsync and find.

The complete script looks something like this;

#!/bin/bash
for d in $(cat /home/user/files-to-move); do  
       rsync --ignore-existing --remove-source-files -Pave "ssh -p ssh-port" \
       /source/dir/$d/ user@remote-host:/destination/dir/$d/ && \
       find /source/dir/$d/ -depth -type d -empty -exec rmdir -v "{}" \;
done  

David Okwii

David Okwii is a Systems Engineer who currently works with Uganda's country code top-level domain registry.

Kampala Uganda http://www.davidokwii.com

Subscribe to oquidave@geek:~ #

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!