Smells like a timeout of some sort to me. Could be a firewall dropping an "idle" connection. How long does the diff backup take to complete? How long from when you kick off the full backup to the time it errors out? Is that timeout consistent? See if you have ServerAliveInterval set in your ssh config, and if not, consider adding it (and related vars).
Cheers, Greg
