Consider: workstation$ eval $(ssh-agent) workstation$ ssh-add ~/.ssh/my_primary_key workstation$ ssh-add ~/.ssh/my_secondary_key workstation$ ssh-add -l xxxx hash /home/user/.ssh/my_primary_key xxxx hash /home/user/.ssh/my_secondary_key
workstation$ ssh -A intermediaryhost intermediaryhost$ ssh-add -l xxxx hash /home/user/.ssh/my_primary_key xxxx hash /home/user/.ssh/my_secondary_key intermediaryhost$ ls ~/.ssh/ <empty> # So, even though these keys *are not on "intermediaryhost": intermediaryhost$ ssh -i /home/user/.ssh/my_primary_key targethostA intermediaryhost$ ssh -i /home/user/.ssh/my_secondary_key targethostB If you do the above ssh's with -vv you will see like: debug1: Will attempt key: /home/user/.ssh/my_primary_key RSA hash explicit agent debug1: Will attempt key: /home/user/.ssh/my_secondary_key RSA hash agent debug1: Will attempt key: /home/user/.ssh/id_rsa RSA hash explicit You can go one step further and configure these using Host/IdentityFile in your ~/.ssh/config *even if the files do not exist on your intermediary machine*.