I ran a few tests, and OpenBSD seems perfectly capable of using those extended characters.... have you tried using doubled quotes?
sftp [EMAIL PROTECTED]:"file with spaces and Umlauts.txt" That should work.. but, spaces and "extended characters" are so "unclean" in the Unix world, it was never designed to use them. -Nix Fan.