Hi Oliver, Are you aware there are other troff implementations than GNU's groff? Neatroff is one. Ali Gholami Rudi wrote it because he wanted better Unicode support for foreign languages, including right-to-left text. He seems very much of your mould in needs.
A good summary of its features is http://litcave.rudi.ir/neatroff.pdf I see UTF-8 hyphenation files mentioned. There's also whole-paragraph formatting and lots of other delights. Rudi's http://litcave.rudi.ir has a Typesetting section past the initial list of recent changes to his software. Feel free to continue discussing neatroff here along with general troff questions. -- Cheers, Ralph.