Both file dialogs should work alike in theory: Return a relative filename if the file is in the document directory or a subdirectory thereof, and return an absolute filename if not. Do you get different behaviour?
Yes, with LyX 1.3.5 qt (Windows port).
Ekkehart