What kind of nvidia graphics card are you talking about? AFAIK, the 'mobile' versions (GeForce 2 Go e.g.) are not supported by XFree's nv driver ... Compiling the sources from nvidia's homepage has - up to now - never been a problem for me; but with kernel 2.4.20 i get loads of unresolved symbols :-(
Steffen