I have evaluated several UI libraries. ExtJS is the nicest, but you can only use it for open source projects, otherwise you have to buy it. QooXDoo looks cool, but wouldn't integrate into web2py very well if you wanted to completely replace jQuery. Then you have YUI (Yahoo User Interface), which is kind of nice, and probably the easiest library to integrate.
But I agree with Phyo that jQuery UI is really the best, easiest option. Plus, some of the jQuery plugins are really nice. Unfortunately, using jQuery's site to search for plugins is terrible. And plugin quality can range from horrible to excellent.