The programs in contrib are free, but many programs are put in contrib because they are not useful in an all-free system. Many depend on non-free packages to be useful.
I think these programs should be kept with the non-free packages. On the other hand, some programs are in contrib for another reason. Perhaps they ought to be on the server with main. Perhaps the contrib category should be split into two category for packages that have some sort of freedom problem (although they are free software), and one for the rest of what is now in contrib.