I have a Shuttle DS437 and DS57U7 for desktop. Fanless, small, and the former in particular is pretty affordable. These are sold as barebones so you only add the components you need -- in your case, probably nothing but RAM.
Do note that the case must stand upright, so they're not as convenient as an APU lying flat in an environment where random people can knock things over.
