Some data collected using VMware 4.5.2 (with Petr's patch 89) which maybe can help track down the memory problem. This is with 2.6.11-rc2-bk2. Took little over an hour for the system to start moving into swap.
Attached gzipped tar file contains 11 files. log which is output of vmstat -a 600 meminfo1 thru 5 slabinfo 1 thru 5 1 is prior to starting a VMware guest, short after a fresh boot. 2 is just after starting a VMware guest. Guest has 128MB assigned. 3 is just after system start to show swap usage. System never normally requires any swap. 4 is just prior to exiting the VMware guest. 5 is after VMware guest closed out. -- Pete Clements
data.taz
Description: gzip compressed data, deflated, last modified: Mon Jan 24 14:54:24 2005, os: Unix