Linux has a (configurable) limit on the neighbor table. I know in RHEL
variants, the default has been 1024 neighbors for a while.
net.ipv4.neigh.default.gc_thresh3
net.ipv4.neigh.default.gc_thresh2
net.ipv4.neigh.default.gc_thresh1
net.ipv6.neigh.default.gc_thresh3
net.ipv6.neigh.default.gc_thresh2
net.ipv6.neigh.default.gc_thresh1
These may be rough guidelines for performance or arbitrary limits
someone thought would be a good idea. Either way, you'll need to
increase the number if you're using IP on Linux.
Although not explicitly stated, I would assume that these computers may
be virtualized or inside some sort of blade chassis (which reduces the
number of physical cables to a switch). Strictly speaking, I see no
hardware limitation in your way, as most top of rack switches will
easily do a few thousand or 10's of thousands of MAC entries and a few
thousand hosts can fit inside a single IP4 or IP6 subnet. There are some
pretty dense switches if you actually do need 1000 ports, but as others
have stated, you'll utilize a good portion of the rack in cable and
connectors.
--Blake
