> On Sun, 9 Jun 2024 12:37:20 -0400 > ma...@mohawksoft.com wrote: > >> Would a stack of RPI5s, controlled by some sort of docker look-alike, >> perform better than a huge VMware server? Would it perform better >> than a large kubernetes cluster? Would they be more secure because >> they are physically separated. > > Not at scale. My employer's current base for our ESX cluster hardware > refreshes this year are Dell R650 rack servers with dual top of the > line Xeon processors and 1TB RAM each; redundant 25 Gbit/s Ethernet; > redundant 32 Gbit/s fibre for connection to the Unity storage system. > Minimum of 3 machines per cluster; with clusters typically 5 to 15 > machines. > > Raspberries Pi can't compete with this.
Yes, that is an impressive amount of compute infrastructure, and 1TB ram and 25g ethernet is impressive. If you need that, sure, but..... Here's the thing. What are you going to use it for? Did you ever see the movie "Money Ball?" Your whole design is kind of what I was pointing out. How many CPU cores do you have? Say you have 144 cores at 2.2ghz (3ghz turbo) per processor, with 288 cores total. What is your memory band-width? Is the system NUMA because I doubt the processor and motherboard are SMP. Now, that architecture will not scale 1:1 because with NUMA different ranges split between the two processors. Yes, Linux does a good job managing this these days, but it is not perfect. I assume since it is going to be an ESX server, it will be sub-divided across a number of virtual machines partitioned for in a way that each VM is configured for a specific work-load. Eight? That means you fibre and Ethernet will be divided across the various VMs. How much does each of those Xeon processors cost? $5K? How much is the RAM? How much is the server? How much power does it consume? What is your max work-load configuration? What is your typical? 72 Raspberry PI5s will cost $7500, have over 1/2 TB RAM, consume about 1KW power. etc. etc. There is a lot of compute power in each of your servers, but there are a hell of a lot of inefficiencies. Virtual machines are inefficient. Kernel scheduling and aligning processors to the VM schedule reduces physical CPU to virtual CPU throughput. A lot of virtual machines fighting for Fibre or ethernet is inefficient. If you are a cloud provider or something like that, great, you can budget this in. I don't know what business to which your servers are targeted, but I don't think it is all that common in the greater scheme of things. I think the days of these systems are numbered. They are very expensive, consume a lot of power, do not use resources (CPU/Memory) very efficiently. Not only that, the redundancy is fairly low. > > -- > \m/ (--) \m/ > _______________________________________________ > Discuss mailing list > Discuss@driftwood.blu.org > https://driftwood.blu.org/mailman/listinfo/discuss > _______________________________________________ Discuss mailing list Discuss@driftwood.blu.org https://driftwood.blu.org/mailman/listinfo/discuss
