> > so that torture focus probably needs some care > > > > in other news i was looking at pine64 and they have an AI agent > > product now. i think it just runs deepseek 7b on an embedded arm chip, > > but they setup heatsinking and preconfigured the model in theory > > https://pine64.com/product/alpha-one-7b-llm-agentic-generative-ai-agent/ > > > > of course like all pine64 products it's out of stock > > > > they say it's based on their starpro at > > https://pine64.com/product/starpro64-32gb-single-board-computer/ which > > of course is also out of stock > > > > this board is risc-v. "powered by an ESWin EIC7700X 1.8GHz Quad-Core > > SiFive P550 64-Bit Processor with IMG AXM-8-256 3D GPU and 20TOPS NPU > > (INT8). The STARPro64 is equipped with 32GB LPDDR5 system memory". > > "PCIe Gen3.0 x4 open ended slot". somebody did a lot of engineering > > work resulting in deepseek actually running on this now it sounds > > like. it looks similar to a normal single board computer in form > > factor, although with some observing there is a PCI slot sticking out > > of it opposite the GPIO pins. > > [... > > > > [so how did they get deepseek running on this thing? the page on it > > has a link to their OS image ... > > https://www.eswincomputing.com/en/bocupload/2024/06/19/17187920991529ene8q.pdf > indicates that there are GPU drivers for normal python-based > frameworks for this CPU ("Pytorch, Tensorflow, PaddlePaddle, ONNX, > etc")
i wonder if the lesson is that if you port a mainstream language model to a small chip and then sell it, everybody will buy it.
