> so that torture focus probably needs some care > > in other news i was looking at pine64 and they have an AI agent > product now. i think it just runs deepseek 7b on an embedded arm chip, > but they setup heatsinking and preconfigured the model in theory > https://pine64.com/product/alpha-one-7b-llm-agentic-generative-ai-agent/ > > of course like all pine64 products it's out of stock > > they say it's based on their starpro at > https://pine64.com/product/starpro64-32gb-single-board-computer/ which > of course is also out of stock > > this board is risc-v. "powered by an ESWin EIC7700X 1.8GHz Quad-Core > SiFive P550 64-Bit Processor with IMG AXM-8-256 3D GPU and 20TOPS NPU > (INT8). The STARPro64 is equipped with 32GB LPDDR5 system memory". > "PCIe Gen3.0 x4 open ended slot". somebody did a lot of engineering > work resulting in deepseek actually running on this now it sounds > like. it looks similar to a normal single board computer in form > factor, although with some observing there is a PCI slot sticking out > of it opposite the GPIO pins. > [... > > [so how did they get deepseek running on this thing? the page on it > has a link to their OS image ...
https://www.eswincomputing.com/en/bocupload/2024/06/19/17187920991529ene8q.pdf indicates that there are GPU drivers for normal python-based frameworks for this CPU ("Pytorch, Tensorflow, PaddlePaddle, ONNX, etc")
