You love machine learning. You want an NVIDIA A100 GPU to experiment with the latest advances. You do not have an NVIDIA A100 GPU. You cannot get an NVIDIA A100 GPU. Did I get that right? It doesn’t take a mind reader to figure that much out. The present artificial intelligence (AI) summer is scorching hot. Breakthroughs in algorithm and hardware design have led to the development of a new class of AI-powered applications that are sparking interest in the field like never before. And that means the latest and greatest GPUs are being bought up by large organizations, leaving none for the rest of us — even if we could find the thousands of dollars they cost between the couch cushions.
Make do with what you have
Fortunately, that does not necessarily mean that the AI hobbyist is out of luck. Many of us have plenty of computing power available to us these days — it is just spread across a lot of devices. You might have a stack of old laptops, a couple aging desktops in the corner, and a drawer full of Raspberry Pis and older smartphones. If you were to somehow mash all of those resources together into a single pool, it might just be enough to run even the latest large language models, like DeepSeek R1 671B, locally on your own hardware.
But how can you do that? An open source software package called exo is seeking to simplify the process. It makes it possible to distribute AI workloads across a large number of heterogeneous computing devices, leveraging the memory and computing power of each as if it were a single unit. In this way, a bunch of old laptops, single-board computers, and smartphones can run even the latest AI algorithms — with or without GPUs.
To use exo, all devices need to be connected to the same local network. The software then automatically discovers all available devices, eliminating the need for a complex configuration process. A dynamic model partitioning strategy then breaks up models into pieces via one of a number of available options which may, for example, run certain model layers on certain devices, according to their capabilities.
That is a lot of Pi
The only hard-and-fast requirement of exo is that enough memory exists across all devices to fit the model that you want to load. That can still be a big ask, however. DeepSeek R1 671B, for instance, requires about 1,342 GB of memory to run. That would take about 168 Raspberry Pis with 8 GB of RAM. But for more modest needs, pooling memory could perhaps get many of us to a couple hundred gigabytes, at least, which would be a big stretch for a single machine.
Full instructions to get exo installed and running are available on GitHub, so be sure to check it out. Not only will it enable you to play with some powerful AI tools, but it will also allow you to make use of some old hardware that is sitting around collecting dust, and that always feels good.