Why We Should Move AI Data Centers Into Space

[Zkuq]

CaptainExplosion said:

Zkuq said:

Of course all this is probably going to require either major processing power improvements or major optimizations to AI models. It's probably going to take some years until local LLMs become a widely practical solution. I'm guessing it won't be until the 30s that local LLMs are going to be more than an enthusiast thing and maybe a fun but ultimately particularly unreliable toy.

Is compression a processing power improvement?

Hmm. I'm not intimately familiar with LLMs, but maybe. I've often heard how e.g. 8 GB of VRAM isn't great for running LLMs locally, but with compression, that VRAM might be able to go a longer way. I'm guessing the main bottlenecks are elsewhere, but perhaps improved compression techniques could also help. I'm really not qualified enough to give more than that. AI itself suggests my line of thinking is correct, but as is often the case, it sounds to be quite a bit more nuanced than that.