By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - General - Why We Should Move AI Data Centers Into Space

ConciousMan said:
CaptainExplosion said:

You can have AI on your local device without data centers?

And aren't some companies working on improving energy efficiency and decreasing water consumption by AI data centers?

Yes, you can run local LLMs model on your PC or something like Spark DGX. But, in the future I believe a smartphone might be able to run a local AI software. They can improve data center efficiency, the problem is that the users are at the mercy of AI solutions providers for output quality. With potential future local based models the user can fine tune and train AI on their own and optimize it for their work.

Of course all this is probably going to require either major processing power improvements or major optimizations to AI models. It's probably going to take some years until local LLMs become a widely practical solution. I'm guessing it won't be until the 30s that local LLMs are going to be more than an enthusiast thing and maybe a fun but ultimately particularly unreliable toy.

Last edited by Zkuq - 10 hours ago

Around the Network
ConciousMan said:
CaptainExplosion said:

You can have AI on your local device without data centers?

And aren't some companies working on improving energy efficiency and decreasing water consumption by AI data centers?

Yes, you can run local LLMs model on your PC or something like Spark DGX. But, in the future I believe a smartphone might be able to run a local AI software. They can improve data center efficiency, the problem is that the users are at the mercy of AI solutions providers for output quality. With potential future local based models the user can fine tune and train AI on their own and optimize it for their work.

You mean energy efficiency?

Zkuq said:
ConciousMan said:

Yes, you can run local LLMs model on your PC or something like Spark DGX. But, in the future I believe a smartphone might be able to run a local AI software. They can improve data center efficiency, the problem is that the users are at the mercy of AI solutions providers for output quality. With potential future local based models the user can fine tune and train AI on their own and optimize it for their work.

Of course all this is probably going to require either major processing power improvements or major optimizations to AI models. It's probably going to take some years until local LLMs become a widely practical solution. I'm guessing it won't be until the 30s that local LLMs are going to be more than an enthusiast thing and maybe a fun but ultimately particularly unreliable toy.

Is compression a processing power improvement?



CaptainExplosion said:
Zkuq said:

Of course all this is probably going to require either major processing power improvements or major optimizations to AI models. It's probably going to take some years until local LLMs become a widely practical solution. I'm guessing it won't be until the 30s that local LLMs are going to be more than an enthusiast thing and maybe a fun but ultimately particularly unreliable toy.

Is compression a processing power improvement?

Hmm. I'm not intimately familiar with LLMs, but maybe. I've often heard how e.g. 8 GB of VRAM isn't great for running LLMs locally, but with compression, that VRAM might be able to go a longer way. I'm guessing the main bottlenecks are elsewhere, but perhaps improved compression techniques could also help. I'm really not qualified enough to give more than that. AI itself suggests my line of thinking is correct, but as is often the case, it sounds to be quite a bit more nuanced than that.



Zkuq said:
CaptainExplosion said:

Is compression a processing power improvement?

Hmm. I'm not intimately familiar with LLMs, but maybe. I've often heard how e.g. 8 GB of VRAM isn't great for running LLMs locally, but with compression, that VRAM might be able to go a longer way. I'm guessing the main bottlenecks are elsewhere, but perhaps improved compression techniques could also help. I'm really not qualified enough to give more than that. AI itself suggests my line of thinking is correct, but as is often the case, it sounds to be quite a bit more nuanced than that.

What are ways to reduce data center energy consumption?

Another thing they should have is noise reduction, but I haven't seen any effective methods for that.



CaptainExplosion said:
Zkuq said:

Hmm. I'm not intimately familiar with LLMs, but maybe. I've often heard how e.g. 8 GB of VRAM isn't great for running LLMs locally, but with compression, that VRAM might be able to go a longer way. I'm guessing the main bottlenecks are elsewhere, but perhaps improved compression techniques could also help. I'm really not qualified enough to give more than that. AI itself suggests my line of thinking is correct, but as is often the case, it sounds to be quite a bit more nuanced than that.

What are ways to reduce data center energy consumption?

Another thing they should have is noise reduction, but I haven't seen any effective methods for that.

There are various ways to improve data center energy efficiency. Like optimizing the software first to achieve more with less energy consumption. Another way is to improve manufacturing packaging and processes. Improving and optimizing microchips architecture is another and most efficient way. Data compression won't add much to energy efficiency in my opinion, it can only reduce memory usage, but AI is more effective when it has even more quality data to learn from.



Around the Network
CaptainExplosion said:
Zkuq said:

Hmm. I'm not intimately familiar with LLMs, but maybe. I've often heard how e.g. 8 GB of VRAM isn't great for running LLMs locally, but with compression, that VRAM might be able to go a longer way. I'm guessing the main bottlenecks are elsewhere, but perhaps improved compression techniques could also help. I'm really not qualified enough to give more than that. AI itself suggests my line of thinking is correct, but as is often the case, it sounds to be quite a bit more nuanced than that.

What are ways to reduce data center energy consumption?

Another thing they should have is noise reduction, but I haven't seen any effective methods for that.

For computers in general, it would be either more optimized software or more powerful/power-efficient hardware. Of course the issue with either is that they both allow more computations to be run, which in turn negates either some or all of the improvements in power consumption. I'm guessing the hardware will end up running at its limits anyway to maximize cost efficiency, or at least that'll be the target.

Another interesting aspect at data center scale is cooling. I don't know how much energy that consumes, but if it's a significant portion of the total, some environmental factors can possibly be utilized (e.g. building data centers in colder climates - my understanding is that my home country, Finland, is quite attractive for data centers because of the cool climate).

Judging by a quick search, it sounds like cooling plays a significant part in data center noise levels. If that's the case, improvements in cooling/reduction in cooling needs ought to improve the noise situation as well. I don't know if data centers typically do usual common mitigation measures, but if they don't, that might also be an option. I imagine it might have a detrimental effect on cooling though, because what doesn't let a lot of noise through probably doesn't let heat through all that well either.

I'm no expert on data centers though, so this is just some things I've run into or thought.

tl;dr: The easiest way is probably building data centers in cooler environments if possible, but I'm not sure how much exactly it helps with power consumption.



TallSilhouette said:

You think data centers are driving up prices now? Wait til we have to launch all that hardware into orbit, to say nothing of cooling components in the vacuum of space with nothing to absorb the heat.

Add to that the need for intermittent thrust to keep them from falling out of orbit, just like the ISS has to every now and then. 

Putting it in space isn't feasible on any human economic, technical, or even physical scale. Cooling alone would be too much of an obstacle. The sheer scale of the required structures for cooling should shoot everyone back down to earth (where the centers will also need to stay). I don't remember the exact figures, but I think just for weight and overall size, the materials for cooling systems would outdo the actual hardware and center-structures by 10x or more.