Pemalite said:
Zkuq said:
If you put related data contiguously, I bet you the chances are it'll be together on the disk as well. The file system is free to do whatever it wants, but most sensible file systems probably try to not scatter data around when they know it belongs together (e.g. if it's a single file, even if that single file comprises of a large number of individual assets). That to me seems way different than e.g. storing each asset as an indivual file, which the file system is much more likely to scatter around because it doesn't know they're related - especially if they're in different directories and could be used in a number of different situations. If you need, say, 2,000 assets for a scene (often a map), and it takes 10 ms to load each one, that's already 20 s in total just for seeking, let alone actually loading the assets. I don't know how many assets you need to load at once in a modern game, but I don't think 2,000 sounds way off for an AAA game. Regardless, even if it's just 200 assets, it's still 20 s just for seeking, then some for loading, some for actually running code etc., and I don't think it's hard to see why loading times might be high if data is scattered all over the place. Developers do all kinds of stuff that's not guaranteed to work, and I'm sure game developers do it just as much if not more than others. I know that in low-level programming languages it's common to consider how exactly data is aligned in memory, how the CPU is likely to cache it etc. even though there might not be any actual guarantees. If it works now and has seemingly little reason to ever change on the target platform, why worry about it not being guaranteed? Stuff like that makes me uneasy personally, because I like guarantees, but that's also something that happens and works to my best knowledge. |
exFAT is not that smart of a file system.
Data can and does scatter over time, game installation content isn't always static... Pieces of data can be created and deleted or modified.
Say for example you install game "A" that is roughly 10,000 Megabytes (10GB) then you install game "B" that is 10,0000 Megabytes after that one.
Then you load up Game A and create a save file and start playing. - That game may compile shaders into a container, that container isn't going to exist in that initial 10,000 Megabyte block of installed data, it's going to exist on the drive after Game B.
Then when you load Game B up, it may decompress some data, that isn't going to exist in the Game B chunk, it's going to exist after the second Game A chunk.
Then say you uninstall Game B, suddenly you have two FREE spaces between two separate Game A chunks.
Let's say you decide to install a 3rd game... That game "C" Just happens to be 20,000 Megabytes, the game is going install 10,000MB into that first free chunk, fill in the second free chunk, then install the rest after the second Game A chunk.
And this is how files and data become fragmented over time.
But imagine this with OS files and data constantly being created and deleted... And this is why Xbox One consoles with a mechanical hard drive tend to deteriorate with performance over time.
Again, game developers don't get to chose where game data is installed, Microsoft (Sony and Nintendo included), doesn't allow that kind of fine-grained access to storage as it can open the system up the hacks/abuse.
Obviously this is no longer an issue with modern devices anymore that use NAND memory. |
Hmm. Aren't those initial 10 GB installations and how exactly they're spread across files likely to be more impactful for performance than whatever gets created after the game is installed? Also, aren't those initial 10 GB also likely to be split into several files by developers even on consoles? Not the whole 10 GB needs to be contiguous for efficient loading, only parts of it need to be contiguous with each other, right? For example, you can have 1 GB file for each map, containing all the assets in the map (some of it probably duplicated from other maps), and the file system is likely to find a contiguous chunk for such a smaller file even when it couldn't fit the whole 10 GB contiguosly. Sure, after enough time, the file system will look like Swiss cheese anyway, but it'll take time before that becomes too problematic.
I have zero experience with developing console games, but personally I'd be really surprised if developers had zero control over how data gets stored. Files are a form of control anyway, so unless the console simply ultimately doesn't support game files, I don't see how files couldn't be used to organize data. I have no idea why the console manufacturers wouldn't allow developers to organize their data this way. Again, I'm fairly confident that each file gets stored contiguously whenever possible, and it's much easier to find contiguous spaces for relatively small files than whole files, so it's probably possible fairly often. I don't have time right now to look this up properly, but AI thinks even exFAT tries to do this, and it does seem like a fairly simple thing to do, so I don't see why it wouldn't.
I'm sure there are many complications (e.g. updates, especially changes to assets), but I'm sure there are more or less effective workarounds to them as well.
