Yeah, but their usage is vaning fast in all their usage scenarios, hence why it's getting more expensive. Also, optical discs ain't interchangeable in production, you need specific presses for specific discs, a Blu-Ray press can't make CDs or DVDs, at least not out of the blue. Discs very much gain from mass production, because the machines to press the discs ain't cheap, and slower production raises costs very fast for them. The problem is that there are so many producers slowing each other down as the demand vanes.

For AAA games, 4-8GB discs are useless, I agree. But for Indies or Retro game collections, they do suffice most of the time. And yeah, we are moving more and more away from discs. Try finding a laptop with an optical drive these days, it's getting more and more difficult. Steaming and digital formats are taking over, and optical discs are more and more becoming a niche product. I'll have to dig to find the source, but afair the sales of Blu-ray discs is close to the one of DVDs in 2000 when VHS tapes where still ruling the market. The economy of scale which is making the optical discs so cheap less and less applies to them.

It won't affect the PS4/XBO anymore, though. But it could affect their successors. Getting more than 100GB on an optical disc is proving to become impractical (UV lasers get stopped by the slightest dust particles, hence why a blue laser is the practical limit for end-users), but with increasingly bigger textures and game worlds, that won't be enough for long anymore.

By contrast, cartridges don't face such a brick wall just yet. Expect them to catch up more and more over the next couple years. I think by end of this year, it won't make much of a price difference between a 25GB Blu-Ray disc and a 16GB cartridge, with the 32GB following close behind.

In fact, I wouldn't be surprised if someone would come up with a new standard next to SD cards (since Secure Digital isn't anywhere near secure) with a similar size, but much more safety. If they could get the Film Industry on board (which will be looking for a medium for 8K movies in the future for sure), then this could quickly replace the optical discs in general usage.

Whilst your post is accurate... There is a successor to Blu-Ray known as the Archival Disk which starts at 300GB - 1 Terabyte of storage.
And Sony even managed to cram 3.3 Terabytes into a disk.

Whilst you are accurate that there are practical physical limits to the lasers... There are ways to cram more data in by reducing crosstalk and error detection... This is a similar issue that mechanical disks have had to deal with for awhile.

Archival discs, as their name implies, are meant for long term archiving of media (like movies) and not intended for consumer level products:

“The development is specifically for professional archiving,” the Panasonic spokesman said. “We are not currently considering optical discs for household consumer use.”

In other words, those are probably too expensive to make or too fragile to warrant consumer products with them. In any case, I don't expect any consumer products, like consoles, with those.

Improved error detection and reduced crosstalk would probably work, but that would also make both the discs and the drives more expensive. Hard drives had to sidestep this by now and are using helium-filled cases (because helium has almost no air friction and is easier to handle than pulling all the air out, creating a vacuum) to allow for faster rotation speeds, and no dust and more discs inside the casings.

So yeah, like I said, more than those 100GB don't seem practical right now. It's technically possible, but very unlikely to be used in consumer products anytime soon.

We are heading to a mostly digital future anyway, so I fail to see the problem.

Since we clearly have a lot of engineers here, I'm curious what the genius solution is instead of cartridges, given the Switch's size and form factor. A full-sized BluRay player is right out. And, mini-discs could possibly end up smaller than 16GB anyway, aside from the potential for failure and scratching from a portable. Or, would you lot rather it was wholly digital only?

Optical disks are becoming antiquated, that’s why they’re going up in price. The market is shrinking and price adjustments need to be made to account for revenue and profit loss to avoid a stock collapse during the transitional phase.

Also, it’s not fair to call Switch game cards “cartridges.” It’s the same as when people called NES cartridges “tapes.”

Cartridges are an obsolete form of technology. They’re a different sort of media utilizing different technology and form factors than Switch game cards.

I think the limitations are starting to hurt the games, developers, and the publishers. Not as much the system.

I believe the read speeds for AD are subpar for gaming use, too - at least, if the game is going to be utilizing the medium's space appropriately.

Besides that, AD is designed to be highly durable - which means the discs are indeed expensive.

Just as you said, the name implies that Archival Disc is intended for archiving data for long-term availability and preservation. Not regular use.

To expand on this:

Cartridges used ROM storage for the games themselves, then battery-backed volatile memory for savegames.

Game cards use flash memory for storage, which is a[n optionally] rewritable distant evolution of ROM, and doesn't require a battery for maintaining saved data. 

Indeed it is possible, they can take features like the error correction and cross-talk reduction and use that to bolster current optical disk capacities.
It won't have an effect on disc prices, drive prices certainly, at-least initially. Most of the work is done on the firmware/software side.

Hard drives before they took the helium road also took the same road as Archival disks, aka. Employed error correction and cross-talk reduction to bolster capacities. (And are still improving on those fronts even today.)
https://en.wikipedia.org/wiki/Hard_disk_drive#Error_rates_and_handling

And before that... We moved from Linear to Perpendicular to now having Shingled layouts. - This is to get around the Supermagnetism issue which limits the physical amount of bits you can have per inch on a disk.

In short, if you cannot make the pitches in the disk smaller, then you work around that.

As for Helium, it's not a necessary requirement to increasing capacities, we are now entering the era of next-generation mechanical disks, so we will have multiple actuators and heat/microwave assisted recording techniques.

Optical disks for portable devices is never a good idea.
They are slow, power hungry and generally not as durable mechanically.

NAND/ROM is probably the best form of media for portable devices other than purely digital.

Archival Disks are an extension of the Blu-Ray format. So similar read speeds should be achievable.
It would help reduce piracy if the next-gen consoles leveraged that technology whilst retaining backwards compatibility with older disk formats.

In saying that, the read speeds are inconsequential anyway, games get installed onto the internal mechanical drive thanks to it's speed and latency advantages.

I'm really not sure what your point is given the platform primarily suffering from an expensive storage medium is Switch. Sometimes Switch ports are more expensive, developers avoid using large carts and publishers are clearly avoiding physical releases. I agree we're moving away from optical discs, but they're still cheaper to produce than Switch carts. In the past platforms like Dreamcast, Gamecube and PSP used unique discs which shows mass production may not be that crucial. I believe optical discs are just inherently cheaper to produce when compared to memory carts that hold a significant amount of storage.

I anticipate the 100GB discs will be used in the 9th gen consoles and will be plenty of storage space for most games. It would help if more games split online and offline content, that way a disc just needs to hold the offline stuff. Maybe by the 10th gen there will be dirt cheap carts that hold a massive amount of storage. Until then though, discs are still the cheap route.