Speaking from the consumer side of things, I have found literally no downsides to cartridges as opposed to discs. Maybe there are some on the manufacturing side, but since the 90s things have improved a lot, so it probably isn't that much of a difference.
Nowadays, most of my PS2 and 360 library is scratched and unplayable, and my cartridges from gameboy onwards still work fine besides Pokemon Yellow, which refuses to boot at all for some reason (probably from the hours I've played and abused it). Not sure if cartridges are feasible for a home console yet (at least for 1080p/4k), but for handhelds they work great. Minimal load times, easy to store in a case, and can survive multiple trips through a washing machine. :D