Ah sideport. I had the Asrock M3A790GHX at one point with 128MB of DDR3 Sideport memory. - But because it clocked at only 1200mhz, it only offered 4.8GB/s of bandwidth verses the system memories 25.6GB/s of bandwidth... So the increase in performance was marginal at best. (I.E. Couple of percentage points.)

On older boards that ran with DDR2 memory that topped out at 800mhz (12.8GB/s of bandwidth) the difference was certainly more pronounced.

On that Asrock board I got more of a performance kick from simply overclocking the IGP to 950Mhz than from turning on Sideport memory, but that is entirely down to the implementation.

In saying that, GPU performance has certainly outstripped the rate of system memory bandwidth increases... I mean heck... The Latest Ryzen notebooks are often still running Dual-Channel DDR4 @ 2400mhz, which is 38.4GB/s of bandwidth, not really a big step up over the Asrock's 25.6Gb/s of bandwidth is it? Yet the GPU is likely 50x more capable overall.

Sideport is great, if implemented well and not on a narrow 16-bit bus. - And you wouldn't even need to use expensive HBM memory to get some big gains.
GDDR5 is cheap and plentiful and on a 32-bit bus could offer 50GB/s which combined with system memory (I assume by a striping method) would offer some tangible gains.
GDDR6 would be a step up even again where 75GB/s should be easy enough to hit... Ideally you would want around 100-150GB/s for decent 1080P gaming.

If they threw it onto a 64-bit bus, then that would double all of those rates, but I would imagine tracing would become an issue, especially on ITX/mATX boards, forcing the requirement of more PCB layers.

Yeah, I had Sideport Memory too on my Asus 880G (don't remember the exact model). And yeah, that 16bit bus sucked, but it was good enough when my GPU failed.

I was talking about HBM simply for one reason: It has a very small footprint. Sure, GDDR5/6 would do also, but would also certainly need more space on a board, which would make it hard to create such boards in ITX or STX sizes. Too bad HMC never got popular, as I think it would have been the best choice for this. And at bit rates of up to 480GB/s, that would have sufficed for a long time in that regard.

Well, 38.4GiB/s is almost 50% more than 25.6GiB/s, but I get your point. Hence why DDR5 will start at DDR5-4400, way above the highest specified DDR4 memory (3200) and supposed to go all the way up to DDR5-6400. iGP/APU will certainly benefit a lot from the bandwith boost, as will some server applications. DDR5-6400 would also bring 51.4GiB/s on a single channel, so over 100GB/s on a dual channel interface, which should be enough for most integrated graphics, at least for now.

Just for comparison, Baffin (RX 460/560) has 112GB/s, so 100GB/s should be enough for 16CU@1200Mhz (RX560 reaches 1300) without choking. Still not enough for that rumored R5 3600G with 20CU, though, unless it would clock at only 800Mhz or so. And no chance even with DDR4-3200, which only delivers half of that bandwith.

People tend to forget Sony/MS will be buying components in the millions of units from the get go.

If I buy 3 of something in the market I can already get a discount. Imagine if I buy 5 milion units at once and the vendor knows I may be needing to buy up to 100M in the next five years? Sony and MS probably gets massive amounts of discount on anything they put in their consoles, specially Sony as the PS consoles are basically guaranteed to go around 100M units made.