PS5's internals should be pretty damn expensive going by their specs, and I'm not sure if dropping the disc drive will be enough to close the gap to Microsoft's more conventional XSX.
They both have 16GB of GDDR6, MS has 10 modules (6x16Gb + 4x8Gb), The PS5 has 8 x 16Gb chips. All chips on platforms are 14Gbs.
They XSX has (or is most likely to have) a bigger APU (360mm2) compared to the PS5s smaller APU (rumored 320mm2). The APU is the single most expensive component in the system.
They are both using an SSD, the PS5 uses 12 (one for each channel) 64GB NAND flash chips vs the XSX 4/8 channel layout. Which also probably uses fewer nand flash chips but of larger capacity.
So if you are suggesting that the PS5 hardware should be more expensive,I am not seeing how. There isn't anything vastly unconventional about the PS5s actual hardware. They just went abt things differently.
Eg. NVMe SSDs typically have 4 channels (sub 2GB/s drives) or 8 channels (3.5GB/s drives), then on servers, you have SSDs with 12 or 16 channels. The PS5 is just using a 12 channel layout, and that s nothing more than a few more traces on the PCB.
Those nand memory chips in the PS4 are so much faster that they need much more expensive chips to reach those speeds, and there are more of them. It's probable that PS5 will need MLC or even SLC chips while the XSX can do with the much cheaper TLC chips. Add to this the way it has been integrated into the system; there's much more to this than just installing faster chips. The way it's integrated sounded like it would need PCBs with more layers, and those quickly add up in price.
The APU was the biggest cost in times of hard drives, but that's not always true with an SSD. Just compare the price of 1TB HDD vs 1TB SSD. You can get a WD Blue for about 35€, but an NVMe SSD of that size will cost you at least around 100€, and more like 150€ minimum if you want something closer to the touted speeds.
So yeah, those Nands and how they are integrated into the PS5 can make a whole lot of difference to the price.