It's been a very long time since I played SOMA (played it basically at launch), so I might get some things wrong.
The monsters do add to the story, they were all scientist and researchers going crazy because of WAU, maybe is not much but I personally liked it. The AI was ok I believe, I remember one section where it was very difficult for me to evade the monster, the freaking thing really followed me everywhere and it was pretty tense moment. I finally had to run and the guy almost catches me on an elevator finishing the area. I didn't like the monster designs though, most of them were kind of lame to be honest.
For the ending sequence I understand what you mean, but I think you have to put into Simon's shoes; he went through hell and when he finally saw some hope he grabbed onto it, deep inside he knew it wasn't going to work, but he was to a point of going crazy so I can't really fault him for having false hopes.
The monster scene you're talking about is scripted. He will be behind you when you're trying to go for the elevator with the chip that allows you to open the door. I hated that part, but because the fucker had some erratic AI behaviour and he got static while blocking the door out of the only place I could escape, which resulted in a cheap death since I couldn't do anything.
Yes, the scientist were becoming more and more paranoid as WAU grew more powerful, but the ARK idea came much, much earlier than when monsters started breaking everything. They were basically a subplot that existed to add monsters to the game, in the pure Frictional Games vein aspect.
And I don't know. Simon really behaved like someone who understood everything without much doubt. When confronted with evidence, he admitted being a robot. When he saw his original memory, he understood he wasn't "real Simon" in a sense. And yet...I dunno. He goes dumb at the very end.