I don't think Nintendo rushed the 3DS or Wii U because of Sony/MS, because why not in that case just wait until 2013, the Wii launched alongside the Playstation 3 without much averse effect.
They had grown accustomed to very high sales the prior 3-4 years, likely pressure from shareholders to maintain that was the key pressure they felt.
Shareholders don't care the reason why sales drop, they just know they were seeing declining sub-20 million shipments and weren't happy with that. If tomorrow iPhone shipments started dropping Apple would have the same issues with their shareholders. Once you set a high bar, the expectation is to maintain it all the time. Shareholders get greedy.
Other issues hit Nintendo hard too. Game development once you get past N64 era (PS2+ tier) becomes more complicated, up to PSX/N64 style games you can still make a lot of games with fairly small main teams, but there's a visual fidelity (more polished 3D capabilities) that occurs when you go to PS2 tier or better. That made game development on the 3DS slower. And Wii U having PS3/360 tier was also another problem as that was even more complicated.
Balancing both systems likely was always going to be impossible for Nintendo.
Other factors like smartphone gaming taking off like a rocket very quickly ... by late 2009 it was still in it infancy but by late 2011/2012, it was everywhere and even tablets like the iPad were massive, massive mainstream hits popping up in households everywhere. That hurt both the 3DS and Wii U, the 3DS because it's lower end style of gaming suddenly looked out of a step with free games on a nice iPad that could do 100 other things better, the Wii U, well all the home stuff the tablet was supposed to do (Sports channel, web browser for the living room, etc. etc.) suddenly looked laughably antiquated.
They bet the farm hard on Nintendogs + cats being an evergreen driver for the 3DS that would drive sales like Nintendogs did for the DS early, and same thing for New Super Mario Bros. U + Nintendo Land for the Wii U ... but all three of these games let them down hard. They waited too long for a Nintendogs sequel, the audience for that had moved on to smartphone gaming, NSMB was too much retro 2D Mario, it was unique the first couple of times, but Nintendo had milked it too quickly with the 3DS also getting NSMB2. Nintendo Land was a decent enough quality for what it was, but mini-game collections weren't selling hardware any more by then either.