I've actually played fairly old games that both lack load time and have impressive AI. The reason these things don't seem to exist right now is because of the focus on graphics. We are past the point where impressive graphics can make a game better. Instead we are at a point where the best graphics only make a game worse. This is because development budgets are the really limiting factor and not machine hardware. The more money that goes into developing graphics, the less there is for other aspects of the game.
That's a valid point. Hopefully, the evolution of development environments will mean that they will need to work less on better-looking products and focus their attention on other aspects of games. Time will tell if this is the case, though I think that a focus on faster memory and CPU's is a clear sign that big changes are coming Let's hope for the best. In many ways this generation was about perfectioning what was done thus far, and IMO they excelled at this.
Vote the Mayor for Mayor!