Since I played Pong back in the days, it's probably my oldest video game. But as many other arcade games (Space Invaders, Asteroids, Pac-Man...) you couldn't really "finish" it.
The oldest game without a proper a story I finished (played all levels) was Soko-Ban.
The oldest game with a story and an ending I played to the finish is King's Quest 1... never finished the Zork-games or most other Infocom text adventures.