I felt the same way while playing Horizon. The towers themselves are great, probably the best implementation of the concept I've seen. They all present a fun (and sometimes unique) challenge, the eventual climb was quick and entertaining, and they don't overstay their welcome. Once i had the 'reward' though, i almost wished i hadn't done it. Some of my drive to explore the area had drained away, and it felt like i should just move on to the next area.
After the first 3 i just stopped climbing them, and opted for more organic exploration. The absence of any reward at all actually made me enjoy the final 2 towers more. I knew i was there because it was fun, rather than because i'd get to see more of a map i'd then not bother exploring properly.
