I have five!
You're kinda right...but not 100% On the "primary" console, anybody can play any game they want, whenever they want. They can even use Xbox Live Gold as much as they want.
On the other consoles, the account that purchased the games needs to be logged in. As long as they're logged in, anybody in any other account can still play the games. Only two consoles can play the games at a time, tho--as long as one of the consoles is the primary console.
I've even played online with my daughter--one digital purchase of the game and one Xbox Live gold subscription (mine) and we were both able to play the same game at the same time, online together!
This is the problem I have come in contact with. If I am logged into the Xbox X using my primary account and my Son wants to play a Minecraft which was purchased with my Account on the Xbox One. In order for him to play Minecraft, my account needs to be logged into his machine. That's a problem since it will log me out on the X. If I am playing a game, using a service like Crunchy Roll or Nextflix, they will stop working.