I think it's more to do with development than wealth. I know here in England 2-3 generations ago families were still very large, and we were probably richer in comparison to the rest of the world back then than we are now. Although that said we still have some very large families on account of people abusing the overly generous welfare system.
Actually, I think it tends to be the opposite. Most of the countries with higher birthrates don't really have any safety nets, so they have to rely on their children to take care of them if something goes wrong or they get old instead as the state won't help them.