I think it's more to do with development than wealth. I know here in England 2-3 generations ago families were still very large, and we were probably richer in comparison to the rest of the world back then than we are now. Although that said we still have some very large families on account of people abusing the overly generous welfare system.
That's because condoms and other ways of birth control weren't very common yet before World War 2. In third world countries, most people still don't have the money and possibilities to buy these things.