I think it is implied that this is the case. The population grows way too fast for it to be a case of natural growth. The "prestige" buildings are also seemingly aimed to "attract" people - statues, inns, theaters, palaces and whatnot.
Wel, if you compare your children who grow up fast, I think there is a little of "natural" birth too 
.
