My Breton battlemage/single mom has her two adopted kids, Sofie and Blaise, at Lakeview Manor with Jordis and Rayya watching them. It's the only Hearthfire house she's built, although she also owns Breezehome, Honeyside and Proudspire Manor.
The only character of mine who's bult any f the others (he actually built all three) was my now retired Bosmer ranger, and he didn't adopt any kids. He originally built all the houses for his spouse, Marcurio, and Marcurio lived at Lakeview Manor for a while. But once my Bosmer joined the TG, he ended up moving Marcurio back to Breezehome as it was just a better location to spend time with him with all the traveling he was ding for the TG questline.
My Breton rogue bought Honeyside, and because he's too lazy to built a house, he moved his spouse, Gilfre, and his kids, Sofie and Blaise, there. Then I decided I would rather have him adopt from Honorhall, since my other Breton has Sofie and Blaise and I wanted something different. So I had to go back to an very early save in the game, so for the time being he's without any family. But I'll probably have him move his future family to Honeyside again--I just can't give up the alchemy lab at Breezehome for a kid's room as this character does a lot of alchemy.