An ancestor of mine has children who were born in Canada, Mchigan, Iowa, Kansas, Missouri and Nebraska. He and his family got around.
Never assume that your ancestor did not move. Just because he was in a specific location in 1850 and 1860 does not mean that he was there in 1855. One of my wife's ancestral families was in Illinois in every census after 1860, but spent two years in Pennsylvania and a year in England after that. Both of these residences took place in off census years and the family was "back" in Illinois for the next enumeration.