I'll admit it. The man that I'm 99% certain is my ancestor's brother had not been located in every census in which he should be enumerated. When going back and finding him in every census, I discovered that in 1870 he had a nephew living with him. A nephew with the same last name--meaning that this nephew is either a son of my ancestor or another brother.
A good clue obtained from going through the every census for the brother of my ancestor.