The best explanation I've heard is that he was at the transition age, when boys start to spend time with the men instead of the women. Given the tradition of those days of men and women travelling separately, his mother thought he was with his father and vice versa... (and he was actually in his Father's house!)