Children have sense but not experience. Experience is a way of independently surviving a physical environment, including other people. I can recall having what in hindsight were sophisticated judgements on hospital staff, aged 2. It was instinctive knowledge of the difference between what they said and the truth, and what their situation believed it was for and was actually doing, and the memory of it doesn't match up with what we think of from a 2 year old.
Babies seem to come with a set of finely tuned emotional instincts which they subsequently apply to actual situations. In hauntings, they'll often refer to an unseen entity as bad mister, or smiley lady, suggesting a knowledge of motive. I suspect young children have a sense of the reality from which they emerged, which may have been fuller than the physical world in which they find themselves. The memory of it fades once it is no longer required for survival, existing only as instinct, which some people block out more completely than others.