I think we use it, just not consciously. It’s not like knowing where north is helps much, but aligning your mental map of an area means a lot

Have you ever come out of a forest or something not where you expected, then had a moment of disorientation before everything snaps into place and you realize exactly where you are?