A book which played a great influence on the development of my own thought when I was a young man was the late cosmologist Carl Sagan's work "The Demon Haunted World". It has been long since I have read this work, and while I by no means subscribe to all of Sagan's science-centric vision, I think it is a useful book full of genuine insight and a good introduction to some of the fundamental methods of logical thought. And how I wish that Sagan had not been taken from the world prematurely, for I cannot help but feel that his nuanced and sympathetic understanding of the interplay between science and religion would have proved a useful anodyne to some of the more strident claims made over these last few decades by partisans on both sides of the issue.
If I am remembering things correctly (for again, it is long since I have actually opened this book and perhaps I unintentionally misconstrue the argument) the main thrust of Sagan's thesis is that mankind long slumbered under the delusion that the processes of the world were driven and controlled by mysterious and elusive powers and intelligences beyond the ken of the mortal mind. Thus, man long lived in fear and ignorance of his environment until the dawning of the scientific revolution provided him with the tools and drive to understand the true impersonal processes which underlie reality. Yet, though people now have the tools to understand and manipulate their environment, many still persist under the delusion that the world is "haunted" by esoteric forces which make of mankind their playthings.
Now, I am not here so much interested in the existence or efficacy of such esoteric "powers" as demons, sprites, spirits, and faeries. But what I will say is that, based on nothing more than an experience of ourselves and our internal processes of mind, I think that the idiom of a demon (or if you prefer, "spirit") haunted world has not, even in a scientific age, outlived its usefulness. For it seems to me that the primary haunting, out of which it seems likely the idea of esoteric powers arose, is none other than the fact that the human person is intrinsically haunted by itself.
What do I mean by this? I mean that the very essence of ourselves--our thought processes, drives, desires, and feelings--are themselves in some fundamental way alien to us. We certainly know our own selves better than we know anyone else, and it is only by this self-knowledge that we are then able to come to understand others (if admittedly imperfectly). Yet, when we look back on the thoughts that we have, or the things that we do, or the concepts that we believe, how often are we perplexed? How often do we ask "why did I do this" or "why did I not do that" or "what was I thinking" or "how could I have believed that"? It seems to me that our temporal condition as beings inevitably drawn through a process of time makes the existent self a fundamentally alien thing from the self that was in the past. For while we can perceive a train of motivations and desires and stimuli constructing a chain of causation, the effects of this become easily disassociated from the self at any given moment. And even more troubling, again a fact drawn from the progression of the person through time, to even look back upon ourselves implies that we are never perceptive of the self that exists at any given moment but only always of the self that existed at some previous moment in time, even if that moment were only a mere moment ago. This is a process perhaps best explicated by Hegel in his "Phenomenology of Spirit".
Thus, because we exist in time, we cannot know ourselves as we are, but can only know that self that was. This disassociation wrought by time disconnects us from ourselves, meaning that we are constantly interacting with a non-existent "phantom" self that follows us through that duration known as the life of a human person. Thus we are "haunted" first and foremost by our very self.
Now I will grant that this is a rather odd way of speaking, and I suspect most of us don't think on the process of memory as an imagining of a phantom spirit. For we are what we are and can be nothing else, and this progression of knowledge through time breeds a familiarity with this process which causes it to bleed into the background of our consciousness. Thus the "haunting" nature of the self is destroyed by our familiarity with it. But this familiarity, though it can hide from the mind the nature of the self's relationship to itself, cannot destroy the ontological nature of this fundamental division. And so this unconsidered haunting seeps into the pre-conscious mind, melding with myth and legend to produce stories replete with mystical phantoms which mankind perceived as the true drivers of the processes which govern the physical world. At least, this is one story of the development of these mysterious "hauntings". Another (not completely divorced from the first) might be that the structure of the minds disassociation exists because it does in fact mimic an aspect of the nature of reality since it would seem that our goal as humans is, at least in part, to break through this inevitable sundering from ourselves into a realm where we can conceive of the self--past, present, and future--as a united and integrated whole. This seems to be what Heidegger references when speaks of a state of Dasein, or immanency of being, or what Leibniz references as the "monad"--an image of the soul as it appears from the integrated divine perspective.
From the perspective of this second story, the "haunting" might best be understood as a relation to those things which lie beyond the standard perception of the self through time. Thus, these things in their immanence have a claim the being (or put another way, "reality") which necessarily transcends the human perception of existence. In this model, far from being the measure of all things, man becomes the spectral image that haunts the "things" that are immanent and already by their nature partake of that integration in time (and perhaps in space).
I am not asserting the "truth" per se of either one of these visions, but I do submit that the idea of a "haunting" of the world, far from being the left-over vestige of an outdated age in humanities development, is a still necessary and vital aspect of our lived experience of ourselves that must be resurrected if we wish to obtain an accurate and nuanced view of the whole of reality. For the more we try to ignore the feeling and experience of being haunted by we-know-not-what, the further we sunder ourselves from a fundamental aspect of our lived experience, and the more alien we become to ourselves. In attempting to escape from the realm of such esoteric powers, the more we give ourselves over to the control of ever more dreadful gods, even if such gods are best understood as un-looked for emanations of the human mind and soul.