Hmm... Not quite. That's true for some ghosts, especially if they were humans. They might have some grudges or some things they didn't get to do in life, so they can become shackled to one spot.

[he looks up at the temple]

So if someone really loved this place, maybe they were unhappy to see it dirty.

But spirits... they can come from a lot of different places. From nature, or from objects. I'm the spirit of my sword. [he places his hand on the hilt of his sword, which rests at his hip] I've just been given this body by my master.