Even if the site to be operated is beneath or below the house, the guardian still will never leave until the items were all been taken out. If the items were already taken out, then he has nothing to guard anymore and thereby he can cross over or be a free spirit. His choice. But if you mean, someone could push him out using "something" before actually finding the items, then that is very very impossible to happen. The entities could be controlled sometime to lessen their disturbance over the above house owner but no one can push them out to leave their task in guarding the items until all were already been taken out. Some treasure hunters, I believe, could luckily find and take out the items with the guardians still on "it" BUT definitely, something is still attached on those items found, and commonly it has its consequences.
The items could sometimes be taken out with the entity still there but not the entity been taken out before finding the items.
The question is... in the underworld, who was tasked to guard the loots? At that very situation, who presently or legally own the loot? The diggers or the guardian? If it is the diggers, then good luck to them.
If it is the guardian who has more right at that moment than the diggers, then find out what is needed to be done and do it.
In understanding more the world of the underworld, watch the movie "Beetlejuice" about their confinement in their own world and the ghost-comedy movie "D' Anothers" the characters were captured in a vortex as spirits and could not go out until the task is done. Some aspect of these movies has element of truth in it (but NOT ALL parts as a whole is true).
Our own belief, our own practice...