Law of self defense is pretty clear. Might not always agree with it, but it is clear. Self defense is allowed to protect yourself or others. No threat, no right to protect. A guy climbing over a fence would not seem to present much of a threat.