I continually have this problem. I dont even try to fish the holes their in just head my boat down stream and get out of their way. 90% of the time they still have some kind of remark. I just quit worrying about it. I used to be confrontational but that just ruined my day. Lots of water to fish so I move on. I go to the river to fish and enjoy myself not fight.