If you had a wrist rocket on board and a bag of marbles you could scare one away that way. When a marble hits the water hard right next to one of their heads (you'll never be able to actually hit one) they go away. You need to do it before they get one of your fish though. And don't get caught doing it!!