No, then the diver is being rude. Le t them know you are fishing in the area. Most people that I dive with or know stay away from fishermen. As usual there is always the exception.