This superstition is amazingly widespread. On the east cost, many, many of the skippers won't let a banana carrying client on the boat. I asked one where this all got started and he said something about a banana boat sinking in some harbor in Hawaii and the fish disappearing for a long time.