Probably more important than washing in between is DRYING in between fish or dunkings into the water.

Constantly wet hands make the exfoliation/sloughing problem a lot worse. The same can be said for trapping a bunch of sweat in latex gloves on a hot day. Too much skin hydration softens and weakens the already damaged skin, enhancing the de-lamination of the layers.... as we eye surgeons like to say, hydrodissection.

Get the gloves off if you are sweating. Dry your hands after they get wet. That prevents overhydration of the skin. Apply some sort of barrier (bag balm, lotion, or gloves if it's not too hot) to maintain normal skin hydration. That strategy has helped me the most.