Lead is not soluble to any reasonable extent in water, so it doesn't get into the water column and affect fish, bugs, birds or people. If you eat eat, I believe the acids in your digestive tract can react with it and make a soluble compound. In most running water, this isn't really an issue. Perhaps someone whose chemistry class has occurred in the current century can be more specific.

In running water, lead settles to the bottom of the gravel. Lead is relatively heavier than the rest of the gravel, and as the current runs and turns rocks, lead ends up sifting to the bottom, out of reach as far as animals are concerned. Think about how gold settles on the crick bottoms, and that's what happens to lead.

Where lead does cause problems is in still water, where lead shot from duck hunters can fall and be ingested by waterfowl. There, when in the bird's gizzard with other gravel, it gets ground up and digested with the rest of the food, and then causes poisoning. That's why lead shot got banned for duck hunting. In England, a lot of the fishing is done in very still water with very small shot, so this shot presents the same risk. I don't think downrigger balls and pencil lead present the same issue.

I don't worry about lead in the rivers. Hydrocarbon pollution is a much bigger deal, among other things.