Interesting discussion, as I have been and avid sailboat racer and an avid fisherman for most of my life. Sailboaters do have the right of way over power boats under all but special circumstances as mentioned overtaking and shipping lanes)and they mostly all know it. Most are courteous enough and aware enough to not push that issue around fishermen, and will give trollers a wide berth. Nobody wants to untangle fishing line from their centerboard or prop shaft.

Sailboat racing is another issue as people get pretty serious and want take advantage of every wind shift or current flow as that's how you win races. My guess is that the dinghy in question was in a race and that's why they "pushed it to the limit".
Sailboat racers get pretty hacked at powerboaters who roar through their racecourse, narrowly missing boats and creating wakes that, at least, impede progress, and at worst can actually damage boats. Sounds simple but you would be amazed at the number of times I've seen that happen. My advice is to give racing sailboats a wide berth and everybody stays happy.

As far as setting up a racecourse in an inconvenient location for other boaters, you may have a good point, but remember, in Puget Sound light sailing winds prevail and sailboats literally cannot make progress against a strong current in light air. Because of that Sailboat racecourses must be set in areas relatively protected from current. That may be why courses are set where they are.

I don't claim for a second that sailors always behave courteously or considerately but maybe this will help folks understand a little more about where they're coming from.