Maybe yes, maybe no.

Trophy hunting for antlered game will generally take males at the "peak" of fitness. As they age, antlers get thinner and lose mass and points. It does seem that the Quality Deer Management, at least as applied to whitetail, works over a span of a decade or two. Can be done, just would take severe restrictions and put most of the harvest on females and really messed-up antlered males.

Horned game has the horns growth throughout life. It is at least theoretically possible to harvest a past-reproductive prime animal and still get good horns.

With predators, the trophy is skull measurements. I don't think the skull gets smaller with age so, again, you can take an animal that has at least had a few years of breeding.

Fish is trickier as some species spawn once which means you do remove the "trophy" genes completely from the next generation. Consequently, severely limiting take and accessing only mature fish could work.