Like DaveD said, I went bigger, then smaller. I've had three boats from 8 to 10'. The bigger one-man boats are fun to row and can handle anything, but they really lack the convenience that makes pontoon boats so nice. They're heavy and more difficult to transport. The 8-9' boats from reputable manufacturers are well made and light. I've settled on a 9' Bronco. It's light enough to carry down a trail but stable enough that I feel confident on the water.
I also prefer the smaller boats for lakes. They can be controlled with fins and sitting closer to the water makes releasing fish easier.