I have a lot of experience fishing from a yak in the Atlantic. When I used to live in boston I had a Perception Acadia decked out for fishing & trolling with rod holders, gps, etc. It is a great way to fish and a lot of fun with anything over 20 lbs on your line. I used the kayak mainly for Striped Bass & Bluefish. Sometimes for sight fishing on the flats off cape cod and sometimes 3-4 miles out in the open ocean. You can safely take a kayak in much rougher water than a small boat, including breaking surf. Kayak fishing is becoming a big thing on the east coast and in California. There several websites devoted to the subject & several mfg now produce fishing-specific boats. I sold the kayak last year to my 65 year old father who is now addicted to yak fishing & owns three kayaks.
As for capsizing, yes it will happen. Main pro of a sit-on-top is that when it flips you just get back on and paddle to shore. No sweat. I had a buddy well offshore when he missed a hit on the surface, jerked the plug out of the water, ducked when it came flying at his head, and went swimming in 40 degree water well out of swimming distance from shore. He was in a sit on top so we flipped it over & he climbed back in, no problem. That said, I definitely don't recommend sit-on-tops for fishing anywhere but california, florida or somewhere like that. You'll freeze. Your legs will get soaked, you are constantly sitting in water that collects in the seating area and the wind blows on your wet legs making it very unpleasant. I can't emphasize this enough: if you buy a sit-on-top you will definitely regret it. Ignore the guys at the shop.
A normal kayak is the only way to go. Always carry a pump, paddle float, and sponge and you will be fine. Practice self rescue in a warm pond a few times and you will see how easy it is. You don't have to know how to eskimo roll. Every year a few people, often fishermen, get in trouble off the cape because the don't carry these things. If you have the paddle float and pump, you'll be back in the boat in less than a minute and paddling away in less than 5. I have done it several times and it really is easy even in cold water.
As for a kayak model, I was happy with the Acadia but the favorite among guys fishing off the cape is the Wilderness Systems Pungo. They make an angler version of this. They're cheap, indestructible. roomy, and very stable. This is probably the model I would go for if I was in the market again. I would also consider rigging a small downrigger on the yak out here. Any recreational boat will work. What you don't want is a long, thin, touring boat or a whitewater boat. The oerfect kayaks are the entry-level models that are wide & stable, but not too wide to be clumsy paddling. With the currents int he sound you will need to be able to paddle at a decent pace.
Here is a link to one of the east coast kayak fishing forums I used to follow.
http://reel-time.com/forum/forumdisplay.php?s=5007278d491a98a3dfb42d5f732b9b97&forumid=34 Here's another site mainly watched by warm-water fishermen.
http://www.kayakfishing.com/ Long post. Brings back some great memories. Good luck. Email me if you have any questions. Again, sit-on-tops are not good for cold water fishing!
Sideoff