I like to eat fish when I can, but more often than not, I find it to be prohibitively expensive. Of course, if I'm being honest, all things considered, it probably costs me at least twice as much to catch fish myself. For that matter, the fish market's a lot more reliable than me, too.

All the suggestions here are good (as far as I know). I give an extra nod to Abu's suggestion, as local markets not only tend to have good, fresh fish, but they also represent money spent locally, which is a cause I support when I can. Of course, some local markets (probably unwittingly) sell stuff they shouldn't, so you do have to be careful. If you are concerned about potentially buying bad karma fish, consult the guide Todd mentioned and ask lots of questions.