Where do you have your gas tank and battery(s). I'd personally run the kicker on the opposite side, because in a smaller, lighter boat like that 50 pounds extra on one side will make you feel out of wack all day long.

Other things that could be screwing with you are: You may have too much bow weight or rearend weight, either will make the boat track differently. Perhaps the boat is too narrow, I prefer a 6' wide bottom on a 18 footer. One thing you can do is while on the river loaded down ask someone next to you if the boat looks flat on the water, if it doesn't make adjustments to compensate. Also, if you have your kicker mounted too far to the outer edge of the boat you'll likely be able to turn one way better than the other and especially in reverse, you'll be able to back only one way. Try and get the kicker as close to the big motor as possible, remember a boat steers best from the middle.
Keith