There are a few important things to research before you leave the house...
First, check the tides and the weather. Some of the best kayaking destinations are passable only at high tide. Driving there, unloading and getting into the water, all to find out that an oyster bed blocks your path, is not a fun experience. Dress in appropriate clothing.
Second, make sure you know your route. Know where the put-ins are, and more importantly, know where the take-outs are. Figure out the logistics - how are you going to get from the take-out (with your kayak) back to where you parked (by the put-in). If you need two cars, plan that ahead of time.
Third, tell someone your plans. Pick a time, later in the day, when you will call to tell them you made it back home. If they don't hear from you, they can alert the proper authorities about the inexperienced kayaker lost somewhere on the river. This is good practice no matter what adventure sport you participate in, particularly if you are going out alone.
Check the rules and regulations for the location, espescially if you are in a state park, national park or conservation area. Some prohibit certain types of containers, beverages or equipment. Some have restricted hours - you don't want to get back late and find out the gate is closed and your car is stuck in the park.
If you plan to fish, make sure you have a state-issued fishing license and know the restrictions and limits on the fish in that area. Catching fish out of season, or of an incorrect size, yields big fines and can seriously damage fish populations. Enjoy the scenery, and the fishing, but don't ever do anything to jeopardize it for the next person.