There are a lot of battery options these days, and Earth 911 has put together a helpful fact sheet about them.
And when you're ready to recycle them, you can use the search option at the top of that page to find places in your zip code that recycle spent batteries.
Throwing batteries in the trash is not an option. Batteries contain toxins which ,will leach into the environment when disposed of in regular trash. The toxins can move into the groundwater near a landfill and end up travelling quite a distance from the original disposal site.
Image courtesy of Cleantechnica.com