There are quite few containers that can be used as inexpensive gardening tools such as seedling starters, planters, and tools.
Here are a few of the things you can save:

Food grade plastic containers in all sizes: Yogurt and sour cream containers, soda/water bottles, fruit cups, milk/water/juice/vinegar jugs, salad, cake/cupcake, and plastic take-out containers, 5 gallon water jugs, buckets, nut/candy containers, parmesan cheese containers, juice bottles, litter and other large food grade containers, coffee containers (K-cups and large containers), vitamin bottles, and others.

Other Things To Save: Styrofoam: Cups, take-out containers, and egg cartons – cans – glass jars – cardboard egg cartons – milk/juice cartons, newspaper, cardboard tubes, and whatever else you find useful!

Plastic cups can be used to plant seedlings. Larger ones can hold a flowering plant and most herbs to full growth. They can be decorated with paint and stickers and given as gifts too. Pop holes in the bottom with a drill or use a candle, nail, and pliers to make holes. (Adult supervision suggested.)

Plastic containers with lids, water bottles, and vitamin bottles are great for storing seeds. Soda bottles can be turned into self-watering planters, or used as a planter.

Milk cartons can be turned into planters that can easily grow a basil/herb plant, a flower, or lettuce. Paint the carton with acrylic paint and decorate with foam stickers or paint a picture. Milk cartons can be cut down and used as temporary plant markers for seedlings.

5 Gallon containers can be turned into a self-watering container garden.

Egg cartons can be used to sort seeds before planting or to start seedlings. 

Milk/water jugs make great watering containers. Simply pop a few holes in the lid with a nail or drill. Use plastic jugs to make plant markers too.

Salad containers can be used to store harvested veggies in the fridge.

Cardboard Tubes can be cut down and used to start seedlings.

Take out containers are great for watering seedlings and to put under plants.

Plasticware – Save your better quality plastic-ware to spoon soil to containers, rake soil to add flower seeds, or to smooth soil over seeds with a knife.

Inspiration Laboratories: Recycled Container Gardening With Kids – Upcycle various containers and household items to use in the garden.

iCreativeIdeas: 40+ Creative DIY Garden Containers and Planters from Recycled Materials – Almost anything can be used as a gardening container! Check out the suggestions on this site!

Note: Make sure all containers are clean before use and are food grade. Containers that are not food grade, or once used for chemicals, should never be reused.