Moving to another city is the ultimate step you can take for long-term savings. Living in a large, major city will always be more expensive than living in a small town because property values are so high. That drives up virtually every other price in the city.
For example, moving from Los Angeles to Buffalo, N.Y. will cut your housing costs in half. Health care will cost 16 percent less, utilities 14 percent less, groceries 13 percent less and transportation 8 percent less [source: C2ER via CNN]. Over several years, you could save tens of thousands of dollars. That said, moving and buying a new home is expensive, so that cuts into your savings. You'll also have to find a job in your new city, and in a city with a low cost of living, you may have to take a pay cut.
In 2009, Forbes listed the best areas in the U.S. to live cheaply. The top city was Manchester, N.H., but note that good cities with low costs of living have certain characteristics. They have medium populations (enough for some job opportunities and cultural institutions), but that population has undergone a decline in the last few decades (which reduces property values due to supply and demand).
If you don't need an urban lifestyle, rural areas usually offer low property values. However, the cost of living can be higher due to lack of access to public transportation and other services you might find in a city.