There are benefits to having an IPO other than raising capital. When a company's stock goes public, it can attract a lot of media attention. This amounts to free advertising for the company. Also, many publishers of stock market information list and track public companies, which means going public can get a company's name "in the books" and in front of investors and stock brokers.
Issuing shares allows employees to hold stock in a company. The employees know that their share of the company will increase in value the more successful the company is. This gives employees more pride in the work they do -- they literally become part owners. This effect can extend to interested business associates, as well. Imagine that the owner of a packaging business buys stock in the bakery we mentioned above when the bakery has its IPO. The packaging business supplies plastic bags to the bakery, so the owner might be willing to give the bakery a deal on the plastic bags in hopes of helping the bakery become more successful (and increasing the value of his stock).
The last benefit to an IPO has to do with rules established by the Securities Exchange Commission (SEC) that strictly regulate public companies to prevent fraud. To go public, a company has to open its accounting practices, sales figures, and marketing plans to anyone who wants to see them. This can make it easier for the company to secure certain kinds of loans and raise money from other investors.