Remember second grade, when your teacher caught you with candy and asked if you had brought enough for the whole class? Or when you had to make valentines for every single one of your classmates? Well, our feelings are just as easily bruised now as they were in elementary school; if we're not invited to something (even an event that we wouldn't want to attend) we perceive it as a slight. Therefore, if you friend one coworker, be prepared to friend them all.
No good, harmless way exists for you to cherry-pick your work friends on Facebook, so either invite everybody at once or at least be ready to accept friend invites as they come in. This goes double for bosses and managers: If they want in, you really don't have much choice but to say yes.
Bear in mind as well that, while refusing to add someone as a Facebook friend creates a socially uncomfortable situation, later deleting a coworker from your friends list amounts to a veritable slap in the face. You're better off tolerating their posts or, if that's more than you can bear, adjusting your settings so that their posts no longer show up in your News Feed.