If you are self-employed, paid in cash, have no home or car loans, no bank accounts, and have never filed or paid income taxes in the past, then it's possible to stay under the IRS's radar. On average, only 2 percent of non-filers get caught [source: Beam]. But if you receive a regular paycheck from an employer, have a mortgage and keep your savings in a bank rather than in garbage bags buried in the backyard, then the IRS will likely catch up with you and the penalties will be stiff.
The trouble starts when IRS computers notice that there are wages, loans and bank accounts associated with your Social Security number but no tax return. After mailing a series of increasingly sternly worded letters, the IRS will file a substitute for return, its own approximation of what you owe. Based on those calculations, the IRS will start applying failure-to-file and failure-to-pay penalties for each month you are late. Both penalties can go as high as 25 percent of your unpaid tax bill [source: IRS].
There is a subculture of tax protesters that employ constitutional objections — the IRS calls them frivolous tax arguments — to justify not paying taxes. The courts have routinely rejected these arguments as tax fraud or tax evasion. The extra penalty for tax fraud is 75 percent of taxes owed, and tax evasion can land you in prison [source: IRS]. Just ask Al Capone.