You Can’t Tax the Rich Without the IRS

The hot policy in Democratic circles these days is raising taxes on the rich. Sen. Elizabeth Warren has a plan to tax “ultramillionaires,” as she calls them. Sen. Bernie Sanders wants to expand the estate tax. Rep. Alexandria Ocasio-Cortez has floated raising the top income tax rate to 70% for those making over $10 million a year.

But before this country raises taxes, it should grapple with something much more prosaic but equally important for tackling inequality: saving the Internal Revenue Service.

Already, wealthy people and corporations easily get around today’s rules. However tough any new laws might seem, they’d soon be undercut.

Slowly and quietly over the past eight years, the IRS has been eviscerated. It’s lost tens of thousands of employees. It has fewer auditors now than at any time since 1953. In real dollars, the agency’s budget has dropped by almost $3 billion since 2010.

Businesses and the wealthy benefit the most from this state of affairs. The largest corporations in America used to be audited every year. That started to change when the cuts began, and today, the audit rate has fallen by half. It’s a similar story for individuals making $10 million or more a year: With twice the chance of escaping IRS scrutiny, the ultrarich are much less likely to lose at the game of audit roulette.

Fixing the problem will require more than increasing the IRS’ budget (though that would certainly help). It’s about having the right personnel with the right skills. Today, the wealthy and corporations have the IRS outgunned. The ultra-affluent — with the help of legions of tax professionals — make domestic income disappear overseas or hide it in a pyramid of partnerships. It’s like trying to take on a modern army while armed with spears and clubs.

The IRS has difficulty tracing the income of the superwealthy or countering their sophisticated arguments about why what appears to be one type of income is actually something else. The agency has also trouble valuing their assets (a problem that, as The New York Times revealed, dates back at least to when Fred Trump was misleading the IRS about how much his buildings were worth). By the public admission of numerous IRS officials, it has long done a poor job of scrutinizing complicated partnerships to understand who owns what portion of what stock.

The top 0.5% of highest-earning Americans account for about a fifth of the income that’s hidden from the IRS, according to one University of Michigan study, or more than $50 billion a year in today’s dollars.

It’s much easier to enforce the tax laws for the bottom 90% of earners. Wages are reported straight to the IRS, and computers can easily check that tax returns accurately report that income. This means that inadequate enforcement of the tax laws necessarily has a regressive effect, liberating those at the top from scrutiny while the masses continue to be tracked by machines.

With its budget slashed, the IRS has pulled back across the board — except for one area where it’s been easier to keep the numbers from falling so much: audits of the poor. More than one-third of all audit targets are recipients of the earned income tax credit, one of the nation’s largest anti-poverty programs. By the hundreds of thousands, IRS computers spit out letters that require low-income taxpayers to prove their eligibility. The counties with the highest audit rates aren’t found in the hedge fund precincts of Connecticut or the lobbyist enclaves of Northern Virginia. No, they’re rural, mostly African American counties in the Deep South.

There’s nothing easy about auditing the rich. Even when the IRS has evidence of flat-out cheating, such as when it discovers a hidden Swiss bank account, it can take years of work to prove its case. Taking such time clashes with one of the core ways the IRS evaluates its managers and agents: how efficiently they open and close audits. As one senior manager put it to us, “If you’re not churning out the exams, you have to explain why you’re not.”

But more often, the affairs of the superwealthy are infernally complicated and the IRS doesn’t have black-and-white proof of evasion. Instead, the agency runs up against what’s called “aggressive” planning in the tax world. To the rich, a tax return is just an opening offer to the government. If they don’t get audited, that’s great. If they do, well, they’re ready for a fight.

In 2009, the IRS decided to take a bold, new approach to auditing the superwealthy. The agency’s idea was to bring specialists together and form a kind of green-eyeshade-wearing Delta Force. It would finally do something it — surprisingly — hadn’t been doing before: look at the entirety of a taxpayer’s empire. That meant examining how his partnerships, limited liability companies, foundations, gifts and overseas operations interact with one another. The ultrarich weave all these together so they end up with an ultralow effective tax rate. “We hadn’t really been looking at it all together, and shame on us,” Steven Miller, a former acting commissioner of the IRS, told us.

Because of budget cuts and lobbying, this elite force never came together. The agency hoped to have a staff of about 240 expert auditors by 2012. But by 2014 it had 96, and by last year the number had fallen to 58. Initially spooked by the group’s ambition, tax professionals who counsel the wealthy now speak of the effort with a kind of pity.

Criminal tax enforcement has also faltered. More than 150 million income tax returns were filed in 2017, but the IRS brought fewer than 800 cases in which someone was charged with making legal income but criminally evading taxes. That’s one reason tax cheats can get away with brazen behavior for so long — until they attract the attention of federal investigators for other reasons. That’s what happened to the former Trump campaign chairman Paul Manafort and the former Trump lawyer Michael Cohen, who pleaded guilty to tax evasion, and the celebrity lawyer Michael Avenatti, who was charged with failing to file taxes at all for several years. (Avenatti denies the charges.)

So, what to do? One 2020 candidate already has a bold proposal to resuscitate the IRS. It’s a plan to pump tens of billions into the agency, enough to fund a second army of agents. That candidate’s name is Donald Trump.

Of course, it was Trump’s party that gutted the agency, costing the country, we estimate, more than $100 billion in lost revenue. The president’s 2020 budget, however, did propose adding $15 billion over 10 years to the IRS enforcement budget, a move that would more than double the agency’s overall budget. The administration estimated this would produce revenue of at least $47 billion. The question arises: Has the administration genuinely had a change of heart? Or is this merely a neat trick to make deficit projections appear smaller?

So far, this is all just talk. It is, though, one of the few areas where Trump and Democrats are in substantive agreement. Democrats in the House recently proposed a $400 million IRS budget increase next year.

Among the Democratic presidential candidates, Warren does mention the IRS in her wealth tax proposal. She calls for an unspecified but “significant increase in the IRS enforcement budget” and a minimum audit rate for taxpayers subject to her “ultramillionaire” tax.

Raising taxes on the rich is a more compelling cause than increasing the operating budget of a bureaucracy most American voters probably hate thinking about. But the boldest tax plan of all starts with salvaging the IRS.