By Michael O'Brien, Political Reporter, NBC News

(NBC) -- A House committee dismissed the IRS official in charge of overseeing the division that took action to single out conservative groups seeking tax exempt status after she invoked her Fifth Amendment rights on Wednesday and refused to testify.

IRS Director of Exempt Organizations Lois Lerner addresses a House committee during a hearing on the agency's targeting of political groups.

Lois Lerner, the IRS director of exempt organizations who has become a center of controversy in the revelations that the tax agency had singled out conservative groups, refused to answer lawmakers' questions during an appearance before the House Oversight and Government Reform Committee. But she defiantly denied having done anything wrong before asserting her rights.

"I have not done anything wrong. I have not broken any laws. I have not violated any IRS rules or regulations," she said in a brief appearance on Capitol Hill. She added that she had not misled Congress during any of her previous testimony before lawmakers.

That denial angered committee conservatives, who said that Lerner had essentially offered testimony, and waived her ability to invoke her constitutional right to not testify. The committee chairman, Rep. Darrell Issa, R-Calif., dismissed Lerner nonetheless, but warned that his panel might again seek her testimony in the future.

The instance marked a dramatic opening to a House hearing into the IRS abuses that was expected to feature abundant political fireworks. The first few minutes of the hearing lived up to that promise.

Rep. Jim Jordan, R-Ohio, for instance linked the Obama administration's assertion that the IRS abuses were limited to rogue employees to its initial assertion following last year's terror attack in Benghazi that it was the outgrowth of a spontaneous protest. (This assertion about Benghazi was eventually proved wrong, and has become another point of contention between the White House and congressional Republicans.)

And Rep. Stephen Lynch, Mass., a Democrat who has at times chafed against other members of his party, raised the prospect of appointing a special prosecutor to investigate the IRS controversy if lawmakers questions' weren't answered to his satisfaction. Previously, demands for a special prosecutor have been limited mostly to a handful of conservative Republicans.