Attorney General Eric Holder said on Tuesday that he recused himself from a case involving a Department of Justice decision to subpoena phone records from Associated Press, the Business Insider reported. Holder also said that the Justice Department has ordered a criminal investigation into the IRS' targeting of different conservative groups applying for tax-exempt status.

Holder called the move by the IRS "outrageous and unacceptable," in a press conference on Tuesday. The attorney general said that the leak being investigated was one of the "top two or three" leaks he has ever seen, claiming it put Americans at risk, regarding the AP phone probe. The Justice Department subpoena stemmed from an AP story, which was published on May 7, 2012, the news company said on Monday. The story dealt with CIA thwarting of an al-Qaeda affiliate in Yemen's plot to detonate an upgrade of the so called "underwear bomb" on a U.S. bound airliner.

Eric Holder told reporters Tuesday that he recused himself last year from any involvement in an investigation of national security leaks, and said the administration has no policy of targeting the news media.

Holder also announced that his department has launched an investigation into revelations that the IRS singled out conservative groups seeking tax-exempt status.  "I have ordered an investigation to be done" to see if criminal activity took place in the case," Holder said. 

"I am confident that the people who are involved in this investigation, who I know for a great many years and who I've worked with for a great many year followed all of the appropriate Justice Department regulations and did things according to DOJ rules."