Two people were taken to the hospital after the crash that set ablaze the seven-story building, Austin fire officials said. The pilot, identified by law enforcement sources as Joseph Andrew Stack, was found dead.
The incident renewed fears of domestic terrorism and gaps in security for private aircraft. The White House said it had no reason to believe there was a link to foreign terrorism and that President Barack Obama was briefed on the situation.
It was probably "a deliberate attack by one individual acting alone," Austin Police Chief Art Acevedo said, downplaying the threat of more attacks.
US Representative Michael McCaul of Texas called it "a deliberate and intentional attack against a federal building."
A rambling note posted on the Internet and signed "Joe Stack" spelled out problems he had with the Internal Revenue Service tax agency. A law enforcement source said authorities were investigating the note.
"Well, Mr. Big Brother IRS man, let's try something different; take my pound of flesh and sleep well," the note said. It was not possible to independently determine the authenticity of the note, dated February 18, or its authorship.
In the note, Stack complained that he ran afoul of U.S. tax laws when he failed to file a return after not earning any income one year. He also said in the note that he had trouble resolving some undocumented income.
CNN reported the pilot had set his own house in Austin on fire before taking off in the plane and crashing it into the building at about 10am CST (1600 GMT). The authorities were also investigating that report, the law enforcement source said.
The plane, identified by the Federal Aviation Administration as a single-engine Piper Cherokee, took off from the airport in nearby Georgetown and was flying under rules that did not require the pilot to maintain contact with air controllers.
A Piper aircraft was registered to Joseph Stack under a Lincoln, California, address, according to the FAA's registry. In the note, Joe Stack said he had moved to Austin from California.
The damaged building housed IRS offices with about 190 employees, including the criminal investigation division as well as auditors and tax collectors, according to agency officials. It sits across the street from an FBI office.
As a precaution, two F-16 fighter aircraft were scrambled after the crash and patrolled above the Texas state capital, according to North American Aerospace Defense Command which is responsible for guarding U.S. airspace.
US Treasury Secretary Timothy Geithner, whose department includes the IRS, expressed concerns about the incident during an appearance in North Carolina. "We are closely monitoring the situation," he said, adding the crash "is of deep concern to me."
The attack is not the first on the IRS, which periodically draws the wrath of taxpayers.
The crash and burned out building also brought back memories of the September 11, 2001, attacks in which hijackers crashed commercial jetliners into the World Trade Center twin towers in New York and the Pentagon outside Washington.
"It's something that has exposed a weakness we've seen since 9-11 -- airplanes can fly into buildings," McCaul said.
In a separate incident on Thursday, a United Airlines flight from Denver to San Francisco was diverted to Salt Lake City because of a threatening note found aboard, according to local airport officials. The passengers, luggage and plane were rechecked and the flight was cleared to continue.