Brown admitted to a few solid tackles on the rugby pitch in his youth and throwing newspapers on the floor but said any anger was largely directed at himself, as he gears up for the general election expected in May.
"If I get angry I get angry with myself," the prime minister told Channel 4 television.
"I throw the newspapers on the floor or something like that.
"Let me just say, absolutely clearly, so that there is no misunderstanding about that: I have never, never hit anybody in my life.
"I don't do these sorts of things.
"In the heat of the moment you say things sometimes and, of course, you do get angry, mostly with yourself.
"Any allegations that have been made about hitting people or anything are completely untrue."
The Observer newspaper published extracts from a book by its correspondent Andrew Rawnsley, which contained allegations about Brown's conduct in his 10 Downing Street office.
It alleged there had been a string of incidents including shouting and swearing at colleagues and thumping car seats that left had staff shaken.
According to the book Britain's top civil servant, Cabinet Secretary Gus O'Donnell, told Brown to change his behaviour after reports that his fiery outbursts had frightened staff.
He reportedly told Brown: "this is no way to get things done".
O'Donnell felt the need "to calm down frightened duty clerks, badly-treated phone operators and other bruised staff" and tell them "don't take it personally", The Observer reported.
However, a spokesman for the Cabinet Office said: "It is categorically not the case that the cabinet secretary asked for an investigation of the PM's treatment of Number 10 staff."
The prime minister's official spokesman said: "These malicious allegations are totally without foundation and have never been put to Number 10."
On Saturday, Brown launched "Operation Fightback", saying while he was "not perfect", voters should think twice before ousting him.
Labour announced its campaign slogan would be "A Future Fair For All".
Although official election campaigning has not yet started, British politics has taken on an increasingly electioneering tone since the start of the year.
Keep up with the latest business news from the region with the Emirates Business 24|7 daily newsletter. To subscribe to the newsletter, please click here.