Several thousand people began fleeing Lower Dir, residents said, a day after security forces killed 26 militants in an offensive after being attacked by Taliban militants.
"I am leaving everything here and taking my family, said Karimullah, a farmer in the Samarbagh area of Lower Dir district.
"We can't take a risk with troops fighting the Taliban."
Lower Dir is about 170 km (104 miles) northwest of the capital Islamabad, and lies on Swat's western flank.
Surging violence across northwest Pakistan and the spread of Taliban influence have heightened concerns about insecurity in nuclear-armed Pakistan, an important US ally whose help is crucial to defeating al Qaeda and stabilising Afghanistan.
Pakistan's allies want to see coherent, decisive action by Islamabad against militants, and analysts say Zardari may want to show some steel before talks in Washington with President Barack Obama and his Afghan counterpart Hamid Karzai on May 6-7.
Lower Dir is part of Malakand division where President Asif Ali Zardari reluctantly sanctioned the imposition of Islamic sharia law this month after a peace deal with radical cleric, Sufi Mohammad, aimed at ending militant violence.
A spokesman for Mohammad, who was released from jail last year after renouncing militancy, said on Monday they were suspending further talks with the government until the Dir operation was stopped.
"We are suspending talks with the government until the military operation in Dir is halted," said Aamir Izat, a spokesman for Mohammad.
Security forces launched the offensive in Lower Dir after militants attacked a convoy of paramilitary troops and 12 children were killed by bomb hidden in a football. Helicopter gunships and artillery targetted militant hideouts in the villages of Lal Qala and Islam Qala. A military spokesman said the bodies of 26 militants were found in the battle zone late on Sunday. Independent casualty estimates were unavailable.
Sporadic artillery fire was heard overnight and on Monday morning and residents saw a helicopter circling the area.
Critics said the approval of sharia law in Swat valley, 125 km (80 miles) northwest of Islamabad, was akin to appeasing the militants, and Mohammad sparked uproar last week by denouncing the parliament, democracy and the Supreme Court as un-Islamic. Emboldened after winning the concession to shariah law, Taliban fighters moved south into the Buner district of Malakand – just 100 km (60 miles) northwest of the Pakistani capital – from the neighbouring Swat valley.
On Friday, with expectations growing of a military operation, Taliban commander Fazlullah, ordered his men to pull back to Swat, but officials and residents said armed fighters who hailed from Buner could still be seen moving around as usual.
Pakistan's interior ministry chief, Rehman Malik, on Sunday called on the Taliban to lay down arms after the security forces launched the operation in Lower Dir.
Keep up with the latest business news from the region with the Emirates Business 24|7 daily newsletter. To subscribe to the newsletter, please click here.