Smoke rose above the Red Mosque after troops began their operation
The army says up to 50 militants and eight soldiers were killed, and about 50 women and children rescued.
Military spokesman Maj Gen Waheed Arshad said militants were firing rockets from the minarets but troops controlled most of the complex.
Students at the mosque and its attached religious schools have waged a campaign for months pressing for Sharia law.
Public anger in the capital mounted after they kidnapped policemen as well as people they considered to be involved in immoral, un-Islamic activities.
Security forces began a full-scale siege of the Lal Masjid (Red Mosque) last week, not long after mosque students abducted seven Chinese workers they accused of running a brothel.
The BBC's M Ilyas Khan in Islamabad says the military operation is a gamble for President Pervez Musharraf who risks a backlash from supporters of those inside the mosque.
In recent days the army has redeployed thousands of troops in north-western Pakistan where pro-Taleban militants opposed to President Musharraf have been carrying out a string of attacks said to be linked to the mosque siege.
Loud explosions and gunfire were heard coming from the mosque area as the assault began at about 0400 (2300 GMT Monday). Troops entered the compound from three directions, exchanging fire with the militants.
Maj Gen Arshad said troops were going from room to room, meeting heavy resistance from those inside, who he said were well armed.
"They have all kinds of weapons, they have machine guns, they have rocket launchers, they have been using grenades - so it's a slow process," he told a news conference.
"About three or four terrorists have occupied the minarets."
The man leading those inside, deputy mosque leader Abdul Rashid Ghazi, was thought to be barricaded in the basement using women and children as human shields, Maj Gen Arshad said.
The military says 20 children have escaped from the mosque. Twenty-six women were rescued by troops, Maj Gen Arshad said.
Reports said one of the women was the wife of mosque leader Maulana Abdul Aziz, who is the elder brother of Mr Ghazi and was caught trying to escape from the compound in a burka last week.
Security forces say they have taken over the building's roof and now control 70% of the mosque complex, which includes a religious school (madrassa) for female students.
About 70 militants have been captured or surrendered so far, the army says.
Security forces sealed off the mosque from journalists and have also kept them away from the hospitals where the wounded are being taken.
It was not clear exactly how many people were left inside the mosque when the assault began.
Mr Ghazi earlier told Pakistan's Geo TV that his mother had been wounded by gunshot.
"The government is using full force. This is naked aggression. My martyrdom is certain now," he said.
Talks reportedly broke down over the militants' demand for an amnesty for all inside the mosque.
The government wants to detain a number of people who are on a wanted list, and also a number of foreigners whom it says are inside.
"We offered him a lot, but he wasn't ready to come on our terms," said former PM Chaudhry Shujaat Hussain, after talks conducted by loudspeaker and mobile phone with Mr Ghazi.
Religious affairs minister Ejaz-ul-Haq, one of the negotiators, said women and children had been locked up on two floors of the Jamia Hafsa religious school, which is attached to the mosque.
As many as five "hardcore terrorists" were inside the mosque, he added, saying that one person killed on the first day of the siege belonged to Jaish-e-Mohammad, an outlawed radical Muslim organisation which has been linked to al-Qaeda.Mr Ghazi denied the presence of any banned extremist groups. He said those inside were students of his religious school.