Mnangagwa took his oath of office in front of tens of thousands of jubilant Zimbabweans who had gathered at a stadium in Harare.
Thousands chanted songs of praises while raising banners carrying messages of hope.
The 75-year-old stalwart of the ruling Zanu-PF party was a key Mugabe confidant for decades until they fell out because of the presidential ambitions of the first lady Grace Mugabe.
He was sacked as vice-president two weeks ago, triggering a political crisis that culminated in Mugabe's resignation on Tuesday.
Mugabe was the world's oldest head of state at the time of his resignation.
Mnangagwa is taking over a tattered economy and a record of human rights abuses albeit with promises of a renaissance in key governance areas.
But he is still associated by many with some of the worst atrocities committed under the ruling Zanu-PF party since the country gained independence in 1980.
Many say he was the country's spymaster during the 1980s civil conflict, in which thousands of civilians were killed.
However, he denies the role in the massacres.
He is widely known as "the Crocodile" - a liberation war nickname that may have stuck because it suited his reputation for ruthless cunning.
He will serve until the end of the presidential term next year.
