This article is more than
6 year oldHailemariam Desalegn told a press conference charges would also be dropped for those still awaiting trial.
The move was designed to allow political dialogue, he said.
Ethiopia is accused by human rights groups of using mass arrests and detention to stifle opposition.
However, this is the first time the government has admitted to having any political prisoners, previously describing them as "criminals", BBC Ethiopia correspondent Emmanuel Igunza says.
Campaign group Human Rights Watch (HRW) has accused the government of using anti-terrorism laws to jail its critics.
The exact number of people in detention is not known. However, thousands have been detained since 2015, when there were widespread protests calling for political and economic reforms, prompting a crackdown.
Just days ago, 19 people linked to the Ginbot 7, deemed a rebel group by the government, were jailed on terrorism-related charges.
Wednesday's announcement included the closure of Maekelawi - a detention facility in the capital, Addis Ababa, which HRW alleged was using illegal interrogation methods back in 2013.
The government denied all allegations of abuses at the centre.
However, it has now decided the prison will become a "modern museum".
A new detention centre will be opened, Mr Hailemariam said, which would comply with international standards.
It is unclear exactly what prompted the decisions, which came after days of consultations within the ruling coalition, the Ethiopian People's Revolutionary Democratic Front (EPRDF).
However, the privately owned Addis Standard newspaper links it to a day of action across social media at the end of December, raising awareness of the plight of political prisoners.
Newer articles
<p>The two leaders have discussed the Ukraine conflict, with the German chancellor calling on Moscow to hold peace talks with Kiev</p>