"We need to get on top of this and find effective ways of dealing with problems of accommodation, of housing, and services in the inner city," Mr Ramaphosa added.
He also commended emergency services, who arrived at the scene 10 minutes after the fire was reported.
Johannesburg city manager Floyd Brink said 200 families were affected by the fire and "all efforts" were taken to provide accommodation.
A spokesman for the emergency services, Robert Mulaudzi, told the BBC that the fire had gutted the building but firefighters had been able to bring out some of the occupants.
He explained that because the building was not properly looked after, makeshift structures and debris had made it hard to search for and rescue people.
A video posted by Mr Mulaudzi to the platform X, formerly known as Twitter, showed fire trucks and ambulances outside the building with burnt-out windows.
Photos from the scene showed covered bodies lined up near the burned building.
One woman told journalists she was outside the building searching for her 24-year-old daughter.
"As soon as I heard the building was burning down, I knew I had to run here to come and look for her," she said.
"Now that I'm here, I'm kept in suspense because I really don't know what is happening. I don't get any direction - so I'm actually very anxious, I don't know if my daughter is alive."
In a visit to the scene, Johannesburg mayor Kabelo Gwamanda said Johannesburg officials would relocate people living in similar "hijacked" buildings in the city, and turn those buildings into social housing. "We are not going there with brute force," he told reporters, "we are trying to apply a sensitive strategy."
Asked whether his administration would take responsibility for the tragedy, Mr Gwamanda said the government was dealing with the issue of cartels hijacking buildings, which was taking place across the city.
In the wake of the fire, many South Africans on social media condemned the online xenophobic attacks that some have made against the victims and survivors of the fire.