The building was sheltering thousands of displaced people at Nuseirat refugee camp in central Gaza, according to the Hamas-run health ministry.
Israel said it had struck several Hamas “terrorists operating in structures located in the area of Al-Jaouni School”.
A local source said the target was a room allegedly used by Hamas police. A spokeswoman for the UN agency for Palestinian refugees (Unrwa) said the claims were "very, very serious" and should be investigated.
The attack comes as hopes rise that a deal between Israel and Hamas is on the horizon, following months of false starts.
Israel has announced it will send a team of negotiators next week to discuss a hostage release deal with Hamas.
It comes after a senior US administration official said Hamas had agreed to "pretty significant adjustments" to its position regarding a potential ceasefire.
A senior Hamas source told the Reuters news agency on Saturday that the group had agreed to begin talks on releasing Israeli hostages 16 days after the proposed first phase of an agreement aimed at ending the Gaza war.
Video from the scene of the Nuseirat school strike shows adults and children screaming in a smoke-filled street covered in dust and rubble, as they run to help the wounded.
Eyewitnesses told the BBC that the attack targeted the upper floors of the school, which is located near a busy market.
The BBC understands that up to 7,000 people were using the building as shelter.
One woman told the AFP news agency how some children were killed as they were reading the Koran when the building was hit.
"This is the fourth time they have targeted the school without warning," she said.
Hamas said five local journalists were among those killed in Israeli attacks on Saturday. Members of their family were also reportedly targeted.
More than 100 journalists have lost their lives in Gaza since the 7 October attacks, according to Reporters Without Borders.
Hamas said the five latest fatalities brings the number to 158.
In a statement, the Israel Defense Forces (IDF) confirmed it had hit the school buildings, saying it had taken "numerous steps" to "mitigate the risk of harming civilians, including the use of precise aerial surveillance and additional intelligence".
Hamas militants were using the location as a "hideout" to carry out attacks against IDF troops, it said.
"Hamas continues to systematically violate international law by exploiting civilian structures and the civilian population as human shields for its terrorist attacks against the State of Israel," it added.
Hamas called the attack a “massacre” on “defenceless displaced civilians”.
Many of the dead and wounded were women, children and the elderly, the group claimed via its English language Telegram channel.
Many schools and other UN facilities have been used as shelters by the 1.7 million people who have fled their homes during the war, which has lasted almost eight months.
"We don’t have all the information yet. Since the war began, we have had more than half of our facilities hit," Juliette Touma, Unrwa's communications director, told the BBC regarding the latest attack.
"Many of them were shelters, and as a result at least 500 people sheltering in those facilities have been killed. Many were women and children."
She added it was not the first time Israel had made such claims, and that they should be investigated.
A previous attack in June on another packed UN-run school in Nuseirat killed at least 35 people.
Local journalists told the BBC at the time that a warplane fired two missiles at classrooms on the top floor of the school.
After that attack, Israel’s military said it had “conducted a precise strike on a Hamas compound” in the school and killed many of the 20 to 30 fighters it believed were inside.
The head of the Unrwa described the June incident as "horrific" and said the claim that armed groups might have been inside a shelter was "shocking" but could not be confirmed.
Israel’s war was triggered by Hamas's unprecedented attack on Israel on 7 October in which Hamas-led gunmen killed about 1,200 people and took 251 others back to Gaza as hostages.
At least 38,098 Palestinians have been killed in Gaza as a result of Israel's offensive, according to the territory's Hamas-run health ministry.
Israel has regularly accused Unrwa of supporting Hamas, which is proscribed as a terrorist organisation by Israel, the UK, US and other countries.
The organisation has rejected this.
In April, a UN investigation found Israel had failed to back up a claim that many of the agency's staff belonged militant groups, but also said it could improve its neutrality, staff vetting and transparency.
Newer articles