The plans for Harrow School's expansion included a new 7,300 square metre sports hall, of which 4,600 sq m would have been built on Metropolitan Open Land, which is mostly undeveloped.
Sadiq Khan's new London Plan, which was published for consultation at the end of 2017, affords the same protection to Metropolitan Open Land that given to green belt areas.
As he rejected the application, the Mayor made it clear he supports the expansion of the school's sporting facilities but said it should not come at the expense of protecting open, green spaces.
Suggesting he “welcomes the greater access offered the local community”, the Mayor invited Harrow School to work with his planning team to develop a new application on the footprint of its existing site.
“I have been clear that protecting London's precious green spaces is one of my top priorities," he said.
“I absolutely support the school's ambition to expand its sporting facilities, and their plans to open them up to the local community for the benefit of people of all ages are to be commended."
He added: “However, I am clear that expansion of this kind must not encroach on open green space, which is one of the capital's most important and cherished assets.
“Nor, in this case, is it necessary to do so, as other options are available.
“I hope the school will rethink its plans and come up with a scheme which allows them to provide a new facility for their pupils and the community without harming the area's precious green spaces.”
As part of the application, the school offered a "land swap", where the land freed up by the demolition of existing buildings would compensate for the land lost by the new buildings.
However, this was deemed to be insufficient as footprint of the demolished buildings only represented a fraction of the land which would have been lost.
The scheme, which was initially approved by Harrow Council in September 2017, also included a new science block, which would not have been built on the designated land.
A spokesman for the school said: "Harrow School is disappointed with the Mayor of London’s decision to overrule Harrow Council on the granting of permission to build new state of the art facilities including sports facilities for which there is a real need in the area and which were proposed to be shared with the local Harrow community.
"Harrow Council agrees with the school that our commitment to provide over 1,300 hours of use of the facilities per year for local primary and secondary schools, clubs and others is an unmissable opportunity to benefit and enhance the lives of very many local residents and schools.
"The school remains grateful to Harrow Council for its strong support for the project and looks forward to continuing to work together in partnership in the future.
"No decision has been taken by the school as to the next steps and it will be seeking guidance from its advisers in the near future."
Keep up to date with the latest news in west London via the free getwestlondon app.
You can set up your app to see all the latest news and events from your area, plus receive push notifications for breaking news.