About 120 homes would be built on allotments opposite Syon Park , under plans from the Duke of Northumberland's estate.
Proposals for the allotment site in Park Road, Isleworth , were unveiled at the weekend, with a planning application expected next month or in January.
They would see the gardening patches replaced with two-and-a-half to three-and-a-half storey terraced housing, containing mostly two-bedroom homes for private rent.
There would be one parking space per flat and two per house at basement level, and trees would be maintained along the site's perimeter.
The duke's estate, which owns Syon Park and the allotment site, says the development is needed to help pay for the upkeep of the grade I-listed Syon House and its grounds.
It has offered to move allotment holders to a new plot inside the park, with the move scheduled to take place next spring.
More than 400 people have signed a petition to list the allotment site, which they say is a haven for rare birds, as an asset of community value.
But even if the application is successful, it is understood it would not prevent development as the land is not being sold by the estate so the community right to bid would not apply.
A spokesman for Northumberland Estates, which manages the duke's business interests, said just over 60 people had attended a public exhibition about the plans at Syon House on Saturday (November 7).
He described public feedback on the day as "balanced" and said the public's views would be taken on board before the final plans are drawn up.
"People welcomed the design and the high-quality of materials which would be used, and they recognised having private rented homes is a positive feature, as opposed to homes which could be bought as an investment," he said.
"Inevitably there were comments about congestion, particularly with the forthcoming closure of Church Street."
Should the plans get the go-ahead, the estate aims to begin construction in August 2016 and complete the buildings in winter 2017.