Sir Martin Moore-Bick invited contributions from those affected by the deadly blaze as he hosted a consultation meeting to give local residents their say on what they want the investigation to cover.
One of the first members of public to speak said: "I don't think you are going to do us justice.
"I'm just watching you here. We need someone who's real."
Applause broke out from the hundreds gathered in the room inside Notting Hill Community Church on Tuesday (July 25) evening.
He continued: "We need justice and we need it fast, and we don't need someone who is going to play around for six weeks.
"Six weeks people have had to wait for this and you are only just looking into this now?"
During his initial address to the room, Sir Martin promised the inquiry would consider the design and construction of the tower, as well as the decisions that were taken in light of recent works done to the tower.
Several people in the room told the former judge he should be mindful of his choice of language and do more than just "consider" these issues.
He replied: "We are talking about the same thing. We are going to investigate and find the facts in relation to the whole course of events.
"And we will do that in a way that means we will get to the bottom of that."
He promised the room that the role of building regulations, the specification of the cladding and insulation, the tower's gas pipes and the role of supervision of works carried out would also be investigated.
Residents echoed concerns voiced at the last meeting with Sir Martin - that the inquiry team should better reflect the "diversity" of those on the west London estate.
At least 80 people died after the fire ripped through the 24-storey tower block in north Kensington, with many of those affected being from BME (black and minority ethnic) backgrounds.
Sir Martin was flanked by his inquiry team including QCs Richard Millett, Bernard Richmond and Kate Grange, inquiry secretary Mark Fisher and a female junior barrister.
Referencing the make-up of the panel, a woman said: "Look at us. Where is that reflected in you?
"Who are you going to put on the panel to inject trust and confidence?", she asked.
Shouting broke out periodically over whether people other than survivors should be able to take the microphone and address the room.
Further yells of "answer the question" were directed at the inquiry head whenever one member of the public finished speaking and a microphone was passed to another.
Responding to appeals for people to be brought to justice, he said: "An inquiry is designed to find out what happened.
"I have no power to do anything in relation to criminal responsibility."
He confirmed that the issue of deregulation would be looked at by the inquiry.
Keep up to date with the latest news in west London via the free getwestlondon app.
You can set up your app to see all the latest news and events from your area, plus receive push notifications for breaking news.