“The British people have clearly spoken,” he said. “They have expressed their democratic will. And there can be no going back now. There can be no middle way. Britain will leave the European Union.”
Describing London as the “beating heart of the country”, he called for a post-Brexit deal that would retain access to the single market.
He said: “To have access to a single market we have got to renegotiate as best we can and that is one of the reasons why it is crucial for London to have a seat at the table because we are the beating heart of the country and London needs to do well.
“When London succeeds, the rest of the country succeeds, too. And when we fail, the entire nation fails.”
However, he dismissed calls for London to be given City State status.
The mayor is nevertheless angling for devolution of fiscal responsibility including tax raising powers, as well as more control over business and skills, housing and planning, transport, health and policing and criminal justice.
Speaking at the Times CEO Summit, Mr Khan added: “I used to be a lawyer and you win cases and you lose cases but, when you lose a case, you have got to accept the jury is always right.
“The referendum is not a situation where you can appeal to the court of appeal.
“We gave the public a choice. And they made a clear choice. And they chose to leave.”
He said that he had spoken to the Mayor of Paris who told him if Britain decided to leave the EU, France would be “rolling the red carpet out” in a bid to attract London business to the French capital.
He said renegotiating to stay within the single market was the top priority when it came to protecting jobs and London’s prosperity.
He added: “The message from executives and entrepreneurs could not be more clear.
“They believe that the crucial thing now is Britain must remain part of the European single market. And I totally agree.
“Five hundred million customers. Nearly half of all British exports. The passporting of our financial services. The foundation of British jobs, wealth and prosperity. Remaining in the single market needs to be priority one, two and three of our negotiation with the EU.”
Mr Khan said the government should hold off for “as long as possible” before triggering Article 50.
He added: “My worry is that unless there is a plan in relation to negotiating with the EU what is the point of serving the notice?
“I think we do need a new Prime Minister, I think David Cameron was right to not to serve notice to quit until we have a new Prime Minister.
“And I hope when we do have a new Prime Minister, he or she will have worked out the game plan and, as importantly, they will speak to us because we have strong views on this and it is right that the government speaks to Scotland, speaks to Northern Ireland, speaks to Wales, but we need a to have seat around the table because London is crucial to our country’s well-being.”
However, Mr Khan refused to be drawn on the crisis engulfing Labour and whether or not Jeremy Corbyn should remain leader of the party.
He said he had spent “every spare moment” since the referendum result speaking to businesses and reassuring them about London’s future and had not had time to get “involved in internal party politics”.
He added: “Jeremy won an election last year with a big mandate and he has got a big job to do and so have I.”
Speaking about potential job losses in the City sparked by Britain’s decision to leave the European Union, he said the Conservative party leadership candidates for Prime Minister needed to “say loudly and clearly” that Britain would remain in the single market as soon as possible.
He added: “Once businesses have that reassurance, it means they don’t need to leave our city. It means people from around the world will still want to invest in our country.”
Mr Khan said that he would not “criticise other cities for trying to pinch our business”, adding “that is what you do if you are the leader of a city” but it was vital that reassurance was given to businesses in London sooner rather than later.
Mr Khan also branded the increase in racist incidents since the Brexit vote as “unacceptable” and called on people to report all abuse to the police.
He added: “If we don’t stamp out ‘low-level stuff’, name-calling, racial abuse, that can lead to more serious stuff and that should have no place in our society.”