The other children, aged eight and ten and who are believed to be his brothers, managed to make it back to shore and alerted rescue services who immediately launched a search for the missing boy.
A helicopter, coastguard boats, divers and police officers scoured the sea and the coastline until night fell, and resumed their hunt on Tuesday morning.
A coastguard official said that there was no hope of finding the boy alive because, even if he had not drowned immediately, he would not have been able to survive more than an hour or two on the chilly waters of the English Channel.
“There have been two tides since he went missing, so the body may have travelled a considerable distance,” an officer involved in the search told the local Nord Eclair newspaper.
The boy’s parents, as well as the other two children who were with him when he disappeared, were in a state of shock and were taken for treatment and counselling in a hospital in Boulogne-sur-Mer.
Local media said that the boys had been gone into the water in an area where swimming was forbidden due to strong currents partly caused by boats entering and leaving the harbour.
The town is a major ferry port with daily crossings to the UK. It is not clear if the British family were staying there or were arriving for a holiday elsewhere in France, or were on the way back home from their summer vacation.