Wildlife activists have confronted National Parks and Wildlife Service officers, claiming more could be done to save the mammal.
A small group is at the Palm Beach wharf and more are believed to be in the Basin, where the whale will be despatched with a lethal injection of anaesthetic.
Experts are defending the decision to put Colin down, saying they have run out of time and options and that he is suffering.
"It's the same as you would do with a dog or cat in pain," one officer said today.
Head of the National Parks and Wildlife Service Sally Barnes last night said Colin's condition had "significantly deteriorated" and the most humane option was to euthanase the calf to "put him out of his misery".
It was believed earlier this morning that the whale might have swum out to sea overnight as he appeared to vanish - but Colin was then located again.
"He was listless, was having trouble breathing was not really staying afloat the way a whale should," Ms Barnes said after vets assessed his condition.
"We had hoped we'd have more time to look at arrangements but we've had to make a very, very difficult decision in the best and most humane interest of the animal to put it down," she said.
The NPWS and other animal welfare groups were supposed to meet last night to discuss ways to help Colin but it was cancelled at the last moment after they realised he was suffering and could not be saved.
The decision to inject the three-week-old with a lethal dose of anaesthetic capped a day of passionate exchanges between environmentalists and NPWS officials on the wharf at The Basin as a whale whisperer and a paediatric doctor both tried to persuade authorities to give Colin a second chance.
But, late in the afternoon, the nation's leading whale vet, David Blyde from Sea World, arrived from the Gold Coast and the decision to put down Colin was made after the whale's condition suddenly deteriorated and his injuries worsened.
Although the lost whale - who had not eaten for at least five days - was not expected to live through the night, officials put off any attempts to euthanase him until daylight.
The decision was made after consultation with Taronga Zoo and animal welfare groups ORRCA and RSPCA.
Once euthanased, the body is expected to be taken to Taronga Zoo for an autopsy to provide scientists with clues as to why it was abandoned by its mother.
Colin had spent the past five days nuzzling boats moored in the cove as if they were its mother."