The Federal Court upheld a decision by Immigration Minister Alex Hawke to cancel his visa.
It means the 34-year-old will miss out on the chance to defend his title at the Australian Open.
Mr Hawke's decision to use his ministerial powers under the Migration Act also means Djokovic is now barred from returning to Australia for the next three years, "except in certain circumstances".
While Mr Morrison did not provide any assurances, when asked if the situation could change if Djokovic got vaccinated, Mr Morrison noted there was a possibility the tennis star could return sooner.
"I'm not going to sort of precondition any of that or say anything that would not enable the minister to make the various calls they have to make," he told Nine radio.
