Midfielder Bellingham and defender Colwill will return to their clubs to continue their rehabilitation, the English FA said in a statement.
Colwill missed Chelsea's Premier League match against Manchester City on Sunday, while Bellingham has been out of Real Madrid's last two matches.
England, who sit atop Group C with 16 points from six matches, have already qualified for Euro 2024. They play eliminated sides Malta and North Macedonia on Friday and Monday.