The steps to remove orphan users from Azure AD are a bit complicated, and I think the only way it can be done is from a local PowerShell session.

Here we go -

1. Install the MSOnline applet

PS C:\Users\administrator> Install-Module MSOnline

NuGet provider is required to continue

PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or

'C:\Users\administrator\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the

NuGet provider by running 'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want

PowerShellGet to install and import the NuGet provider now?

[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"):

Untrusted repository

You are installing the modules from an untrusted repository. If you trust this repository, change its

InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from

'PSGallery'?

[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): Y

2. Connect to the MSOnline service

PS C:\Users\administrator> Connect-MsolService

3. Remove users one by one

PS C:\Users\administrator> Remove-MsolUser -UserPrincipalName my-user@my-domain.com

Confirm

Continue with this operation?

[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"):