Yes you could.
When you downgrade from an Unlimited plan to another (remove White Label), the custom domain will be removed too. This allows it to be chosen by another coach who may want it.
If you're worried about losing your custom domain, you will need to remain on an Unlimited plan and leave White Label added to your account. We will not be able to get it back for you if another coach chooses to use it while you're gone sorry.
Thanks
FITR