I think that the cooldown should go both ways. You should be allowed to change your mind during cooldown and return to your original realm. Then get charged the 150 gems for any more changes. Would solve more than one problem.