I also get the change done in less then 30 seconds. It does however take some strategic thinking before hand.
I make sure there is no extra gear floating around and keep some of the heroes in that I use for banish.
That way there is no need for inventory space.
Just make sure you know which heroes has what gear on and drop the right ones on eachother.
This just takes some preparing and consistency when dressing your heroes before they change to banish form.