ok i tried to ignore this but i'd like to give my 2 cents after all...
first off the bat, i'd like to point out that you are comparing a KIA with a Benz here: both will bring you to your goal line, but one will do it faster and in style, while the other is simply cheaper.

then you can see and compare for yourself by simply clicking the power up arrow on your sword and/or spirit, it will tell you your current power, the future power and the power if you pay the gems for the 100% upgrade.
furthermore, they have absolutely NO mats in common, meaning you should collect enough mats for at least 1 sword and 1 spirit, if not, then i would always prefer the spirit since its more versatile and can be used by your staff and sword bearers (obviously not the one you will upgrade but the others that are "leftover").