Never seen it happen, removed saints from gold before.
It does not automatically use your card normally, you have to be the one to use it.
Since the use for the card is not far below removing the saint, perhaps you accidentally pressed it.
Otherwise it sounds like a gap in memory or a fluke to me.