Same thing happened to me yesterday when I was moving my gems around, but refreshing the game seemed to fix the problem whenever they disappeared.