It looks like, from my admittedly not exact checking, that the loss of a shard causes a relevant buff's strength to decrease. For example, losing a fire shard causes my sovereign's burning blade strength to decrease by two. However gaining a new shard does not have the reverse effect, causing existing buffs to increase by the relevant amount. You still have to dispel and recast the buff to get the new shard to effect the buffs.
