This would introduce some complexity to equipment selection, making it less obvious that any one item is "best" for any given action (e.g., fire spells).
Example: A combat tiara that imparts a charisma bonus, but also enhances the combat bonus of other items; so, that one item alone does not in itself provide the ideal bonus, but it acts as part of a set, and swapping it out alone won't necessarily be the best choice.