Neutral Shards? imho, no.
And one could say "But if I specialize myself and don't find such shard, I'm screwed!". Yes, and if you find it you rock.
Which is all well and good in real life but in a game who wants a 75% chance of being screwed from turn 0? As I understand it, shards are supposed to be rare and powerful. However in games, rare and powerful often = unbalanced. On the other hand, if shards aren't rare and powerful then they are boring. If shards remain rare then something like this idea from Robert might work.
Or... a way that can be implemented right now:
Let spells get 3/4 of their bonus (damage or whatever) from the correct shards but
1/4 from the total number of all shards.
If you balance the numbers right then a character specializing in fire magic but with a non-fire shard could still have more powerful fire magic than a non-specialized character with a fire shard. That way, instead of being screwed by your starting position, you just don't rock as hard as you could have with a fire shard.
Alternatively (or maybe additionally), If shards become more common to insure that everybody is at least somewhat close to every elemental shard, then the cost of bringing a shard under your control should be much much higher. How about if shards cost essence to control? That would make you think twice about which shards to control and if you wanted to focus on all of one element or not. As it is now, the cost of 100g to build on a shard is trivial. So, you just go around building on any shard you can find. If the cost was much higher, it would be a serious decision as to what shard you would build on. As I said before, this game needs the opportunity for players to make more meaningful decisions. Not only what to build but also what not to build.
Of course this is all moot if there is no reason to specialize in less than all of the spell books. As I see it, if there is the ability to select spell books then there should be a real reason to not select all of them. Otherwise, why bother to have spell book selection at all. There also needs to be a reason to select all of them if thats how you would like to play. It should be a strategic decision that has real game changing consequences either way.