I have the same impression as the OP. Almost every game sees several repeats of the same item. (I hate rusty vambraces, btw. Really? That's notable treasure?) A couple of quick thoughts:
(1) It could be the way that SD seeds the RNG--do they reseed it with every call, or prepopulate a table of randoms to access? Maybe they could try something different.
(2) It could be (as suggested) that the pool of items is too small. My personal solution for that would be a tool similar to Master of Magic's "itemmake", which allowed you to access any loot model in the game and design a new loot item by giving it X number of properties. Then it writes that item to the .xml loot file. The number and type of properties were weighted, so the more powerful the item, the more powerful the lair/quest needed to find it. I would handily pay for DLC that let me do this. Itemmake was one of the best little secrets of MoM. I know you can do this already by editing the .xml tables, but having a simple GUI for hopping through the models and picking properties would be nice.
(3) Adopt a diablo/torchlight/borderlands approach to loot, with a base item and 1-3 prefixes and suffixes depending on where/how it was found. I just started a new game and the first 3 green huts I hit all gave me an "Assassin's Hood". Lame. But not so bad if I found three hoods with different properties (Assasin's Hood, Hood of the Earth, and General's Hood of Leadership, for example). Since there's little difference in in-game appearance (except for flaming weapons, etc), I'm not sure why this wouldn't be a feasible approach to help the game stay fresh (other than the work involved, obviously).
Anyway, that's my $.02. If they will do that and give me the option to close my borders so other players can't cross without permission (instead of forcing me to jump into the diplomacy screen *every time* they try to send their pioneers across), I would be a very happy camper.