I've seen this extensively when I modded games. I'm not saying that mods are at fault. What I'm saying is that if the pool of items is much larger, then the chance of getting the same item twice or three times should be smaller. And yet I frequently get the same item drop over and over.

I see this all the time. 

Yeah, I've walked through the code on this because I've had the same feeling from time to time but it appears to be working.  At about the point where I've gotten 3 Stone Maces in a row I also think that it seems very unlikely.  But it certainly isn't always the mace, and it doesn't mean that I will always get a mace next (it seems like whenever I'm watching for it it never happens).

So looking at the code it seems to just be that sometimes you will randomly roll the same item multiple times.  Though I'll keep my eyes out for it in the future.

One big offender that I have changed for 1.2 is the Shattered Statue goodie hut. Previously it gave out 1 of 5 items (instead of any common item).  Because of that if you got a lot of Shattered Statues around you it was a lot more liekly you would get a bunch of those items (Stone Maces, Escape Scrolls, Hate Stones).

Still wishing you guys would check out the issue I'm having with my mods, where the game will show one item and then award another item in quest dialogs and after-battle reports. Tired of getting hairstyles when finishing a quest, so to speak.

I had a game lately where I found 3 hate stones in a row and I frequently get the impression, that i find the same items over and over again. Maybe something in the random number generator is causing this? There are random number algorithms that generate not so random numbers if you always reseed them with a new seed instead of generating several numbers from one seed. I think the Java Random class behaves like that. If you always create a new object to get a new number the numbers are not as good. If you reuse one object to create all the numbers they a better distributed. They don't using Java but maybe a algorithm that behaves similar.

^This is a layer of strategic depth FE could definitely use.

I don't know how it is coded so I can make any specific statements...  but I'd have a look at how the code seeds the random number generator.  Is it possible the code is passing the same seed for subsequent calls?

http://www.cplusplus.com/reference/cstdlib/srand/

The pseudo-random number generator is initialized using the argument passed as seed.

For every different seedvalue used in a call tosrand, the pseudo-random number generator can be expected to generate a different succession of results in the subsequent calls torand.

Two different initializations with the same seedwill generate the same succession of results in subsequent calls torand.

If seed is set to 1, the generator is reinitialized to its initial value and produces the same values as before any call torandorsrand.

In order to generate random-like numbers,srandis usually initialized to some distinctive runtime value, like the value returned by functiontime(declared in header <ctime>). This is distinctive enough for most trivial randomization needs.

Correct. But a predetermined sequence of turns is sometimes desired; for example it may be desirable to seed the RNG with the same seed after reload. This could be done to ensure a sequence can be repeated if desired and to prevent the most simple form of reload-until-you-get-what-you-want.

The hard way to prevent it is to randomize all drops on map creation and keeping it as part of the map save.

If I could make a wish I would like to have both options: (1) Same RNG seed on reload , and (2) Save loot items with map.

Perhaps a more-elegant solution would be to have the system pull items from a table without replacement; e.g. once you get the stupid stone mace once it doesn't pop again so you never, ever, ever, ever have to look at the damn thing and read the joke, mildly funny the first time, annoying the second, and now as horrible as an innocent little girl in a scary movie who you just know is demon-possessed...

Sorry, I get carried away sometimes.  Point being, pull items from a table and flag 'em as passed out already.  If that's too restrictive for generic bits of armor, put multiple copies in the table.  

But how difficult would it be for SD to increase the number of low level items, thus decreasing repetition?

It would be pretty easy low hanging fruit IMO. They don't have to be very creative either. For exmple, a variety of low level swords or staffs each with a different minor benefit. One would have +5% magic resistence, another might have ignore 20% of opponents defense, another might have +2 dodge, etc, etc.

If I can create stuff by adding/altering the xml files, it should be a very easy thing for SD to do.

I have seen this as well a lot.