I think a solution to the pioneer spam would be pioneers not placing down a city immediately, but rather an improvement that would then need to be built up using their home city's build queue while also consuming growth. These pre-settlements would also cause nearby monsters to attack before they were complete, so throwing up cities in frontier territory is dangerous unless you're committed to their defense.
This method thus makes expansion more gradual and the only early-on crazy land grab would be with Outposts for those trying to lay a wider claim.
Allows players to expand without the tooth and nail fighting for city spots? Check.
Lets players focus on different early game strategies with pros or cons to each? Check.
Allows larger empires? Check.
Emphasizes more caution when placing a new settlement? Check
Moving the increase production costs away from the pioneer unit itself allows players to send them out for other purposes and doesn't make the loss of a single pioneer devastating by comparison. By separating the city construction from the pioneer cost, it also allows future tampering (such as increased city time costs per city) without knocking everything else out of place in the process. Allow only one pre-settlement per city, or even one pre-settlement at a time, eliminate pre-settlements in another player's ZoC, and I think the problem is solved with minimal need to mess with other aspects of the game.
The only issue somebody brought up was the AI being unable to defend the pre-settlements but... if the AI can't check monster strength within 3 or so tiles and send armies to a single location until completed, well, let's just say that that AI is one with a bit too many issues for proper release. I'd say knocking monsters off their lairs would be exploitable but, as things stand, placing a ZoC under them makes them get off their butts and wander around already. At least if they're attacking instead of doing... whatever they normally do... it's immediately dangerous to place a city next to them.