In addition to not getting any archers for building castles/forts, all wall upgrades for Fortresses claim that I get +2 Defender groups, but I only ever get one for building that improvement.
This makes castles/forts extremely unappealing, because they say I get +3 units defending my city but I only actually get one.