I had more than one panel worth of units garrisoned in a town so that you have to press the arrow to scroll right.
When I pressed V to move them out of the town it ejected them all and then only the "second" panel of units still existed (or could be found on the map). The first panel was overwritten when the second group was ejected. Thus I lost all but two units.