In a recent game I had a caravan attacked and selected the retreat option. The caravan was then stuck in location permanently for the rest of the game. It could not be selected to move, merged with another unit or any other trick I could think of. It just sat there permanently non-moving!
Caravans should probably not be allowed to retreat from combat but should be an automatic casualty unless they have a defender with them.