Correct me if I'm wrong, but don't troops move much faster on the roads the caravans produce? That's the main advantage for me. The only reason I really put Caravans in the field is to make those roads so I can reinforce areas as necessary when teleporting isn't an option.
That't the main reason to build caravans everywhere, the other reason is they provide food.
Quoting cpl_rk,
reply 7
City spam + caravans from each city = no food problem.
Theoretically yes - but in that game where i was starved of food, it was a large map and a long distance between any resources. So I'm not going to get even one full run out of a caravan once established because the animals are going to be falling over each other in the stampede to kill it!
It's not theoretical, it's actual. I typically endgame large maps with 50+ food with over 100 cities. If you want to see my last game I posted it in the save games thread, can't think of the title right of the thread right off-hand. I had +96 food on turn 433 with over 100 cities and only 2AI left to crush in the game. Just, build a bunch of cities, caravans in each, and send them to food producing cities, you won't have to worry about food anymore. Of course, don't massivly be building houses everywhere either, just in the important cities.
Of course, this is going to change in 1.1, apparently caravans won't affect food anymore but gold instead, so enjoy it while you can 
