Food in general seems to be messed up, sometimes I am lucky and get a settlement with 2 fertile tiles, then its ok.

What really needs to be done is each settlement needs to have a default food production amount at a given city level to support defense forces and the population.

Bonus fertile tiles where "large scale" farms are built would be for army and further defensive force expansion.

City level 1  =   +1 food

City level 2  =   +1.5 food

City level 3  =   +2 food

City level 4  =   +2.5 food

City level 5  =   +3 food

Something like that maybe ?

yep , I know you can discover fruit orchards for sure, 25% food bonus if I recall...

Well, not only can you find more resources, and research better food production, but the problem usually happens when you take over enemy cities. For some reason, so far the AI doesn't seem to always need food, and they place cities all over the place. Capturing them can decrease your available food by a lot.

And since food is a scarce thing, you have to be really careful about placing cities not next to a harvesting place.

Besides... you're supposed to be low on food, remember? Part of the environment 

I always tend to play Empire and until now I had never any real problems with food, even if I had 20+ cities of lvl 3+.

There isn't much to do other than picking three or four cities to specialize on producing food. There you do the following:

- Use both building with the +25% food production
- Use as many caravans as possible to increase the food output
- Try to get a few of those champions who give each a bonus of +20% and put them in the city with the highest output

With this I usually get each city to produce something around 20 - 40 food and more, which can keep many other cities running.

Other than that, even without caravans you should use every oasis and fertile land, for a few of them can also sum up in a nice supply of food.

One last thing: There is no need to bring any city above lvl 3 but two or three at most, since Empire has very few building requiring a higher city lvl. Three or four houses are enough. And destroy the huts in conquered enemy cities - the AI is fond of building way too much of them.

I usually raze enemy cities unless they're in a good location.  The AI will sometimes put cities the minimum five tiles away, for no other reason than to grab a clay mine or other useless resource.  And currently, since one stack of squads can march across the land and bludgeon everyone to death (and the AI likes to leave Sovereigns unprotected) I don't do defense.

And razing things is just fun anyway.

The Empire gives up the bonus food tiles in the adventuring tree for an extra caravan per settlement.  Generally this means you're going to be more food constrained in the early game, but less as the game progresses.  My typical remedy is to go capture a Kingdom capital ASAP.  Of all the basic resources in the game, Food is the one I have never had any shortage problems.

Before.  A city that produces 4 food, has two houses, irrigation (+25%), and medium length caravan (+50%) nets out at +5 food to the global pool.