I've finished 2 games so far, here's a listing of the bugs / balance issues I've found so far:

I have to say, the improvements in Beta 3 are great!  Weapons seem more unique, the first strike problem was nicely fixed, everything is heading in the right direction!  

Bugs / Annoyances:

XP is still not balanced (but it's better).  I killed 3 groups of 5 mites and got 7xp, then killed one group of 3 mites and got 3xp.  If anything, killing more should be more xp per mite

Pathing is still very poor - when I have a unit leave a city to go somewhere, it often doesn't leave by the closest point in the city.  This will matter even with 1 tile cities, as it might exit on the far side.  And units don't use roads very intelligently, especially when caravans are on the roads

Still can't view the city when selecting what upgrade building to give them - that would be really nice to be able to do

An emperor that was immobilized surrendered, and was still immobilized, but in the middle of nowhere.  He got killed and was double immobilized in my city.

The 3-turns per unit construction minimum sucks.  And it causes weird things, like being able to rush them complete and GAIN gold when you've added more production than the unit costs but the 3 turn limit is in the way

When I design units, I want to see how many of each resource I have so I can plan correctly.

On a large map with dense monsters, by season 63 I had 4 cities, and had run out of things for my heroes to do anywhere near my territory.  At that point I could declare war and steamroll the AI, go ranging really far for monsters to level on, or just click next turn a lot.

It would be really nice if you could choose where improvements like apiaries and farms send their extra grain, or maybe have an economic tech that allows you to create a trade route that sends grain from one city to another.

Balance Issues

What decides how far apart armies start in tactical battles?  It seems random.

Early game - I have a 44 population city with a workshop, but it takes me 6 seasons to make 3 clubs and give them to some peasants?  In general, the long training time for units is absurd, but the 3 turn minimum is even worse.  Why would it take my 700 person city with the Great Mill and 6 materials 3 turns to train pioneers or those same clubmen?

Leveling heroes - far too often the abilities I want aren't present, preventing me from shaping the heroes as I want.  Please consider going to a talent-tree like approach.  Random isn't fun.

Using enemy roads means you can chain capture cities with insanely high movement

Too much grain is required for a max level city - 8 is virtually impossible to get

Even on Hard (my next game will be at ridiculous)  the game was won well before even half of the techs were researched. 

AI Issues

AI armies often get "stuck" and never move again for the rest of the game.  There was one place in my last game where Pariden had like 6 armies and they never moved from there in 100+ turns

A hero with 1 hp cast berserk on himself and died

A hero kept trying to cast things with a cast time after having been counter-spelled (twice)

Drakes don't use fire breath

AI still wastes time with buffs when they're completely unnecessary.  For the same mana, a fire bolt could have killed the only enemy, but the AI wants to cast Growth or Burning Blade first

It's been said that one advantage the human has is the ability to reload.  Maybe, on harder levels, let the computer cheat if the human does.  Every reload within a minutes of a save (not counting crashes) lets the AI remove immobilized and a penalty from a hero / sovereign, or save up a removal for when it needs it, or prevent a wandering monster from destroying a town.