In most games of this nature the arrow keys will scroll the map. I tend to play with one hand on the arrow keys for map movement and the other on the mouse for unit commands. It is really annoying when you accidentally move a unit where you don't want to because of this. Perhaps a setting in the options.
By the way, even with no stacks selected the arrow keys will still move a stack. This must be a bug.