When I zoom in it zooms toward my mouse cursor, but in such a way that the mouse cursor moves on the map, so the destination of the zoom keeps changing and I end up far from where I intended to zoom to.
The fix should be simple. The camera/viewpoint should move in a straight line toward the mouse cursor and the view direction should stay more or less constant. This should cause the cursor to stay still relative to the map.