I think I realized what's the issue here.
It's not a bug, but very bad gameplay mechanic.
Explained in this thread.
In short, you can't build anything in tiles that is 4 or less space away from other city, distance counting from city walls borders, not city center.
When I built lost library in northern city, enveloping it in its city walls, it invalidated all tiles that Raaminobl could build on, including those that already have improvements (so if existing ones are destroyed, new ones can't fill same place).