Don't think there anywhere to look these things up yet, But as for your examples, Lumber mill need a Old forest (or some such) a special tile. Doesn't help with trees. Quarries also need special tile I believe. Marble I think it's named.

They removed listing all improvements that you can't build due to lacking pre-reqs because they couldn't find a way to show what you need to build them, which is kind of odd.

So yes, right now it only shows what you can build. It is confusing and hopefully they figure out how to show you what you need to be able to build each improvement.