Do settling next to river or sea matters ?

Settling next to a river allows you to build a pier on the river tile which gives you some bonuses to food and gildar, as you upgrade it.

Settling next to the sea does not give any bonuses as far as I know.

I have been wondering why not set it up that docks can be used on beaches thereby making ocean side settlements more attractive and advantageous.  Would you just need to change the restricted tiles to include beach right?

Setting up next to the sea does not add any buildings. It does, however, give you a safe area, fewer approaches to your settlement.Well, at least until the Krackens migrate over...

Can you build to a river and build the pier? Similarly can you build to a forest and build the mill?

In the scenario, yes (although some say that this ceases to be possible after a save and reload). In the sandbox game, no.

Parrotmath created a mod which will permit you to do so in the sandbox game, however. Look for the snaking mod here:

https://forums.elementalgame.com/444851/page/1/

  Settling next to a river, where there is also forest around, can result is you not being able to build anything other than a pier - EVER.  You can't clear forest either (for some reason).  You're just stuck.    I've played thru a game for over 100 turns and never could build anything but pier and upgrades to that (dock and such), except for whatever resources eventually came within my city limits (iron, crystal). 

that sounds like a bug. arent't you the guy who also had the poblem with the one per faction and world achievement buildings? maybe something is wrong with your installation? you could try verifying file integrity (steam function) or just plain uninstalling and re-installing the game.