No. The point of a non-aggression pact is that you can't cancel it and declare war. What would be the point if you could?
The point would be have some assurance that you don't need to guard a border for a while. Germany had a non-aggression pact with the Soviet Union until July of 1941 (when they invaded).
That said, I see what you're saying in game-play terms; it would seem a bit trifling. A way to improve this would be to allow one to break a non-aggression pact at the cost of influence and lowering your relationship with everyone, because other factions see you as less trustworthy.