They don't actually pull you into their square. Instead, if your army had a destination set, they automatically move towards it after being attacked. I consider this to be a bug. If you don't leave any armies with destinations at the end of your turn, you won't see this happen.