When the import start from the asset store it will overwrite all the actions and dll's automagically?
Yes. Just import all the files.
Backup the Actions folder if you've edited any actions, or made/downloaded custom actions.
Last time i did erase all the PM folder, and then i start running into problems..
Erasing the PM folder is generally not a good idea unless you're trying to completely remove Playmaker from a project.
Importing files on top of the current installations should work.
Occasionally Unity throws errors when updating scripts/dlls, but restarting Unity should fix them. If the PlayMakerEditor.dll and PlayMaker.dll somehow end up at different versions it can cause more serious problems, but you should be able to manually replace the dlls in explorer/finder if this happens. This should be extremely rare (e.g. if Unity crashes during an import).
Is it safe anyway to upgrade an existing project?
That's the idea, but it's *always* a good idea to backup projects before upgrading in case something goes wrong!!
Problems are more likely when upgrading from a much older version, or from a beta version (since sometimes implementations change over the course of a beta).
I would also recommend against updating too much at once - e.g., update to Unity 3.5 first, check the project, then update to Playmaker 1.3.x. Otherwise it can be hard to track down the actual cause of any problems.
And you should consider every update carefully - if you're about to ship your product you probably don't want to change too much!
If you really want to take a look at what's changed, use a diff program to compare the new Playmaker install against the previous install. There are many good free diff programs online.
The best option of all is to use version control. That gives you a backup and diff as part of your regular workflow. Personally I can't work without it!!