Hi,
I've done that in a previous project. My Agents move freely and do things like they want like a SIM... they walk and stop or go straight to a destination. You need to do a fsm manager inside your agent GO or prefeb with some send/receive global call from objects. I can't tell you how precisely to do it but it's faisable. Your destination objects need to send a call that they are free and agents must receive that call to go to them. Good luck