Yes, indeed the player has a 2D collider.
That actually got me thinking and I found out it is a mistake from my end. One of the objects sending the ray cast had a too big collider that it covered the ray cast. Which brings a question, how can I put a collider on a raycaster object without interferring with the raycaster?
However, there is another problem that surfaced. The detection doesn't still happen right away; it happens after moving around the object for a while.
So for example, I have raycaster and target. If there are no objects between raycaster and target, the raycaster doesn't detect the target. Even if I left it there for a few minutes. However if I move the target around, the raycaster picks it up after a while (not right away either).
If I put an object between the target and the raycaster it still doesn't detect the obstacle at all. Even though I have in the setup two cases where one detects the obstacle and puts in the debug window that it sees the target and the other puts out that it sees the obstacle.
Any idea how to resolve this? It is pretty weird really.