Do a Google search on "UAV Collision Avoidance". UAV is what you might casually call a drone. There is a ton of active research.

Visual avoidance is promising because it has low weight and power requirements and as you point out the target to avoid can be completely passive.

Visual collision avoidance is part of the very active research in "Machine Vision" which is what you might call "Robotic Vision", the sort of thing that Google's self-driving cars exploit.