Those are rare circumstances. Self driving cars will be deployed when they are safely able to handle 99.9% of driving and they have ways to call home for instructions from remote monitors when they encounter a rare situation,who can then click on an interface to specify where to go. That's how Waymo does it currently.