Someone could do arbitrage on this by creating a pipeline of "high quality" tenants that they can market to landlords as less likely to skip out on rent or become problem tenants: Ivy-League graduates, employees of well-known companies, etc.
Not really. The property quoted in the article was purchased in 1974 and California brought in strict limits on property tax increases in 1978.
So the property taxes are probably trivial. The house acts as an investment with high returns and favorable tax status. They can sell it or take out a mortgage if they ever need money.
Given that burning down the property to avoid a long eviction struggle isn't unheard of, the owner is perfectly rational to decide not to rent it out.
Highly unlikely that 10,000 landlords with an extraordinary desire to maximize their return are being "irrational". If you rent out the property in San Francisco, you effectively lose control of what you can do with it. By not renting it out, you maintain control, and you can rent to family, make use of it yourself. I'm guessing that airbnb also is an attractive option now to get around the price-control laws.
<cynical thought>
But I suspect renting it to family/AirBnB - and not declaring the income - ends up looking very much like "leaving the building vacant" on official records...
If you rent out property for residential use anywhere in the US, you do lose control of what you can do with it, outside the bounds of your leasing agreement and local law regarding the topic.
Most jurisdictions recognize the need for a person to have a safe, personal, private space in order to have a happy and productive life. It is a recognition of this fact that causes most places to agree that a man's home is his castle, even if he is renting his home from another.