Essentially what you are arguing is that historical preservation limits the supply of new housing, and that this limited supply coupled with high demand pushes up housing prices and reducing housing affordability. However, I think the role of historical preservation in this is overstated. Take the case of the condo boom in Toronto over the last decade. Certainly investment in new housing stock in downtown is at or near an all time high, but does this mean that housing in downtown Toronto is becoming more affordable? Hulchanski's "Three Cities" would appear to indicate otherwise. Perhaps investment itself is a cause of increasing affordability problems.
The notion that private market forces can produce "affordable" housing I think only works with a very limited notion of affordability - measuring affordability as the price of a dwelling relative to the price of another dwelling). A more holistic definition of affordability is one that measures the price of a dwelling relative to income. Under this definition, it is the case the private market forces cannot produce affordable housing as it is simply uneconomical.
Well, from a simplistic assessment, what you're saying seems to make sense. But I think the problem is a little bit more deep than that. Investment typically leads to falling prices. In almost all industries. Agriculture, electronics, textiles, etc. More investment means more productive capacity, economies of scale and all that.
Housing is no different, except for one thing: leverage.
When I can go and get a pre-approved mortgage for $750,000 and you can too, there's little standing in the way between you and I bidding up a $620,000 property to $700,000 if we both want it. But that's no investment. That's debt sloshing around the market, which is a result of loose monetary policy on the part of the government -- or, more simply: money printing.
Frshly created money through loose interest rates, go chasing assets. And they all too often end up chasing speculative instruments: real estate, stocks, commodities, etc.
So, investment does not lead to a drop in affordability: price inflation through debt monetization is.
Economic history is on my side, too: from the period of 1830 to 1971, house prices tracked wage trends. It was only in the post Bretton-Woods era of massive credit expansion, have we seen a sustained trend of housing price to income divergence.