Well part of it is certainly the fact that the city insists on building subways the most expensive way possible. There are also no doubt other bureaucratic reasons and also the English speaking world in general can no longer build public transit for a reasonable cost anymore. That's not just me saying it either there was an actual study done that Canada, Australia, New Zealand, and the US have some of the highest transit construction costs on earth for some reason. If I had to narrow down the big culprits though, I would say its: choosing the most expensive construction methods (e.g deep bores), to many consultants, to many regulations, lack of institutionalized knowledge, an unwillingness to look at examples from abroad ("we are special snowflakes and things that work somewhere else won't work here", and "every solution needs to be invented here"), and a genuine fear of upsetting a vocal minority.
To me bringing down construction costs will require loosening regulations, ending the consultant grift, building institutional knowledge (both by eliminating consultants and doing more work in house, looking at you Metrolinx), looking at other countries to see how they solved problems (many solutions they use will work here if we try hard enough), using cheaper alignments and construction methods (e.g cut & cover, at-grade, and elevated), and lastly a little iron-fisted rule to eliminate opposition and end the fear of angering a small number of home/business owners. That latter won't happen if politicians view the future only as far as the next election and fear loosing more then seeing the bigger long-term picture. Short-term thinking won't get us anywhere and that's why the bulk of Toronto transit construction occurred during a period when the incumbent party (PC's) had little fear of loosing the next election (until they did). Lastly we need to take a page out of LA's play book and stop building lines in a one-off manner. We need to have a comprehensive 30 year plan that everyone agrees on and has political support at the municipal and provincial levels. This plan isn't just a list of nice to haves but an actual costed out plan that we all agree will be built out in its entirety instead of in a piecemeal fashion. If we have to we should follow California and just legislate the plan making its construction law.