St Louis got the 1904 games because they were hosting the World's Fair at the same time. The games were spread out over 6 months, so it wasn't your typical two week Olympics that we've come to know.
Vancouver isn't a looming disaster. In fact, other than the issues over the athlete's village (and those are being rectified) things are going well, and they completed construction on every venue already. A lot of venues are open and already used. It will be a very successful Olympics, there's nothing to suggest otherwise really. Even the issues of poverty and homelessness have been put to the forefront and the recent BC Budget is addressing this issue. Other good things like transit improvements and the improvements to the Sea-to-Sky Highway will leave a really strong legacy.
The fears over budgets and getting into debt are easy to throw out there because other Olympics have had trouble managing their budgets. Over 7 years any multi-million/billion dollar project, whether it's an Olympics or a building, is going to exceed the budget set. It's just the reality of fluctuating economies. Hell, look at oil prices over the last couple years. They rose by 300% and then fell by the same number. How does a preliminary budget project that? And that's just oil. Steel and other production materials fluctuate as well.
The summer Olympics in particular can really change the direction of a city, and they tend to benefit second tier cities moreso than large ones. Cities like London and Atlanta are already large economic engines. They really didn't need an Olympics to validate themselves. However, cities like Barcelona, Beijing and Sydney have really benefited from the international attention. Also, once the infrastructure is in place, the possibility of hosting other world-class events like World Championships are much easier to win. I'm not talking about hosting a football World Cup (which will never happen in Canada), but just World Championships in one of the many sports that are found in the Olympics.
Also, some of you make it sound like after the games are gone, the infrastructure will sit unused forever. Elite athlete training centres are very important and contribute not just to the quality of athletes (which has an economic benefit as well) but to things like research and development or coaching and training which can result in better programs for youths. A Central Canada Centre of Excellence is needed especially considering the size of Ontario's economy and population.
I think if done properly an Olympics can really be a catalyst for creating a stronger image of Toronto. By then our Waterfront will be done (it's not an "either or" situation, nor are we waiting for an Olympics to change things any longer) and we'll have some other important new pieces of infrastructure like Transit City and just maybe the DRL. An Olympics really would be a great way of showing the world the new Toronto that we've built. It's worked for other cities of comparable stature so I don't see why Toronto would align itself with the Atlantas of the Olympic world.