How about this: We buy one or more of the Ontario Place "pods" and convert it to residential. You could put 10 or 12 units in each pod with a shared rooftop terrace. Great views, on the water. Park the boat underneath.
Personally, the Cabin "D" next to the John St. Roundhouse would suit me fine as a weekend retreat. Move it back to trackside (maybe up near Strachan) and I could use it as a hobby house, or a place to watch trains go by.
Any of the interlocking buildings (John St., Cherry St. or Scott St.) would be fine as well. John St. suffers a bit under the Skydome pedestrian bridge, but Cherry and Scott have nice bay windows. The trains can switch themselves in and out of Union.