I didn't find the Terrace Galleries offensive to look at. They had a reserved, aloofness to them - set back from Bloor beyond a little patch of green and some railings -that oozed "classy". That reno created some nice skylights, for instance ( some of which remain, above the Gallery of Chinese Architecture ), that brought in a subtle play of natural light ( as Gehry has done at the AGO ). But they were a departure from a sensible long-range plan to expand the pedestrian promenade throughout the building, and make busy Bloor Street the entrance once more, so it made sense to remove them as part of a larger development plan to revive the institution. As a bonus, a couple of galleries were added on a new fourth floor and a fancy restaurant on a new fifth floor.
The Crystal galleries are large, versatile spaces, not custom designed for what goes in them. Gehry's AGO addition is rather the opposite. Many of the large artworks in the AGO's contemporary galleries wouldn't fit into the smaller galleries, some with low ceilings, that house the Thomson collection; the small Krieghoffs and Milnes would get lost if moved to the contemporary galleries; the Henry Moore and Frum Collections are also housed in custom-designed spaces. Whereas the Crystal is a single wing, the AGO addition is nuanced, with several distinctive components ( the south wing and the Dundas wing could be from different buildings they're so unmatched ) and that reflects the different needs of the two institutions - quite successfully, I think.
It seems that whenever people don't like a building they call it a "car dealership". Both the Crystal and the Four Seasons Centre - buildings about as dissimilar as you can get - have been called that recently on this forum. But there are some well designed car dealerships in Toronto ( two near me, facing onto the Don valley, for instance ... ) that appear to be every bit as practical - as design solutions - as our various new cultural buildings have turned out to be.