Normally, I would concede that you are right, but here's the thing; Every time a service similar to this was suggested on this forum, it was met with derision. It was always said that a train doesn't need to go everywhere and that a bus could do the job. Why the logical inconsistency?
Different people have different opinions. Personally I am on the side that we need to restore passenger service on as many lines as possible throughout Canada, because we run the risk of boxing ourselves into a corner in some cases by building up bus services. I also oppose highway expansion in general, so I'm not in favour of a model that would be highway-based, especially when in so much of rural Canada, there are still a number of barely-used rail corridors which provide unparalleled access to small downtowns. Rail service direct to rural town centres would help correct the "spreading out" many towns have suffered from in the 20th century and be a useful trigger for downtown revitalization, improvement of other modes of transport, and local business potential.
I think it's a mistake to make these comparisons because service abandonments happened for many different reasons, and I think the campaign for service restoration is in any case a unique one -- for example, the NEORN and First Nations proposals around restoring Algoma passenger rail which would emphasize regional connectivity, something a lot of North American transit fans struggle with conceptually I think, because a regional network model doesn't follow the dynamic of suburban radial lines feeding into a metropolitan core. I think a lot of transit planning fans in North America have an instinctive preference for that model because it's what is familiar to them, especially with GO and Toronto, but it's not really applicable to a lot of Canada except in limited cases like Toronto, Montreal, and maybe cities like London. This stuff needs to serve the area and take local needs into account, such as the recognition that the Northlander was poorly marketed in the context of cottage country around Bracebridge and Gravenhurst, as opposed to the way the Sudbury-White River train has been extensively marketed as an outdoor tripper/fisher/hunter train to access areas that aren't connected to the road system (implication, you can go fishing in areas that won't be mobbed by people and fished or hunted out). Similarly our population dynamics are shifting toward formerly suburban seniors selling up and becoming exurban and with the likelihood they'll be able to continue to drive dropping every year, there need to be solutions for these people. Similarly with COVID-19 we've seen the strong desire for people to get out of cities in the summer and travel to Niagara, Georgian Bay, the Bruce Peninsula, etc. along with concerns from local municipalities in these areas about being overrun with illegally parked cars. These areas could benefit from a pattern of high service levels in the summer, lower service levels in the winter, and depending on the actual place, a mix of bus and train service (or very short trains) to adjust for ridership. There is no way these trains (or buses) would be terminating at Union.
I think especially with the building up of new suburban rail hubs in the GTA, plans like RER/GO Expansion, and even plans for bespoke branchline services like Cambridge-Guelph, we're looking at a future where the hurdle for rail service is a lot lower, and if battery EMUs become widespread, the potential to reactivate rural branchlines for passenger service is strong, and have those services connect at the nearest major intercity hub where a transfer to Via would be possible. I think if anything, UT is a bit too stuck in the idea that every train needs to terminate at a major metropolitan centre, but if we are going to grow our rail system then in the future, we can start looking at how railway stations in 2nd or 3rd tier Ontario cities can become hubs of their own. Especially with the Corridor, we need to reinforce that "spine" as much as possible, but at the same time, a train to Gaspe could easily serve a similar "weekender" role, and probably did before its cancellation. What is the lesson we are teaching, say, university students in Montreal who enjoy the outdoors but don't own a car and ride the subway every day, if they need a car to travel to Gaspe? Our rail network has shrunk and lost so many branchline services over the past 40 years and the lesson can't be that we need to reduce it further around a single urbanized mega-corridor between Toronto and Montreal, or we'll just hasten the already-near terminal infliction of cars and highways on rural Canada, a highway system that kills quite a lot of people one way or another. We'll also teach younger people that they have to choose between living inexpensively and being car-free, and that cars are for people who can find work and afford to live in Toronto, not people who want to stay in their hometown, work in industries that are rural by nature. We'll teach people that travelling requires a car (we really already have, especially with Greyhound gone).
I think the issue with operating a reduced passenger rail system in a Canadian environment is the rule of thumb that fringe industries are expensive to run. You have fewer bidders on contracts for things like track maintenance, it's harder to shop around for the best price or design of cars and you're stuck with a few manufacturers (look at the disaster with Bombardier). You don't have a solid core of professionals in the passenger rail industry and end up bringing them in from other countries or other industries, and in both cases that can be expensive and a bad fit. It's harder to find trades workers experienced in the industry. You have very few opportunities for economies of scale. For example, if Canada was producing passenger rail cars near-continuously and there were a number of competing manufacturers, I imagine things would be a lot more cost effective, and plants like Thunder Bay wouldn't be going through endless cyclical layoffs that decimate their ability to fulfill large contracts (e.g. TTC) on time when they do come around. We need to scale up our passenger rail industry to meet our population growth and try to bring more people onto transit, and trying the old tactic of substituting buses for rail service, seen pretty much everywhere as an inferior option and a sign of degrading service, is not the way.