I think improvement should be in order.
Sure many will claim it is beautiful as it is and things should remain intact. That's just backward thinking - these people will prefer everything to remain unchanged. They would prefer Toronto to be all Victorian houses outside the CDB. Toronto islands have such great potential to be a more accessible urban park. Its current status is by no means satisfactory.
The islands first needs to be more accessible. Even living downtown, south of Queen st, I find it too troublesome to get to the islands. Central Park in New York, Hyde Park in London, or Luxemburg park in Paris are all extremely accesible. The islands being islands is partly the issue, but not quite. Just to get to Bay/Queen's Quay is enough trouble, and you need to pass a lot of unpleasant things. Most would need to take a subway to union, and transfer to a streetcar underground...
Second, the island needs to offer more value. Right now, it doesn't live up to its potential, not even 50% of it. Leaving it as natural as it is is not the way to go. A lot of upkeep and beautification work needs to be done. Much of the land remains unused (I don't mean building KFC's or condos of course) and needs skillful design. We are blessed with such a big and beautiful island right to the edge of downtown, yet it only functions as a getway from the city and its cars, noise and buildings. Simply not enough. We should expect more.
As to the islands residents, they would always selfishly prefer noboy ever steps into the island, or no development is ever done so that they can have it all to themselves. Forget what they want. If they want pure quietness, why not move out of the GTA, there is plenty out there.