Any federal exercise of its residual power under Section 91 would be heavily litigated by the provinces. Keep in mind that the section reads:
It shall be lawful for the Queen, by and with the Advice and Consent of the Senate and House of Commons, to make Laws for the Peace, Order, and good...