Chris Patten joined former UK Prime Minister Sir John Major in supporting three independent candidates who had the Tory whip removed for rebelling on Brexit.