Sam Gyimah was this month effectively expelled from the Conservatives along with 20 of his colleagues.