Former business secretary departs from Lib Dem position to argue that public opinion is behind post-Brexit immigration controls