British voters across all party allegiances think there should be an annual cap on the number of migrant arrivals after a Brexit.