At the time of the referendum on the UK’s membership of the EU in 2016, the big issue was immigration.