The longest serving special adviser in the Northern Ireland Office has said there would be "a comfortable majority for the Union" if a border poll was held tomorrow.