The African Union on Friday called on Britain to withdraw from the Chagos Islands and end its "continued colonial administration" there after a U.N. deadline for it to do so expired.