The president is refusing to back down from his comments that there were 'very fine people' on both sides.