the creation of "people of color" as an identity category basically serves to unite the interests of people who are treated insensitively by whites, which is, to be fair, a fairly enormous swath of people.
As long as the overwhelming majority of people in the category have experienced the effects of residential segregation, the category remains, I think, relatively stable and useful. If that were to change, I think the category would probably fall apart.