Large groups of people have always fallen through the cracks among racial categories — often by design.