Abstract

Large language models (LLMs) have great potential for social benefit, but their general-purpose capabilities have raised pressing questions about bias and fairness. Researchers have documented significant disparities in model output when different demographics are specified, but it remains unclear how more systematic fairness metrics—those developed in technical frameworks such as group fairness and fair representations—can be applied. In this position paper, we analyze each framework and find inherent challenges that make the development of a generally fair LLM intractable. We show that each framework either does not logically extend to the general-purpose LLM context or is infeasible in practice, primarily due to the large amounts of unstructured data and the many potential combinations of human populations, use cases, and sensitive attributes. These inherent challenges would persist even if empirical roadblocks were overcome, but there are still promising practical directions, particularly the development of context-specific evaluations, standards for the responsibility of LLM developers, and methods for iterative and participatory evaluation.