Introduction
For decades, research evaluation has relied on rankings as its primary output. Institutions, researchers, and policymakers have become accustomed to interpreting performance through ordered lists, where position is assumed to reflect quality.
This model persists because it is simple. Rankings compress complex realities into a single comparative structure that is easy to communicate and operationalize.
Yet this simplicity comes at a cost.
As demonstrated in previous analyses, the aggregation of multiple indicators into composite scores obscures the multidimensional nature of research. Rankings, built upon these scores, inherit and amplify these limitations. They reduce diversity to position, context to uniformity, and interpretation to ordinal comparison.
If research is inherently multidimensional, then its evaluation cannot be meaningfully represented through a single axis.
This requires a shift in paradigm—from rankings to profiles.
1. The Structural Limits of Rankings
Rankings are designed to order entities along a single dimension. This requires that all relevant aspects of research performance be translated into a unified scale.
In practice, this translation involves:
aggregating heterogeneous indicators
normalizing across disciplines and contexts
assigning weights to different dimensions
These steps do not eliminate complexity—they suppress it.
As a result, rankings produce outputs that are:
reductive, by collapsing multiple dimensions into one
non-diagnostic, by failing to explain why positions differ
context-insensitive, by treating diverse entities as comparable
Rankings answer the question: Who is ahead?
They do not answer: In what way? and why?
2. The Consequences of Ranking-Centered Evaluation
The dominance of rankings shapes not only how research is evaluated, but also how it is conducted.
When performance is defined by position:
institutions optimize for rank rather than substance
researchers prioritize indicators that influence scores
diversity of research strategies is reduced
This leads to systemic effects:
homogenization of research outputs
reinforcement of existing hierarchies
marginalization of context-specific contributions
In this sense, rankings are not neutral instruments.
They actively structure behavior.
3. What Profiles Change
Profiles offer an alternative representation of research performance. Instead of compressing multiple indicators into a single value, they preserve their structure.
A profile is not a score.
It is a configuration of dimensions.
By presenting indicators side by side, profiles allow evaluation to become interpretive rather than reductive.
They enable users to ask:
What are the strengths across different dimensions?
Where are the limitations?
How do patterns vary across contexts?
This transforms evaluation from comparison to understanding.
4. The Structure of a Research Profile
A robust evaluation profile is organized across distinct, analytically meaningful dimensions. While specific frameworks may vary, a structured profile typically includes:
Productivity
Output volume, publication patterns, and temporal distribution.
Impact
Citation-based indicators, field-normalized influence, and knowledge diffusion.
Collaboration
Co-authorship structures, institutional networks, and international engagement.
Integrity Signals
Indicators related to research practices, transparency, and reproducibility.
Contextualization
Field normalization, career stage, institutional environment, and resource conditions.
Each dimension is presented independently, yet interpreted collectively.
This structure preserves complexity without sacrificing clarity.
5. Interpretation Over Calculation
The shift from rankings to profiles is not only technical—it is epistemological.
Ranking-based systems prioritize calculation:
they seek definitive outputs.
Profile-based systems prioritize interpretation:
they support informed judgment.
In a profile framework:
indicators are not endpoints, but inputs
evaluation is not automated, but guided
conclusions are not imposed, but derived
This aligns evaluation with the nature of research itself—complex, contextual, and multidimensional.
6. Implications for Evaluation Systems
Adopting profiles requires changes at both system and institutional levels.
For evaluation platforms:
replace composite scores with structured indicator sets
provide visualization tools for multidimensional interpretation
ensure transparency in indicator construction
For policymakers:
move away from rank-based decision criteria
incorporate profile-based analysis in funding and assessment
recognize diversity in research contributions
For institutions:
evaluate performance across dimensions rather than positions
align incentives with balanced research development
avoid over-reliance on external rankings
This transition does not eliminate comparison.
It reframes it within a more valid analytical structure.
Conclusion
Rankings have dominated research evaluation because they offer simplicity. However, this simplicity is achieved through the suppression of structure, context, and diversity.
Profiles provide a more appropriate alternative. By preserving multiple dimensions of performance, they enable evaluation to move from numerical compression to structured understanding.
The future of research evaluation lies not in refining rankings, but in replacing them.
Evaluation should not ask where an entity stands.
It should ask what it represents.