Introduction
Research evaluation systems have undergone significant methodological evolution. From improving data coverage and refining indicators to addressing issues of weighting, transparency, and contextual interpretation, substantial effort has been invested in making evaluation more robust.
More recently, the limitations of composite scores have been exposed, and alternative models—such as multidimensional profiles—have been proposed to better represent the complexity of research performance.
Yet a fundamental question remains insufficiently addressed:
What happens after evaluation?
Metrics are often treated as endpoints—final outputs that directly inform decisions. Funding allocations, institutional rankings, hiring outcomes, and policy directions are frequently derived from evaluation results with minimal additional interpretation.
This conflation of evaluation and decision-making introduces a critical risk.
Evaluation describes.
Decision selects.
When the distinction between the two collapses, metrics begin to substitute judgment rather than inform it.
1. The Misuse of Metrics
In many research systems, metrics are used not as analytical tools, but as decision mechanisms.
This misuse takes several forms:
treating scores as definitive indicators of quality
applying thresholds without contextual justification
relying on rankings as proxies for complex assessments
In such cases, the evaluation process is effectively reduced to a rule:
higher score → better outcome.
This transformation is appealing because it simplifies decision-making. However, it also removes the need for interpretation, justification, and accountability.
Metrics, in this context, become instruments of automation rather than tools of analysis.
2. Evaluation Is Not Decision
To understand the limitations of metric-driven decisions, it is necessary to distinguish clearly between evaluation and decision-making.
Evaluation
descriptive in nature
multidimensional
concerned with analysis and representation
Decision
normative in nature
reductive by necessity
concerned with selection and action
Evaluation produces structured information.
Decision requires judgment under constraints.
No evaluation system, regardless of its sophistication, can resolve the trade-offs inherent in decision-making. These trade-offs must be made explicitly.
When metrics are used to bypass this process, decisions become opaque and unaccountable.
3. The Risks of Direct Translation
When evaluation outputs are directly translated into decisions, several structural risks emerge.
Loss of Context
Profiles and indicators that were designed to preserve context are reduced to simplified criteria.
Hidden Trade-offs
Decisions inevitably involve prioritizing certain dimensions over others. When metrics are used mechanically, these trade-offs remain implicit.
False Objectivity
Decisions appear objective because they are based on numbers, even though the interpretation of those numbers is not transparent.
Accountability Gaps
Responsibility shifts from decision-makers to metrics, making it difficult to question or challenge outcomes.
In such systems, metrics do not support decisions—they conceal them.
4. What Responsible Decision Systems Require
If metrics are to play a constructive role in policy and institutional decision-making, they must be embedded within structured decision frameworks.
A responsible decision system requires:
Structured Interpretation
Evaluation outputs must be interpreted, not applied. Profiles should be read across dimensions, not reduced to single criteria.
Context Integration
Decisions must account for disciplinary, institutional, and temporal contexts that metrics alone cannot capture.
Explicit Trade-offs
Where priorities conflict, decision-makers must articulate which dimensions are being emphasized and why.
Documented Reasoning
Decisions should be accompanied by explanations that connect evaluation inputs to final outcomes.
Accountability Mechanisms
Decision-makers—not metrics—must remain responsible for the choices made.
5. The Role of Metrics in Decision-Making
Metrics are indispensable to research evaluation. They provide structure, consistency, and comparability.
However, their role must be properly defined.
Metrics should:
inform decisions by organizing relevant information
support interpretation by highlighting patterns and relationships
enable comparison within appropriate contexts
Metrics should not:
determine outcomes autonomously
replace human judgment
obscure the reasoning behind decisions
The distinction is critical.
A metric can guide a decision.
It should never become one.
6. From Profiles to Decisions: A Layered Model
To bridge evaluation and decision-making, it is useful to conceptualize the process as a set of structured layers:
Layer 1: Data
Raw research outputs and metadata.
Layer 2: Metrics
Constructed indicators derived from data.
Layer 3: Profiles
Multidimensional representations of performance.
Layer 4: Interpretation
Analytical reading of profiles within context.
Layer 5: Decision
Final selection based on explicit criteria and priorities.
Each layer serves a distinct function.
Collapsing these layers leads to methodological and governance failures.
Maintaining separation ensures that decisions remain informed, interpretable, and accountable.
7. Implications for Policy and Institutions
Reframing the relationship between evaluation and decision-making has significant implications.
For policymakers
avoid using rankings or thresholds as standalone decision tools
require interpretive frameworks alongside evaluation outputs
For funding bodies
incorporate qualitative and contextual considerations in allocation processes
document decision rationales transparently
For institutions
resist over-reliance on external metrics for internal decisions
develop internal capacities for profile-based interpretation
This shift does not complicate decision-making unnecessarily.
It restores its legitimacy.
Conclusion
Advances in research evaluation have improved how performance is measured, represented, and interpreted. However, these advances risk being undermined if evaluation outputs are used uncritically in decision-making.
Metrics are powerful tools—but they are not decision-makers.
The integrity of research evaluation depends not only on how metrics are constructed, but on how they are used. Responsible systems recognize that evaluation informs decisions; it does not replace them.
The future of research governance lies in maintaining this distinction.