Crowd management lifecycle

System design & procurement

Operator training & pilots

Live deployment & monitoring

Post-event review & audit

The nine design principles (derived from systematic review of 46 studies)

Explainability

AI outputs must be derivable in a way understandable to operators

TransparencyInterpretabilityJustice

Traceability

Outputs must be sourced to specific inputs or actions

ResponsibilityInterpretability

Rights protection

Systems must not infringe fundamental rights of individuals

EqualityPrivacyFreedom

P4 · Priority

Human primacy

AI must not overtake human decision-making — operators retain final authority

AutonomyInformed consentFreedom

Reliability

Systems must perform in an intentional and expected manner

ControlReliabilityTrust

Environmental integrity

Systems must not harm the physical environment

Env. protectionCommon good

Human flourishing

AI use must promote societal goods and well-being

BeneficenceHuman welfareCommon good

P8 · Priority

Physical safety

Outputs must not cause or risk physical harm to people in a crowd

Non-maleficenceSecuritySafety

P9 · Priority

Psychological safety

Outputs must not risk psychological or dignity harm to persons

Non-maleficenceHuman dignityBeneficence

Implementation guidelines for group decision-making

Principle	Value basis & rationale	Design rule	Implementation hints	Group decision implications
Safety & reliability first	Safety and non-maleficence; systems must not cause physical or psychological harm and should perform reliably.	Incorporate risk management across the AI lifecycle; implement conservative defaults and manual override.	Stress-test on crisis scenarios; include redundant sensors; monitor health in real time; integrate cybersecurity controls.	Define thresholds for human override; allocate responsibility to a safety officer; ensure consensus on emergency actions.
Explainability & transparency	Transparency and explainability support accountability and fairness; they build public trust.	All recommendations must be accompanied by human-understandable rationales, uncertainty estimates and audit trails.	Use interpretable models; provide post-hoc explanations; expose input features and confidence scores; maintain accessible logs.	Ensure group members understand AI outputs; require explanations before acting; document dissent and overrides.
Human oversight & agency	Respect for autonomy and informed consent requires that humans remain in control; EU law mandates oversight.	Design AI as decision support, not a decision maker. Always allow human operators to override automated actions.	Build interfaces highlighting override options; implement human-in-the-loop workflows; incorporate “stop” buttons.	Establish protocols for override; distribute authority across the team; ensure accountability when deferring to AI.
Fairness & non-discrimination	Fairness and equality demand avoiding bias and treating individuals equitably.	Use representative data; test for disparate impact; calibrate models to minimise false positives/negatives.	Implement fairness metrics; retrain regularly; employ bias mitigation techniques; involve diverse stakeholders.	Communicate fairness considerations; ensure decision-makers understand bias; create paths for appeals.
Privacy & data governance	Privacy protects individual rights and fosters trust; crowd-safety AI processes sensitive data.	Collect only necessary data; apply data minimisation and purpose limitation; adopt privacy-enhancing technologies.	Use anonymisation or differential privacy; restrict access; delete data after use; publish clear notices.	Inform decision-makers about data provenance; avoid decisions on personal data; involve privacy officers.
Accountability & responsibility	Assigning responsibility ensures harms can be redressed and obligations are met.	Define clear accountability across the AI supply chain and within crisis teams.	Use contracts requiring documentation and model cards; assign risk assessors; maintain event logs.	Clarify accountability when acting on AI outputs; record decisions; establish review boards for incidents.
Common good & sustainability	Interventions should promote the common good, protect the environment and enhance societal welfare.	Evaluate societal impacts and long-term consequences; align with sustainability and community values.	Include environmental impact assessments; engage communities; design adaptable systems; gather feedback.	Encourage decisions considering collective welfare; integrate ethicists and community representatives.

Known gap: AI ethics literature consistently identifies values but rarely implements post-deployment verification. AI-Compass explicitly targets this gap: these principles must be tested empirically (D2) and audited continuously — not declared once at design time.

Priority principles for crowd safety deployments

Context-specific priorities (public security & policing value fingerprint)

Human primacy in real-time crisis decisions (P4)

AI tools present recommendations, not directives. Override mechanisms must be visible and operable under operational stress. Operators must be trained against over-reliance — time pressure is when AI deference risk is highest.

Explainability at the point of use (P1 + P2)

Speed cannot sacrifice explainability. Every AI alert must include a concise rationale legible to a trained operator in the field. Post-event full traceability from inputs to outputs is required for accountability review.

Fairness and non-discrimination (P3)

Crowd-monitoring AI carries elevated risk of encoding historical biases in demographic risk scoring. Equality must be empirically tested during development and periodically audited in deployment. Legal obligation under EU AI Act for high-risk public safety AI.

Applying VSD: tripartite methodology across the lifecycle

Conceptual

Identify direct and indirect stakeholders. Map value tensions (e.g. safety vs. privacy in surveillance). Apply normative frameworks to clarify what is at stake.

Stage 1: Design & procurement

Empirical

Engage operators, affected communities, and legal experts. Use value scenarios and participatory design to surface context-specific concerns.

Stage 2: Training & pilots

Technical

Examine how system features support or inhibit the nine principles. Design for explainability, traceability, and human override mechanisms.

Stages 3 + 4: Deployment & review

Regulatory alignment

EU AI Act (2024)

Art. 13–14 | Transparency & explainability (P1, P2)

Art. 14 | Human oversight for high-risk AI (P4)

Art. 10 | Bias testing in training data (P3)

Art. 15 | Accuracy, robustness & security (P5, P8)

Art. 17 | Quality management & logging (P2)

Value-Sensitive Design Principlesfor Crowd Safety AI

Value-Sensitive Design Principles
for Crowd Safety AI