Imagine filling in an official form only to discover that the system refuses your real name, your real address, or your actual legal status — not because the law says so, but because the software won’t let you proceed.
Digital systems are supposed to help us comply with the law — but what if they start defining it? As governments and companies increasingly rely on software to enforce regulations, we face a growing problem: Shadow Compliance. These are rules and restrictions enforced by software-tools, APIs, or platforms through their implementation — not by legislation, democratic process, or public debate.
When Software Makes The Law
Compliance is no longer just a matter of reading the law. It's a matter of what software-tools allow. From tax filing portals and procurement systems to content moderation algorithms, software increasingly decides what is permissible — or not. These decisions are often invisible, unaccountable, and shaped by technical, not legal, logic.
Examples of Shadow Compliance
- ✘ A form won’t submit unless you tick a GDPR consent box — even if that consent isn't legally required in your case.
- ✘ An e-invoicing API enforces tax fields based on outdated or overly strict interpretations.
- ✘ A platform automatically deletes content flagged by AI as “potentially harmful” — without context or recourse.
- ✘ A procurement tool excludes bidders because their documents don’t match a predefined (but non-mandatory) format.
Why It Happens
- Technical overreach: Developers encode interpretations of rules to ensure consistency — but lock out nuance.
- Risk aversion: Companies over-implement restrictions to avoid liability — not necessarily to follow the law accurately.
- Lack of legal context: Software teams operate without legal guidance, leading to rigid or incorrect implementations.
- Automation pressure: Compliance processes are scaled via software — sacrificing flexibility and judgment.
Policy reality 2025: Many aspects of shadow compliance do not arise from (misinterpreted) laws, but from procurement guidelines, default settings, and tool validation rules. Even though AI regulations and platform laws call for transparency, risk management, and human oversight, the actual implementation decisions often remain invisible. A remedy would be mandatory documentation from requirement to concrete implementation (requirement, solution approach, implementation including validation steps; alignment of the implementation with the intended purpose of the respective legal norm), traceable change logs, and easily accessible complaint and override channels directly within the system.
The Hidden Risks
- ⚠ Citizens and businesses may be denied services or rights without legal basis.
- ⚠ Organizations may assume they’re “in compliance” while enforcing non-binding or even false interpretations.
- ⚠ Accountability becomes murky: Who is responsible for the rule — the regulator, the developer, or the vendor?
„When rules are enforced by machines, they stop being questioned — even when they’re wrong or against the law.“
Real World Case – University Systems and the Name Trap
In public digital systems, we often confuse what’s legally required with what’s technically expected. When forms reject real identities because they don’t fit predefined input masks, we no longer enforce the law — we enforce the database schema.
A striking example can be found in university application systems. Both applicants and administrative staff are often required to enter a "first name" and a "last name" — not because the law demands it, but because the software enforces it. Input fields are mandatory, and the system rejects submissions that don’t comply with this structure. Individuals with only one legal name — as is common in countries like Indonesia, parts of India, or Myanmar — are forced to invent artificial data just to pass validation.
This technical assumption of a Western name format propagates throughout the system. Once the data is stored in this rigid structure, it feeds into downstream processes that rely on it being “correct.” If an administrator bypasses the form and inserts a single name directly into the database to reflect the legal reality, the system may break. Certificate generation in examination management may fail, as templates and their processing logic depend on the presence of both first and last names. Even the university email system may be unable to generate an address, as it expects to concatenate given and family names into formats like firstname.lastname@university.edu.
In effect, a bureaucratic fiction is enforced by code. What is technically expected becomes more powerful than what is legally true. The result is silent exclusion: identities that do not conform to the system’s structure are either rejected or forcibly reshaped. This is not legal enforcement — it is software-enforced conformity.
Toward Transparent Compliance
Shadow compliance isn’t just a technical problem — it’s a democratic one. To address it, we need:
- ✓ Clear separation between law and tool enforcement
- ✓ Legal reviews of compliance logic in software workflows
- ✓ Human override paths and transparent error reporting
- ✓ Multi-stakeholder governance of compliance platforms
Conclusion
Digital tools can support the enforcement of rules — but they must not define them.
When user interfaces and code start shaping their own compliance, we need checks and balances that ensure both legal accuracy and democratic accountability.
Otherwise, we risk building systems that are technically precise — but legally unfounded and ethically questionable.
Shadow compliance may be invisible — but its consequences are not. It’s time to bring these rules without legal legitimacy into the light.
Further Reading & Sources
- Mireille Hildebrandt – “Law As Computation in the Era of Artificial Legal Intelligence. Speaking Law to the Power of Statistics” (2017)
- Cathy O’Neil – “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy” (2016)
- European Commission – “Proposal for a Regulation on Artificial Intelligence (AI Act)” (2021)
- Tarleton Gillespie – “Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media” (2018)
- Shoshana Zuboff – “The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power” (2019)
