AI Registers 101
A short guide to designing and deploying ADS registers that enable meaningful transparency
Series on Responsible AI
Important note on nomenclature: While often referred to as "AI" registers, the more appropriate term should be "Algorithmic Decision System" (ADS) registers.
What is a register? And how can it enable transparency?
A register is an official record of information. In its simplest form, it can provide visibility over algorithmic systems in use, which is the most basic form of transparency. It can also be designed to enable meaningful transparency that meets stakeholder information needs. It’s the latter use case that I explore throughout this article.
Meaningful transparency is an important concept in AI ethics. It means delivering the information pertinent to each stakeholder group in the format that is best suitable for their understanding. This is especially critical in the context of "high-risk" systems that can directly or indirectly impact benefits, punishments, or opportunities an individual may receive and where meaningful transparency is key for enabling agency and accountability.
As discussed in my previous article, algorithmic policy and regulation is largely lagging. There are currently no requirements in place for organizations to make disclosures on algorithmic systems in use. The first major development to challenge this status quo is the draft EU AI Act (2021), which will mandate organizations to create and maintain inventories of AI systems and risk-mitigation measures. If implemented, these mandated disclosures will likely not achieve meaningful transparency for impacted groups due to their limited scope, but they denote a helpful step forward to providing basic visibility.
Despite the lack of regulation, a number of local governments, such as those of Amsterdam, Antibes, Helsinki, Nantes and New York City, have taken steps towards creating their own algorithmic registers. These first generation registers vary in levels of sophistication and completeness. They also generally fall short of enabling the "ideal" of meaningful transparency but nevertheless represent a good step forward towards ADS transparency.
There is a lot that we can learn from this first generation of algorithmic registers. One interesting finding based on practitioner interviews is that a register can help incentivize the ADS owner to improve their internal governance and controls. It can also enable citizen-driven accountability mechanisms, which are crucial given the current regulatory landscape.
How to design and deploy an algorithmic register that enables meaningful transparency?
While a register in itself may seem to be a simple tool, most of the heavy lifting to launch one has to do with the governance and processes required to maintain and share ADS disclosures.
This guide walks through the critical steps of establishing an ADS register in the hopes of enabling wider adoption of this transparency tool.
Step 1: Agree on register goals
Start with a shared understanding of success at an organizational level. This will be your yardstick to measure performance and improve your register with each iteration.
Some motivational goals include:
- Tracking planned, deployed and retired ADS under a standard reporting framework;
- Meeting citizen information needs and enabling agency for impacted groups;
- Creating a strong governance around ADS implementations;
- Enabling compliance with existing (or future) regulation.
Step 2: Identify and understand stakeholder needs
The next step is to identify all relevant internal and external stakeholders and understand their information needs related to the set of algorithmic decision-making systems owned by your organization .
Internal stakeholders include teams responsible for the commissioning, development, deployment, use and maintenance of the algorithmic-decision system(s). They can also include senior leadership and other ADS teams.
External stakeholders can include regulatory and oversight bodies (if mandated), public-interest groups (advocates, journalists, researchers), as well as individuals impacted directly or indirectly by the algorithmic decision-making system(s).
Conducting user research studies is a valuable way to uncover complex information needs and test communication modalities with stakeholders.
For example, individuals impacted by the algorithmic decision-making system may have varying levels of understanding of algorithms and computational processes. Some may not be well versed on these topics at all, others may have introductory notions, and yet others may be curious to poke around at the code themselves. As impacted individuals, they are likely to be primarily concerned with understanding their own outcome and having opportunities for remedial.
Public-interest groups, on the other hand, are composed of individuals who typically have technical expertise in assessing algorithmic systems and their impact. Their primary goal from a register is to have the information required to validate the responsible use of an algorithmic system.
This study on citizen information needs in the UK is a helpful reference for conducting similar user research. One interesting finding from the study is that citizens are generally unconcerned with algorithmic transparency information unless they happened to personally have a problem with a certain algorithmic system. They also expect transparency information to be made public, regardless of their personal interest in it, as a means to enable public-interest groups to examine the information on their behalf and raise any potential concerns.
Step 3: Define appropriate engagement model
Once all relevant stakeholders are identified, you then have to define the appropriate model to engage with each stakeholder on the algorithmic decision system(s) in use by your organization.
Internally, there should be clear roles and responsibilities assigned, as well as formal hand-overs and review touchpoints between different teams that commission, design, develop, maintain, and use the system.
Externally, approvals or consultations with government bodies may be needed depending on regulation. Impacted individuals and public-interest groups also should be engaged regularly. I offer a deep-dive into these citizen engagement models in this article.
It is important to take a lifecycle-based approach in designing this engagement model as algorithmic systems can be regularly updated post-deployment. One of the main criticisms of the EU AI Act is that it treats ADS providers as manufacturers launching one-off products that can be assessed once, at launch, which seriously overlooks the extent to which algorithmic systems evolve post-deployment.
The most important decisions regarding an algorithmic system are usually taken pre-system development. The engagement touchpoints and assessments conducted at this stage are crucial to safeguard the responsible development and use of the algorithmic system.
Below is an illustrative example of an engagement model for a high-risk algorithmic system that requires regulatory approvals and adequately engages with civil society to minimize the risk of harm.
Step 4: Create and adopt an ADS disclosures framework
At each step of the algorithmic system lifecycle, there is important information to be captured about the process in place and the outcomes the system is achieving.
Each disclosure should have an assigned owner and should be reviewed by other stakeholders to validate the information shared.
Below is a detailed view of recommended disclosures over a system’s lifecycle, including disclosure owners and recommended touchpoints.
The insights uncovered during stakeholder interviews and user research (step 2) should help inform communication guidelines for each disclosure.
Each disclosure should be written and illustrated in a way that best enables key stakeholder understanding and agency.
The most complete set of disclosures should be logged in an internal register. Subsets of this information can be shared publicly to best meet citizen information needs. Regulators, auditors, and potentially public-interest groups, should be given privileged access to relevant private information through data-sharing agreements.
This is what we call multi-level disclosures.
System owners may not be comfortable sharing source code and other elements they may view as "trade secrets" to the public. There are also fears that sharing the minute details of how a system works could lead to system gaming. Multi-level disclosures allow system owners to share the information most relevant for each stakeholder needs with critical information (such as source code) provided through privileged access channels where the stakeholder evaluating this information can share their assessment results (but not the source code for example).
; Right: ADS Architecture Schema for Youtube's Recommendation Engine (geared towards technical users), Source](https://towardsdatascience.com/wp-content/uploads/2022/04/1rBjC3-r609MCJWjBNTTFxg.png){kind=link}
Let’s consider the disclosures related to the architecture of an algorithmic decision-making system – which can help relevant stakeholders understand how the system works. For example, individuals impacted by an automated parking system that issues fines for misconduct would like to know how the system works without having to necessarily understand the technical stack powering the ADS. They are interested in understanding how the ADS arrives at a decision to issue a fine, which parts of the process are reviewed by humans, and what the process to contest a potentially incorrect decision is. Other stakeholders might like to have access to how the system works from a technical standpoint. That means understanding the processing steps and methods used, and potentially getting access to the code and test data.
Different sets of information are required to enable meaningful transparency for each concerned stakeholder. The register should enable capturing all of this information and disclosing appropriate subsets of it to each stakeholder.
Going back to our automated parking system example. A public register should include an easy-to-understand flowchart explaining how the system reached the decision to issue a fine (or not), and clearly outlining which parts of the process are performed or monitored by humans. Auditors and public-interest groups can request access to the code and detailed technical architecture documentation to perform assessments. This is what multi-level disclosures looks like in practice.
Step 5: Set transparency requirements from procured services
Delivering transparency can become complex when you do not have full ownership of the sub-systems in place. In fact, it is very common for owners of algorithmic systems to procure third-party services (such as data and AI platforms with built-in models, out-of-the-box models, or dedicated enterprise solutions).
Most public sector entities have specific procurement guidelines and some countries have even gone further to institute specific guidelines when it comes to procuring data and AI-related services (e.g. UK). We are starting to see the public sector institute specific algorithmic transparency requirements from their third-party providers. For example the City of Amsterdam requires that its suppliers conform with "procedural transparency" (i.e. information required to establish adequate safety processes) and provide "technical transparency" (i.e. source code) to auditors when required.
Private organizations should leverage their buying power to select vendors that can adhere with transparency requirements and work together on achieving them. The third-party provider(s) should also be accounted for when designing the engagement model supporting your register.
Step 6: Design register UI/UX
Design plays a crucial role in enabling meaningful transparency for diverse stakeholder needs.
A diverse set of stakeholders should be able to easily access the register and find the information they need in a modality that best serves their understanding and agency.
The register design should follow a human-centric iterative approach that engages with core stakeholders and integrates their feedback into the design process.
Some key deliverables here include:
- Defining user journeys for each stakeholder persona;
- Defining key user interactions;
- Designing the register interface accounting for disclosure architecture, user journeys, and key interactions.
It’s important to take edge cases into consideration here. For example, how might the user experience differ for a low-risk vs a high-risk ADS? How might that impact the design of the register?
Step 7: Progressive roll-out and testing of algorithmic systems on the register
Select a few ADS candidates for the beta release of your register. Ideally, these ADS should be in the early stages of the lifecycle (commissioning phase) to ensure the right information is captured with supporting processes and governance.
Treat your register like a product. Start with an MVP, test, measure performance against set goals, and iterate.
After the first system goes live on the register, collect feedback from relevant internal and external stakeholders and track progress against register goals. Typically, several iteration cycles will be required to get the register and the processes supporting it right.
Once you have a stable ADS onboarding mechanism in place, it is time to evangelize to the rest of the organization and include all ADS in the register.
Towards broader register implementations
Most organizations have some internal log of ADS in use that may or may not be centralized in one place. It is however very uncommon for organizations to share this information publicly (even subsets of it).
In this guide, I attempted to lay out the motivation behind evangelizing algorithmic registers and provide guidelines to enable its implementation. Strong incentive mechanisms will need to be in place to enable broader register adoption (likely through regulation). I do hope that the prospect of future regulatory change and growing pressure by civil society and public-interest groups will be a motivating factor for organizations to take on a proactive approach when it comes to enabling algorithmic transparency.
Registers are a useful tool in that effect; they incentivize setting in-place both internal and external governance to ensure the responsible use and deployment of Algorithmic Decision Systems.
Going deeper: participatory approaches to enabling algorithmic transparency
In the next article, I explore how to engage citizens in enabling the responsible use of algorithmic decision-making.
Part of a series on Responsible AI based on my graduate thesis "Beyond the Black Box" (MIT 2021). The ideas presented were developed based on the feedback and support of several practitioners with direct experience of regulating, deploying, and assessing AI systems. I am sharing and open-sourcing my findings to enable others to easily study and contribute to this space.
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
