USA: An Overview of the DIU's AI Guidelines
The Defense Innovation Unit ('DIU') released, on 15 November 2021, Responsible Artificial Intelligence Guidelines1 ('the AI Guidelines') in order to integrate the U.S. Department of Defense's ('DoD') Ethical Principles for AI2 ('the Ethical Principles') into the DIU's commercial prototyping and acquisition programs. The AI Guidelines provide a step-by-step framework for artificial intelligence ('AI') companies, DoD stakeholders, and program managers to align with the Ethical Principles.
The DIU highlighted that fairness, accountability, and transparency are considered at each step in the development cycle of an AI system, and outlined that the AI Guidelines are the result of their 2020 initiative to implement the Ethical Principles. To do so, the DIU drew upon best practices from government, non-profit, academic, and industry partners, and explored methods for implementing these principles in several of its AI prototype projects.
However, although the AI Guidelines aim to facilitate, and should be viewed as complementary to a company's internal ethics review and related testing and evaluation procedures, they do not, in every situation, offer a universally reliable way to address and amend any shortcomings that may be faced, such as biased data, inappropriately selected algorithms, or poorly defined applications.
The Aim of the AI Guidelines
The DIU noted that the AI Guidelines aim to provide a clear, efficient process of inquiry for personnel involved in AI system development such as, program managers, commercial vendors, and government partners, to accomplish the following goals:
- 'ensure that the Ethical Principles are integrated into the planning, development, and deployment phases of the AI system lifecycle;
- effectively examine, test, and validate that all programs and prototypes align with the Ethical Principles; and
- leverage a process that is reliable, replicable, and scalable across a wide variety of programs'.
In addition, the AI Guidelines are framed within three major phases of the technical lifecycle of an AI system: planning, development, and deployment.
The planning phase
The DIU outlined that in the planning phase, personnel from the government agency that are requesting the AI system collaborate with the program manager to define the AI system's functionality, the resources required to create it, and the operational context into which it will be deployed. In this respect, the DIU further detailed that the AI Guidelines require the following questions to be answered before proceeding to the development phase:
- Have tasks, quantitative performance metrics, and a baseline against which to evaluate system performance been clearly defined?
- Has ownership of, access to, provenance of, and relevance of candidate data/models been evaluated?
- Are end users, stakeholders, and the responsible mission owner identified?
- Has harms modelling to assess likelihood and magnitude of harm been conducted?
- Has the process for system rollback and error identification/correction been identified?
In addition to these questions, the DIU stated, in its AI Guidelines, that the planning phase workflow ensures that AI is appropriate for the task, and is only applied after the following applies:
- 'other methods, for example human-driven solutions, have been evaluated;
- success metrics and baselines are well-scoped;
- appropriate data (e.g., high quality, accurate, representative data, etc.) is acquired to support the capability;
- stakeholders, mission owners, and end users (including potentially impacted populations) are appropriately considered and consulted;
- detailed risk assessments and harms modelling are conducted; and
- processes for reverting from a malfunctioning system and identifying or addressing system errors are pre-emptively prescribed'.
The development phase
Subsequently, and in the development phase, the planned AI system is further built out by DoD and/or company personnel. To do this, the AI Guidelines list additional questions to be answered prior to moving on to the final phase of the AI system technical lifecycle:
- Has a plan been created to prevent the intentional or unintentional manipulation of data or model outputs, as well as identifying who is responsible for implementing this plan?
- Have procedures and reporting processes for system performance and post deployment monitoring been defined, along with identifying who is responsible for implementing these procedures?
- Have roles/persons with the power to make and certify necessary changes to the capability been designated?
- Has an appropriate plan/interface to verify individual outputs of the system been deployed?
- Have roles/positions for government and/or third-party system audits been defined and assigned?
To facilitate the answering of these questions within the development phase, the DIU explains, in its AI Guidelines, that the focus of this development phase is on:
- 'mitigating the potential negative impact of data or model manipulation;
- delineating metrics and indicators for post-deployment monitoring;
- explicitly assigning authority to make changes to the capability;
- enabling users to understand how each system output is generated; and
- planning for routine system auditing'.
The deployment phase
In the final phase of the AI system technical lifecycle, the deployment phase, the DoD or company personnel will make use of the AI system in an operational setting. Additionally, the AI Guidelines describe concrete sets of continuous evaluation procedures that must be scoped and performed on an ongoing basis throughout an AI system's lifecycle, and with respect to the deployment phase, the workflow in this phase focuses on:
- 'continuous task and data validation that ensures the original task specification and data inputs are still valid and secure;
- functional testing that evaluates whether the capability still performs the desired task sufficiently well to be operationally useful; and
- harms assessment and quality control to make certain that potential negative impacts on stakeholders are constantly reassessed and mitigated when necessary'.
More specifically, each of these three steps in the workflow of the deployment phase can be assessed in the following way:
- Continuous task and data validation:
- Are tasks still properly defined?
- Are data inputs qualitatively and quantitatively assessed and protected against interference/manipulation?
- Functional testing:
- Is the capability still meeting the desired functional goals?
- Are performance deviations identifiable and rectifiable?
- Do processes exist to rollback malfunctions?
- Harm assessment and quality control:
- What do post deployment monitoring and auditing show?
- What is learned from conducting continuous harms modelling?
- Is continuing to identify responsible mission users to handle these concerns taking place?
The DIU noted that following a year of developing, testing, and iterating on the AI Guidelines, many lessons were learnt. These lessons include setting customer and vendor expectations, spending time on metrics, accounting for the fit of technology and tasks where high risk applications are best paired with low risk technology and vice versa, incorporating industry best practices where appropriate, and investing time and resources in documentation.
In this respect, the AI Guidelines indicate that setting customer and vendor expectations requires following an assessment of the issues laid out in the AI Guidelines, 'accompanied by a realistic estimation of long-term resourcing and sustainment requirements', which will 'enable pragmatic decision-making about the potential costs, benefits, and risks of developing an AI based system for a given application'. Importantly, the DIU highlighted that 'program managers should be clear about responsible AI expectations from day one, vendors should be encouraged to come forward with performance issues, and that and DoD partners should remember that any flaw in an AI system represents an operational risk to DoD personnel'.
With respect to the lesson of investing time and resources in documentation, the DIU outlined that this involves the process of working through the AI Guidelines on real projects, which has 'reinforced the importance of precise, descriptive, documentation to the viability and efficacy of AI capabilities for the DoD'. Furthermore, the DIU noted that 'in order for a user to be confident in leveraging a capability for a given mission, they must not only be convinced that the system works as intended, but also that the user's problem corresponds exactly to what the AI system was built to solve'.
Finally, the DIU highlighted the importance of incorporating industry best practices. Specifically, the DIU specified that the DoD organisations can learn from commercial sector advancements in ethical AI development. In particular, the DIU noted that '[m]any of the AI solutions sought by the DoD have parallel applications in the private sector, where commercial organisations are independently developing ethical frameworks to guide and inform their AI development activities'.
The DIU highlighted that the AI Guidelines are a useful starting point for operationalising the Ethical Principles and that they will continue collaborating with experts and stakeholders from government, industry, academia, and civil society to further develop the AI Guidelines.
Alexander Fetani, Senior Privacy Analyst
1. Available at: https://assets.ctfassets.net/3nanhbfkr0pc/acoo1Fj5uungnGNPJ3QWy/6ec382b3b5a20ec7de6defdb33b04dcd/2021_RAI_Report.pdf
2. Available at: https://www.ai.mil/docs/Ethical_Principles_for_Artificial_Intelligence.pdf