Robotics at the Crossroads: A Discussion of Ethical Considerations,Moral Implications,and Inclusive Design

Technologies (Elizabeth Phillips)

The deployment of increasingly autonomous technologies like generative Artificial Intelligence (AI) is significantly outpacing our ability to fully study and understand its implications for the people who will use them. In Human Factors a core tenant in the study of automated and autonomous systems has been that such technologies will not simply “supplant human activity; rather change the nature of the work that humans do (Parasuraman et al., 1997 p. 231).” However, there are many open questions surrounding the type of work humans will do either in concert with autonomous machines or in service of such machines and their implications on human well-being at large. For instance, it is well known that many machine learning models which are the backbone of AI systems are trained on data that reflects bias and can result in discriminatory outcomes especially for underrepresented groups (Mehrabi et al., 2021). Further, huge amounts of this data are predominantly annotated, moderated, and flagged by hidden workforces of low-wage, under-protected people. At times these workers are responsible for moderating some of the worst content from the internet including hate speech and sensitive images (Heaven, 2020). Similarly, historical increases in human productivity, efficiency, and economic value have followed technological progress. Humans produce more “things” of economic value than at any other time in human history largely aided by advances in technology. Yet, individual compensation for such economic value has not increased in kind (Hernandez et al., 2016). Thus many important questions surround who is really being serviced by such technologies? What are their impacts on the people upon which these technologies depend? What are the long-term consequences of reallocating humans to perform only the worst forms of work? And by enabling people to be more productive are we actually increasing their quality of life, economically or otherwise? The purpose of this talk will be to discuss the implications that autonomous technologies pose for the people who build them, make use of them, and are impacted by them.

Autonomous Vehicles and Ethical Decision Making (Alex Chaparro)

One of the main motivations behind the introduction of autonomous vehicles (AV) is the possibility of reducing vehicle crashes due to driver factors such as inattention, fatigue, experience, or aging. Although a decrease in crashes may be realized this new technology will also create new types of crashes, due to the limits of the technology, unexpected system failures, and new patterns of interactions between AV’s and other road users including pedestrians. The public reactions to AV crashes may affect the wider acceptance of AV’s and ramifications for other emerging autonomous transportation systems like urban air mobility. AVs are equipped with sensors and algorithms that will make decisions that will affect human lives. These decisions in some cases will involve the AV choosing to protect someone or something during an emergency and AV will act as an artificial moral agent making decisions with ethical consequences. Social concerns about how AVs should behave ethically in unavoidable crashes has motivated discussion amongst vehicle manufacturers, the general public, and policymakers about the decision-making of AV’s in situations where it must make an ethical decision. Understanding public’s beliefs and expectations concerning the behavior of AV’s when confronted with an ethical decision is critical to inform the design of software governing the response of AV’s. This presentation will discussion ethical aspects of autonomous vehicles including safety and the relevance of broader ethical considerations pertaining to security, privacy, transparency, algorithmic fairness, reliability, and other factors. The presentation will also explore the role of human factors professionals in addressing these ethical issues.

Opportunities for Inclusive and Ethical Design in the U.S. Army (Thomas J. Alicia)

Traditionally, military equipment was not designed with diversity or inclusivity of their users in mind. Anthropometric studies undertaken by the US Army in the 1960s only included active-duty males in the sample (White & Churchill, 1971) and it wasn’t until 1988 that the anthropometric measurements were updated to include women. Despite this, Army aviation has historically been such a male-dominated career path that the seats and cockpits are designed to accommodate the 5th – 95th percentile male only.

Today the U.S. Army is more diverse than ever and the pilots and operators of next-generation aircraft will be representative of the current Army aviation demographics. The cockpit, seats, controls and displays must be developed to be accessible and usable for all pilots to achieve maximum workforce efficiency and flexibility.

Additionally, modern aircraft are more complex and capable than ever before, including the ability to establish teaming relationships with Unmanned Aerial Systems (UAS) that can perform specific mission tasks autonomously. This concept is called Manned-Unmanned Teaming (MUM-T) and is an area of significant focus and research for future military operations.

The consideration of ethics and decision-making in these MUM-T systems is of paramount importance. The advanced capabilities integrated in the next generation of aircraft are envisioned with the primary objective of increasing lethality and survivability of the Warfighter. This inherently means making time-sensitive decisions where mistakes can lead to mishaps or unintended casualties. Much of the information coming into the aircraft to base these decisions on goes to an Air Mission Commander (AMC), whose role is to be in command of all aircraft in a flight. The unit commander designates this responsibility based on recent aviation experience, maturity, judgment, situation awareness abilities, and understanding of the commander’s intent (Army Regulation 95-1, 2018). In a MUM-T mission, it is often the AMC who directs the UAS assets to perform their tasks as part of the larger team. Ensuring the system is providing the AMC with the most accurate and up-to-date information possible is the first step to ensuring sound decision making throughout the chain of command during Army aviation operations.

Military researchers and practitioners in Human Factors and Human Systems Integration domains work to design and implement technology across the entire R&D spectrum. Efforts can range from basic research of concepts such as workload and situation awareness to applied integration and testing of interconnected equipment on board an aircraft. As system designers, it is our responsibility to develop a system that can provide an AMC with the information they need when they need it, without overloading or distracting them. As the Army transitions to the next generation of advanced aircraft teaming arrangements and capabilities there are opportunities throughout the development lifecycle for researchers to inject good design to meet these goals, minimize mishaps, and increase mission effectiveness.

Ethics and Inclusion in Product Design & Development (Jennifer Teves)

In a world where AI and robotics have become ubiquitous in people’s lives, Product Experience Researchers have a unique role to play in the process of creating products and services that people use daily. While individuals in these roles are not the people writing codes and building algorithms for technologies like AI and robotics, they are the researchers who look at the use cases and impacts of these technologies to the humans who would eventually use and interact with it when it gets translated into a product.

Research teams working in technology should treat ethics, diversity, equity, and inclusion as our non-negotiables in doing research. This means that we ensure that products are first built as a response to an unaddressed need; and, that technologies in the form of products or services that humans interact with should have the goal of improving the lives, removing the barriers, and building a world that is inclusive of all the people living in it – particularly groups that have historically been excluded, ignored, and underrepresented. Specifically, in an inclusive product research environment, each product or service is assessed through the feedback of a neurodiverse group representative of different race, ethnicity, age, disabilities, gender identities, language spoken, faith/religion, and other demographic variables that have historically alienated or underrepresent a subset of the population (Rempel & Hoy, 2019). Considering the needs of underrepresented groups, particularly individuals with disabilities, have a great potential to inspire better designs of products (Pullin, 2011).

According to Holmes (2018), “inclusion is about challenging the status quo”. Inclusive research practices should be part of the rigor of the scientific research practice if we were to move the needle on equity. Not doing so is synonymous to our inability to correct for our blind spots, which could result to building technologies that are unethical, or worse, harmful for the population that it is supposed to serve.

Design Equity (Ericka Rovira)

Human-robot interaction (HRI) as a discipline likely has more questions than answers when it comes to equitable design. Some of these important questions are explored below. Is technology value free? And if not, what values, or rather whose values are highlighted, and who’s are downgraded?

The need to develop equitable designs is growing given the many examples of bias appearing in algorithms and autonomous systems ranging from hiring algorithms to hand soap dispensers or policing facial recognition systems. Indeed, it is important for researchers and designers of AI and robotic platforms to ensure that technology is not reproducing social inequities. And further, until there is sufficient evidence that autonomous systems are not reproducing social inequities, we should insist that such technologies decision-making influence be limited.

How do we design equitable autonomous systems? Based on the current reading of the literature, I propose the following 3 approaches. First, equity audits take an in-depth view of the fairness of an organization’s policies and practices to ensure fairness. Traditionally, equity audits have been used in educational settings and corporate organizations. Similarly, equity audits may be used by researchers and designers of autonomous systems to ensure fairness. Ruha Benjamin (2019) suggests the following questions to guide an equity audit of technology: Is the design team representative? Who benefits by the design? Who is harmed by the design? If a specific technology is scaled, does it introduce new inequities or exacerbate old ones?

Second, as suggested by Ostrowski, Walker, Das, Yang, Breazeal, Park, and Verma (2022), given the interdisciplinary nature of HRI it is increasingly important that experts in diversity equity and inclusion (DEI) are invited to collaborate with roboticists in the design of robot experimentation, the analysis of data from such experimentation, and in the design of robots.

Third, in human factors we often have an application section in our work. Ostrowski et. al. (2022) suggest that authors should include the societal implications of their work in publications. The purpose of this talk is to welcome other considerations that Human Factors researchers and professionals should take to ensure design equity in robots and autonomous systems.