Can AI develop Decency?
A recent AI mishap prompted me to notice, that AI may actually - in some cases - help us to act more human than our average day-to-day dealings with each others would suggest.
In light of all the things going wrong with AI (halluzinations, bias, wrong priorities, etc.) this may sound overly optimistic or even naiv. And that is a fair evaluation. However, as we can see from this incident, the way AI behaves (meaning: interacts with us by overing answers based on probabilities) can be trained to do something that may help us, to become better.
Here is what happened, how this matters and what new conclusions could be derived from it.
What happened
Triggering this discussion was a conflict between Air Canada and a customer regarding a misleading explanation of the airline's bereavement fare policy by its chatbot. The passenger, needing to book a flight from Vancouver to Toronto due to a family bereavement, was erroneously informed by the chatbot that he could book a flight immediately and request a refund for a bereavement rate within 90 days. Following this advice led to the passenger's refund request being denied, as Air Canada's actual policy does not permit retroactive bereavement rate adjustments after booking.
Despite efforts and evidence presented by the passenger showing the chatbot's misleading guidance, Air Canada initially refused to issue a refund, instead offering a $200 coupon. The passenger then took the issue to Canada's Civil Resolution Tribunal, where Air Canada defended itself by arguing it should not be liable for the chatbot's misinformation, implying the chatbot was a separate legal entity.
The tribunal, overseen by Christopher Rivers, decisively ruled in favor of the customer. It criticized Air Canada's defense, emphasizing the unreasonable expectation that customers verify information provided by the chatbot against other areas of the website. The passenger was awarded a partial refund of $650.88 CAD from the original fare of $1,640.36 CAD, plus additional damages for interest and tribunal fees.
Following the tribunal's decision, which Air Canada has agreed to comply with, it was noted that the airline's chatbot service appeared to be disabled on their website, suggesting a measure taken in response to the case's outcome. This incident highlights a significant moment of legal examination regarding the responsibility of companies for the actions of their chatbots.
Here is the article that is the basis of this summary: https://arstechnica.com/tech-policy/2024/02/air-canada-must-honor-refund-policy-invented-by-airlines-chatbot/
Bottom line: the AI promised something that would be a decent offer, whereas the business would not want to consider such an offer, but in fact goes to court.
This might be a case where an AI has learned to choose the more ‘decent’ answer.
Coincidence, you might say, a random result, another glitch or just another good reason to do better. But lets face it: the AI did what it was trained to do.
Let´s get back to that thought a little later, but first see, how this fits in to the contemporary discussion.
Here is an overview of what´s following:
- Reliability vs Confidence: Examining the distinction between how often AI systems perform as expected (reliability) and our belief in their ability to make correct decisions (confidence).
- Erosion of Trust: Highlighting how failures and biases in AI systems can lead to a decrease in public trust and skepticism towards technology.
- Future of AI in Decision-Making and Ethical Governance: Exploring the evolving role of AI in shaping decisions across various sectors and the importance of ethical guidelines to govern its use.
- Light at the end of the tunnel: Acknowledging the potential for AI to demonstrate, that it can evolve beyond biases and errors towards making ethically sound and benevolent decisions.
- Our Mission: Get the training right!: Emphasizing the critical importance of careful and ethical selection of training data to improve AI outcomes and ensure technology serves humanity positively.
Reliability vs Confidence
The incident involving Air Canada's chatbot and a misleading explanation of the airline's bereavement fare policy underscores a broader challenge in the realm of artificial intelligence: the impact of AI errors on consumer trust and the implications for the technology's acceptance and deployment. AI systems, from chatbots to decision-making algorithms, have the potential to significantly enhance efficiency and user experience across various sectors. However, when these systems falter—whether by disseminating incorrect information, exhibiting bias, or making unsafe recommendations—the fallout can undermine public confidence in AI, prompting skepticism about its reliability and efficacy.
Mishaps over the past years:
- Microsoft's AI Chatbot Tay (2016) - Tay, an AI chatbot released by Microsoft on Twitter, started producing offensive content after learning from user interactions. This happened online, specifically on Twitter, in March 2016. The damage was a public relations issue for Microsoft, showcasing AI's potential to amplify harmful biases. As a consequence, Tay was taken offline within 24 hours; the incident sparked debates over AI ethics and the need for safeguards.https://www.theguardian.com/world/2016/mar/29/microsoft-tay-tweets-antisemitic-racism and https://www.makeuseof.com/lessons-microsoft-learned-tay-ai-disaster/
- Autonomous Uber Vehicle Fatality (2018) - A self-driving Uber car was involved in the first fatal collision with a pedestrian. This happened in Tempe, Arizona, in March 2018. The damage raised safety and ethical concerns regarding autonomous vehicles. As a consequence, there was a temporary halt to Uber's autonomous testing and increased scrutiny and discussion on the safety of self-driving technology.https://www.theverge.com/2023/7/31/23814474/uber-self-driving-fatal-crash-safety-driver-plead-guilty
- IBM Watson for Oncology's Inaccurate Recommendations (2018) - Watson for Oncology made unsafe and incorrect cancer treatment recommendations. This happened with global implications, reported in 2018. The damage highlighted the limitations of AI in complex medical decision-making, posing a potential risk to patient safety. As a consequence, there was criticism of IBM and a reevaluation of AI's role in healthcare, emphasizing the need for validation against clinical outcomes.https://www.advisory.com/daily-briefing/2018/07/27/ibm
- Facebook's News Feed Algorithm Changes (2017) - AI-driven algorithms prioritized divisive and sensational content, contributing to misinformation. This happened globally, with changes implemented around 2017. The damage was an erosion of trust in social media as reliable information sources, impacting public discourse. As a consequence, Facebook announced algorithm changes and increased efforts to combat misinformation.https://techcrunch.com/2017/01/31/facebook-authentic-news
- Amazon's AI Recruitment Tool Bias (2018) - Amazon's recruitment tool exhibited bias against women for technical roles. This happened in 2018, used internally by Amazon. The damage sparked a broader debate on ensuring AI fairness and neutrality in hiring practices. As a consequence, Amazon discontinued the program, highlighting challenges in deploying AI for HR processes.https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html
- Google Photos Racial Misidentification (2015) - Google Photos' AI algorithm incorrectly labeled African American users. This happened in 2015. The damage was a public outcry over racial insensitivity, highlighting AI systems' potential to propagate biases. As a consequence, Google apologized and fixed the algorithm, prompting discussions on ethical AI development.https://algorithmwatch.org/en/google-vision-racism/
- Flash Crash Attributed to Algorithmic Trading (2010) - High-frequency trading algorithms caused a significant stock market crash. This happened in the U.S. stock market on May 6, 2010. The damage included a temporary massive market value loss and shaken investor confidence. As a consequence, there were increased regulatory measures on algorithmic trading to prevent future crashes.https://incidentdatabase.ai/cite/28/
- UnitedHealth Algorithm Bias (2019) - An algorithm used by UnitedHealth was found to be less likely to refer Black patients for improved care programs. This happened in the United States, highlighted in a 2019 study. The damage was racial bias in healthcare recommendations, highlighting disparities in healthcare access. As a consequence, there was a reevaluation of using AI in healthcare, emphasizing the need for unbiased algorithms.https://www.theguardian.com/society/2019/oct/25/healthcare-algorithm-racial-biases-optum
- COMPAS in the Criminal Justice System - COMPAS software used for assessing recidivism risk was criticized for potential racial biases. This happened in various U.S. courts, with criticisms emerging prominently around 2016. The damage was ethical concerns regarding fairness and transparency in parole and bail decisions. As a consequence, there was sparked debate and calls for ethical guidelines, transparency in AI use in the justice system.https://blogs.chatham.edu/dsethics/cases/compas/ and https://www.theatlantic.com/technology/archive/2018/01/equivant-compas-algorithm/550646/
- Apple Card Credit Limit Bias Allegations (2019) - Apple Card faced allegations of gender bias in credit limit decisions. This happened in the United States, reported in November 2019. The damage raised questions about AI-driven financial decision-making and the potential for unconscious biases. As a consequence, an investigation by regulatory authorities was prompted, sparking discussions on accountability in AI algorithms.https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html
- Denial of Social Services Due to Automated Systems - Automated decision-making systems wrongly denied individuals benefits in Arkansas. This happened in Arkansas, with incidents reported over several years leading up to 2020. The damage was arbitrary reductions in essential services for disabled individuals, highlighting the risks of AI in welfare. As a consequence, there was criticism of the use of AI without safeguards, calling for transparency and appeal mechanisms.https://www.theverge.com/2018/3/21/17144260/healthcare-medicaid-algorithm-arkansas-cerebral-palsy
MIT has also issued a paper on the coming war on hidden algorithms: https://www.technologyreview.com/2020/12/04/1013068/algorithms-create-a-poverty-trap-lawyers-fight-back/
Erosion of Trust
"Sorry, the system won´t allow me to do that"
The timing of such AI missteps, including the incident with Air Canada's chatbot, could hardly be more inopportune, arriving at a moment when people are already grappling with the frustrations of red tape and bureaucratic entanglements in various aspects of their lives. In an era marked by digital transformation, consumers increasingly encounter the phrase "sorry, the system won't allow me to do that" from service agents, a refrain that symbolizes the often rigid and impersonal nature of automated systems. This growing reliance on technology for decision-making and customer service has amplified the impact of AI errors, making them not just inconveniences but poignant reminders of the potential dehumanization in our interactions with service providers.
The discontent arising from such encounters is more than just frustration with a single flawed AI interaction; it reflects a deeper concern over being ensnared in a web of automation that lacks empathy and flexibility. As individuals navigate a landscape dense with bureaucratic procedures, the expectation for technology, particularly AI, is to offer relief and efficiency, not compound existing challenges with errors and misinformation. This backdrop of bureaucratic fatigue makes AI failures particularly galling, as they betray the promise of technology as a tool for simplification and empowerment.
The repeated missteps of AI, exemplified by incidents like the Air Canada chatbot debacle, underscore a pivotal challenge for the future of technology: maintaining and restoring trust in systems designed to simplify and enhance our lives. In an era where automation and artificial intelligence increasingly mediate our interactions with the world, the imperative to ensure these systems are not only efficient but also reliable and user-friendly has never been more critical. Each failure not only impacts the immediate stakeholders but also contributes to a broader societal skepticism towards technological solutions, threatening to stall progress and innovation.
Consequences and Solutions
The erosion of trust could lead to heightened scrutiny and, in some cases, precipitate a knee-jerk regulatory response from governments aimed at curbing AI's use, potentially stifling innovation and imposing constraints that may not address the root causes of these mishaps. Such incidents highlight the urgent need for comprehensive strategies to bolster AI's reliability, including the development of ethical guidelines, transparency in AI operations, and mechanisms for accountability.
In this context, ensuring AI systems are designed with an emphasis on user-centric values, transparency, and the capacity for human oversight is not just beneficial—it's imperative for maintaining public trust and acceptance of AI technologies. Addressing these concerns proactively is crucial for preventing a backlash against automation and preserving the potential of AI as a force for positive change in society.
In response to these challenges, providers and stakeholders in the AI ecosystem are increasingly focusing on improving the robustness and accountability of AI systems. Efforts include investing in AI research that prioritizes fairness, accuracy, and transparency, implementing rigorous testing phases to detect and correct biases before deployment, and developing more sophisticated natural language processing algorithms to enhance chatbots' understanding and response accuracy. Moreover, the incident with Air Canada's chatbot illustrates the legal and ethical responsibility companies have for their AI systems' actions, signaling a shift towards greater accountability.
The continuous acceptance and widespread deployment of AI technology hinge on addressing these concerns effectively. As AI becomes more integrated into everyday services, ensuring these systems are trustworthy, understandable, and capable of acting in the users' best interests is paramount. By adopting a proactive approach to AI development that prioritizes ethical considerations and user welfare, companies can mitigate the risks associated with AI mishaps and foster a more resilient foundation for the technology's future growth and integration into society.
Balancing Act: Navigating the Future of AI in Decision-Making and Ethical Governance
Growing dependence
As we move towards an era where decisions become increasingly automated, guided by the outputs of AI models, we're standing on the brink of a significant transformation in how we interact with technology and make choices in our personal and professional lives. These AI systems, with their complex networks of calculations, are not human; they don't possess consciousness or emotions but operate through probabilities derived from vast databases of numbers, simulating a form of learning akin to a young child absorbing information from the environment. Despite their mechanical and impersonal nature, the influence of AI on decision-making processes is profound and expanding.
The trajectory we're on suggests a future where AI-driven decision-making permeates every sector of society, from healthcare diagnosing and treatment recommendations to legal judgments and financial advising, further blurring the lines between human intuition and algorithmic predictions. As AI models become more sophisticated, the accuracy and reliability of their outputs could lead us to rely more heavily on these systems, potentially relegating human decision-making to a supervisory role. This shift promises efficiency and the mitigation of human biases in decisions, yet it also raises critical ethical and practical concerns.
The risk of over-reliance on AI for critical decisions could lead to scenarios where the limitations of AI are not fully accounted for, such as the inability of these systems to understand context or moral nuances in the same way humans do. Furthermore, as decision-making processes become more automated, there's a potential for diminishing human oversight, making it imperative to establish robust frameworks for accountability, transparency, and explainability in AI systems.
Moreover, the growing automation of decisions could exacerbate existing inequalities if AI systems are trained on biased data sets, leading to outcomes that systematically disadvantage certain groups. To mitigate these risks, there's a pressing need for multidisciplinary efforts involving ethicists, technologists, policymakers, and community stakeholders to guide the development and deployment of AI, ensuring these systems are equitable, just, and aligned with societal values.
In conclusion, the progression towards more automated decision-making heralds a new era of efficiency and potential, but it also necessitates a reevaluation of our relationship with technology. Balancing the benefits of AI-driven automation with the need for human-centric values and ethical oversight will be one of the defining challenges of our time, shaping the future of society in the digital age.
Real Concerns
This all is rooted in current trends and theoretical projections about the future of artificial intelligence and its integration into various sectors of society. The discussion touches on real concerns and possibilities based on the capabilities and limitations of AI as understood today. Here are some key points to consider:
- AI's Decision-Making Role: AI systems are indeed becoming more integrated into decision-making processes across numerous fields, including healthcare, finance, and legal systems. This trend is based on the capability of AI to analyze large datasets and identify patterns or probabilities that may not be immediately apparent to human analysts.
- Ethical and Practical Concerns: The concerns about over-reliance on AI, the potential for AI to exacerbate existing inequalities, and the need for ethical oversight are genuine. These issues are widely discussed in academic, technological, and policy-making circles. The development of AI systems that are fair, transparent, and accountable is a major focus of current research and debate.
- Limitations of AI: AI does indeed have limitations, particularly in understanding context, moral nuances, and the broader social implications of its outputs. AI systems operate based on the data they are trained on, and their "decisions" are predictions based on patterns in that data. They lack human intuition and empathy, which are often crucial in complex decision-making scenarios.
- Future Projections: While the projection about AI taking a more central role in decision-making and potentially relegating human decision-making to a supervisory role is speculative, it is grounded in the current trajectory of AI development and deployment. However, the exact outcome will depend on a multitude of factors, including technological advancements, regulatory decisions, and societal attitudes towards AI.
- Multidisciplinary Efforts for AI Governance: The call for a multidisciplinary approach to AI governance, involving ethicists, technologists, and policymakers, is a real and pressing need recognized by many experts in the field. Ensuring AI systems are developed and used in ways that are ethical and beneficial to society is a complex challenge that requires input from diverse perspectives.
While the specific future of AI and its role in society is uncertain and subject to change, the themes and concerns discussed here are based on real and current issues in the field of artificial intelligence. The accuracy of these projections will of course depend on how technology evolves and how society chooses to manage and integrate AI systems. At the moment though, it appears that the exponential developments are not slowing down.
The Black-Box Problem
As we venture further into the age of artificial intelligence, a growing concern is the widening gap between human understanding and the complex inner workings of AI systems. This distance, often referred to as the "black box" problem, highlights a fundamental challenge: as AI algorithms become more sophisticated and autonomous, deciphering their decision-making processes becomes increasingly difficult for humans.
The intricate layers of neural networks and the vast amounts of data they process mean that even the creators of these systems can struggle to explain why a particular decision was made.
This opacity raises significant issues, especially in critical applications where understanding the rationale behind an AI's decision is essential for trust, accountability, and ethical oversight.
Bridging this gap is not just a technical challenge but a crucial societal task to ensure that as machines become an integral part of our decision-making infrastructure, they do so in a way that aligns with human values, ethics, and the capacity for comprehension. Addressing this issue demands concerted efforts in developing explainable AI (XAI) technologies and regulatory frameworks that prioritize transparency, without sacrificing the efficiency and benefits that AI brings to complex problem-solving.
Light at the end of the tunnel
The incident at Air Canada, where a chatbot provided misleading information regarding the airline's bereavement fare policy, might initially seem to underscore the pitfalls of AI. However, it also illuminates a compelling avenue for optimism: the potential for AI to make decisions that are not only accurate but also inherently more humane, provided it receives the proper training. This perspective advocates the power of training data as a pivotal factor in shaping AI behavior, suggesting that with meticulously curated inputs, AI systems can indeed reflect values that prioritize decency and empathy.
Drawing from the idea that AI models have, on occasion, demonstrated biased behavior or unexpected emergent capabilities, it's plausible to consider that these systems can also learn to favor decisions that embody a more compassionate and equitable approach. The suggestion that AI could have contributed to a more humane outcome in scenarios like the financial crisis of 2008 is intriguing. It posits that if AI had been trained on data encapsulating more ethical decision-making frameworks, perhaps the algorithms could have highlighted alternatives that mitigated the crisis's severity or even averted it altogether.
The Air Canada incident, therefore, can be viewed not merely as a failure but as a case study emphasizing the significance of training data in AI development. It underscores the idea that the outcomes of AI systems are deeply entwined with the quality and nature of their training data. This realization carries a profound implication: by investing in the careful selection and curation of training data that embodies ethical principles and human values, developers can steer AI towards making decisions that are not only logical but also morally sound and socially beneficial.
Our Mission: Get the training right!
The realization that emerges from the Air Canada chatbot incident transcends its specific circumstances, casting a spotlight on a profound and universal truth: the essence and outcomes of artificial intelligence are fundamentally shaped by the training data it is fed. This understanding is not merely a technical observation but a clarion call for a more conscientious and principled approach to the creation and nurturing of AI systems. It is a passionate plea for the right training—a commitment to embedding our highest human values into the very fabric of AI.
Embarking on this path requires a deliberate and ethical approach to selecting training data. It begins with the recognition that every dataset is a reflection of the values it embodies and the biases it may inadvertently perpetuate. Therefore, the selection process must be guided by principles of diversity, fairness, and inclusivity, ensuring that AI systems are exposed to a wide spectrum of human experiences and perspectives. This approach not only mitigates the risk of biases but also enriches AI's capacity for empathy and understanding, laying the groundwork for decisions that resonate with the best of human judgment.
Moreover, the process of training AI must be iterative and reflective, open to constant evaluation and recalibration based on outcomes and feedback. It should involve a multidisciplinary team of experts—not only data scientists and AI developers but also ethicists, sociologists, and representatives from the communities most likely to be impacted by the AI's decisions. This collaborative effort ensures that AI systems are not developed in isolation but are continually aligned with evolving societal norms and ethical standards.
Additionally, transparency in AI training processes is crucial. By making the methodologies and data sources behind AI systems accessible and understandable, we can foster trust and accountability. This transparency allows for a broader societal engagement in discussions about the role and impact of AI, ensuring that these technologies are shaped not just by the few but by the many, reflecting a collective vision of the future we aspire to.
This pursuit of embedding our highest human values into AI through meticulous training is not merely an aspiration; it is an imperative for the technological age. As AI becomes increasingly interwoven into the fabric of daily life, influencing decisions that affect everything from individual opportunities to global economies, the stakes could not be higher. The quality of the training data and the ethical frameworks guiding AI development become the bedrock upon which the trustworthiness and benevolence of our technological counterparts rest.
Engaging in this endeavor requires more than technical expertise; it demands a cultural shift in how we conceive of AI development. It calls for a proactive stance in which technology creators actively seek out diverse voices and experiences, ensuring that AI systems learn from the full spectrum of human existence. This diversity is not just a bulwark against bias; it is a wellspring of creativity and innovation, opening AI to new possibilities and perspectives that might otherwise remain unexplored.
To facilitate this shift, educational institutions, industry leaders, and policy makers must come together to foster an ecosystem that values ethical AI training as a core competency. Curriculum and training programs should not only focus on the technical aspects of AI but also on the ethical considerations, equipping the next generation of AI practitioners with the tools they need to navigate the complex moral landscape of AI development.
Moreover, regulatory frameworks must evolve to support and enforce ethical training practices, setting standards that ensure AI systems are developed with accountability and public welfare in mind. These regulations should encourage transparency, requiring companies to disclose the datasets used for training AI and the ethical guidelines followed during development. Such measures would not only bolster public trust in AI but also incentivize companies to adhere to best practices in AI training.
In essence, the call for the right training is a call to action for all involved in the development and deployment of AI. It is an invitation to imbue our technology with our most cherished human qualities—compassion, fairness, and wisdom. By committing to this path, we have the opportunity to harness the incredible potential of AI to not only solve complex problems but to do so in a way that uplifts humanity, bridges divides, and fosters a more just and understanding world. The journey toward realizing this vision is fraught with challenges, but it is also filled with the promise of a future where technology and humanity converge in harmony.