Unit 5: Ethics and Challenges of AI

I. Orientation

Artificial intelligence ethics examines how AI systems affect people, society, institutions, and the environment. It is grounded in the principle that an AI system should be useful without causing unjustified harm, violating rights, or hiding responsibility. Ethical AI combines technical accuracy with fairness, transparency, privacy, safety, accountability, and human oversight.

• AI system: A system that uses data and computational methods to make predictions, recommendations, classifications, or decisions.
• Stakeholder: Any person or group affected by AI, including users, workers, customers, communities, developers, and governments.
• Ethical risk: A possibility that an AI system may cause harm, discrimination, loss of privacy, deception, or unfair treatment.
• Human oversight: The ability of qualified people to monitor, question, correct, or stop an AI system.
• Accountability: The requirement that identifiable people or organizations accept responsibility for an AI system’s effects.
• Responsible design: Building AI with safety, fairness, privacy, explainability, and social consequences considered from the beginning rather than after deployment.

II. Bias — Unequal or distorted outcomes

A. Bias

Bias in AI occurs when a system produces systematically unfair or inaccurate results for certain people or groups. It commonly arises from training data, design choices, labels, or the environment in which the system is used.

• Biased data: Historical data may reflect existing discrimination. If a hiring dataset contains mostly successful male applicants because earlier managers favored men, a model trained on it may learn patterns associated with male applicants rather than genuine ability.
• Underrepresentation: Facial-recognition systems trained mainly on lighter-skinned faces may perform less accurately on darker-skinned faces. A lower accuracy rate for one group can create unequal risks in policing or identity verification.
• Label bias: Human judgments used as labels may be subjective. Predicting “risk of reoffending” from arrest records can reproduce unequal policing patterns because an arrest is not identical to actual criminal behavior.
• Measurement bias: A system may use an unsuitable proxy. Health-care algorithms that use past medical spending as a measure of medical need can underestimate need in communities that historically received less care.
• Fairness measures: Statistical parity, equal opportunity, and calibration measure different aspects of fairness. Improving one measure can sometimes reduce another, so the relevant social purpose must be stated clearly.
• Reduction of bias: Useful measures include representative datasets, subgroup testing, documentation of data sources, fairness audits, and human review of high-impact decisions. Removing sensitive attributes alone does not guarantee fairness because other variables may act as proxies.

III. Surveillance — Monitoring through AI

A. Surveillance

AI surveillance uses cameras, sensors, biometric systems, online monitoring, or data analysis to observe and infer information about people. Its ethical acceptability depends on necessity, legality, proportionality, and control over collected data.

• Facial recognition: A camera can compare a face with a watchlist. A false match may lead to questioning, detention, or denial of access, especially when the system is used in public spaces without meaningful consent.
• Predictive monitoring: AI can combine location data, purchases, messages, or browsing behavior to infer habits. A prediction about suspicious activity is an inference, not proof of wrongdoing.
• Privacy loss: Continuous collection can reveal religious attendance, medical visits, political meetings, or personal relationships. Even individually harmless data can become sensitive when combined.
• Consent and purpose limitation: People should know what data is collected, why it is needed, how long it is retained, and who can access it. Using workplace attendance data later to evaluate political activity violates the original purpose.
• Power imbalance: Employees, students, migrants, and citizens may be unable to refuse monitoring by employers, schools, border agencies, or governments.
• Safeguards: Strong safeguards include data minimization, encryption, short retention periods, access logs, independent authorization, accuracy testing, and a clear right to challenge automated surveillance decisions.

IV. Job loss — Automation and changing work

A. Job loss

AI can replace some tasks, reduce demand for certain occupations, and transform nearly all workplaces. The central ethical issue is not only whether jobs disappear, but who receives the benefits and who bears the costs.

• Task automation: A language model may draft routine reports, while software may classify invoices or schedule deliveries. Automation of a task does not always eliminate an entire occupation, but it can reduce the number of workers required.
• Unequal impact: Routine clerical, customer-service, transport, and production roles may face greater automation pressure than jobs requiring complex physical work, social trust, or unusual judgment.
• Deskilling and surveillance: AI productivity tools can narrow workers’ discretion and measure keystrokes, response times, or delivery routes. Workers may become responsible for decisions while having little control over the system.
• Distribution of gains: Companies may gain efficiency while workers experience layoffs, lower wages, or insecure contract work. Ethical deployment requires considering affected communities, not only shareholder returns.
• Human transition: Reskilling, paid training, redeployment, unemployment protection, and consultation can reduce harm. A worker displaced by an automated claims system should have access to training for roles such as exception handling or customer support.
• Human-centered automation: AI should augment human abilities where possible. In aviation, medicine, or infrastructure control, qualified people must retain authority to intervene rather than merely approve machine outputs.

V. Misinformation — False or misleading AI-generated content

A. Misinformation

Misinformation is false or misleading information, whether shared intentionally or unintentionally. AI increases its scale by generating convincing text, images, audio, and video rapidly; deliberately deceptive content is often called disinformation.

• Deepfakes: Synthetic audio can imitate a public figure’s voice, while synthetic video can place a person’s face in a fabricated scene. The danger increases during elections, emergencies, or financial transactions.
• Automated scale: A single system can produce thousands of personalized messages. This makes false claims harder to identify and allows attackers to target different communities with different narratives.
• False authority: Fluent language, realistic images, or fabricated citations can make incorrect output appear reliable. A confident AI answer is not evidence that its claim is true.
• Harms: Misinformation can cause panic, damage reputations, influence voting, promote unsafe medical practices, or enable fraud. A fake emergency announcement may cause people to ignore genuine warnings later.
• Detection limits: Watermarks, provenance records, platform labeling, and forensic tools can help, but detection is imperfect because generation methods evolve quickly.
• Responsible response: Users should verify important claims through independent, authoritative sources, while platforms and developers should slow mass abuse, preserve evidence, label synthetic media where appropriate, and provide correction mechanisms.

VI. AI and human rights — Protecting dignity and freedom

A. AI and human rights

AI must operate consistently with human rights, including privacy, equality, freedom of expression, due process, safety, and human dignity. Human rights provide limits on what may be automated even when automation is technically possible.

• Right to privacy: Collecting biometric or behavioral data without a lawful and necessary purpose can interfere with private life. Privacy requires more than hiding data; it includes control over how personal information is used.
• Equality and non-discrimination: An automated decision about credit, employment, housing, education, or welfare should not disadvantage people because of protected characteristics or unjustified proxies.
• Due process: A person affected by an AI decision should receive understandable reasons, notice of the decision, access to relevant evidence, and a meaningful opportunity to appeal. “The algorithm decided” is not an adequate explanation.
• Freedom of expression: Content-ranking and moderation systems can shape whose speech is seen. Errors may silence legitimate political criticism, while weak moderation may amplify threats or harassment.
• Safety and dignity: Medical, legal, and public-sector AI can affect life-changing outcomes. Human review is essential when errors may cause physical injury, loss of liberty, homelessness, or denial of essential services.
• Rights-based design: Impact assessments should identify affected groups before deployment. High-risk systems need testing, monitoring, incident reporting, and the possibility of suspension when harms exceed acceptable limits.

VII. Importance of responsible AI use — Trustworthy deployment

A. Importance of responsible AI use

Responsible AI use is important because AI decisions can operate at large scale, remain difficult to inspect, and affect people who never chose to interact with the system. Ethical principles must therefore be converted into operational practices.

• Accuracy and safety: A medical diagnostic model should be tested on data resembling its real users, with sensitivity and specificity reported for relevant groups. Accuracy alone does not show whether errors are safe.
• Transparency: Organizations should document the system’s purpose, data sources, limitations, performance, and responsible personnel. A model card or system report can make these details reviewable.
• Explainability: Explanations should match the decision’s stakes. A rejected loan applicant needs meaningful factors and an appeal route, not merely a technical description of neural-network layers.
• Accountability: Roles should be assigned across developers, deployers, managers, and regulators. Audit logs can record the input, model version, output, human override, and final action.
• Privacy and security: Data should be minimized, access-controlled, encrypted, and deleted when no longer necessary. Security testing should consider prompt injection, data leakage, model theft, and unauthorized use.
• Human oversight: People must be trained to recognize automation bias—the tendency to accept a machine recommendation uncritically. Oversight is real only when the reviewer has time, authority, and information to disagree.
• Continuous monitoring: Ethical review continues after launch. A system may become unfair when populations, laws, data patterns, or social conditions change, so organizations need complaint channels, audits, and incident-response procedures.

VIII. Real-life problems caused by unethical AI — Concrete consequences

A. Real-life problems caused by unethical AI

Real-world cases show that unethical AI produces legal, financial, emotional, and physical harms rather than merely theoretical concerns.

• Hiring discrimination: Amazon discontinued an experimental recruiting tool after it learned patterns from historical resumes that favored male applicants for technical roles. The case illustrates how past workforce imbalance can become an automated disadvantage.
• Facial-recognition error: Robert Williams was wrongfully arrested in Detroit in 2020 after a facial-recognition match identified him as a suspect. The incident demonstrates why a probabilistic match cannot be treated as conclusive evidence.
• Risk-assessment concerns: The COMPAS system used in U.S. criminal-justice settings became part of a public debate after reporting alleged different error patterns across racial groups. The lesson is that fairness depends on the chosen definition, data, context, and consequences of errors.
• Welfare and public services: Automated fraud-detection systems may classify benefit recipients as suspicious using incomplete or incorrect data. A mistaken suspension can remove food, housing, or medical support before a person can appeal.
• Misinformation and fraud: Voice-cloning tools can imitate relatives or executives in urgent payment requests. A short fabricated audio message may cause financial loss because listeners trust a familiar voice.
• Health-care inequality: If training data reflects unequal access to treatment, an AI system may rank some patients as less urgent even when their medical need is comparable. This can delay diagnosis or treatment.
• Surveillance harms: Mass biometric monitoring can chill protest and association. People may avoid lawful meetings when they believe attendance will be recorded and linked to their identity.
• Accountability lesson: Each case involves more than a faulty algorithm: organizations selected the data, defined the objective, deployed the system, and decided how much human review was sufficient. Ethical responsibility therefore belongs to the complete sociotechnical system, not only the software.