Unit 5: Ethics and Challenges of AI - Subjective Questions

Ethical bias in AI refers to systematic and unfair preferences or disadvantages produced by an AI system toward certain individuals or groups.

Bias can enter at several stages:

• Data collection: Training data may underrepresent particular communities or contain historical discrimination.
• Data labeling: Human labelers may apply inconsistent or prejudiced judgments.
• Algorithm design: The choice of features, objectives, or performance measures may favor one group.
• Model training: An algorithm may learn patterns that reflect existing social inequalities.
• Deployment: A system may be used in a context different from the one for which it was designed.
• Feedback loops: Biased predictions may influence future data and reinforce the original bias.

Therefore, bias is not only a technical issue; it is also a social and organizational issue requiring continuous monitoring and human oversight.

The major causes of bias in AI include:

• Historical bias: Past decisions may reflect discrimination based on gender, race, caste, disability, or socioeconomic status.
• Representation bias: Some groups may be insufficiently represented in the training data.
• Measurement bias: The selected features may not accurately represent the quality or behavior being predicted.
• Algorithmic bias: Model design choices may produce unequal outcomes.
• Evaluation bias: Testing a system on a narrow population may hide poor performance for other groups.

The consequences include:

• Unfair rejection of loan, employment, or educational applications.
• Incorrect medical diagnosis for underrepresented populations.
• Discriminatory policing or facial recognition outcomes.
• Loss of public trust in AI systems.
• Legal violations and damage to human dignity.

Bias can be reduced through diverse datasets, fairness testing, independent audits, transparency, and meaningful human review.

AI-powered surveillance uses technologies such as facial recognition, biometric analysis, location tracking, and behavior prediction to monitor people. Although it may support public safety, it creates serious ethical concerns:

• Loss of privacy: Individuals may be monitored without their knowledge or consent.
• Mass surveillance: Continuous monitoring can affect entire populations rather than specific suspects.
• Lack of consent: People may not know how their images, voices, or movements are being collected.
• Misidentification: Facial recognition systems can wrongly identify innocent people.
• Chilling effect: People may avoid lawful speech, protests, or associations because they feel watched.
• Power imbalance: Governments or organizations may use surveillance to control vulnerable communities.
• Data misuse: Collected information may be shared, sold, or exposed through security breaches.

Ethical surveillance requires necessity, proportionality, clear laws, limited data retention, transparency, and independent oversight.

Legitimate AI surveillance and unethical mass surveillance differ in purpose, scope, accountability, and safeguards.

Ethical surveillance must be necessary, proportionate, accurate, secure, and subject to human rights protections.

AI can automate routine physical and cognitive tasks in areas such as manufacturing, transportation, customer service, data entry, and financial analysis. As a result, some existing jobs may disappear or require fewer workers.

The wider social effects include:

• Unemployment and income loss for workers whose tasks are automated.
• Skill mismatch when workers do not have the abilities required for new AI-related jobs.
• Wage inequality if highly skilled workers benefit while others face reduced opportunities.
• Regional inequality when automation affects particular industries or locations.
• Workplace stress caused by constant performance monitoring or fear of replacement.
• Creation of new jobs in AI development, system maintenance, data governance, and human-centered services.

AI does not affect all jobs equally. Ethical adoption should include reskilling, fair transition policies, worker participation, social protection, and the redesign of jobs so that humans and AI complement one another.

Job displacement occurs when an AI system performs tasks previously completed by human workers, reducing the need for certain roles. For example, automated checkout systems may reduce the demand for some cashier positions.

Job transformation occurs when AI changes the tasks, skills, or responsibilities within an existing occupation. For example, a doctor may use an AI diagnostic tool but still interpret results, communicate with patients, and make final decisions.

Job creation occurs when new occupations or industries emerge because of AI. Examples include AI auditors, data curators, model safety specialists, and prompt or workflow designers.

These effects may occur together. A responsible approach should:

• Provide affordable reskilling and lifelong learning.
• Protect workers from unsafe or unfair monitoring.
• Involve employees in workplace AI decisions.
• Share productivity gains fairly.
• Preserve meaningful human roles where judgment, empathy, and accountability are essential.

AI contributes to false or misleading information in several ways:

• Generative AI: It can create realistic but false text, images, audio, and videos.
• Automated content production: Large volumes of misleading material can be produced quickly and cheaply.
• Personalized targeting: Recommendation systems may show users content that confirms their existing beliefs.
• Bots and fake accounts: Automated accounts can amplify false claims and create the appearance of public support.
• Synthetic media: Deepfakes can falsely show a person saying or doing something.
• Hallucinated content: AI systems may confidently produce inaccurate information.

Misinformation is false information shared without necessarily intending to deceive, whereas disinformation is deliberately created or distributed to mislead. Both can damage elections, public health, social harmony, and trust in reliable institutions.

Deepfakes are synthetic audio, images, or videos generated or modified using AI. They can produce serious social and political effects:

• Election manipulation: Fake statements by candidates may influence voters.
• Damage to reputation: Individuals may appear to participate in activities that never occurred.
• Public confusion: People may become unable to distinguish authentic evidence from fabricated media.
• Extortion and harassment: Deepfake intimate content can be used to threaten or humiliate victims.
• Diplomatic conflict: Fabricated statements may create tension between countries.
• Decline of trust: Genuine recordings may be dismissed as fake, creating a so-called liar's dividend.

Possible safeguards include provenance labels, cryptographic authentication, media literacy, rapid fact-checking, platform accountability, and laws that protect victims while respecting freedom of expression.

Human rights provide an ethical and legal framework for designing and using AI. AI systems can support rights, but they can also threaten them.

Important rights affected by AI include:

• Right to privacy: Threatened by unauthorized data collection and surveillance.
• Right to equality: Threatened by biased decisions.
• Right to freedom of expression: Affected by automated censorship or manipulation.
• Right to work: Affected by automation and unfair employment practices.
• Right to health: Influenced by the safety and accuracy of medical AI.
• Right to due process: Threatened when people cannot challenge automated decisions.
• Right to human dignity: Harmed when people are reduced to scores or predictions.

A human-rights-based AI approach requires necessity, proportionality, fairness, transparency, accountability, privacy protection, accessibility, and effective remedies for affected individuals.

Responsible AI means developing and using AI in ways that benefit people while minimizing harm. Important principles include:

• Fairness: Systems should avoid unjust discrimination and provide equitable outcomes.
• Transparency: People should receive understandable information about how significant decisions are made.
• Explainability: Decisions should be explainable at a level appropriate to the context.
• Accountability: Organizations and identifiable humans must be responsible for system outcomes.
• Privacy: Personal data should be collected lawfully, minimally, and securely.
• Safety and reliability: Systems should be tested, monitored, and designed to fail safely.
• Human oversight: Humans should retain authority over high-impact decisions.
• Inclusiveness: Systems should be accessible and designed with diverse communities.
• Sustainability: Energy use and environmental effects should be considered.

These principles must be implemented through policies, technical controls, audits, training, and continuous monitoring.

Transparency means providing information about an AI system's purpose, data, limitations, and operating conditions. Explainability means giving understandable reasons for a particular output or decision.

They are important because they:

• Help users determine whether a system is suitable for a task.
• Allow affected people to challenge errors or unfair decisions.
• Support audits and regulatory compliance.
• Increase trust when explanations are accurate and meaningful.
• Help developers identify unexpected behavior and bias.
• Clarify who is responsible for decisions.

For example, if an AI system rejects a loan application, the applicant should receive relevant reasons such as insufficient verified income rather than an unexplained score. However, explanations should not expose sensitive security information or create a false impression that a complex system is perfectly understandable. Transparency must be combined with privacy, security, and accountability.

Privacy and data protection ensure that information about people is collected and used respectfully and securely. They are essential because AI systems often require large datasets containing personal, sensitive, or inferred information.

Important practices include:

• Purpose limitation: Collect data for a specific and legitimate purpose.
• Data minimization: Use only the information necessary for that purpose.
• Informed consent: Explain collection and use in understandable language where consent is required.
• Access and correction: Allow individuals to view and correct relevant data.
• Security: Protect data from unauthorized access, loss, and misuse.
• Retention limits: Delete information when it is no longer necessary.
• Anonymization or pseudonymization: Reduce direct identification risks.
• Privacy impact assessments: Identify risks before deployment.

Privacy protection should be built into the complete AI lifecycle rather than added after a system has been released.

Human oversight means that qualified people can understand, supervise, intervene in, and if necessary stop an AI system. It is especially important in high-risk areas such as healthcare, criminal justice, employment, education, credit, and critical infrastructure.

Human oversight is necessary because:

• AI may produce inaccurate or biased results.
• A model may encounter situations not represented in its training data.
• Context, compassion, and moral judgment may be required.
• Affected individuals need a way to appeal decisions.
• Responsibility must not be hidden behind an automated system.

Effective oversight requires trained operators, clear authority to override the system, meaningful explanations, performance monitoring, escalation procedures, and documented accountability. Human involvement must be genuine; simply placing a person near a system without time, information, or authority is not adequate oversight.

A well-known problem is the misidentification of people by facial recognition systems, particularly when the training data contains limited representation of certain racial or gender groups. An incorrect match may lead to police questioning, arrest, loss of employment, or public humiliation.

The ethical problems include:

• Discrimination: Error rates may differ significantly between demographic groups.
• Violation of due process: A computer-generated match may be treated as evidence without sufficient verification.
• Lack of consent: People may be scanned in public without meaningful choice.
• Irreversible harm: A false accusation can damage a person's reputation and safety.

Remedies include:

• Testing accuracy separately across demographic groups.
• Improving training data and independent evaluation.
• Requiring human verification and corroborating evidence.
• Restricting use in high-risk contexts.
• Providing notice, appeal procedures, and compensation for harm.
• Conducting regular audits and publishing performance results.

An AI hiring system may learn from historical recruitment records. If previous hiring decisions favored a particular gender, social group, institution, or employment history, the model may reproduce those patterns. It may also use proxy variables, such as postcode or career gaps, that indirectly represent protected characteristics.

Possible harms include:

• Qualified candidates being rejected unfairly.
• Existing workplace inequality being reinforced.
• Applicants being unable to understand or challenge decisions.
• Privacy violations caused by excessive personal data collection.

Ethical improvements include:

• Reviewing training data for historical and representation bias.
• Removing unjustified proxy features.
• Testing selection rates and error rates across groups.
• Using job-related and validated criteria.
• Informing applicants when AI is used.
• Providing human review and an appeal process.
• Auditing the system throughout deployment.

AI should assist recruitment professionals rather than make unreviewable final decisions about people's livelihoods.

AI in healthcare can assist diagnosis, treatment planning, and patient monitoring, but errors may cause serious physical and emotional harm. Problems can occur when training data excludes certain populations, medical records contain errors, or clinicians rely too heavily on model outputs.

Potential harms include:

• Misdiagnosis or delayed diagnosis.
• Unequal quality of care for underrepresented patients.
• Privacy breaches involving sensitive health data.
• Incorrect treatment recommendations.
• Lack of explanation for decisions affecting patients.

Safeguards include:

• Clinical validation using diverse and representative datasets.
• Independent testing before deployment.
• Continuous monitoring for performance changes.
• Human review by qualified healthcare professionals.
• Clear communication of limitations and uncertainty.
• Strong cybersecurity and privacy controls.
• Informed patient communication where appropriate.
• Incident reporting, correction, and accountability procedures.

Medical AI should support professional judgment rather than replace responsibility for patient care.

An AI system that provides decisions without understandable reasons is often described as a black-box system. This creates ethical challenges, especially when decisions affect rights, opportunities, or safety.

The main challenges are:

• Accountability gap: It becomes difficult to identify who is responsible for an outcome.
• Limited contestability: Affected people may be unable to challenge an incorrect decision.
• Hidden bias: Unfair patterns may remain undetected.
• Loss of professional judgment: Users may accept an output without questioning it.
• Reduced trust: People may reject useful systems when they cannot understand them.

Possible responses include using interpretable models where practical, providing post-hoc explanations carefully, documenting data and design choices, conducting independent audits, maintaining human review, and giving affected people access to appeal and correction mechanisms. In high-impact contexts, accuracy alone is not sufficient; legitimacy and accountability are also required.

An AI impact assessment is a structured process for identifying, evaluating, and reducing the possible effects of an AI system on people, society, and the environment before and during deployment.

Major steps include:

1. Define the purpose: State what the system will do and why it is needed.
2. Identify stakeholders: Consider users, affected individuals, vulnerable groups, workers, and the wider public.
3. Map data and processes: Document data sources, model inputs, outputs, and decision pathways.
4. Identify risks: Examine bias, privacy, safety, security, misinformation, labor, and human rights risks.
5. Evaluate severity and likelihood: Consider who may be harmed, how seriously, and how often.
6. Plan mitigations: Use data improvement, access controls, human review, testing, and restrictions.
7. Consult stakeholders: Obtain feedback from experts and affected communities.
8. Monitor after deployment: Track incidents, performance, complaints, and changing risks.

The assessment should be updated whenever the system, data, or use context changes.

Responsible AI requires shared responsibility because no single group controls every stage of an AI system.

• Developers: Should use suitable data, test for bias, document limitations, protect security, design for safety, and report known risks.
• Organizations: Should define legitimate purposes, provide resources for governance, train staff, conduct impact assessments, monitor systems, and provide remedies.
• Governments and regulators: Should establish clear standards, protect fundamental rights, support independent oversight, enforce accountability, and regulate high-risk uses.
• Users and professionals: Should understand system limitations, avoid blind reliance, protect personal data, verify important outputs, and report harmful behavior.
• Affected communities: Should have opportunities to participate in design, consultation, and complaint processes.

Cooperation among these groups is essential because technical safeguards alone cannot solve social, legal, and ethical problems.

Large-scale AI systems may require substantial computing power for training and operation. This can consume electricity, water for cooling, and raw materials used in hardware production.

Environmental and social concerns include:

• Energy consumption: Training and operating models may contribute to greenhouse gas emissions, especially when electricity comes from fossil fuels.
• Water use: Data centers may consume water for cooling in regions where water is scarce.
• Electronic waste: Frequent hardware replacement can create toxic waste.
• Resource extraction: Mining materials for servers can harm ecosystems and communities.
• Unequal distribution of benefits: Companies or wealthy regions may gain advantages while local communities bear environmental costs.
• Labor concerns: Data preparation and content moderation may involve low-paid workers exposed to disturbing material.

Responsible development should measure environmental impact, improve energy efficiency, use renewable energy where possible, extend hardware life, protect workers, and disclose relevant sustainability information.