Unit 6: AI for Intelligent Systems and Robotics - Subjective Questions

Computer vision is a branch of artificial intelligence that enables computers and robots to acquire, process, interpret, and understand information from images and videos.

Importance in intelligent systems and robotics:

• It allows robots to perceive their surroundings through cameras and other visual sensors.
• It supports object recognition, obstacle detection, navigation, and human-robot interaction.
• It enables intelligent systems to inspect products, analyze medical images, monitor environments, and recognize activities.
• In robotics, computer vision converts raw visual data into useful information for decision-making and control.

A typical computer vision pipeline includes image acquisition, preprocessing, feature extraction or representation, classification or detection, and action generation.

Image classification assigns an input image to one or more predefined categories.

General process:

• Capture an image using a camera or sensor.
• Preprocess the image by resizing, normalizing, or reducing noise.
• Extract visual features or learn them automatically using a deep neural network.
• Pass the features to a classifier.
• Produce a class label and, often, a confidence score.

For example, a robotic system may classify an image as containing a human, a vehicle, a table, or an obstacle. Applications include identifying objects for grasping, recognizing road signs in autonomous vehicles, sorting materials, detecting defects, and identifying safe landing or movement areas.

Object detection identifies the objects present in an image and determines their locations, usually by drawing bounding boxes around them. It generally produces an object class and a confidence value for each detected region.

Difference from image classification:

• Image classification assigns a label to the entire image.
• Object detection identifies multiple objects and localizes each one.
• Classification answers what is in the image, whereas detection answers what and where.

Object detection is useful in robotics because a robot must know both the identity and position of objects. It supports collision avoidance, object picking, human tracking, navigation, surveillance, and interaction with objects in changing environments.

Computer vision systems used by robots face several practical challenges:

• Lighting variation: Changes in brightness, shadows, and reflections can alter the appearance of objects.
• Occlusion: An object may be partly hidden by another object.
• Viewpoint changes: Objects look different when observed from different angles or distances.
• Background complexity: Similar colors or cluttered scenes can make segmentation difficult.
• Real-time requirements: Robots must process visual data quickly enough to respond safely.
• Sensor limitations: Cameras may suffer from noise, low resolution, or restricted fields of view.
• Generalization: A model trained in one environment may perform poorly in a new environment.

These challenges can be addressed using diverse training data, data augmentation, sensor fusion, efficient models, and continuous evaluation in realistic conditions.

Natural Language Processing (NLP) is the area of AI concerned with enabling computers to understand, process, generate, and respond to human language.

Important NLP tasks include:

• Tokenization and text preprocessing.
• Speech recognition and conversion of speech into text.
• Part-of-speech tagging and syntactic analysis.
• Named entity recognition.
• Sentiment and intent analysis.
• Machine translation and text generation.

In intelligent systems and robotics, NLP provides a natural communication interface. Users can give spoken or written instructions, ask questions, request information, or control a robot using ordinary language. NLP therefore improves accessibility, usability, and human-robot collaboration.

A chatbot for a robot usually contains the following components:

• Input interface: Receives text or speech from the user.
• Speech recognition: Converts spoken language into text when necessary.
• Natural language understanding: Identifies the user's intent and extracts important entities or parameters.
• Dialogue manager: Maintains conversation context and selects the next response or action.
• Knowledge base or service layer: Retrieves information or accesses external services.
• Natural language generation: Produces a suitable response.
• Speech synthesis: Converts the response into spoken language.
• Robot control interface: Converts commands into physical actions.

For example, a user may say, Move to the kitchen. The system recognizes the navigation intent, identifies the destination, checks the robot's map, and sends a movement command while reporting progress to the user.

Rule-based chatbots:

• Use predefined rules, keywords, and decision trees.
• Are predictable and easy to test.
• Work well for limited and clearly defined tasks.
• Fail when users use unexpected wording or ask questions outside the rules.

AI-based conversational systems:

• Use machine learning and NLP models to interpret varied language.
• Can identify intents, maintain context, and generate flexible responses.
• May improve through training and interaction data.
• Can produce incorrect, ambiguous, or unsafe responses if not properly controlled.

For simple robot commands, rule-based methods may be sufficient. For service robots that support open-ended conversation, AI-based systems are more capable. Practical systems often combine both approaches to obtain flexibility together with safety and reliability.

Conversational AI in robots has several limitations and safety concerns:

• Ambiguity: Human language may have multiple interpretations.
• Context failure: The system may misunderstand references, previous statements, or user intentions.
• Speech difficulties: Noise, accents, dialects, and poor microphones can reduce recognition accuracy.
• Incorrect responses: A language model may provide factually wrong or unsuitable information.
• Privacy risks: Voice recordings and personal conversations may expose sensitive data.
• Unauthorized commands: A robot may execute instructions from an untrusted person.
• Bias: Training data may cause unfair or culturally inappropriate responses.
• Physical consequences: A misunderstood command can lead to unsafe movement.

Safety measures include user authentication, confirmation for critical actions, restricted command sets, logging, privacy protection, human supervision, and emergency stop mechanisms.

Reinforcement learning (RL) is a machine learning approach in which an agent learns to make decisions by interacting with an environment and receiving feedback.

• Agent: The learner or decision-maker, such as a mobile robot.
• State: A representation of the current situation, such as the robot's position, sensor readings, and battery level.
• Action: A choice made by the agent, such as moving forward, turning, or grasping an object.
• Reward: A numerical feedback signal indicating how desirable an outcome is.
• Environment: The world in which the agent operates.

The robot tries to learn a policy that selects actions producing high long-term reward. For example, reaching a destination may generate a positive reward, while hitting an obstacle may generate a large negative reward.

The exploration-exploitation trade-off concerns how an agent chooses between trying unfamiliar actions and using actions that are already known to work.

• Exploration: The robot tries new actions or routes to discover potentially better strategies.
• Exploitation: The robot selects the best-known action to obtain a high immediate or expected reward.

For example, a delivery robot may exploit a familiar route that is usually fast. However, it may explore an alternative route to discover that it is safer or less congested. Excessive exploration can be dangerous or inefficient, while excessive exploitation can prevent the robot from finding better solutions.

A common strategy is an -greedy policy. With probability , the robot explores, and with probability , it chooses the currently best-known action.

Q-learning estimates the value of taking action in state . This value is represented by and indicates the expected long-term return.

The basic update rule is:

where:

• is the reward received after taking action .
• is the new state.
• represents a possible next action.
• is the learning rate.
• is the discount factor.
• is the best estimated future value.

The term inside brackets is the temporal-difference error. The robot repeatedly updates its estimates after interactions and eventually selects actions with high Q-values. For example, a navigation robot can learn which movement is most useful in each location while avoiding actions that lead to collisions.

Reinforcement learning can train an autonomous robot to select navigation actions based on its observations.

Application procedure:

• Define the state using location, orientation, obstacle distances, target direction, and velocity.
• Define actions such as moving forward, turning left, turning right, or stopping.
• Design rewards that encourage reaching the goal and discourage collisions, excessive energy use, and unnecessary delay.
• Allow the robot to interact with a simulation or physical environment.
• Update the policy or value function after each action.
• Test the learned policy under different maps and conditions.

A successful policy should reach destinations efficiently while maintaining safety. Training in simulation followed by controlled real-world testing is often preferred because it reduces physical damage and improves learning efficiency.

An autonomous robot is a machine that can perceive its environment, make decisions, and perform tasks with limited or no continuous human control.

Main components include:

• Sensors: Cameras, lidar, ultrasonic sensors, GPS, inertial sensors, and touch sensors.
• Perception: Interprets sensor data and identifies objects, obstacles, and environmental conditions.
• Localization and mapping: Estimates the robot's position and represents its surroundings.
• Planning: Selects paths, tasks, and actions to achieve goals.
• Control: Converts plans into motor commands.
• Learning and adaptation: Improves behavior when conditions change.
• Communication and supervision: Supports human interaction and status reporting.
• Safety systems: Detects failures, limits dangerous actions, and enables emergency stopping.

Autonomy results from the coordinated operation of these components rather than from a single AI algorithm.

Sensor fusion combines information from multiple sensors to produce a more accurate and reliable estimate of the robot's environment or internal state.

A camera provides rich visual information but may be affected by darkness or shadows. Lidar provides accurate distance measurements but may not identify object appearance. GPS provides global position outdoors but may be unavailable indoors. Combining these sensors compensates for their individual weaknesses.

Benefits of sensor fusion:

• Improved localization and mapping.
• Better obstacle detection.
• Greater robustness under changing environmental conditions.
• Reduced uncertainty and sensor-specific errors.
• More reliable decision-making.

Fusion may be performed at the raw-data level, feature level, or decision level. The selected method depends on computational resources, sensor characteristics, and task requirements.

An intelligent robot can integrate the three technologies as follows:

• Computer vision interprets camera images to identify objects, people, landmarks, and obstacles.
• NLP interprets human instructions and converts them into goals or tasks.
• Reinforcement learning selects actions and improves behavior through feedback.

For example, if a user says, Bring me the red cup, NLP identifies the task and object description. Computer vision searches the environment for a red cup and estimates its position. The planning and control system moves the robot toward the object, while reinforcement learning can help it select safe movement and grasping actions. Feedback from sensors and task success updates the robot's behavior.

Integration requires a common representation of goals, reliable communication between modules, real-time processing, and safety checks before physical actions are executed.

Localization determines where the robot is located within an environment. Mapping creates or uses a representation of that environment, including walls, objects, and free spaces. Path planning determines a safe and efficient route from the current position to a desired destination.

These functions are closely related:

• A robot requires a map to plan meaningful routes.
• It requires localization to know its position on the map.
• It must update both estimates when new sensor information becomes available.

Common planning methods include graph search, potential fields, sampling-based planning, and optimization-based methods. Reliable localization, mapping, and planning allow robots to navigate without collisions, respond to changes, and reach goals efficiently in indoor, outdoor, and industrial environments.

Supervised learning:

• Uses labeled examples containing inputs and desired outputs.
• Can train a robot to classify objects or recognize gestures.
• Requires a suitable and sufficiently large labeled dataset.

Unsupervised learning:

• Uses data without predefined labels.
• Can discover clusters, patterns, or unusual sensor behavior.
• Helps robots organize observations and detect anomalies.

Reinforcement learning:

• Learns through interaction and reward feedback.
• Is suitable for sequential decisions such as navigation, manipulation, and control.
• May require extensive exploration and careful reward design.

A robot may use all three methods together: supervised learning for perception, unsupervised learning for discovering patterns, and reinforcement learning for selecting actions.

AI-powered robots create benefits but also raise important concerns:

• Safety and accountability: Responsibility must be established when an autonomous robot causes harm.
• Privacy: Robots with cameras and microphones can collect personal information.
• Bias and fairness: Training data may lead to unequal treatment of people or environments.
• Employment: Automation may replace or significantly change certain jobs.
• Security: Connected robots can be hacked or manipulated.
• Transparency: Users may need explanations for important decisions.
• Human dignity and autonomy: Robots should support people without inappropriate surveillance or manipulation.

Responsible development requires privacy by design, security testing, human oversight, transparent policies, unbiased evaluation, clear regulations, and mechanisms for reporting and correcting failures.

Sim-to-real transfer is the process of transferring a robot policy or model learned in a simulated environment to a physical robot.

Importance:

• Simulation allows rapid training without damaging expensive hardware.
• Large numbers of experiences can be generated at low cost.
• Dangerous situations can be tested safely.
• Different environments, objects, and weather conditions can be created easily.

However, simulations do not perfectly represent the real world. Differences in friction, lighting, sensor noise, motor response, and object behavior create a reality gap.

Methods for reducing the gap include:

• Domain randomization.
• Accurate physical modeling.
• Adding realistic sensor noise.
• Fine-tuning with real-world data.
• Using adaptive and robust control methods.

Effective transfer improves the safety, speed, and cost-efficiency of robot learning.

Important future trends include:

• Embodied AI: Systems that learn through interaction between perception, reasoning, and physical action.
• Multimodal models: Models that combine vision, language, audio, and sensor data.
• Collaborative robots: Robots designed to work safely and naturally with humans.
• Edge AI: Processing data locally on robots to reduce delay and protect privacy.
• Continual learning: Robots that adapt to new tasks and environments without forgetting previous skills.
• Swarm robotics: Groups of robots coordinating to perform complex tasks.
• Explainable and trustworthy AI: Systems that provide understandable reasons for decisions.
• Soft and bio-inspired robotics: Robots with flexible structures and adaptive movement.
• Cloud and robot learning platforms: Shared training and knowledge across multiple robots.

These trends aim to make robots more capable, adaptable, efficient, safe, and useful in real-world environments.