Unit 6: Intelligent Robotics and Applications - Subjective Questions

Machine vision is the use of cameras, sensors, image-processing algorithms, and computing systems to enable a robot to acquire and interpret visual information about its environment.

The major stages are:

• Image acquisition: A camera or vision sensor captures an image of the workspace. Proper illumination and lens selection are important.
• Preprocessing: Noise removal, contrast enhancement, filtering, and image normalization improve image quality.
• Segmentation: The object of interest is separated from the background using thresholding, edge detection, or region-based methods.
• Feature extraction: Features such as edges, corners, color, texture, shape, and key points are identified.
• Recognition and classification: The extracted features are used to identify or classify objects.
• Pose estimation: The position and orientation of the detected object are calculated.
• Decision and control: Visual information is supplied to the robot controller for tasks such as inspection, navigation, sorting, or pick-and-place.

Thus, machine vision acts as the robot's visual sensing and interpretation system.

2D machine vision represents a scene in terms of image coordinates, intensity, and color, while 3D machine vision also measures depth and surface geometry.

2D vision

• Uses conventional grayscale or color cameras.
• Processes features such as edges, contours, patterns, and colors.
• It is relatively inexpensive, fast, and easy to implement.
• It cannot directly determine depth and may perform poorly when objects overlap or their height varies.
• Applications include barcode reading, surface inspection, object counting, and planar pick-and-place.

3D vision

• Uses stereo cameras, structured light, time-of-flight cameras, or LiDAR.
• Produces point clouds or depth maps containing spatial information.
• It can estimate object dimensions, volume, distance, position, and orientation.
• It is more robust for randomly arranged objects and navigation in complex environments.
• It requires greater computation, calibration, and cost.
• Applications include bin picking, obstacle avoidance, autonomous navigation, and dimensional inspection.

Key comparison

• Data: 2D provides width and height; 3D provides width, height, and depth.
• Complexity: 2D processing is generally simpler than 3D processing.
• Lighting sensitivity: 2D systems often depend heavily on controlled lighting, whereas active 3D sensors may handle textureless objects better.
• Accuracy considerations: Both systems require calibration, but 3D systems additionally require accurate depth calibration and coordinate transformation.

The choice depends on task complexity, required accuracy, environmental conditions, and cost.

Artificial intelligence enables robots to make decisions, learn from data, and operate in uncertain or changing environments.

The perception-planning-action cycle consists of:

• Perception: The robot collects data using cameras, LiDAR, encoders, microphones, and force sensors. AI methods identify objects, speech, obstacles, and environmental conditions.
• State estimation: Sensor data is combined to estimate the robot's position and the state of its surroundings.
• Planning: AI selects a goal, generates a path, schedules actions, or determines an appropriate manipulation strategy.
• Action: Motors and actuators execute the selected plan through control commands.
• Feedback and learning: The outcome is measured and used to correct errors or improve future decisions.

AI techniques used in robotics include:

• Machine learning and deep learning
• Computer vision
• Natural-language processing
• Knowledge representation and reasoning
• Search and optimization
• Reinforcement learning

For example, a warehouse robot detects a package, identifies its location, plans a collision-free path, grasps it, and updates its model if the grasp fails.

Supervised learning

A model learns a mapping from input data to known labels or target outputs.

• Robotic applications: Object recognition, defect classification, grasp-quality prediction, and human-gesture recognition.
• Advantages: It can provide high accuracy when sufficient labeled data are available.
• Limitations: Data labeling is expensive, and performance may decrease in conditions not represented in the training set.

Unsupervised learning

A model discovers structures or patterns in unlabeled data.

• Robotic applications: Sensor-data clustering, anomaly detection, terrain grouping, and discovering object categories.
• Advantages: It reduces dependence on labeled datasets and may reveal unknown patterns.
• Limitations: Learned groups may not correspond to useful real-world categories, and evaluation is difficult.

Reinforcement learning

An agent learns a policy by interacting with an environment and receiving rewards or penalties. Its objective is to maximize the expected cumulative return:

where is the reward and is the discount factor.

• Robotic applications: Grasping, locomotion, path planning, and adaptive control.
• Advantages: It can learn complex behavior without explicit programming of every action.
• Limitations: Physical training may be slow or unsafe, reward design is difficult, and learned policies may not generalize.

A practical intelligent robot may combine all three approaches and use simulation or transfer learning to reduce real-world training risks.

An autonomous mobile robot performs a task with limited human intervention by sensing its environment, estimating its state, planning actions, and controlling its motion.

Main architectural components

• Sensors: Cameras, LiDAR, ultrasonic sensors, wheel encoders, inertial measurement units, and GPS collect environmental and motion data.
• Perception: Raw sensor data are processed to detect obstacles, landmarks, free space, and objects.
• Localization: The robot estimates its pose relative to a map or reference frame.
• Mapping: A geometric, topological, semantic, or occupancy-grid representation of the environment is created.
• SLAM: Simultaneous localization and mapping allows the robot to build a map while estimating its own position within it.
• Path planning: A global planner finds a route from the initial position to the goal. Algorithms such as Dijkstra's algorithm, A*, and rapidly exploring random trees may be used.
• Local planning and obstacle avoidance: Sensor updates are used to respond to moving obstacles and deviations from the global route.
• Motion control: Linear and angular velocity commands are generated so that the robot follows the planned trajectory.
• Task planner: High-level goals are converted into ordered actions.
• Safety supervisor: Emergency stopping, speed limits, and fault monitoring override unsafe commands.

The components operate in a feedback loop. Reliable autonomy requires accurate sensors, robust software, real-time communication, and safe behavior under uncertainty.

• Teleoperated robot: A human directly controls most robot movements from a local or remote station. Examples include some bomb-disposal and underwater robots.
• Semi-autonomous robot: The human assigns goals or approves major decisions, while the robot independently performs functions such as stabilization, navigation, or obstacle avoidance.
• Fully autonomous robot: The robot senses, decides, and acts with minimal human intervention within its defined operating conditions.

The appropriate level of autonomy depends on:

• Task complexity and predictability
• Safety consequences of failure
• Quality and reliability of sensors and AI
• Communication delay and availability
• Legal and ethical requirements
• Need for human judgment or accountability
• Environmental uncertainty
• Cost and computational resources

Full autonomy is not always preferable. In medical, defense, and emergency applications, meaningful human supervision may be necessary even when the robot has advanced autonomous capabilities.

Robot communication allows sensors, controllers, actuators, other robots, human operators, and cloud platforms to exchange commands and data.

Important requirements

• Low latency: Essential for real-time control and collision avoidance.
• Reliability: Messages must reach the correct destination without corruption or unacceptable loss.
• Adequate bandwidth: Cameras and LiDAR require more bandwidth than simple sensor nodes.
• Determinism: Industrial control may require predictable message-delivery times.
• Scalability: The network should support many robots and devices.
• Security: Authentication, encryption, and access control prevent unauthorized operation.
• Interoperability: Devices from different manufacturers should exchange information through common interfaces.

Common communication methods

• CAN and serial communication: Frequently used within robots for sensor and actuator communication.
• Industrial Ethernet and fieldbus: Used for deterministic factory control.
• Wi-Fi and private cellular networks: Provide wireless connectivity for mobile robots.
• Bluetooth and Zigbee: Suitable for short-range or low-power applications.
• MQTT: A lightweight publish-subscribe protocol often used in IoT systems.
• Robot middleware: Platforms such as ROS provide message passing, services, and reusable software interfaces.

Protocol selection depends on range, data rate, latency, mobility, environmental conditions, and safety requirements.

Wired communication

• Provides high reliability, stable bandwidth, and relatively low latency.
• Is less affected by radio interference and is easier to secure physically.
• Restricts mobility and may suffer from cable wear, tangling, or connector failure.
• Is commonly used in fixed industrial robot cells and internal robot networks.

Wireless communication

• Supports mobile robots, flexible layouts, and remote monitoring.
• Is easier to extend across warehouses, farms, and outdoor areas.
• May experience interference, signal blockage, variable latency, packet loss, or cyberattacks.
• Is commonly implemented using Wi-Fi, cellular networks, Bluetooth, or specialized radio links.

Safe handling of failures

• Use message acknowledgments, checksums, retries, and sequence numbers.
• Provide redundant communication channels for critical systems.
• Monitor connection quality with heartbeat or watchdog messages.
• Store essential control functions locally rather than depending entirely on the cloud.
• Define a safe state, such as controlled stopping or reduced-speed operation, when communication is lost.
• Encrypt data and authenticate users and devices.

A robot should never continue a hazardous operation solely on the assumption that a delayed command will arrive.

The Internet of Robotic Things combines robotics, Internet of Things technologies, cloud or edge computing, communication networks, and artificial intelligence. Connected robots can share sensor data, coordinate actions, and access remote computing services.

Typical architecture

1. Physical layer: Robots, sensors, actuators, identification devices, and smart machines.
2. Connectivity layer: Wi-Fi, cellular networks, Ethernet, gateways, and messaging protocols.
3. Edge layer: Nearby processors perform low-latency filtering, inference, and control.
4. Cloud or platform layer: Data storage, large-scale analytics, fleet management, and model training.
5. Application layer: Dashboards, remote supervision, predictive maintenance, and task scheduling.

Benefits

• Centralized monitoring and fleet coordination
• Sharing of maps and learned knowledge
• Predictive maintenance based on operating data
• Remote software updates and diagnostics
• Improved resource utilization and scalability

Applications

• Connected warehouse fleets
• Smart factories
• Agricultural field robots
• Hospital service robots
• Inspection and disaster-response systems

Challenges

• Cybersecurity and unauthorized control
• Privacy of video, location, and user data
• Network delay and service interruption
• Interoperability between manufacturers
• Large data volumes and energy consumption
• Dependence on cloud infrastructure

Critical safety functions should remain available locally so that a network failure does not create an unsafe condition.

A collaborative robot, or cobot, is designed to perform tasks in a workspace shared with human workers, subject to a suitable risk assessment and protective measures.

Cobots

• Often use force, torque, proximity, or vision sensors.
• Commonly provide hand-guided teaching and intuitive programming.
• Usually operate at controlled speeds and forces near people.
• Are suitable for flexible, low-volume, or frequently changing tasks.
• Common applications include assembly assistance, machine tending, inspection, and packaging.

Conventional industrial robots

• Are optimized for high speed, payload, precision, and repeated production.
• Commonly operate inside guarded cells separated from workers.
• Usually require specialized programming and fixed layouts.
• Are suitable for welding, painting, high-speed handling, and heavy material movement.

A cobot is not automatically safe in every application. Its tools, payload, workpiece edges, speed, and surrounding machinery may still create crushing, impact, cutting, or trapping hazards.

Principal collaborative modes

• Safety-rated monitored stop: The robot stops when a person enters the collaborative space and resumes only when safe conditions are restored.
• Hand guiding: An operator directly guides the robot using a suitable control device and enabling mechanism.
• Speed and separation monitoring: The robot adjusts its speed or stops according to the measured distance from a person.
• Power and force limiting: Robot design and control restrict impact energy, force, pressure, and torque.

Risk-assessment procedure

1. Define the application: Specify the robot, end effector, payload, workpiece, environment, and intended human interactions.
2. Identify hazards: Consider impact, crushing, trapping, sharp tools, hot surfaces, electrical hazards, unexpected startup, and dropped loads.
3. Estimate risk: Evaluate the severity of harm, probability of exposure, and possibility of avoidance.
4. Reduce risk by design: Eliminate sharp edges, reduce mass and speed, limit force, and avoid trapping spaces.
5. Add safeguards: Use monitored stops, scanners, emergency stops, protective devices, and safe control functions.
6. Provide information and training: Establish operating procedures, warning signs, maintenance rules, and worker training.
7. Validate and document: Test safety functions under normal operation and foreseeable faults.
8. Review periodically: Reassess risk after tooling, software, payload, or layout changes.

Collaboration is safe only when the complete application, not merely the robot arm, has been assessed.

Major hazards

• Collision or impact from moving robot links
• Crushing and trapping between the robot and fixed objects
• Unexpected startup caused by software, sensor, or communication faults
• Ejected or dropped workpieces
• Sharp, hot, electrical, hydraulic, pneumatic, chemical, or radiation hazards
• Human error during programming, cleaning, or maintenance
• Cyberattacks that modify commands or disable safety functions

Layered safety measures

• Inherently safe design: Reduce mass, speed, force, sharp edges, and accessible trapping points.
• Physical safeguards: Install fences, interlocked gates, covers, and fixed barriers.
• Presence detection: Use light curtains, safety scanners, pressure mats, and vision systems.
• Safety-related control: Apply emergency stops, safe torque off, speed monitoring, position limits, and redundant controllers.
• Operating procedures: Use lockout-tagout, permit systems, safe teaching modes, and maintenance checklists.
• Training and personal protective equipment: Train operators in hazards, emergency response, and system limitations.
• Inspection and validation: Test stopping performance, interlocks, sensors, and fault responses.
• Cybersecurity: Restrict access, authenticate updates, segment networks, and record security events.

Safety follows the principle of risk reduction: eliminate hazards where possible, apply engineered controls, and use administrative controls only for remaining risks.

A protective device must be positioned far enough from the hazard to ensure that the robot reaches a safe condition before a person can reach the dangerous area.

Assume that:

• A person approaches at speed .
• Total reaction and stopping time is .
• An additional allowance for sensor penetration, uncertainty, or reach is .

During the total stopping time, the person may travel:

Therefore, a simplified minimum separation distance is:

The total time may be written as:

where is sensor response time, is controller response time, and is mechanical braking time. Hence:

Factors affecting the distance

• Maximum possible human approach speed
• Detection-zone geometry and sensor resolution
• Communication and controller delays
• Robot speed, payload, inertia, and brake condition
• Tool and workpiece dimensions
• Measurement uncertainty and system tolerances
• Possibility of reaching over, under, or around the safeguard
• Applicable safety standards and validated stopping-time measurements

In an actual installation, the value must be determined using the relevant standard and measured worst-case stopping performance rather than relying only on a theoretical calculation.

Main ethical issues

• Safety and non-maleficence: Robots should not create unreasonable risks to people or the environment.
• Privacy: Cameras, microphones, health data, and location data may reveal sensitive information.
• Bias and fairness: Training data can cause unequal recognition, service, or treatment of different groups.
• Transparency: Users should know when they are interacting with a robot and understand the basis of important decisions.
• Accountability: Responsibility must be assigned among designers, manufacturers, owners, programmers, and operators.
• Human autonomy: Robots should support informed human choices rather than manipulate or unnecessarily replace judgment.
• Employment effects: Automation may displace jobs while creating new skill requirements.
• Dual use: Technologies developed for beneficial purposes may also be used for surveillance or weapons.
• Environmental impact: Manufacturing, energy consumption, batteries, and electronic waste must be considered.

Responsible decision framework

1. Identify stakeholders and intended benefits.
2. Identify foreseeable harms, misuse, and affected rights.
3. Test data and models for bias, reliability, and explainability.
4. Apply privacy-by-design and security-by-design principles.
5. Keep meaningful human oversight for high-impact decisions.
6. Define responsibility, audit trails, and complaint mechanisms.
7. Validate performance in realistic conditions.
8. Monitor the deployed system and correct newly discovered problems.

Ethical robotics requires continuous governance throughout design, deployment, maintenance, and retirement—not merely compliance testing at the end.

Robots in industrial automation perform repetitive, hazardous, precise, or high-speed production tasks. Common applications include welding, painting, assembly, machine tending, palletizing, packaging, and inspection.

Components of a robotic cell

• Industrial robot manipulator and controller
• End effector such as a gripper, welding gun, or spray tool
• Sensors and machine-vision equipment
• Fixtures, feeders, conveyors, and auxiliary machines
• Programmable logic controller for cell coordination
• Human-machine interface for monitoring and commands
• Safety fences, interlocks, scanners, and emergency stops
• Communication network and production-data system

Benefits

• High productivity and repeatability
• Improved quality and reduced process variation
• Continuous operation
• Better worker safety in hazardous tasks
• Traceability and collection of production data
• Flexible reprogramming for product changes

Limitations

• High initial investment and integration cost
• Need for skilled programming and maintenance
• Downtime caused by component or software failure
• Difficulty handling highly unstructured tasks
• Safety and cybersecurity requirements
• Possible workforce displacement and retraining needs

Successful automation requires technical feasibility, economic justification, safe integration, and planning for human roles.

Applications

• Surgical robots: Assist surgeons with precise instrument control and minimally invasive procedures.
• Rehabilitation robots: Support repetitive therapeutic movements and gait training.
• Prosthetic and assistive robots: Restore or augment movement for people with disabilities.
• Hospital service robots: Transport medicines, samples, meals, and supplies.
• Telepresence robots: Connect remote specialists with patients and clinical teams.
• Disinfection robots: Use ultraviolet light or other methods to disinfect controlled spaces.

Advantages

• High precision, stability, and repeatability
• Smaller incisions and potentially faster recovery in suitable procedures
• Reduced physical workload for clinical staff
• Quantitative monitoring of rehabilitation progress
• Remote access to specialist expertise

Limitations and concerns

• High purchase, maintenance, and training costs
• Possibility of mechanical, software, or network failure
• Need for sterilization and compatibility with clinical workflows
• Patient privacy and cybersecurity risks
• Unclear responsibility when errors occur
• Risk of overreliance on automation

Medical robots generally assist rather than replace qualified professionals. Clinical validation, informed consent, human oversight, and fail-safe operation are essential.

Applications of agricultural robots

• Autonomous tractors and field vehicles
• Robotic seeding and precision fertilizer application
• Mechanical or targeted chemical weed control
• Fruit and vegetable harvesting
• Crop and soil monitoring using ground robots or drones
• Livestock monitoring, feeding, and milking
• Sorting, grading, and packaging of produce

Benefits

• Reduces repetitive manual labor
• Enables precise application of water, fertilizer, and pesticides
• Supports continuous crop monitoring
• May reduce waste and environmental impact
• Improves data-driven farm management

Technical challenges

• Uneven terrain, mud, dust, rain, and changing illumination
• Variability in plant size, shape, color, and maturity
• Occlusion by leaves and closely packed crops
• Need to handle delicate produce without damage
• Reliable localization where GPS is weak or obstructed
• Long operating hours, energy limits, and field maintenance
• Safety around workers, animals, and conventional machinery
• Economic affordability for small farms

Agricultural robots require rugged mechanical design, robust vision, adaptive manipulation, safe navigation, and task-specific AI.

Robots in logistics transport goods, retrieve inventory, sort packages, load or unload items, perform palletizing, and assist workers in order fulfillment.

Automated guided vehicles

• Usually follow predefined paths marked by wires, magnetic tape, reflectors, or fixed navigation features.
• Operate effectively in stable and highly structured environments.
• Are predictable and relatively straightforward to manage.
• Require infrastructure or route modifications when the layout changes.
• May stop when an obstacle blocks the fixed route.

Autonomous mobile robots

• Use LiDAR, cameras, maps, and localization algorithms to navigate.
• Plan and modify routes dynamically.
• Can move around obstacles and adapt more easily to layout changes.
• Require more complex software, sensing, fleet coordination, and cybersecurity.

Warehouse benefits

• Faster and more consistent material movement
• Reduced walking distance for workers
• Improved order accuracy and inventory visibility
• Flexible scaling during periods of high demand

Challenges

• Congestion and multi-robot traffic management
• Interaction with people and manually operated vehicles
• Battery charging and fleet scheduling
• Communication reliability
• Safe operation near shelves, ramps, and loading areas

Fleet-management software assigns tasks, prevents route conflicts, monitors batteries, and coordinates robots with warehouse-management systems.

Applications

• Unmanned ground, aerial, surface, and underwater vehicles
• Reconnaissance, mapping, and border surveillance
• Explosive-ordnance disposal and mine detection
• Search and rescue in collapsed, radioactive, or contaminated areas
• Transport of equipment and supplies
• Inspection of hazardous infrastructure

Operational benefits

• Reduces direct human exposure to dangerous environments
• Provides persistent observation and access to confined areas
• Carries sensors beyond normal human capability
• Supports faster mapping and situational awareness
• Performs repetitive or physically demanding missions

Operational limitations

• Communication jamming, loss, or delay
• Sensor errors, poor visibility, and uncertain identification
• Cyberattacks and hostile takeover
• Limited battery life and difficult field repair
• Reduced performance in unfamiliar terrain
• Automation bias among human operators

Ethical concerns

• Civilian privacy and mass surveillance
• Accountability for incorrect identification or harmful action
• Escalation and lowered barriers to conflict
• Bias in recognition and threat-assessment algorithms
• Delegation of lethal decisions to autonomous systems

Defense and emergency robots should use secure communication, clear rules of engagement, traceable decisions, rigorous testing, and meaningful human control, especially when actions may cause injury or loss of life.

Major emerging trends

• Generative and multimodal AI: Robots can interpret combinations of language, images, sound, and sensor data, enabling more natural instructions and general-purpose task planning.
• Foundation models for robotics: Large pretrained models may transfer knowledge across robots and tasks, reducing application-specific training.
• Edge AI: AI inference is performed on or near the robot, reducing latency and dependence on cloud connectivity.
• Digital twins: Virtual models are synchronized with physical robots for simulation, monitoring, optimization, and predictive maintenance.
• Soft robotics: Flexible materials and compliant actuators support safe interaction and delicate object handling.
• Swarm robotics: Many simple robots coordinate to perform exploration, monitoring, or transport tasks.
• Humanoid and mobile-manipulation robots: Systems combine navigation and dexterous manipulation for human-designed environments.
• Bio-inspired and micro-robotics: Designs imitate biological movement or operate at very small scales, including possible medical applications.
• Cloud robotics and IoRT: Connected robots share data, computation, maps, and learned models.
• Human-robot collaboration: Improved sensing and intuitive interfaces allow people and robots to work more closely.

Likely influence

These trends may increase flexibility, autonomy, personalization, and deployment in unstructured environments. However, they also increase concerns about safety validation, employment transition, privacy, cybersecurity, energy consumption, bias, and accountability. Future systems will require technical innovation together with standards, workforce education, ethical governance, and effective human oversight.