AnatomyOfARobot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Section I. INTRODUCTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 2. Anatomy of a Robot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A robot can be thought of as a computer on wheels: it is distinguished\n",
    "from a traditional computer by its physical, movable \"body\", and is\n",
    "distinguished from a traditional machine by its computational \"brain.\"\n",
    "\n",
    "The body of a robot moves in some 2D or 3D space, by means of applying\n",
    "forces to the robot's own body or its environment. The body can have a\n",
    "wide range of forms, ranging from a ground vehicle with wheels to a\n",
    "complex humanoid robot. Most robots are usually made of metal or some\n",
    "other rigid material, where rigid sections known as *links* are\n",
    "connected by movable *joints* at their articulation points.\n",
    "\n",
    "> **Link**. A rigid component of a robot, often attached to other links via joints.\n",
    "\n",
    "> **Joint**. An articulation point between two robot links.\n",
    "\n",
    "The \"brain\" of a robot consists of one or more computers running\n",
    "software that process inputs from its *sensors* and computes outputs\n",
    "that drive the robot's *actuators*. Communication between the processor,\n",
    "sensors, and actuators is accomplished via wires or sometimes wireless\n",
    "communication.\n",
    "\n",
    "Each of these components is analogous to some component of a biological\n",
    "system. Links are analogous to bones; joints are analogous to, well,\n",
    "joints; sensors are analogous to sensory organs; actuators are analogous to\n",
    "muscles, and wires are analogous to nerves. Just as there are millions\n",
    "of biological species with diverse, sometimes exotic external and\n",
    "internal structures, the range of possible robot designs is endless.\n",
    "\n",
    "***************\n",
    "\n",
    "|![fig:RobotAnatomya](figures/introduction/pioneer.png)| ![fig:RobotAnatomyb](figures/introduction/asimo.jpg) |\n",
    "|:-----------------------------------------------------|:-----------------------------------------------------|\n",
    "| **Pioneer mobile robot**                             | **Honda Asimo humanoid robot**                       |\n",
    "| 1 link (body), 0 joints                              | 72 links (incl. fingers), 26 joints                  |\n",
    "| Actuated by wheels (differential drive)              | Actuated by electrical motors + speaker              |\n",
    "| Sensors: SICK laser, sonar, wheel encoders           | Sensors: stereo cameras, joint encoders, gyroscope, inertial measurement unit, foot pressure sensors |\n",
    "\n",
    "<div class=\"figcaption\"><a name=\"fig:RobotAnatomy\">Figure 1</a>. Two robots with very different \"anatomy\".</div>\n",
    "\n",
    "***************\n",
    "\n",
    "The capabilities of robots have been changing fluidly over the years, as\n",
    "researchers invent new materials, fabrication strategies, circuits,\n",
    "computers, sensors, and actuation devices. Indeed, keeping up with the\n",
    "state-of-the-art can be exhausting!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Links and joints\n",
    "\n",
    "Like the bones of an animal, a robot's links offer structural support\n",
    "and protection of internal parts. Most often, some sort of metal is used\n",
    "to provide strong structural support, although some robots use other\n",
    "materials like wood, plastic, or composites. Surfaces that are meant to\n",
    "interact with the environment should be shaped appropriately for the\n",
    "desired task, and may be coated in rubber or another high-friction\n",
    "material to ensure consistency of contact.\n",
    "\n",
    "Joints in robots typically either allow rotation or translation.\n",
    "Rotation usually occurs about a single hinge-joint axis. Ball-and-socket\n",
    "joints like in the human hip or shoulder are much rarer due to the\n",
    "difficulty of fabrication, lubrication, and actuation. Translational\n",
    "joints are fairly common in the form of linear slides used in gantries\n",
    "and Cartesian positioning stages, and in pistons used in many hydraulic\n",
    "and pneumatic robots. To ensure precise movement and durable operation,\n",
    "the joint should be constructed from high-strength materials and\n",
    "high-precision manufacturing. Friction reduction along joints is usually\n",
    "provided using lubrication, bearings, or bushings.\n",
    "\n",
    "The links and joints of a robot also make up its outward appearance, and\n",
    "for some applications, significant effort is devoted to designing a\n",
    "pleasing appearance. To an engineer this may seem like a frivolous\n",
    "concern, but the success of a consumer product often rests on the appeal of\n",
    "its design! Robots that are designed for direct human interaction, such\n",
    "as haptic devices or exoskeletons, should also be designed with\n",
    "ergonomic concerns in mind.\n",
    "\n",
    "It should be noted that although links are mostly rigid, all materials\n",
    "do flex and resonate to some extent. This can be problematic for\n",
    "high-precision applications. Flexibility can also be exploited for\n",
    "certain applications, such as impact-tolerant end-effectors. Soft rubber\n",
    "is often used in robot fingertips to encourage a larger surface area to\n",
    "be in contact. Moreover, some researchers have developed *soft robots*\n",
    "whose bodies are designed explicitly to be flexible, like octopuses or\n",
    "elephant trunks. For flexible links and soft robots, the traditional\n",
    "link-and-joint modeling framework that we will cover in depth in this\n",
    "book does not directly apply. Instead, more complex continuum mechanics\n",
    "models should be used to analyze and predict their behavior."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sensors\n",
    "\n",
    "Sensors are devices that measure some physical quantities of the world and encode the signals into electrical (typically digital) form.  There are many kinds of sensors, and more are being invented every year.  Common sensor types in robotics are given below.\n",
    "\n",
    "| Type | Examples | Purpose |\n",
    "|:-----|:---------|:--------|\n",
    "|Proprioceptive | joint encoders (relative or absolute), joint limit contact switches | Measure the position of a link relative to another. |\n",
    "| Inertial      | gyro, accelerometer, IMU | Measure the acceleration, velocity, or position of a link. |\n",
    "| Tactile       | contact switch, force/torque sensor (load cell), scale, pressure sensor, tactile array | Detect external forces on the robot's body. | \n",
    "| Visual        | monocular camera, stereo camera, laser rangefinder, depth sensor | Measure light intensity from the robot's surroundings.  Some sensors estimate distance from the device to objects in the world via triangulation or time-of-flight.  |\n",
    "| Other         | GPS, sonar, temperature (especially motor temperature), gas sensors, battery voltage |  |\n",
    "\n",
    "Most of the time, sensors are mounted to a link of the robot, and move while the robot moves.  The raw data produced by the sensor is often not particularly useful in itself (e.g., cameras will give you pixel colors rather than object identities), and perception algorithms must process this data into meaningful quantities.  It is important to build accurate models of the physics behind the sensor (and methods for \"inverting\" these physics) in order to help a robot correlate sensor signals into an understanding of the physical world.\n",
    "\n",
    "The position and orientation of the sensor on the link is extremely important to be able to make sense of sensor data. A laser sensor may be saying that an obstacle is 1m away, but is the obstacle ahead, behind, to the left, or above the robot?  How close is too close before the robot ends up colliding?  We will look into this issue further  when we discuss calibration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Actuators and transmissions\n",
    "\n",
    "An actuator is a mechanism that generates a force / torque given electrical signals; a transmission is a mechanism that applies an actuator's forces / torques to the robot's links.  Most commonly on industrial robots, actuators are electric motors while the transmissions are some sort of gearing system (e.g., gearbox).  On mobile robots, the transmissions from actuators to the ground are actually the wheels (or tracks).  It is also common to think of (and purchase) the actuator and transmission as a unit, such as with servomotors.\n",
    "\n",
    "Common actuator types include:\n",
    "\n",
    "- *Electric motors*: stepper motors, brushed / brushless DC motors, AC motors. \n",
    "\n",
    "- *Pneumatic actuators*. High variation in size and strength.\n",
    "\n",
    "- *Hydraulic actuators*. Very strong.  \n",
    "\n",
    "- *Chemical (combustion) actuators*: internal combustion engines in automobiles and prop planes; turbines in jet planes.\n",
    "\n",
    "- *Piezoelectric actuators*.\n",
    "\n",
    "Actuators can be rotary (revolves around an axis) or linear (translates about an axis).  Each type and model of actuator has many subtle differences that should be taken into account when fine-tuning a robot's design, e.g., in the power source, power / weight ratio, control bandwidth, maximum speed, stroke length, continuous / peak torque, complexity of electronics and maintenance, etc.  Most often, a roboticist will choose between electric, pneumatic, and hydraulic motors.  Electric motors come in a plethora of models with high variation in size, strength, and speed, and are convenient to use. Pneumatic actuators have a higher power-to-weight ratio, and lightweight pistons can be placed on extremities far from heavy pumps using air lines, but require maintaining complex pump / valve systems.  Hydraulic actuators are stronger and heavier, and have similar drawbacks. \n",
    "\n",
    "There are also a variety of problems that arise when an actuator is pushed beyond its safe operating limits.  Electric motors *burn out* when the temperature produced by the motor cannot be dissipated quickly enough, and which burns through the insulation on the motor's wiring.  Pneumatic and hydraulic systems can *blow seals* where the internal pressure overcomes the strength of the seals and gaskets on the pneumatic / hydraulic lines or cylinders. In hydraulic systems, leaks can be quite messy!\n",
    "\n",
    "Transmission systems can be quite diverse.  Common forms of transmission include:\n",
    "\n",
    "- *Gearing*: gearboxes, lead screws, harmonic drives.  Can convert rotary to linear motion and vice versa.  Must be located directly at the joint.\n",
    "\n",
    "- *Cables*: like tendons in biological systems, cables can connect a motor to a far-off link. Usually outputs linear motion, and can only pull.\n",
    "\n",
    "- *Wheels / tracks*: used for mobile robots.  Can be arranged in *differential drive*, *skid-steer*, and *Ackermann steering* arrangements.  Omnidirectional (Mechanum) wheels are available on some platforms.\n",
    "\n",
    "- *Pneumatic / Hydraulic lines*: connect a pump / valve system to a pneumatic / hydraulic cylinder.\n",
    "\n",
    "- *4-bar linkages*: static bars connected by joints, used to convert rotary to linear motion and vice versa.\n",
    "\n",
    "- *Flywheels*: use changes of angular momentum to control orientation, used primarily in space robotics.\n",
    "\n",
    "- *Propellers*, *aelierons*, and *control surfaces*: deflect a surrounding gaseous / liquid medium to propel the robot, used primarily for aerial and water vehicles.\n",
    "\n",
    "- *Thrusters*: eject mass at high speed to exert force using Newton's Law, used primarily in space robotics.\n",
    "\n",
    "- *Feet*, *hands*, *wings*, etc: bioinspired methods to touch the environment / deflect the surrounding medium to propel the robot.\n",
    "\n",
    "\n",
    "When  designing or choosing a transmission systems, a designer must be cognizant of the:\n",
    "\n",
    "- *Gear reduction*: force multiplier from input to output.\n",
    "\n",
    "- *Backlash*: a dead-band about which force is not productively generated when the motion of direction is switched. \n",
    "\n",
    "- *Internal friction*: losses of motive force.\n",
    "\n",
    "- *Backdrivability*: the ability for forces on the output shaft to be reflected on the input shaft. High backdrivability implies low backlash and low internal friction.\n",
    "\n",
    "- *Longevity and durability*: cheap gear teeth may wear down, materials in cables may exhibit creep (stretch), and tires wear down over time.  Transmission systems are often the first parts of robots to break!\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cognitive architectures\n",
    "\n",
    "The title of this section is a bit misleading: robots don't perform\n",
    "\"cognition\" as humans and animals do. However, they do indeed need to\n",
    "process sensor information and make decisions about how to act. This\n",
    "processing is accomplished via one or more computers, rather than\n",
    "biological brains. The nature of a robot's computing hardware is\n",
    "relatively unimportant to its behavior, as long as there are sufficient\n",
    "computing resources available. Instead, what has greater impact is the\n",
    "*form and function of information processing elements*, which is\n",
    "referred to as a *cognitive architecture*.\n",
    "\n",
    "(The terminology used in this chapter is highly influenced by Chapter 2\n",
    "in Stuart and Russell's excellent textbook *Artificial Intelligence: a\n",
    "Modern Approach*.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sense-plan-act framework\n",
    "\n",
    "A cognitive architecture resides in the robot's mind, which controls the\n",
    "robot's body, which interacts with the outside world. The body and\n",
    "external world form the information processing *environment* of the\n",
    "mind. (Note that the mind cannot alter the outside world directly: in\n",
    "other words, no telepathy is allowed!) The role of a robot's mind is to\n",
    "receive sensor information (sense), do some processing (plan) and to\n",
    "output commands to the body's actuators (act). This perspective is also\n",
    "known as the *sense-plan-act* framework, and is illustrated in <a href=\"#fig:SensePlanAct\">Fig. 1</a>.\n",
    "\n",
    "******************\n",
    "\n",
    "![fig:SensePlanAct](figures/introduction/sense_plan_act.png)\n",
    "\n",
    "<div class=\"figcaption\"><a name=\"fig:SensePlanAct\">Figure 1</a>. The sense-plan-act framework structures how the robot's \"mind\" processes information from its sensors (Sense), calculates an action (Plan), and then performs the action using its body (Act).  In turn, the body interacts with the environment by applying physical forces and receiving data from sensing devices.\n",
    "</div>\n",
    "\n",
    "******************\n",
    "\n",
    "Imagining one's brain severed\n",
    "from one's body at the spinal cord, the sense-plan-act framework views\n",
    "the brain's role as processing the input and output that pass through\n",
    "the nerves at the spinal cord. The brain is a highly developed organ\n",
    "that processes information, makes decisions, learns, and adapts in such a\n",
    "way to generate the infinite richness of human behavior. When we program\n",
    "a robot's software, we likewise control the contents and functions of\n",
    "its mind. The question is, how do we construct a mind so that the robot\n",
    "behaves intelligently (or at least, usefully)?\n",
    "\n",
    "The notion of \"useful task\" is defined external to the robot itself, and\n",
    "is selected by the robot's designer. We will discuss techniques and\n",
    "issues with measuring performance in later chapters on system\n",
    "engineering; for now we shall assume there is some existing quantifiable\n",
    "performance metric, like tracking a curve as closely as possible, or\n",
    "having a low risk of colliding with pedestrians, etc.\n",
    "\n",
    "<a id=\"reflexive\"></a>\n",
    "First, we will consider a *simple reflexive* architecture, in which the\n",
    "inputs are converted to outputs via a set of fixed operations (*rules*).\n",
    "Examples of this architecture include servomotors, automatic doors, some\n",
    "factory automation components, and other simple electromechanical\n",
    "devices. The input processing can be sophisticated, such as those found\n",
    "in camera autofocus systems. We will cover other sensor processing\n",
    "methods in Section V. Often, simple reflexive machines are\n",
    "not even thought of as \"robots\" because they operate only on the data\n",
    "that is immediately provided to them, and do not derive any richer\n",
    "understanding.\n",
    "\n",
    "<a id=\"model-based-reflexive\"></a>\n",
    "In the *model-based* reflexive architecture, the robot estimates the\n",
    "current *state* of its environment using history. State represents an\n",
    "understanding of the robot's body and world, which may include\n",
    "information outside of what is currently observed. To implement such an\n",
    "architecture, the robot needs both *memory* of previous state(s) and a\n",
    "*model* of how its environment changes &mdash; that is, how the world behaves\n",
    "on its own and how its body interacts with the world. The architecture\n",
    "contains a *state estimator* that performs continual state updates.\n",
    "However, it still chooses its actions by fixed rules as a function of\n",
    "its state. Nevertheless, high-performance behavior can be obtained by 1)\n",
    "an accurate model, 2) an accurate state estimator, 3) carefully designed\n",
    "rules. Examples of this architecture include automatic stabilization for\n",
    "quadcopters, aircraft autopilots, and driver assistance systems in\n",
    "modern automobiles (e.g., automatic braking and lane keeping). We will\n",
    "cover dynamic models in detail in Section IV and state estimation in Section VI.\n",
    "\n",
    "<a id=\"deliberative\"></a>\n",
    "The next level of architecture complexity adds prediction and\n",
    "deliberation into the fold. A *deliberative* architecture uses a\n",
    "*predictive model* to estimate how its future actions alter the\n",
    "environment as well as a *performance metric* to choose which course of\n",
    "action is the best. To generate possible future actions in a systematic\n",
    "way, a *planner* computes many potential options and their outcomes. By\n",
    "maximizing performance, a deliberative architecture is designed\n",
    "explicitly to achieve some notion of *optimality*. (Contrast this with a\n",
    "reflexive device, which can perform well only by coincidence or shrewdly\n",
    "designed rules.) These techniques can generate excellent behaviors if 1)\n",
    "the predictive model is accurate, 2) the planner discovers a\n",
    "high-quality sequence of actions quickly, and 3) the metric encoded by\n",
    "the robot is truly a good measure of performance (as judged by human\n",
    "observers). Section III covers planning in detail.\n",
    "\n",
    "<a id=\"learning\"></a>\n",
    "Finally, a *learning* architecture incorporates sensor data over time to\n",
    "automatically adapt components of its predictive models, state estimators,\n",
    "and/or planners.  The amount of learning can be relatively small, such as\n",
    "auto-calibration of a robot's sensors to correct for sensor drift, or large,\n",
    "such as reinforcement learning to adapt the robot's behavior according to\n",
    "observed outcomes.  Section VI discusses machine learning, calibration, and\n",
    "reinforcement learning techniques to be used in such architectures.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perception, planning, and control; hierarchical architectures\n",
    "\n",
    "The Plan component of the Sense-Plan-Act framework above can typically be broken down into three components: \n",
    "\n",
    "- *Perception*.  From sensor data and memory of the past, construct a representation of all the knowledge the robot has available about itself and its surroundings.  When there is uncertainty, either construct a best guess, or explicitly model how much uncertainty is available.\n",
    "\n",
    "- *Planning*.  Use the robot's knowledge representation and dynamics models to decide what the robot should do now and in the future.  Outputs reference motions or feedback policies.\n",
    "\n",
    "- *Control*. At a high feedback rate, execute the reference motions generated by planning by outputting actuator signals.  Use immediate sensor feedback to correct for minor disturbances.\n",
    "\n",
    "In a complex intelligent robot, such as an autonomous vehicle, these components are individually quite sophisticated.  To optimize the behavior of each component requires deep knowledge, skills, and effort on the part of its engineering team.\n",
    "\n",
    "Although these components are described as though they would be separate stages of a pipeline, they actually interact at multiple levels based on the timing requirements of behavioral decisions.  The planning component is usually broken down so that mission-level decisions are made at the highest layer, which are then broken into subtasks that should be achieved on the order of seconds to minutes, which are then finally broken down into continuous reference motions at sub-second granularity.  In a sense, the control component is the lowest level of the planning hierarchy.\n",
    "\n",
    "The perception component may also be hierarchical as well. Some sensor signals, like joint encoder values, are processed minimally to be used by the controller.  Others, like vision data, are processed with computationally expensive, relatively slow operations like map-building and object recognition so that mission planning can make significant decisions about the robot's behavior.  A very common architecture is shown below: \n",
    "\n",
    "******************\n",
    "\n",
    "![fig:HierarchicalArchitecture](figures/introduction/hierarchical_architecture.png)\n",
    "\n",
    "<div class=\"figcaption\"><a name=\"fig:HierarchicalArchitecture\">Figure 2</a>. A hierarchical architecture is broken into perception vs decision-making components, which are vertically divided between slow (e.g., mapping, object recognition, planning, trajectory optimization) and fast (e.g., state estimation, tracking control) components.  Common inputs (sensors and input devices) and outputs (actuators and displays) also operate at different rates.\n",
    "</div>\n",
    "\n",
    "******************\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The end-to-end approach\n",
    "\n",
    "In contrast to the perception-planning-control pipeline, the philosophy of *end-to-end* behavior attempts to compress the entire Plan component into a single monolithic function.  Because robot behavior is usually too complex to tune by hand, this philosophy is usually approached via the perspective of *end-to-end learning*, where the robot itself adjusts the behavior of the Plan component to achieve better outcomes.  Recently, this is becoming more a plausible approach given the success of deep neural networks in computer vision and related machine learning fields.\n",
    "\n",
    "However, there is still a long way to go before end-to-end robots are practical, and in fact there are *no commercially-significant robots in the world today that are developed using an end-to-end approach*.  Why might this be the case?  One may be that human scientists and engineers have developed exceptionally precise models for robot behavior that can't be matched using model-free approaches.  Another reason may be that the development process for the classical pipeline does a better job of exploiting the ingenuity of human engineering teams by imposing clear roles, responsibilities, and performance metrics for each component.  Indeed, it is very hard to determine why a system built entirely on machine learning made a mistake, which makes debugging and development much more difficult.  A final reason could be that the machine learning community simply does not yet have the right model for robotic cognition, and some breakthrough awaits just around the corner!  (Personally speaking, I enjoy the robot engineering process, and hope for the sake of myself, my colleagues, and my students that this latter scenario does not occur any time soon.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "### Key takeaways\n",
    "\n",
    "* A robot's body is defined by links, joints, sensors, and actuators.  Links are typically rigid (but not always, in the case of soft robots).\n",
    "* Proprioceptive, inertial, visual, and tactile sensors are the most common sensors on modern robots.\n",
    "* The sense-plan-act framework connects the robot's cognition, encapsulated as the intermediate plan step, with its sensors and actuators which interact with the external world.\n",
    "* In the classical cognitive architecture, the \"plan\" step is further broken down into perception (building representations of the state of the robot and the external world), planning (using those representations to make decisions about what to do next), and control (converting decisions into actions).\n",
    "* Hierarchical architectures break down the perception and decision-making stages into slow, medium, and fast levels which interact at different rates.  Here, planning and control are grouped together into the single decision-making stage: planning is just thought of as the slow / medium levels, while control is the fastest level.\n",
    "* In the end-to-end cognitive architecture, the \"plan\" step is a complex neural network trained from data, usually reinforcement learning or imitation learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Glossary\n",
    "* [Link](#Links-and-joints): A rigid component of a robot, usually connected to other links by joints.\n",
    "* [Joint](#Links-and-joints): An articulation point between two links.\n",
    "* [Sensor](#Sensors): A device that measures a quantity in the physical world and reports it in electrical form. \n",
    "* [Actuator](#Actuators-and-transmissions): A source of controllable force.\n",
    "* [Cognitive architecture](#Cognitive-architectures): The form and function of information processing elements (software) that run on a robot's computing hardware to process sensor input and generate actuator output.\n",
    "\n",
    "Sensing\n",
    "* [Proprioception](#Sensors): Sensing that measures the position of a link relative to another.\n",
    "* [Inertial sensing](#Sensors): Sensing that measures the position of a link relative to a fixed (world) frame.\n",
    "* [Tactile sensor](#Sensors): Sensing that measure forces applied to links.\n",
    "* [Vision sensor](#Sensors): Sensing that measures light (electromagnetic radiation), whether in the visible spectrum or not. \n",
    "\n",
    "Actuation\n",
    "* [Transmission](#Actuators-and-transmissions): A device that connects an actuator to forces on one or more links.\n",
    "* [Gear reduction](#Actuators-and-transmissions): The force multiplier from input to output.  The force multiplier is accompanied by a proportional reduction in velocity.\n",
    "* [Backlash](#Actuators-and-transmissions): A \"dead-band\" about which force is not productively generated when the motion of direction is switched.  Usually caused by spacing between gear teeth.\n",
    "* [Backdrivability](#Actuators-and-transmissions): The degree to which an actuator and its corresponding joint can be moved via the application of external forces.  \n",
    "\n",
    "Cognition\n",
    "* [Sense-plan-act framework](#Sense-plan-act-framework): A classical framework by which cognitive architectures are modeled.  The cognitive (computational) aspects of the robot lie within the \"plan\" block, while the other blocks represent input and output from the physical environment.\n",
    "* [Reflexive agent](#reflexive): An agent that operates according to fixed rules that process sensor inputs into actuator outputs.  A [model-based reflexive agent](#model-based-reflexive) performs some degree of perception to process sensor inputs into estimated physical quantities.\n",
    "* Model-based agent\n",
    "* [Perception-planning-control architecture](##Perception,-planning,-and-control;-hierarchical-architectures): A classical division of responsibilities in the \"plan\" block of the sense-plan-act framework.  Perception processes the sensing input into a world representation, planning makes decisions, and control converts those decisions into actuator outputs.\n",
    "* [Hierarchical architecture](#Perception,-planning,-and-control;-hierarchical-architectures): A division of the perception-planning-control pipeline into parallel slow and fast tracks depending on the computational complexity of components.  Control is typically treated as \"fast planning\".\n",
    "* [End-to-end learning](#The-end-to-end-approach): A more recent framework that eliminates the perception-planning-control framework in favor of a mapping from sensor input to actuator output, learned either through reinforcement learning or imitation learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}