Вы находитесь на странице: 1из 34
ENPM808F: Robot Learning Lecture 1: Robots, Machine Learning, Autonomy and the Brain

ENPM808F: Robot Learning

Lecture 1:

Robots, Machine Learning, Autonomy and the Brain

Questions §   Question 1: Why do you want to take this course? §  

Questions

§

 

Question 1: Why do you want to take this course?

§

 

Question 2: What background do you have in the field of

robotics?

 

§

 

Question 3: Have you taken any courses in the following

areas?

 

§ Artificial Intelligence

§ Robot Modeling and/or Simulation

 

§ Machine Learning or Pattern Recognition

§ Control Theory

 

§ Optimization

 

§ Robotic Planning

 

§

Question 4: Do you have any experience programming robots, either with actual hardware or in simulation?

What is a robot? Origin: Karel Capek's play R.U.R. ( Rossum’s Universal Robots), 1921 Czech:

What is a robot?

Origin: Karel Capek's play R.U.R. (Rossum’s Universal Robots), 1921 Czech: robota servitude, forced labor

Robots), 1921 Czech: robota – servitude, forced labor “A machine (sometimes resembling a human being in

“A machine (sometimes resembling a human being in appearance) designed to function in place of a living agent, esp. one which carries out a variety of tasks automatically or with a minimum of external impulse.” --- Oxford English Dictionary

“A reprogrammable, multifunctional manipulator designed to move materials, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks.”

--- The Robot Institute of America

through various programmed motions for the performance of a variety of tasks.” --- The Robot Institute

Some Real Robots

Some Real Robots Baxter Wind Turbines Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot

Baxter

Some Real Robots Baxter Wind Turbines Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot

Wind Turbines

Inspection Robot

Some Real Robots Baxter Wind Turbines Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot

Hubo

Some Real Robots Baxter Wind Turbines Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot

Warehouse Automation Robots

Turbines Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars

Sony AIBO

Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover
Inspection Robot Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover

iRobot PackBot

Hubo Warehouse Automation Robots Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover Asimo Quadrotor

da Vinci Surgical Robot

Robots Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge

Mars Rover

Sony AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV

Asimo

AIBO iRobot PackBot da Vinci Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New

Quadrotor

PackBot da Vinci Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New Horizons Probe

DARPA Urban Challenge

Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New Horizons Probe Stickybot IRobot Roomba

AUV

New Horizons Probe

Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New Horizons Probe Stickybot IRobot Roomba

Stickybot

Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New Horizons Probe Stickybot IRobot Roomba

IRobot Roomba

Surgical Robot Mars Rover Asimo Quadrotor DARPA Urban Challenge AUV New Horizons Probe Stickybot IRobot Roomba

Big Dog

Some Fictional Robots

Some Fictional Robots

What is Machine Learning?

“[Machine learning is] a computer program that improves its performance at some task through experience.“ --- Tom Mitchell

“Machine learning is the study of data - driven methods capable of mimicking, understanding and aiding human and biological information processing tasks .“ --- David Barber

“Machine learning algorithms can figure out how to perform important tasks by generalizing from

examples .”

--- Pedro Domingos

“At a basic level, machine learning is about predicting the future based on the past. “ --- Hal Daume

Why is Machine Learning Important?

Availability of large amounts of data

Memory is cheap

Expansion of computational resources

Better algorithms

Automation of intelligence enhances economic opportunities

Machine Learning

Forms of Machine Learning:

Supervised (all training data labeled)

Semi - supervised Learning (some labeled data)

Reinforcement Learning (reinforcement signal)

Unsupervised Learning (no labeled training data)

Machine Learning

Examples of Supervised Learning:

Regression

Neural Networks

Decision Trees

Machine Learning

Examples of Semi- Supervised Learning:

Transductive Learning

Clustering (with labels)

Machine Learning

Examples of Reinforcement Learning:

Markov Decision Processes (MDPs)

Temporal Difference Methods

Q Learning

Dynamic Programming

Machine Learning

Examples of Unsupervised Learning Methods:

K- means clustering

Anomaly detection

Expectation - Maximization (EM)

Principal Component Analysis

Machine Learning

Key Concepts:

Induction

Generalization

Convergence

Overfitting

Scalability

Data Requirements

Machine Learning

Some Commonly Used Machine Learning Algorithms:

Regression (e.g., Gaussian process regression)

Decision Tree Methods (e.g., CART, C4.5)

Artificial Neural Networks (e.g., multilayer perceptrons)

Bayesian Networks (e.g., Bayesian Belief Nets)

Case-Based Reasoning

K-Nearest Neighbor Classifiers

Support Vector Machines

Radial Basis Functions

Evolutionary Algorithms

Hidden Markov Models

E-M Algorithm

Kohonen Self-Organizing Maps

Temporal Difference Learning

Q-Learning

Deep Networks (e.g., Boltzmann machines, CNNs)

Ensemble Methods (e.g., Boosting, Bagging, Decision Forests)

What are Autonomous Robots?

“Robots that have ability to make decisions on their own without input from human operators.” --- S.K. Gupta

“To be autonomous, a system must have the capability to independently compose and select among different courses of action to accomplish goals based on its knowledge and understanding of the world, itself, and the situation.” --- Defense Science Board 2015 Summer Study on Autonomy

Simple Control Loop

Simple Control Loop

Simple Control Loop
Robot Control Loop Sense Think Act

Robot Control Loop

Sense Think Act
Sense
Think
Act
Poll Sensors Robot Control Loop Goals Filter Sensor Data Sense Think Policies and Behaviors Fuse
Poll Sensors
Poll Sensors

Robot Control Loop

Goals
Goals
Filter Sensor Data Sense Think Policies and Behaviors Fuse Sensor Data Constraints Act Actuation Controller
Filter
Sensor Data
Sense
Think
Policies and
Behaviors
Fuse Sensor
Data
Constraints
Act
Actuation
Controller
Environment
Lag
Gains
OODA Loop

OODA Loop

OODA Loop
James S. Albus § Pioneer in Motor Control Learning and Control of Robotic Systems §

James S. Albus

§ Pioneer in Motor Control Learning and Control of Robotic Systems

§ May 4, 1935 – April 17, 2011

§ Ph.D. in Electrical Engineering from the University of Maryland, College Park, 1972

§ 1957 - 1973 worked for NASA on Project Vanguard at the Naval Research Laboratory

§ March 17, 1958, Vanguard 1 became the 2 nd artificial satellite successfully placed in Earth orbit by the United States. It was the first solar-powered satellite, and is the oldest artificial satellite still in space.

§ 1973 - 2008 worked at the National Bureau of Standards (NBS) which changed its name in 1980 to the National Institute of Standards and Technology (NIST)

(NBS) which changed its name in 1980 to the National Institute of Standards and Technology (NIST)
(NBS) which changed its name in 1980 to the National Institute of Standards and Technology (NIST)

Vanguard 1 Satellite

James S. Albus § 1971 , publishes a new theory of cerebellar function that modified

James S. Albus

§ 1971 , publishes a new theory of cerebellar function that modified and extended a previous theory published by David Marr in 1969 .

§ 1975, based on his cerebellar model, Albus invents a new type of neural net computer, the Cerebellar Model Articulation Controller (CMAC ).

§ 1980s, Albus co - invents the Real - Time Control System (RCS). The 4D/RCS Reference Model Architecture is a reference model for military unmanned vehicles, which describes how software components of military unmanned vehicles should be identified and organized.

§ 1992, Albus invents and develops a new generation of robot cranes ( Robocrane ) based on six cables and six winches configured as a Stewart platform .

§ 2009, Albus extends the reference model to a cognitive architecture for Intelligent Multi - Agent Systems.

model to a cognitive architecture for Intelligent Multi - Agent Systems. Cerebellar Perceptron, 1971 CMAC Diagram,

Cerebellar Perceptron,

1971

model to a cognitive architecture for Intelligent Multi - Agent Systems. Cerebellar Perceptron, 1971 CMAC Diagram,

CMAC Diagram, 1975

4D/RCS Reference Model Architecture ( Albus , 2002) 4D/RCS reference model architecture for an unmanned

4D/RCS Reference Model Architecture ( Albus, 2002)

4D/RCS Reference Model Architecture ( Albus , 2002) 4D/RCS reference model architecture for an unmanned vehicle

4D/RCS reference model architecture for an unmanned vehicle contains many layers of computational nodes each containing elements of sensory processing, world modeling, value judgment, and behavior generation.

4D/RCS Node

4D/RCS Node

4D/RCS Node
4D - RCS Control Loop (Simplified)

4D - RCS Control Loop (Simplified)

4D - RCS Control Loop (Simplified)
4D -RCS Control Loop

4D -RCS Control Loop

4D -RCS Control Loop
The Cerebellum The cerebellum (Latin for "little brain") is the region of the brain that

The Cerebellum

The Cerebellum The cerebellum (Latin for "little brain") is the region of the brain that is

The cerebellum (Latin for "little brain") is the region of the brain that is responsible for motor control.

The cerebellum responds to movement commands initiated in the cerebral cortex.

cerebellum.gif, Wikimedia Commons

Cell Types and Connections of the Cerebellum Gray706.png, Wikimedia Commons

Cell Types and Connections of the Cerebellum

Cell Types and Connections of the Cerebellum Gray706.png, Wikimedia Commons

Gray706.png, Wikimedia Commons

Cell Types of the Cerebellum • Purkinje cells lie at the heart of the cerebellar

Cell Types of the Cerebellum

Purkinje cells lie at the heart of the cerebellar circuit, and are distinguished by receiving more synaptic inputs than any other cell type in the brain.

Granule cells are the smallest and most numerous cells in the brain. They receive all of their inputs from mossy fibers, and by outnumbering the mossy fibers (by a factor of 200 - 1 in humans) may serve as sparse encoders of that information before it is passed along to Purkinje cells.

Mossy fiber cells form excitatory synapses with the granule cells and the cells of the deep cerebellar nuclei . Each mossy fiber makes contact with about 400 – 600 granule cells .

Climbing fiber cells are thought to signal errors in motor performance, and according to the computational models of Marr and Albus , provide the primary teaching signal used for learning in the cerebellum.

Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K

Granule

Cells

Mossy

Fiber

Cells

Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K

200M

Purkinje

Cells

Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K
Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K

15M

40,000M

Deep

Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K

Nuclear

Cells

Granule Cells Mossy Fiber Cells 200M Purkinje Cells 15M 40,000M Deep Nuclear Cells 75K

75K

Cerebellar Perceptron Cerebellar Perceptron: P, Purkinje cell; B, basket cclls ; S, stellate b cells.

Cerebellar Perceptron

Cerebellar Perceptron Cerebellar Perceptron: P, Purkinje cell; B, basket cclls ; S, stellate b cells. Each

Cerebellar Perceptron: P, Purkinje cell; B, basket cclls ; S, stellate b cells. Each Purkinje

cell has inputs of the type shown. Citation: James S. Albus , A Theory of Cerebellar Function, Mathematical Biosciences 10 , pp. 43 , (1971 ) .

Cerebellar Model Articulation Controller Citation : James S. Albus , A New Approach to Manipulator

Cerebellar Model Articulation Controller

Cerebellar Model Articulation Controller Citation : James S. Albus , A New Approach to Manipulator Control:

Citation: James S. Albus , A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller, Journal of Dynamic Systems, Measurement, and Control , pp. 225, September 1975.

References • David Marr, “A Theory of Cerebellar Cortex,” Journal of Physiology , 202: 437

References

David Marr, “A Theory of Cerebellar Cortex,” Journal of Physiology , 202: 437 - 470.

James S. Albus , “A T heory of Cerebellar F unction,” Mathematical Biosciences , 10 :25 – 61, 1971.

James S. Albus , “A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller,” Journal of Dynamic Systems, Measurement, and Control , pp . 225, September 1975 .

James S. Albus , et al ., “4D /RCS: A Reference Model Architecture For Unmanned Vehicle Systems Version 2.2 .,” 2002.

Reading Assignments for 06/06 • James S. Albus , “A Theory of Cerebellar Function,” Mathematical

Reading Assignments for 06/06

James S. Albus , “A Theory of Cerebellar Function,” Mathematical Biosciences , 10:25 – 61, 1971.

James S. Albus , “A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller,” Journal of Dynamic Systems, Measurement, and Control , pp. 225, September 1975.

Tom Mitchell, “The Discipline of Machine Learning,” CMU- ML - 06 - 108 (CMU Technical Report), July 2006.

Pedro Domingos , "A few useful things to know about machine learning," Communications of the ACM , 55.10:78 - 87, 2012.

Homework Due by 4pm, 06/06 1) Identify 10 real robots (not shown in class) and

Homework Due by 4pm, 06/06

1) Identify 10 real robots (not shown in class) and 10 fictional robots. Explain the role fictional robots play in the research and development of new capabilities for real robots.

2) What motor control functions are performed by the cerebellum and cerebrum, respectively, in the human brain?

3) What role is played by the climbing fibers in the cerebellum, and why is their function controversial?