AUTONOMOUS SYSTEMS

INTRODUCTION

Pedro U. Lima
Instituto Superior Técnico/Instituto de Sistemas e Robótica
September 2017

Course Handouts
All rights reserved

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Systems

What is an Autonomous System?

Autonomous Systems 2017© Pedro Lima Introduction 2

 Autonomous Systems

AUTONOMY

From Greek αυτονομία autonomia,

noun of quality from αυτόνομος autonomos “independent,

living by one’s own laws”,

from αυτο auto- “self” + νόμος nomos “custom, law”.

In Wikipedia

Autonomous Systems 2017© Pedro Lima Introduction 3

 Autonomous Systems

SYSTEM

From late Latin systēma, from Ancient Greek σύστημα

(sustēma), “organised whole, body”),

from σύν (sun), “with, together”) + ἵστημι (histēmi), “I stand”).

A collection of organized things.

A way of organizing or planning.
A whole composed of relationships among the members.

In Wikipedia

Autonomous Systems 2017© Pedro Lima Introduction 4

 Autonomous Systems

AUTONOMOUS SYSTEM

A collection of organized things, composed of relationships

among them, and capable of living independently by its own
laws

Autonomous Systems 2017© Pedro Lima Introduction 5

 Autonomous Systems

AUTONOMOUS AGENT

An agent is anything that can be viewed as perceiving its

environment through sensors and acting upon that
environment through effectors (Russel & Norvig, 1995).

Jennings and Wooldridge have three key concepts for an

agent:
• situatedness,
• autonomy,
• flexibility

Autonomous Systems 2017© Pedro Lima Introduction 6

 Autonomous Systems

AUTONOMOUS AGENT (Jennings & Wooldridge, 1998)

Agent is a computer system, situated in some environment, receiving

sensory inputs from its environment and performing actions which
change the environment in some way.
An autonomous agent should be capable of flexible autonomous
action in order to meet its design objectives.
• Flexibility includes that the agent responds to changes it perceives
in the environment, takes the initiative where appropriate and is social
with other artificial agents and humans.
• Autonomy means that the agent should be able to act without the
direct intervention of humans or other agents and that it should have
control over its own actions and internal state.

Autonomous Systems 2017© Pedro Lima Introduction 7

 Autonomous Systems

AUTONOMOUS AGENTS

• should be able to learn from experience

• should be able to interact and even cooperate with other autonomous
agents
• should have means of handling the uncertainty associated to its
sensor inputs and the effects of its actions over a surrounding ,
possibly dynamic, environment
• should have means to take rational decisions, leading to some goal

Autonomous Systems 2017© Pedro Lima Introduction 8

 Autonomous Systems

AUTONOMOUS ROBOTS

Autonomous robots are robots (embodied situated agents)

which can perform desired tasks in unstructured
environments without continuous human guidance. Many
kinds of robots have some degree of autonomy. Different
robots can be autonomous in different ways.
In Wikipedia

Autonomous Systems 2017© Pedro Lima Introduction 9

 Autonomous Systems

AUTONOMOUS ROBOTS have the ability to

• Gain information about the environment.

• Work for an extended period without human
intervention.
• Move either all or part of itself throughout its operating
environment without human assistance.
• Avoid situations that are harmful to people, property, or
itself.
• An autonomous robot may also learn or gain new
capabilities like adjusting strategies for accomplishing its
task(s) or adapting to changing surroundings.
In Wikipedia

Autonomous Systems 2017© Pedro Lima Introduction 10

 Autonomous Systems

• Examples of Autonomous Systems

– Autonomous Mobile Robots (individuals or teams)

– Mobile and Static Sensor Networks
– Intelligent Homes
– Autonomous Cars
– Satellites
– Spacecraft
– Software agents
• Information search
• Computer viruses

– Humans, other animals!

Autonomous Systems 2017© Pedro Lima Introduction 11

 Autonomous Mobile Robots – Historical Evolution
1998
1950
1997

2001
1970-2
2002

2003

1977 1997

1996

1989 2004
2008

1993
2012 2010

2014
2016

2011
Autonomous Systems 2017© Pedro Lima Introduction
 Autonomous Mobile Robots – Historical Evolution

Grey Walter (UK, University of Bristol) built

1950 three wheeled, turtle like, mobile robotic vehicles.
These vehicles had:
a light sensor,
touch sensor,
propulsion motor,
steering motor, and a two vacuum tube
analog computer.

Elsie

Even with this simple design, Grey demonstrated

that his turtles exhibited complex behaviors. He
called his turtles Machina Speculatrix after their
speculative tendency to explore their environment.

The Adam and Eve of his robots were named

Elmer and Elsie ( ELectro MEchanical Robots,
Light Sensitive. )

http://en.wikipedia.org/wiki/William_Grey_Walter

Autonomous Systems 2017© Pedro Lima Introduction 13

 Autonomous Mobile Robots – Historical Evolution

1969
Shakey (Stanford University)
was the first mobile robot to be
controlled by vision

Shakey was set (relatively) simple

tasks to solve:
• To recognize an object using vision
• Find its way to the object
• Perfom some action on the object
(for example, to push it over)

See a recent talk on Shakey at ICRA

2015 by Peter Hart:
http://icra2015.org/conference/plenaries-keynotes#hart

Autonomous Systems 2017© Pedro Lima Introduction 14

 Autonomous Mobile Robots – Planetary Rovers (1)

Mars rovers: Sojourner

NASA JPL Rovers
http://marsrovers.jpl.nasa.gov/home/

Autonomous Systems 2017© Pedro Lima Introduction 15

 Autonomous Mobile Robots – Planetary Rovers (2)

Mars rovers: Spirit and Opportunity

In Mars
since 2004

Autonomous Systems 2017© Pedro Lima Introduction 16

 Autonomous Mobile Robots – Planetary Rovers (3)

Mars rovers: Curiosity

In Mars
since Aug
2012

Autonomous Systems 2017© Pedro Lima Introduction 17

 Autonomous Mobile Robots – Planetary Rovers (4)

Mars rovers: Curiosity

Autonomous Systems 2017© Pedro Lima Introduction 18

 Autonomous Mobile Robots – Industrial

AGVs and LGVs

AGV, Efacec – Transformers factory LGV, Soporcel Paper factory

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Industrial

Mobile Manipulators

KUKA YouBot MetraLabs SCITOS G5

with Schunk arm

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Industrial

KIVA Systems / Amazon

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Industrial

BAXTER / Rethink Robotics

BAXTER

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Indoors

SIGA – Ydreams + IdMind

Vaccuum cleaner Roomba

Helpmate
Wheelesley-robotic wheelchair WowWee ROVIO, USA
Autonomous Systems 2017© Pedro Lima Introduction
 Autonomous Mobile Robots – Field

PackBot, iRobot, USA

Shrimp, EPFL, Switzerland RAPOSA, ISR/IST + IdMind, Portugal

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Aerial and Marine

Passarola, ISR/IST, Portugal

AIR

Helicopter
Quadcopter, UAVision, Portugal

UNDERWATER OCEAN

Infante – ISR/IST, Portugal Delfim – ISR/IST, Portugal

Autonomous Systems 2017© Pedro Lima Introduction
 Autonomous Mobile Robots – Aerial Quads

Check also this TED Talk by Raffaello d’Andrea about quads:

http://www.ted.com/talks/raffaello_d_andrea_the_astounding_athletic_power_of_quadcopters?language=pt

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Autonomous Cars (1)

Cybercar, EU project

Google Driverless Car

! ATLAS Car, U. Aveiro, Portugal

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Autonomous Cars (2)

Google Driverless Car

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Autonomous Cars (3)

Google Driverless Car

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Autonomous Cars (4)
Mercedes Driverless Car

Mercedes-Benz has successfully completed the first autonomous long-distance drive ever, involving both town and cross-
country traffic, using near-production-standard sensor systems. The Mercedes-Benz S 500 INTELLIGENT DRIVE research
vehicle covered 100 kilometers from Mannheim to Pforzheim, Germany, under real traffic conditions and complex situations
including traffic lights, roundabouts, [from https://www.youtube.com/watch?v=AihC5flC-38, 18 Sep 2013]
Autonomous Systems 2017© Pedro Lima Introduction
 Autonomous Mobile Robots – Autonomous Cars (5)

Mercedes Driverless Car

Reprinted from IEEE Spectrum, July 2014

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Autonomous Cars (6)

Mercedes Driverless Car

Reprinted from IEEE Spectrum, July 2014

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Legged (1)

AIBO, Sony
ASIMO, Honda

QRIO, Sony

PARO, Japan

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Legged (2)

Atlas, Boston Dynamics

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Legged (3)

WildCat quadrupede, Boston Dynamics

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Mobile Robots – Assistant

Robotic Assistant/
Cognitive humanoids

• Bio-inspired
• Cognitive Systems
• Antropomorphic
• Robotics and
Neurosciences

iCub, FP6 RobotCub project BALTAZAR, ISR/IST , Portugal

Autonomous Systems 2017© Pedro Lima Introduction 36

 Robot Networks

Autonomous Systems 2017© Pedro Lima Introduction 37

 Robot Networks – Soccer Robots

Autonomous Systems 2017© Pedro Lima Introduction 38

 Autonomous Mobile Robots – Quad Teams

Autonomous Systems 2017© Pedro Lima Introduction

 Spacecraft Formations

Autonomous Systems 2017© Pedro Lima Introduction 40

 Static + Mobile Sensor Networks

Increased resolution from robot action

Low resolution data obtained

from remote sensing

Presence of substance 1 detected

!
Autonomous Systems 2017© Pedro Lima Introduction 41
 Static + Mobile Sensor Networks

distributed
sensors
(cameras)

ubiquitous
comms

Robot
assistants
(cognitive,
social)
networked
personal
assistants

Autonomous Systems 2017© Pedro Lima Introduction 42

 Autonomous Networked Robot Systems
ISR SocRob@Home

ISR Co-Bot, ISR/IST , Portugal

Autonomous Systems 2017© Pedro Lima Introduction 43

 Home Sensor Networks

Autonomous Systems 2017© Pedro Lima Introduction

 Robots + Sensors in Urban Scenarios

CMU-Portugal MAIS+S (Multiagent Intelligent Surveillance System)

Autonomous Systems 2017© Pedro Lima Introduction

 Robots + Sensors in Hospitals

FP7 MOnarCH (Multi-Robot Cognitive Systems Operating in Hospitals)

(Coordinated by ISR/IST)
• Goal: Insert social robots in social environments
• Case study: IPOLisboa’s pediatric ward

Autonomous Systems 2017© Pedro Lima Introduction

 Robots + Sensors in Hospitals

FP7 MOnarCH (Multi-Robot Cognitive Systems Operating in Hospitals)

(Coordinated by ISR/IST)

Autonomous Systems 2017© Pedro Lima Introduction

 MBOT Evolution

Autonomous Systems 2017© Pedro Lima Introduction

 MBOT (1)

Interaction Kinect Camera

with microphones

LEDs Face Expression Video Projector

Navigation
Servo motors
Asus Xtion camera
shoulders and head

Touch Screen

Laser range finder

Autonomous Systems 2017© Pedro Lima Introduction

 MBOT (2)

RFID reader

Voice and Sounds

LEDs “emotional” state

Body Touch
Capacitive cells

LEDs robot state

Autonomous Systems 2017© Pedro Lima Introduction

 Autonomous Systems – Non-Robotic Systems

1996: IBM Deep Blue computer defeats chess

world champion Garry Kasparov

2011: IBM Watson defeats JEOPARDY former

winners Brad Rutter and Ken Jennings

2016: Google Deep Mind AlphaGo defeats GO

world champion Lee Sedol

2011
Autonomous Systems 2017© Pedro Lima Introduction 51
 Autonomous Systems – New Challenges

• Growing robots, inspired in humans cognition

• Bio-inspired robot collectives
• Robot societies using social sciences concepts
…

Autonomous Systems 2017© Pedro Lima Introduction 52

 Autonomous Systems – Ethical Challenges

• Autonomous cars ethical dilemmas

• Threats to employment:
• companies should pay for redeployment or training of replaced
workers
• or taxes should be charged to companies which strongly invest on
automation
•…
Automation can bring several advantages to society,
including more wealth:
the issue is to make sure it is distributed by all, not just a few
53
Autonomous Systems 2017© Pedro Lima Introduction
 Funded by the
©RoCKiN Project, contract no FP7-601012
Autonomous Systems 2017© Pedro Lima European Union
Introduction
 RoCKIn@Home

• Improving the quality of home life

for the people of Europe

• Set in Granny Annie’s apartment,

the robots will help keep her healthy
and doing the things she loves

• Robots are benchmarked on 3 tasks

and 3 functionalities

• Practice testbed set up at IST

Autonomous Systems 2017© Pedro Lima Introduction 55

 RoCKIn@Home TBM 1
Getting To Know My Home

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Home TBM 2
Welcoming Visitors

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Home TBM3
Catering for Granny Annie Needs

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Home - Functionalities

• Object Perception

• Navigation

• Speech Understanding

Autonomous Systems 2017© Pedro Lima Introduction 59

 RoCKIn@Work

• Ensuring the continued

competitiveness of Europe’s
manufacturing industry
• Set in the RoCKIn & RoLLIn factory, a
robot will assist with the assembly
of a drive axel - one component of
the robot itself and therefore a step
towards self- replicating robots.

• Robots are benchmarked on 3 Tasks

and 2 Functionalities.
• Practice testbed set up at BRSU

Autonomous Systems 2017© Pedro Lima Introduction 60

 RoCKIn@Work TBM 1
Assembly Aid Tray for Force Fitting

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Work TBM 2
Plate Driling

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Work TBM 3
Prepare Box for Manual Assembly Step

http://rockinrobotchallenge.eu
Autonomous Systems 2017© Pedro Lima Introduction
 RoCKIn@Work - Functionalities

• Object Perception

• Object Manipulation

• Control

Autonomous Systems 2017© Pedro Lima Introduction 64

 Robot Competitions and Challenges

DARPA Robotics Challenge 2015

Autonomous Systems 2017© Pedro Lima Introduction 65

 Robot Competitions and Challenges (2)

MBZIRC 2017

Autonomous Systems 2017© Pedro Lima Introduction 66

 Components of an Autonomous System

An autonomous system (AS) is a combination of various physical

(hardware) and computational (software) subsystems:
– Acting and Moving – how does an AS actuate over its environment
– Sensing – how does an AS measure properties of itself (e.g., its
location) and of its environment
– Reasoning and Planning – how does na AS map the measurements
into actions, possibly planning short- and long-term actions and motions
– Communication – how does a robot communicate with an outside
operator and with other robots; how does it interact naturally with
humans?

Classical cycle: SENSING – PROCESSING - ACTING

Autonomous Systems 2017© Pedro Lima Introduction 67