IAI : Building Intelligent Agents

John A. Bullinaria, 2005

1. 2. 3. . 5. $. '. +. /.

Rational Agents Intelligent Agents Simple Reflex Agents Reflex Agents !ith an Internal State "oal Base# Agents %tilit& Base# Agents (a)hine *earning ,he Agent -n.ironments 0omponents of an AI Agent

Rational Agents
2An agent is an&thing that )an 3e .ie!e# as perceiving its en.ironment through sensors an# acting upon that en.ironment through actuators.4 5Russell 6 7or.ig, page 328

Stimulus

Receptors/ Sensors

Agent

Effectors/ Actuators

Response

It is no a))i#ent that this #iagram is of exa)tl& the same form as the #iagram for the intera)tion of the human ner.ous s&stem !ith its en.ironment. A rational agent is one that a)ts in a manner that )auses it to 3e as su))essful as it )an. 9hat )ounts as su))essful o3.iousl& #epen#s on !hat !e re:uire the agent to #o. 9e nee# to #etermine an appropriate performance measure in ea)h )ase. ;ne has to 3e )areful in )hoosing that measure, as there are often man& )onfli)ting )riteria that nee# to 3e ta<en into a))ount.
! 12

Ideal Rational Agents

Being rational also #epen#s on !hate.er the agent has per)ei.e# in the past, !hat the agent <no!s a3out its en.ironment, an# !hat a)tions it is a3le to perform. An ideal rational agent is one that ta<es !hate.er a)tion is expe)te# to maximise its performan)e measure on the 3asis of the e.i#en)e pro.i#e# 3& its per)eptual histor& an# !hate.er 3uilt1in <no!le#ge it has. 0arr&ing out a)tions to o3tain useful e.i#en)e for future a)tions is all part of rationalit&. =or example, it is usuall& rational to !ant to learn> If an agent?s a)tions are 3ase# onl& on its 3uilt1in <no!le#ge, an# not on its o!n experien)e !ith its en.ironment, then !e sa& that the agent la)<s autonom&. An autonomous agent !ill supplement its 3uilt1in <no!le#ge !ith its o!n a):uire# 5or learne#8 <no!le#ge in or#er to a)t appropriatel&. It is often a goo# AI strateg& to 3uil# s&stems@agents that ha.e enough <no!le#ge to get them starte#, an# then lea.e them to learn the rest.

Intelligent Agents
Agents are )omprise# of an architecture 5e.g. a )omputer8 plus a program that runs on that ar)hite)ture. In this mo#ule !e are primaril& intereste# in #esigning the programs. In #esigning intelligent s&stems there are four main fa)tors to )onsi#erA B A " Percepts C the inputs to our s&stem Actions C the outputs of our s&stem Goals C !hat the agent is expe)te# to a)hie.e Environment C !hat the agent is intera)ting !ith

9e shall )onsi#er four t&pes of agent s&stem of in)reasing sophisti)ationA 1. 2. 3. . Simple Reflex Agents Reflex Agents !ith an Internal State "oal 3ase# agents %tilit& 3ase# agents

PAGE Descriptors vs. PEAS Descriptors

PAGE Descriptors are not the uni:ue !a& of #es)ri3ing intelligent s&stems. ;ne popular alternati.e in.ol.es PEAS DescriptorsA B A S Performance C ho! !e measure the s&stem?s a)hie.ements Environment C !hat the agent is intera)ting !ith Actuators C !hat pro#u)es the outputs of the s&stem Sensors C !hat pro.i#es the inputs to the s&stem

9e )an see that there is a )lear an# simple mapping 3et!een PAGE an# PEASA Performance Environment Actuators Sensors Pecepts Actions Goals Environment

9hi)h #es)riptors !e use ma<es little #ifferen)e to the s&stems !e 3uil#.

Examples of Intelligent Agents

Agent !pe
(e#i)al #iagnosti) s&stem Satellite image anal&sis s&stem Bart1pi)<ing ro3ot Refiner& )ontroller Intera)ti.e -nglish tutor

Percepts
S&mptoms, test results, patient?s ans!ers Bixels of .ar&ing intensit& an# )olour Bixels of .ar&ing intensit& an# )olour ,emperature, pressure an# )hemi)al rea#ings ,&pe# !or#s

Actions
Duestions, test re:uests, treatments, referrals Fispla& a )ategorisation of the s)ene Bi)< up parts an# sort them into 3ins ;pen an# )lose .al.es, a#Gust temperature Fispla& exer)ises, suggestions, )orre)tions

Goals
Eealth& patients, minimise )osts 0orre)t image )ategorisation Bla)e parts into )orre)t 3ins (aximise purit&, &iel#, safet& (aximise stu#ent?s exam results

Environment
Batient, hospital, staff Images from or3iting satellite 0on.e&or 3elt !ith parts, 3ins Refiner&, staff

Set of stu#ents, exam papers

! 1$

Simple Reflex Agents

=in#ing the A)tion appropriate for a gi.en set of Ber)epts in a loo<1up ta3le is )learl& going to 3e impossi3le for all 3ut the most tri.ial situations, 3e)ause of the prohi3iti.e num3er of entries su)h a ta3le !oul# re:uire. Eo!e.er, one )an often summarise portions of the loo<1up ta3le 3& noting )ommonl& o))urring input@output asso)iations !hi)h )an 3e !ritten as condition-action rulesA if {set of percepts} t"en {set of actions}

,hese are sometimes also )alle# situation-action rules, productions, or if-then rules. "enerall&, large sets of rules li<e these, e.g. if it is raining t"en put up umbrella

ma& together pro#u)e a)tions that appear intelligent, an# the& )an 3e implemente# .er& effi)ientl&. Eumans ha.e man& learne# responses an# innate reflexes of this form. Eo!e.er, !e shall see that their range of appli)a3ilit& is a)tuall& .er& narro!.
! 1'

Reflex Agents #it" an Internal State

;ne pro3lem !ith Simple Reflex Agents is that their a)tions #epen# onl& on the )urrent information pro.i#e# 3& their sensors. If a reflex agent !ere a3le to <eep tra)< of its pre.ious states 5i.e. maintain a #es)ription or 2mo#el4 of its pre.ious histor& in the !orl#8, an# ha# <no!le#ge of ho! the !orl# e.ol.es, then it !oul# ha.e more information a3out its )urrent state, an# hen)e 3e a3le to ma<e 3etter informe# #e)isions a3out its a)tions. Impli)it in the <no!le#ge a3out ho! the !orl# e.ol.es !ill 3e <no!le#ge of ho! the agent?s o!n a)tions affe)t its state. It !ill nee# this so that it )an up#ate its o!n state !ithout ha.ing to rel& on sensor& information. 9e sa& that su)h reflex agents ha.e an internal state. ,he& !ill ha.e a))ess to information@pre#i)tions that are not )urrentl& a.aila3le to their sensors. ,his !ill gi.e them more s)ope for intelligent a)tion than simple reflex agents agents.
! 1+

Goal Based Agents

-.en !ith the in)rease# <no!le#ge of the )urrent state of the !orl# pro.i#e# 3& an agent?s internal state, the agent ma& still not ha.e enough information to tell it !hat to #o. ,he appropriate a)tion for the agent !ill often #epen# on !hat its goals are, an# so it must 3e pro.i#e# !ith some goal information. "i.en <no!le#ge of ho! the !orl# e.ol.es, an# of ho! its o!n a)tions !ill affe)t its state, an agent !ill 3e a3le to #etermine the )onse:uen)es of all 5or man&8 possi3le a)tions. It )an then )ompare ea)h of these against its goal to #etermine !hi)h a)tion5s8 a)hie.es its goal, an# hen)e !hi)h a)tion5s8 to ta<e. If a long se:uen)e of a)tions is re:uire# to rea)h the goal, then Search an# Planning are the su31fiel#s of AI that must 3e )alle# into a)tion. In simple reflex agents, su)h information !oul# ha.e to 3e pre1)ompute# an# 3uilt into the s&stem 3& the #esigner. "oal 3ase# agents ma& appear less effi)ient, 3ut the& are far more flexi3le. 9e )an simpl& spe)if& a ne! goal, rather than re1programming all the rules.
! 1/

$tilit! Based Agents

"oals alone are not enough to generate high :ualit& 3eha.iour. ;ften there are man& se:uen)es of a)tions that )an result in the same goal 3eing a)hie.e#. "i.en appropriate )riteria, it ma& 3e possi3le to )hose the H3est? se:uen)e of a)tions from a num3er that all result in the goal 3eing a)hie.e#. 9e tal< a3out the utility of parti)ular states. An& utilit& 3ase# agent )an 3e #es)ri3e# as possessing a utility function that maps a state, or se:uen)e of states, on to a real num3er that represents its utilit& or usefulness. 9e )an then use the utilit& to )hoose 3et!een alternati.e se:uen)es of a)tions@states that lea# to a gi.en goal 3eing a)hie.e#. 9hen there are )onfli)ting goals, onl& some of !hi)h )an 3e a)hie.e#, the utilit& is a3le to :uantif& the appropriate tra#e1offs. 9hen there are se.eral goals, none of !hi)h )an 3e a)hie.e# !ith )ertaint&, the utilit& allo!s the li<eliness of su))ess to 3e !eighe# up against the importan)e of the goals.
! 110

%ac"ine &earning
9e )an see that per)epts shoul# not onl& 3e use# for generating an agent?s imme#iate a)tions, 3ut also for impro.ing its a3ilit& to a)t in the future, i.e. for learning. In pra)ti)e, learning )an )orrespon# to an&thing from tri.ial memorisation, to the )reation of )omplete s)ientifi) theories. 9e )on.entionall& )lassif& learning into three in)reasingl& #iffi)ult )lassesA Supervised &earning C learning !ith a tea)her 5e.g. the s&stem is tol# !hat outputs it shoul# pro#u)e for ea)h of a set of inputs8 Reinforcement learning C learning !ith limite# fee#3a)< 5e.g. the s&stem must pro#u)e outputs for a set of inputs an# is onl& tol# !hether the& are goo# or 3a#8 $nsupervised learning C learning !ith no help 5e.g. the s&stem is gi.en a set of inputs an# is expe)te# to ma<e some <in# of sense of them on its o!n8 (a)hine learning s&stems )an 3e set up to #o all three t&pes of learning.
! 111

&earning Agents
,he most po!erful agents are a3le to learn 3& a)ti.el& exploring an# experimenting !ith their en.ironment. A general learning agent has four 3asi) )omponentsA 1. 2. 3. . "e Performance Element C !hi)h ta<es in per)epts an# #e)i#es on appropriate a)tions in the same !a& as a non1learning agent. "e 'ritic C !hi)h uses a fixe# stan#ar# of performan)e to tell the learning element ho! !ell the agent is #oing. "e &earning Element C that re)ei.es information from the )riti) an# ma<es appropriate impro.ements to the performan)e element. "e Pro(lem Generator C that suggests a)tions that !ill lea# to ne! an# informati.e experien)es 5e.g. as in )arr&ing out experiments8.

7ote that not all learning agents !ill nee# a pro3lem generator C a tea)her, or the agent?s normal mo#e of operation, ma& pro.i#e suffi)ient fee#3a)< for learning.
! 112

"e Agent Environments

9e ha.e tal<e# a lot a3out intelligent agents, 3ut little a3out the en.ironments !ith !hi)h the& intera)t. ,here are fi.e parti)ularl& useful #imensions along !hi)h one )an )ategoriIe the properties of an agent?s en.ironmentA Accessi(le vs. Inaccessi(le C An en.ironment is accessible to an agent if the agent?s sensor& apparatus gi.es it a))ess to the )omplete state of the en.ironment. ,he agent !ill not then nee# to maintain an internal state to <eep tra)< of the !orl#. Deterministic vs. )on*deterministic C ,he en.ironment is deterministic if its next state is #etermine# )ompletel& 3& its )urrent state an# the a)tions of its agents. A #eterministi) 3ut ina))essi3le en.ironment ma& appear non1#eterministi) to the agent. (ost real en.ironments are so )omplex that, in pra)ti)e, the& ha.e to 3e treate# as non1#eterministi). Discrete vs. 'ontinuous C A discrete en.ironment has a limite#@finite num3er of #istin)t, )learl& #efine# per)epts an# a)tions.
! 113

Episodic vs. )on*episodic C In an episodic en.ironment the agent?s experien)e )an 3e #i.i#e# into 2episo#es4 )onsisting of the agent per)ei.ing an# then pro#u)ing a)tions that #epen# onl& on that episo#e. Su3se:uent episo#es #o not #epen# on pre.ious episo#es, an# so the agent )an limit ho! far it nee#s to thin< ahea#. Static vs. D!namic C A static en.ironment #oes not )hange !hile an agent is #eli3erating. ,he agent !ill then ha.e no nee# to <eep )he)<ing the !orl# !hile it is #e)i#ing on an a)tion, nor !orr& a3out the passage of time. It is )lear that #ifferent t&pes of en.ironment !ill re:uire #ifferent t&pes of agents to #eal !ith them effe)ti.el&. ,he har#est en.ironments to )ope !ith !ill 3e ina))essi3le, non1#eterministi), )ontinuous, non1episo#i), an# #&nami). ;ften !e run an environment program for our agents to intera)t !ith. ,his gi.es ea)h agent its per)epts, re)ei.es an a)tion from ea)h agent, an# then up#ates the en.ironment. It ma& also <eep tra)< of the performan)e measures for ea)h agent.
! 11

'omponents of an AI Agent
9e no! ha.e a goo# i#ea of the )omponents that nee# to 3e 3uilt into an AI agentA 1. 2. 3. . A means to infer properties of the !orl# from its per)epts. Information a3out the !a& the !orl# e.ol.es. Information a3out !hat !ill happen as a result of its possi3le a)tions. %tilit& information in#i)ating the #esira3ilit& of possi3le !orl# states an# the a)tions that lea# to them. "oals that #es)ri3e the )lasses of states !hose a)hie.ement maximises the agent?s utilit&. A mapping from the a3o.e forms of <no!le#ge to its a)tions. An a)ti.e learning s&stem that !ill impro.e the agents a3ilit& to perform !ell.

5.

$. '.

9e shall 3e returning to these issues throughout the remain#er of this mo#ule.

! 115

+vervie# and Reading

1. 2. 3. . 5. $. ,he )on)epts of agent an# rational agent !ere intro#u)e#. ,he four BA"- #es)riptors are Ber)epts, A)tions, "oals, an# -n.ironment. In)reasingl& sophisti)ate# agent t&pes areA Simple reflex agents, Reflex agents !ith an internal state, "oal 3ase# agents, an# %tilit& 3ase# agents. 9e loo<e# at the (a)hine *earning )omponents that )an lea# to impro.e# agent performan)e. =i.e prin)ipal #istin)tions )on)erning agent en.ironments areA A))essi3ilit&, Feterminism, Fis)reteness, -piso#i)ness, an# Stati)ness. 9e en#e# 3& summarising the main )omponents re:uire# of an AI agent.

Reading 1. Russell 6 7or.igA Se)tions 2.1, 2.2, 2.3, 2. , 2.5. J,his le)ture ties in )losel& !ith this material C the other 3oo<s #o not )o.er it .er& !ell.K
! 11$