Chapter 2 Heuristic Search Techniques

Ms Smita Selot

Contents
A framework for describing search methods is provided and several general purpose search techniques are discussed. All are varieties of Heuristic Search:
Generate and test Hill Climbing Best First Search (A*) Problem Reduction (AO*) Constraint Satisfaction Branch and bound Means-ends analysis

Heuristic
Is a technique that improves the search method Like our tour guides Improve the quality of search path Example general purpose heuristicnearest neighbourhood Applying it to Travelling salesman problem
Select a city arbitrarily Select the next city closest to current city and not visited yet Repeat step 2 till all cities have been visited

Travelling salesmen problem

Start from A

A 6 4 C

3 B 5 2 D 1

Next city it B as cost is minimum From B select D From D select C Finally A-B-D-C This executes in time proportional to N*N better than !N
A 3 4 B 5 C C 2 D 1 C 6 D

Heuristic
Word heuristic come from greek word heuriskein meaning to discover Heuristics is a technique that improves the efficiency of search process by sacrificing the claims of completeness Good heuristic leads in promising direction Bad heuristic will yield inefficient search path Heuristic function is a function that map from problem state descriptions to measure of desirability, represented as numbers They describe goodness of a state Heuristics are not always as general as TSP problem h=f(x,y) where f can be squaring, identity, function

Heuristics
Heuristics are important in real life problem solving process- why?
Rarely we look for optimum solutions
Good solutions do well People are satisfier rather than optimizers Seek solution which satisfy requirements and quit as they find one- example parking space

Approximation produced may not be very good in worst case. Worst case hardly occur in real time Analysing heuristic- leads to better understanding of the problem

Heuristic function
There are 2 ways in which domain specific , heuristic knowledge can be incorporated in rule based search procedure
In rule themselves, where rule depicts set of sensible moves instead of all moves A heuristic function that evaluates individual problem states and determine the degree of desirability
Problem state description Measure of desirability ( number)

Heuristic function maps the two

Heuristic function
Can be simple or complex At times
higher values - good position Lower values good position

Objective of heuristic function : is to guide search process in most profitable direction by suggesting path to follow when more than one are available Good heuristic function less time in search Bad heuristic function will not improve the search Cost of computing heuristic overweighs the effort saved in search process There is trade-off between cost of evaluating hf and the saving in search time that function provides

Problem characteristics
Heuristics are applicable to variety of problems and effectiveness of heuristic depends on class of problem Before applying the heuristic we must analyse the problem under following heads
Is problem decomposable? Can solution step be ignored? Is problem universal predictable? Is good solution to the problem is obvious? Is desired solution a state or path to state? Is knowledge required to solve problem or constraint the search

Problem characteristics
Is problem decomposable?
Take two problems
Integration Block domain problem

Can solution step be ignored?

Ignorable in which solution step can be ignored eg(Mathematical theorem proving) Recoverable in which solution step can be undone (8 puzzle) Irrecoverable in which solution step cannot be undone eg ( chess) Control strategy required may be
simple for ignorable requiring backtracking for recoverable Require great effort in taking decision in irrecoverable- planning

Problem characteristics
Is the universe predictable?
8 puzzle can use planning to identify the exact solution Bridge exact planning not possible so navigate several plans and then choose Difference is due to certain outcome and uncertain outcome Hardest type of problem is one with uncertain outcome and irrecoverable
playing bridge controlling robot arm lawyer helping to defend client from murder charge

Problem characteristics
Is good solution to problem is obvious
Example problem using logic (FOPL) and TSP In logic one may ask question and interested in answer here optimum path does not matter In TSP best path matters Difference is any path solution or best path solution Best path problem is computationally harder to achieve

Is solution a state or path

Eg semantic analysis in NLP require a state as goal Water jug problem requires a path

What is the role of knowledge

Even if unlimited computing power is available Knowledge about good moves , will always help search strategy to perform better Example Chess or newspaper content to decide which party will win next election

Problem classification
When actual problems are analysed , we find that there are several broad classes into which problem falls So classes can be associated with generic control strategy for solving the problem Example classification problem
Medical diagnosis ,fault diagnosis in vehicle

Propose and refine

Design and planning process

Problem classification
Various problem solving method No single way to solve all kind of problem Each problem may not require a complete new strategy Instead
Analyse the problem Sort the various problem solving method by kinds of problem which they solve We can solve the new problem

Problem and problem space

1.Define the problem Problem definition and state space representation of problem 2. Analyse the problem Problem characteristic PRODUCTION SYSTEM

3.Create knowledge base 4. Find best control strategy

Production system
A good way to describe operations that can be performed in a search for solution to problem A production system consist of A set of rules One or more knowledge base A control strategy A rule applier

Production system characteristics

Can production system be represented by set of characteristics What relation exists between problem types and types of production system

Classes of production system

Monotonic production system is a production system in which application of a rule never prevents later application of another rule that could have been applied when at the time the first rule was selected . Non monotonic system never allows this Partially commutative production system if application of particular sequence of rules transform state x to state y , then any permutation of those rules also transform state x to y Commutative production system is one which is both partially commutative as well as monotonic

Categories of production system

Monotonic Partially commutative Not partially commutative Theorem proving Chemical synthesis Non monotonic Robot navigation Bridge

Issues in design of search programs

Every search process is traversal in tree structure It finds path connecting initial and final state Tree could be constructed from rules that define allowable moves in state space It is too large and most of it need never be explored

Issues in design of search programs

Hence instead of building an explicit tree and then searching it Most search program represent tree implicitly in the rules And generate those part that are explicitly explored by algorithm.

Issues in design of search programs

Direction in which to conduct the search
Forward
Initial to final moving left to right in rules

backward reasoning
Final to initial moving right to left

How to select applicable rules (matching)

Production system spend most of the time looking for rules to apply

How to represent each node

Generate-and-Test
Algorithm:
1. Generate a possible solution. For some problems, this means generating a particular point in the problem space. For others it means generating a path from a start state 2. Test to see if this is actually a solution by comparing the chosen point or the endpoint of the chosen path to the set of acceptable goal states. 3. If a solution has been found, quit, Otherwise return to step 1.

Generate-and-Test
It is a depth first search procedure since complete solutions must be generated before they can be tested. In its most systematic form, it is simply an exhaustive search of the problem space. Operate by generating solutions randomly. Also called as British Museum algorithm If a sufficient number of monkeys were placed in front of a set of typewriters, and left alone long enough, then they would eventually produce all the works of Shakespeare. Dendral: which infers the struture of organic compounds using NMR spectrogram. It uses plan-generate-test.

Generate-and-Test

Problem

solution

valid solution

Generator

Tester

Rejected solution

Eg: Mapping pictures

Hill Climbing
Is a variant of generate-and test in which feedback from the test procedure is used to help the generator decide which direction to move in search space. The test function is augmented with a heuristic function that provides an estimate of how close a given state is to the goal state. Computation of heuristic function can be done with negligible amount of computation. Hill climbing is often used when a good heuristic function is available for evaluating states but when no other useful knowledge is available.

Simple Hill Climbing

Algorithm: 1. Evaluate the initial state. If it is also goal state, then return it and quit. Otherwise continue with the initial state as the current state. 2. Loop until a solution is found or until there are no new operators left to be applied in the current state:
a. b. Select an operator that has not yet been applied to the current state and apply it to produce a new state Evaluate the new state
i. ii. If it is the goal state, then return it and quit. If it is not a goal state but it is better than the current state, then make it the current state. iii. If it is not better than the current state, then continue in the loop.

Simple Hill Climbing

The key difference between Simple Hill climbing and Generate-and-test is the use of evaluation function as a way to inject task specific knowledge into the control process. Is one state better than another ? For this algorithm to work, precise definition of better must be provided.

Steepest-Ascent Hill Climbing

This is a variation of simple hill climbing which considers all the moves from the current state and selects the best one as the next state. Also known as Gradient search

Steepest-Ascent Hill Climbing

Algorithm: 1. Evaluate the initial state. If it is also a goal state, then return it and quit. Otherwise, continue with the initial state as the current state. 2. Loop until a solution is found or until a complete iteration produces no change to current state:
a. b. Let SUCC be a state such that any possible successor of the current state will be better than SUCC For each operator that applies to the current state do: i. Apply the operator and generate a new state ii. Evaluate the new state. If is is a goal state, then return it and quit. If not, compare it to SUCC. If it is better, then set SUCC to this state. If it is not better, leave SUCC alone.

c.

If the SUCC is better than the current state, then set current state to SUCC,

: Hill-climbing
This simple policy has three well-known drawbacks:
(a)

1. Local Maxima: a local maximum as opposed to global maximum. 2. Plateaus: An area of the search space where evaluation function is flat, thus requiring random walk. 3. Ridge: Where there are steep slopes and the search direction is not towards the top but towards the side.

(b)

(c)

Figure 5.9 Local maxima, Plateaus and ridge situation for Hill Climbing

Hill-climbing
In each of the previous cases (local maxima, plateaus & ridge), the algorithm reaches a point at which no progress is being made. A solution is to do a random-restart hill-climbing - where random initial states are generated, running each until it halts or makes no discernible progress. The best result is then chosen.

Figure 5.10 Random-restart hill-climbing (6 initial values) for 5.9(a)

Solution to problems of Hillclimbing

Backtrack to some earlier node-this may give a
different direction. A path which was equally promising can be chosen .It requires maintaining list of path followed .Good for handling local maxima .

Make a big jump-to reach a new section of search .Good

for dealing with plateaus.

Apply two or more rules before doing the testmoving in several direction at once .Good for ridges

Best First Search

Combines the advantages of both DFS and BFS into a single method. DFS is good because it allows a solution to be found without all competing branches having to be expanded. BFS is good because it does not get branches on dead end paths. One way of combining the two is to follow a single path at a time, but switch paths whenever some competing path looks more promising than the current one does. At each step of the BFS search process, we select the most promising of the nodes we have generated so far. This is done by applying an appropriate heuristic function to each of them. We then expand the chosen node by using the rules to generate its successors

Best First Search

Similar to Steepest ascent hill climbing with two exceptions: In hill climbing, one move is selected and all the others are rejected, never to be reconsidered. This produces the straightline behavior that is characteristic of hill climbing. In BFS, one move is selected, but the others are kept around so that they can be revisited later if the selected path becomes less promising. The best available state is selected in the BFS, even if that state has a value that is lower than the value of the state that was just explored. This contrasts with hill climbing, which will stop if there are no successor states with better values than the current state.

OR-graph
It is sometimes important to search graphs so that duplicate paths will not be pursued. An algorithm to do this will operate by searching a directed graph in which each node represents a point in problem space. Each node will contain:
Description of problem state it represents Indication of how promising it is Parent link that points back to the best node from which it came List of nodes that were generated from it

OR-Graph
Parent link will make it possible to recover the path to the goal once the goal is found. The list of successors will make it possible, if a better path is found to an already existing node, to propagate the improvement down to its successors. This is called OR-graph, since each of its branches represents an alternative problem solving path

BFS
Step 1 A B Step 4 A B C D B Step 2 A B
3

Step 3 A C D

1 Step 5 A C D E 4 F 6

F 6

5 I

4 J

F 6 1

Implementation of OR graphs
We need two lists of nodes:
OPEN LIST
Nodes that have been generated and have had the heuristic function applied to them but which have not yet been examined. OPEN is actually a priority queue in which the elements with the highest priority are those with the most promising value of the heuristic function.

CLOSED LIST
Nodes that have already been examined. We need to keep these nodes in memory if we want to search a graph rather than a tree since whenever a new node is generated, we need to check whether it has been generated before.

BFS
Step 1 A B Step 4 A B C D B Step 2 A B
3

Step 3 A Close 5 D
3

1 Step 5 A C D

open
E 4 F 6

F 6

5 I

4 J

F 6 1

Algorithm: BFS
Given : s-starting state, n-any node, g- goal state, o-set of operators, c()-cost function

1. 2.

Start with OPEN containing just the initial state

initialization open = {s} closed={ } c{s}=0

Until a goal is found or there are no nodes left on OPEN do

{ fail if open ={ }; terminate if n G then terminate with success } a. Pick the best node on OPEN select : select n from open b Generate its successors Expand Generate the successor of n using set of operators- o

b. For each successor do:

i. If it has not been generated before, evaluate it, add it to OPEN, and record its parent. V succ m , if m ~ {open U closed} set C(m)=C(n)+C(n,m) insert m in open If it has been generated before, change the parent if this new path is better than the previous one. In that case, update the cost of getting to this node and to any successors that this node may already have. if m {open U closed} set C(m) = min{ C(m),C(n)+C(n,m)} if C{m} has decreased and m {closed} move it to open if m {open} change the cost

ii

BFS : simple explanation

It proceeds in steps, expanding one node at each step, until it generates a node that corresponds to a goal state. At each step, it picks the most promising of the nodes that have so far been generated but not expanded. It generates the successors of the chosen node, applies the heuristic function to them, and adds them to the list of open nodes, after checking to see if any of them have been generated before. By doing this check, we can guarantee that each node only appears once in the graph, although many nodes may point to it as a successor.

Examples (2): 8-puzzle

f1(T) = the number correctly placed tiles on the board: 1 3 2
f1
8 5 6 4 7

=4

f2(T) = number or incorrectly placed tiles on board:

gives (rough!) estimate of how far we are from goal 1 3 6 2 4 7

f2

8 5

=4

Most often, distance to goal heuristics are more useful !

44

Examples (3): Manhattan distance

f3(T) = the sum of ( the horizontal + vertical distance that each tile is away from its final destination):
gives a better estimate of distance from the 1 5 2 goal node
f3
8 3 4 7

= 1 + 4 + 2 + 3 = 10

45

Heuristics: Example
8-puzzle: h(n) = tiles out of place

Goal state

1 2 3 8 6 7 5 4

h(n) = 3

1 8 7

2 6

3 4 5

Example - cont
1 2 3 8 6 7 5 4
h(n) = 3 h(n) = 2 h(n) = 4 h(n) = 3

1 8 7

3 6

1 2 3 8 6 4 7 5

1 2 8 6 3 7 5 4

1 2 3 8 6 7 5 4
h(n) = 3

h(n) = 3

h(n) = 2

h(n) = 4

1 8 7

3 6

1 2 3 8 6 4 7 5
h(n) = 3 h(n) = 1

1 2 8 6 3 7 5 4 1 2 3 8 6 4

1 2 8 6 7 5

A* Algorithm
BFS is a simplification of A* Algorithm Presented by Hart et al Algorithm uses:
f: Heuristic function that estimates the merits of each node we generate. This is sum of two components, g and h and f represents an estimate of the cost of getting from the initial state to a goal state along with the path that generated the current node. g : The function g is a measure of the cost of getting from initial state to the current node. h : The function h is an estimate of the additional cost of getting from the current node to a goal state. OPEN CLOSED

A* Algorithm
1. Start with OPEN containing only initial node. Set that nodes g value to 0, its h value to whatever it is, and its f value to h+0 or h. Set CLOSED to empty list.
Initialize open = {s} closed = { } g(s)=0 f(s)=h(s)

2.

Until a goal node is found, repeat the following procedure: (a)If there are no nodes on OPEN, report failure.
fail if open={ } terminate with failure

A* Algorithm
(b)Otherwise pick the node on OPEN with the lowest f value. Call it BESTNODE. Remove it from OPEN. Place it in CLOSED. Select Select minimum cost state n from OPEN Save n in CLOSE (c )See if the BESTNODE is a goal state. If so exit and report a solution. terminate if n G terminate with success and f(n)

A* algorithm
d)Otherwise, generate the successors of BESTNODE For each of the SUCCESSOR, do the following: expand: SUCC m of n if m ~ {open U close } set g(m) = g(n)+C(n,m) set f(m)=g(m)+h(m) Insert m in open set the link to parent node if m {open U close } set g(m)=min[g(m),g(n)+C(n,m)] set f(m)=g(m)+h(m) if f(m) has decreased and m {close } move it to open if f(m) has decreased and m {open} update the cost and change the link to parent node

Observations about A*
Role of g function: This lets us choose which node to expand next on the basis of not only of how good the node itself looks, but also on the basis of how good the path to the node was. h, the distance of a node to the goal. If h is a perfect estimator of h, then A* will converge immediately to the goal with no search.

8-Puzzle Evaluation Function

f(n) = d(n) + W(n)

W(n) counts the number of displaced tiles in that database associated with the node n d(n) is the depth of the node in the search tree

Playing 8-Puzzle
4 2 1 7 8 6 3 4 5

2 1

8 6 7

3 4 5

2 1 7

8 6

3 4 5

2 1 7

8 6 5

3 4

Playing 8-Puzzle
4 2 1 7 6 8 3 4 5

2 7

8 1 6

3 4 5

2 1 7 8 6

3 4 5

2 1 7

8 4 6

3 5

6 2 7

8 1 6

3 4 5

2 7

8 1 6

3 4 5

2 1 8 7 6

3 4 5

2 1 7

3 8 6 4 5

Playing 8-Puzzle
5 1 7 2 8 6 3 4 5

1 7

2 8 6

3 4 5

1 2 8 7 6

3 4 5

1 7

2 8 6

3 4 5

A* special cases f(n)=g(n)+h(n)

When h(n) = 0. => Uniform Cost Method
Search only depends on g

When g(n) = 1, h(n) = 0 => Breadth First When g(n)=0 => Best First Search/A* If non-admissible heuristic
g(n) = 0, h(n) = 1/depth => depth first

One code, many algorithms

Behavior of A* search Underestimation and overestimation Underestimation: If we can guarantee that h never over estimates actual value from current to goal, then A* algorithm is guaranteed to find an optimal path to a goal, if one exists. Consider the following example.

 Assume the cost of all arcs is 1.

We see that f (E) = 5 = f (C) Suppose we resolve in favor of E, the path currently we are expanding. Clearly expansion of F( f = 6) is stopped and we will now expand C. Thus we see that by underestimating h(B), we have wasted some effort but eventually discovered that B was farther away than we thought. Then we go back and try another path, and will find optimal path.

Overestimation:
Now consider another situation. We expand B to E, E to F and F to G for a solution path of length 4.But suppose that there a direct path from D to a solution giving a path of length 2. We will never find it because of overestimating h(D). We may find some other worse solution without ever expanding D. So by overestimating h, we can not be guaranteed to find the cheaper path solution.

A Overestimated (1+3) B (2+2) E (3+1)F (4+0) G

(1+4)

(1+5) D

Properties of A*
A* generates an optimal solution if h(n) is an admissible heuristic and the search space is a tree: h(n) is admissible if it never overestimates the cost to reach the destination node A* generates an optimal solution if h(n) is a consistent heuristic and the search space is a graph: h(n) is consistent if for every node n and for every successor node n of n: h(n) c(n,n) + h(n) h(n)
n d c(n,n) n If h(n) is consistent then h(n) is admissible h(n)

Frequently when h(n) is admissible, it is also consistent

Admissibility of A*
A search algorithm is admissible, if for any graph, it always terminates in an optimal path from initial state to goal state, if path exists. If heuristic function h is underestimate of actual value from current state to goal state, then the it is called admissible function. So, we can say that A* always terminates with the optimal path in case h(x) is an admissible heuristic function.
Monotonicity: A heuristic

function h is monotone if 1. states Xi and Xj such that Xj is successor of Xi h(Xi) h(Xj) cost (Xi, Xj) i.e., actual cost of going from Xi to Xj 2. h (goal) = 0

Gracefull Decay of Admissibility

If h rarely overestimates h by more than , then A* algorithm will rarely find a solution whose cost is more than greater than the cost of the optimal solution. Under certain conditions, the A* algorithm can be shown to be optimal in that it generates the fewest nodes in the process of finding a solution to a problem.

AND-OR graphs
AND-OR graph (or tree) is useful for representing the solution of problems that can be solved by decomposing them into a set of smaller problems, all of which must then be solved. One AND arc may point to any number of successor nodes, all of which must be solved in order for the arc to point to a solution.
Goal: Acquire TV Set

Goal: Steal a TV Set

Goal: Earn some money

Goal: Buy TV Set

AND-OR graph examples

A B 9 C 3 D B A 38 4 17 E 5 F 10 G 3 C D

9 H 4 I

27 J 15 10

Problem Reduction
FUTILITY is chosen to correspond to a threshold such than any solution with a cost above it is too expensive to be practical, even if it could ever be found. Algorithm : Problem Reduction 1. Initialize the graph to the starting node. 2. Loop until the starting node is labeled SOLVED or until its cost goes above FUTILITY:

a.

Traverse the graph,

starting at the initial node following the current best path, accumulate the set of nodes that are on that path and have not yet been expanded or labeled as solved.

b.

Pick one of these nodes

expand it. If there are no successors, assign FUTILITY as the value of this node. Otherwise, add its successors to the graph for each of them compute f. if f of any node is 0, mark that node as SOLVED.

Problem reduction
c. Change the f estimate of the newly expanded node to reflect the new information provided by its successors. Propagate this change backward through the graph.

Note:This propagation of revised cost estimates back up the tree was not necessary in the BFS algorithm because only unexpanded nodes were examined. But now expanded nodes must be reexamined so that the best current path can be selected.

AO* algorithm
GRAPH consist of node representing initial state call this state as INIT
Let G={s}

Until INIT is labelled SOLVED or h > FUTILITY repeat the following

select for expansion one of unexpanded node from marked sub tree and call it NODE n Expand For every successor m of n
Add succ m to graph If succ m is SOLVED , mark it as SOLVED If not SOLVED compute h(m)

Back propagation of the newly discovered information

Call cost revision(n)

AO* algorithm contd..

Cost Revision(n) :Let S be set of nodes that have been SOLVED or
whose value has been changed. Repeat following steps till S is empty

Initialize S ={n} Select from S a node m (select a node possibly whose descendent occurs in G).Remove m from S If s={ } return Compute the cost of each arc emerging from m
If m is AND node with succ r1, r2, r3 ........... rk
Set h(m)= h(ri)+C(m,ri) Mark each succ of m If every succ is SOLVED , mark m as SOLVED

If m is OR node with succ r1, r2 , r3 ............. rk

Set h(m) = min( h(ri)+c(m,ri)) Mark the best succ If best succ is SOLVED mark m as SOLVED

If cost of m has changed then change must be propagated , hence insert all ancestor of m in S for which m is marked successor

Some observations
Cost must be propagated to all path backwards . Hence all ancestor must be added to S- why? Problem may come if graph contain cycles At time longer path may be better than shorter path Disadvantage : No interaction amongst subgoals

Constraint Satisfaction
Constraint Satisfaction problems in AI have goal of discovering some problem state that satisfies a given set of constraints. Design tasks can be viewed as constraint satisfaction problems in which a design must be created within fixed limits on time, cost, and materials. Constraint satisfaction is a search procedure
that operates in a space of constraint sets. The initial state contains the constraints that are originally given in the problem description. A goal state is any state that has been constrained enough where enough must be defined for each problem. For example, in crypt arithmetic, enough means that each letter has been assigned a unique numeric value.

Constraint Satisfaction
Constraint Satisfaction is a two step process:
First constraints are discovered and propagated as far as possible throughout the system. Then if there still not a solution, search begins. A guess about something is made and added as a new constraint Propagation can then occur with this new constraint

First step propagation occurs from fact that there are

dependencies amongst constraint These dependencies occur because many constraints involve more than one object and many object participate in more than one constraint

Constraint propagation also occurs due to presence of inference rules that allow additional constraints to be added which are inferred form the given ones.

Algorithm: Constraint Satisfaction

1. Propagate available constraints.
a. b.
i. ii.

Set OPEN to set of all objects that must have values assigned to them in a complete solution. Then do until an inconsistency is detected or until OPEN is empty:
Select an object OB from OPEN. Strengthen as much as possible the set of constraints that apply to OB. If this set is different from the set that was assigned the last time OB was examined or if this is the first time OB has been examined, then add to OPEN all objects that share any constraints with OB. Remove OB from OPEN.

iii.

2.

If the union of the constraints discovered above defines a solution, then quit and report the solution.

Algorithm: Constraint Satisfaction

3. 4. 5. If the union of the constraints discovered above defines a contradiction, then return the failure. If neither of the above occurs, then make a guess at in order to proceed. To do this loop until a solution is found or all possible solutions have been eliminated:
a. b. c. Select an object whose value is not yet determined select a way of strengthening the constraints on that object. Recursively invoke constraint satisfaction with the current set of constraints augmented by strengthening constraint just selected.

Constraint Satisfaction: Example

Cryptarithmetic Problem: SEND +MORE ----------MONEY Initial State: No two letters have the same value The sums of the digits must be as shown in the problem Goal State: All letters have been assigned a digit in such a way that all the initial constraints are satisfied.

Cryptasithmetic Problem: Constraint Satisfaction

1. 2.

The solution process proceeds in cycles. At each cycle, two significant things are done:
Constraints are propagated by using rules that correspond to the properties of arithmetic. A value is guessed for some letter whose value is not yet determined.

A few Heuristics can help to select the best guess to try first: If there is a letter that has only two possible values and other with six possible values, there is a better chance of guessing right on the first than on the second. Another useful Heuristic is that if there is a letter that participates in many constraints then it is a good idea to prefer it to a letter that participates in a few.

Solving a Cryptarithmetic Problem

Initial state
M=1 S= 8 or 9 O = 0 or 1 -> O =0,S=9 N= E or E+1 -> N= E+1 C2 = 1,C3=0 N+R >8 E<> 9 N=3 R= 8 or 9 2+D = Y or 2+D = 10 +Y

C4 C3 C2 C1 S END + MORE M ONEY

E=5 E=2
N=6; 6+R=5 or 15 R=8,9=> R=8

C2= 0
2+D =Y N+R = 10+E R =9 S =8

C2= 1
2+D = 10 +Y D = 8+Y D = 8 or 9 C2+N+R=14 E=4;Conflict

C2= 0
E=5

C2= 1
C2+N+R=15

D=8
Y= 0 ; Conflict

D=9
Y =1 ; Conflict

D+5=10+Y D=7;Y=2

Game Playing
Games as search problems Efforts in developing game playing algorithm
Charles Babbage and Bowden thought of extending analytical engine to play chess Shannon wrote a paper describing program that can play chess Alan Turing described chess playing algo, which could not be built Samuel succeeded in building first significant operational game playing program
checkers it could play and learn from mistake

Game Playing
Two main reason that game appears to be a good domain for machine intelligence: Provide a structured task in which it is easy to find success or failure Did not require large amount of knowledge Second reason is true only for small games (tic-tac-toe) Example Chess
Average branching factor = 35 On an average game one player can make 50 moves To examine a complete tree we would have to examine 35100 positions

Game Playing
Search Algorithm like generate and test
Generate: generates all possible moves Tester: Evaluates each move

In Chess all 35 moves are genearte and tester must evaluate each of them A better solution is to have
Improved generator which will produce only good path Improved test procedure so that best moves will be recognized first

Game Playing
Use Plausible move generator in which small number of promising move are generated With such a generator, test procedure can spend more time in evaluating these nodes. Search procedure will find a path till goal is reached

Game Playing
Goal: winning / loosing state Hence, like heuristics , use static evaluation function to find the best node Tic-tac-toe = number of winning moves for X- number of winning moves for O Chess: add number of blacks(w), add number of whites(W) quotient W/BTwo concern parameter in game search is depth and branching factor Even with plausible generator, in games like chess, it is not possible to search till goal is found.

MinMax Algorithm
Idealization and simplification:
Depth first depth limited search Two players Alternate moves
MAX player MIN player

Available information:
Perfect: chess, chequers, tic-tac-toe (no chance, same knowledge for the two players) Imperfect: poker, Stratego, bridge

Game representation
Searching a game tree using OR graph with two types of nodes
Min Node: select minimum cost successor Max Node: select maximum cost successor Terminal nodes: winning or loosing states Usually impossible to search upto terminal nodes

Max

Min

Can propagate W or L from bottom up to root node Difficult is state space is large Can be done for simple game like tic-tac-toe

Game playing-MinMax
Games like chess ;complete state space cannot be analyzed We expand the nodes to certain depth (ply) Apply heuristic function Knowledge of game is required to calculate the cost Apply minmax: its like look ahead strategy Also called zero sum game as
if A gets more B will get less If A win B will loose

Search with an opponent

Search with an opponent

Trivial approximation: generating the tree for all moves Terminal moves are tagged with a utility value, for example: +1 or -1 depending on if the winner is MAX or MIN. The goal is to find a path to a winning state. Even if a depth-first search would minimize memory space, in complex games this kind of search cannot be carried out. Even a simple game like tic-tac-toe is too complex to draw the entire game tree.

Search with an opponent

Search with an opponent

Search with an opponent

Heuristic approximation: defining an evaluation function which indicates how close a state is from a winning (or losing) move This function includes domain information. It does not represent a cost or a distance in steps. Conventionally:
A winning move is represented by the value+. A losing move is represented by the value -. The algorithm searches with limited depth.

Each new decision implies repeating part of the search.

Minimax

Minimax-value(n): utility for MAX of being in state n, assuming both players are playing optimally =
Utility(n), maxs Successors(n) Minimax-value(s), mins Successors(n) Minimax-value(s), if n is a terminal state if n is a MAX state if n is a MIN state

Example: tic-tac-toe
e (evaluation function integer) = number of available rows, columns, diagonals for MAX - number of available rows, columns, diagonals for MIN MAX plays with X and desires maximizing e. MIN plays with 0 and desires minimizing e. Symmetries are taken into account. A depth limit is used (2, in the example).

Example: tic-tac-toe

Example: tic-tac-toe

Example: tic-tac-toe

 It is depth first depth limited search

The minimax algorithm

Uses plausible move generator to generate the promising successor(moves/nodes) Uses Static evaluation function with large values indicating good values for us +10 -------------------------- -10 Win for us win for opponent Value generated by static evaluation function is backed up to parents node Since static evaluation function is an estimate , it is better to carry search more than one ply in game tree look ahead
99 99

The minimax algorithm

Our goal is to maximize our move Opponent goal is to minimize our move Hence the name minmax algorithm A (-2) (-6) B C (-2) (-4) D

E (9)

F (-6)

G (0)

H (0)

I (-2)

(-4) (-3)

100

Initial call to minmax will be minmax(current,0,p1)-if player 1 starts otherwise minmax(current,0,p2)

100

The minimax algorithm

Requirement: MOVEGEN (Position Player) generates list of nodes representing the moves that can be made by player in position STATIC(Position , Player) returns a number representing goodness of position with respect to player When to stop recursion ? Uses function DEEP ENOUGH which returns true if search can be stopped False otherwise Return value by the algorithm Value : backed up value Path itself Hence a structure containing both the values is returned

101

101

Minmax (position, depth, player ) { Step1:If DEEP-ENOUGH(position,player) then return structure VALUE=STATIC(position , player) PATH=nil Step2:Else { SUCCESSORS=MOVE_GEN(position,player) Step3: if (SUCCESSOR= empty) return structure // same as DEEP_ENOUGH Step4: if (! Empty(SUCCESSORSvalue )) { BEST_SCORE=min that STATIC can generate For every SUCC of SUCCESSOR 4(a) { RESULT_SUCC= MINMAX(SUCC, depth+1,OPPOSITE(player) 4(b) NEW-VALUE= - VALUE(RESULT_SUCC) 4(c) if NEW-VALUE > BEST_SCORE (i) { BSET_SCORE=NEW-VALUE (ii) BEST-PATH= SUCC + PATH(RESULT_SUCC) path from CURRENT to SUCC . }// end if }// end for
102

 Step5: value=BEST_SCORE path=BEST_PATH return structure }// end if }// else }// minmax

Minmax(A,0,p1) 1 false 2 move-gen(A,p1) 3F 4 succ=(B,C,D) for B result_suc=minmax(B,1,p2) 1F 2 move-gen(b,p2) 3 F 4 succ=(E,F,G) for E result_succ=minmax(E,2,p1) 1. T val=static(E,p1)=9 path=nil return (path=nil,val(9)) newval=-9 best-score=-9 best-path=E return(BP,BS)

The minimax algorithm

The algorithm first recurses down to the tree bottom-left nodes

105

and uses the Utility function on them to discover that their values are 3, 12 and105 8.

The minimax algorithm

A B

Then it takes the minimum of these values, 3, and returns it as the backed-up value of node B. 106 Similar process for the other nodes. 106

The minimax algorithm

The minimax algorithm performs a complete depth-first exploration of the game tree. In minimax, at each point in the process, only the nodes along a path of the tree are considered and kept in memory.

107

107

The minimax algorithm

If the maximum depth of the tree is m, and there are b legal moves at each point, then the time complexity is O(bm). The space complexity is:
O(bm) for an algorithm that generates all successors at once O(m) if it generates successors one at a time.

108

108

The minimax algorithm: problems

For real games the time cost of minimax is totally impractical, but this algorithm serves as the basis:
for the mathematical analysis of games and for more practical algorithms

Problem with minimax search:

The number of game states it has to examine is exponential in the number of moves.

Unfortunately, the exponent cant be 109 eliminated, but it can be cut in half. 109

Alpha-beta pruning
It is possible to compute the correct minimax decision without looking at every node in the game tree. Alpha-beta pruning allows to eliminate large parts of the tree from consideration, without influencing the final decision.

110

110

Alpha-beta pruning

The leaves below B have the values 3, 12 and 8. The value of B is exactly 3. It can be inferred that the value at the root is at least 3, because MAX has a choice worth 3.
111 111

Alpha-beta pruning

C, which is a MIN node, has a value of at most 2. But B is worth 3, so MAX would never choose C. Therefore, there is no point in looking at the other successors of C.
112 112

Alpha-beta pruning

D, which is a MIN node, is worth at most 14. This is still higher than MAXs best alternative (i.e., 3), so Ds other successors are explored.
113 113

Alpha-beta pruning

The second successor of D is worth 5, so the exploration continues.

114 114

Alpha-beta pruning

The third successor is worth 2, so now D is worth exactly 2. MAXs decision at the root is to move to B, giving a value of 3
115 115

Alpha-beta pruning
Alpha-beta pruning gets its name from two parameters.
They describe bounds on the values that appear anywhere along the path under consideration:
= the value of the best (i.e., highest value) choice found so far along the path for MAX = the value of the best (i.e., lowest value) choice found so far along the path for MIN
116 116

Alpha-beta pruning
Alpha-beta search updates the values of and as it goes along. It prunes the remaining branches at a node (i.e., terminates the recursive call)
as soon as the value of the current node is known to be worse than the current or value for MAX or MIN, respectively.

117

117

Alpha-beta bounds
Alpha bound of j
Maximum current value of all max ancestor of j Exploration of a min node ,j, is stopped when its value equals or falls below alpha In min node we update beta

Beta bound of j
The minimum current of all MIN ancestor of j Exploration of a max node j is stopped when its value equals or exceeds beta In a MAX node we update beta

Alpha-beta pruning
MAX {, }

If Vi > , modify If Vi , pruning Return

Vi

MIN

{, }

If Vi < , modify If Vi , pruning

Vi

Return

and bounds are transmitted from parent to child in the order of node visit. The effectiveness of pruning highly depends on the order in which successors are examined.

{alpha = -, beta = +}

{3, +} {-, +}

{-, 3} {-, +}
D

3 {-, 3}
D A

{3, +}

{3, +}

{3, +}
D F G H

The I subtree can be pruned, because I is a min node and the value of v(K) = 0 is < = 3

{3, +}

{3, +} {3, +}
B F G H D J A

{3, +} {3, 5}

3
D

5 {3, +}

3 5
F

{3, 5}

5
A

4 4

3
D

The G subtree can be pruned, because G is a max node and the value of v(M) = 7 is > = 5

Minmax _A_B(position, depth, player ,UT,PT) {

Step1:If DEEP-ENOUGH(position,player) then return structure VALUE=STATIC(position , player) PATH=nil Step2:Else { SUCCESSORS=MOVE_GEN(position,player) Step3: if (SUCCESSOR= empty) return structure // same as DEEP_ENOUGH Step4: if (! Empty(SUCCESSORSvalue )) { BEST_SCORE=min that STATIC can generate For every SUCC of SUCCESSOR 4(a) { RESULT_SUCC= MINMAX_A_B(SUCC, depth+1,OPPOSITE(player),-PT,-UT) 4(b) NEW-VALUE= - VALUE(RESULT_SUCC) 4(c) if NEW-VALUE > PT (i) { PT=NEW-VALUE (ii) BEST-PATH= SUCC + PATH(RESULT_SUCC) path from CURRENT to SUCC . }// end if (d) if PT >= UT then stop examining this branch and return value=PT path=BEST-PATH }// end for Step5: value=PT path=BEST_PATH return structure }// end if }// else

Some observation in alpha beta

Effectiveness of alpha beta depends upon the order in which the nodes are considered If nodes are perfectly ordered then
If x= number of terminal nodes in alpha beta at depth d And y= number of terminal nodes without alpha beta at depth d/2 Then x=2*y Doubling the depth only at cost of double the number of terminal nodes implies a significant gain

This idea can be extended to cut off path that are only slightly better than current path
Exploration of subtree is terminated that offers little possibility of improvement , hence called futility cut off

Additional refinements
Waiting for Quiescence
At times great difference in value is observed as we move from one level to another Our decision to change path should not occur on such short term measures

Secondary search
If total expansion was upto n level we can expand one node an extra level to reassure the path-singular extension

Book moves maintaining the catalogue of all moves (list of possible moves
Incorporates knowledge and search together

Final comments about alphabeta pruning

Pruning does not affect final results. Entire subtrees can be pruned, not just leaves. Good move ordering improves effectiveness of pruning. With perfect ordering, time complexity is O(bm/2).
Effective branching factor of sqrt(b) Consequence: alpha-beta pruning can look twice as deep as minimax in the same amount of time.
125 125

Summary
Four steps to design AI Problem solving: 1. Define the problem precisely. Specify the problem space, the operators for moving within the space, and the starting and goal state. 2. Analyze the problem to determine where it falls with respect to seven important issues. 3. Isolate and represent the task knowledge required 4. Choose problem solving technique and apply them to problem.

Summary
What the states in search spaces represent. Sometimes the states represent complete potential solutions. Sometimes they represent solutions that are partially specified. How, at each stage of the search process, a state is selected for expansion. How operators to be applied to that node are selected. Whether an optimal solution can be guaranteed. Whether a given state may end up being considered more than once. How many state descriptions must be maintained throughout the search process. Under what circumstances should a particular search path be abandoned.