An Improvement to the Scout Tree Search

Algorithm
by Alexander Reinefeld
Tsan-sheng Hsu
tshsu@iis.sinica.edu.tw
http://www.iis.sinica.edu.tw/~tshsu
1
Introduction
It looks like alpha-beta pruning is the best we can do for a
generic searching procedure.
What else can be done generically?
Alpha-beta pruning follows basically the intelligent searching behav-
iors used by human when domain knowledge is not involved.
Can we nd some other knowledge behaviors used by human in
searching?
Intuition: One a MAX node
Suppose we know currently we have a way to gain at least 300 points
at the rst branch.
If there is an ecient way to know the second branch is at most
gaining 300 points, then there is no need to search the second branch
in detail.
Is there a way to search a tree approximately?
Is searching approximately faster than searching exactly?
The same intuition holds for a MIN node.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 2
Scout procedure
Invented by Judea Pearl in 1980.
It is possible to verify whether the value of a branch is great
than a value v or not in a way that is faster than knowing its
exact value.
procedure TEST(J: position, v: value, >: condition)
determine the successor positions p
1
, . . . , p
d
if d = 0, then
return TRUE if value(J) > v
return FALSE otherwise
for i := 1 to d do
if J is a MAX node and TEST(p
i
, v, >) is TRUE then return TRUE
if J is a MIN node and TEST(p
i
, v, >) is FALSE then return FALSE
if J is a MAX node then return FALSE
if J is a MIN node then return TRUE
Condition can be stated as by properly revising the algorithm.
For the condition to be < or , we need to switch conditions for the
MAX and MIN nodes.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 3
EVAL for Scout
Algorithm SCOUT(J: position)
determine the successor positions p
1
, . . . , p
d
if b = 0, then return value(J) else
v = SCOUT(p
1
)
for i := 2 to d do
if J is a MAX node and TEST(p
i
, v, >) is TRUE then
v = SCOUT(p
i
)
if J is a MIN node and TEST(p
i
, v, ) is FALSE then
v = SCOUT(p
i
)
return v
TCG: NegaScout, 20071126, Tsan-sheng Hsu 4
Performance of Scout
Show great improvements on depth > 3 for games with small
branching factors.
It traverses most of the nodes without evaluating them preciously.
Few subtrees remained to be revisited to compute their exact mini-max
values.
Experimental data shows
Scout favors skinny games, that is games with high depth-to-width
ratios.
One depth = 5, it saves over 40% of time.
Maybe bad for games with high branching factors.
Move ordering is very important.
The rst branch, if is good, oers a great chance of pruning further
branches.
A node may be visited more than once.
First visit is to test.
Second visit is to nd its exact value.
Q: Can information in the rst search be used in the second search?
TCG: NegaScout, 20071126, Tsan-sheng Hsu 5
Discussions for SCOUT I
EVAL and TEST may visit less nodes than alpha-beta.
max
min
max
min
5
8
15
10
0
5
8
15
10
0
J
J
K K
SCOUT
ALPHABETA
Assume TEST(J, 5, >) is called by the root after the rst branch is
evaluated.
It calls TEST(K, 5, >) which skips Ks second branch.
TEST(J, 5, >) fails after returning from the 3rd branch.
No need to do Scout for the branch J.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 6
Discussions for SCOUT II
EVAL and TEST is not always better than alpha-beta.
max
min
max
min
max
ALPHABETA
5
10
0
25
20
8
[10, infinity]
[10,25]
[10,25]
[10,25]
TEST[10,A,>]: true
TEST[25,B,>]: false
TEST[0,C,>]: true
SCOUT
5
10
0
25
20
8
A
B
C
D
TEST[8,D,>]: false
TCG: NegaScout, 20071126, Tsan-sheng Hsu 7
Discussions for SCOUT III
When the rst branch of a node has the best value, then TEST
scans the tree fast.
For TEST to return TRUE for a subtree T, it needs to evaluate at
least
one child for a MAX node in T, and
and all of the children for a MIN node in T.
If T has a xed branching factor b and uniform depth d, the number of
nodes evaluated is (b
d/2
).
Compared to alpha-beta pruning whose cut o comes from
bounds of search windows.
It is possible to have some cut-o for alpha-beta as long as there are
some move ordering are good.
For SCOUT to run well, it requires that the rst branch needs to be
the best branch.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 8
Alpha-beta revisited
Denitions:
An alpha-beta search failed-high means it returns a value that is larger
than its upper bound .
An alpha-beta search failed-low means it returns a value that is smaller
than its lower bound .
Null window search:
Using alpha-beta search with the window (m, m + 1).
The result can be either failed-high or failed-low.
Fail high means the return value is at least m + 1.
Equivalent to TEST(J, m, >) is true.
Fail low means the return value is at most m.
Equivalent to TEST(J, m, >) is false.
Versions:
Fail hard alpha-beta Nega-Max version: F2
Fail soft alpha-beta Nega-Max version: F3
Brute force Mini-Max version: F
Always nds the correct answer according to the Mini-Max formula.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 9
Fail hard version
Original version.
Algorithm F2(position p, integer alpha, integer beta)
determine the successor positions p
1
, . . . , p
d
if d = 0, then return f(p) else
begin
m := alpha
for i := 1 to d do
t := F2(p
i
, beta, m)
if t > m then m := t
if m beta then return(m)
end
return m
Properties:
alpha < beta
F2(p, alpha, beta) alpha if F(p) alpha
F2(p, alpha, beta) = F(p) if alpha < F(p) < beta
F2(p, alpha, beta) beta if F(p) beta
F2(p, , +) = F(p)
TCG: NegaScout, 20071126, Tsan-sheng Hsu 10
Fail soft version
Algorithm F3(position p, integer alpha, integer beta)
determine the successor positions p
1
, . . . , p
d
if d = 0, then return f(p) else
begin
m :=
for i := 1 to d do
t := F3(p
i
, beta, m)
if t > m then m := t
if m beta then return(m)
end
return m
Properties:
alpha < beta
F3(p, alpha, beta) alpha if F(p) F3(p, alpha, beta) alpha
F3(p, alpha, beta) = F(p) if alpha < F(p) < beta
F3(p, alpha, beta) beta if F(p) F3(p, alpha, beta) beta
F3(p, , +) = F(p)
TCG: NegaScout, 20071126, Tsan-sheng Hsu 11
Comparisons between F2 and F3
Both versions nd the corrected value v if v is within the
window (, ).
Both versions scan the same set of nodes during searching.
F3 nds a better value when the value is out of the search
window.
Better means a tighter bound.
When it fails high, F3 normally returns a value that is higher than that
of F2.
Never higher than that of F!
When it fails low, F3 normally returns a value that is lower than that
of F2.
Never lower than that of F!
F3 saves about 7% of time when transposition table is used to
store searched results.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 12
NegaScout
Intuition:
Try to incooperate SCOUT and alpha-beta together.
The searching window of alpha-beta if properly set can be used as
TEST in SCOUT.
Using a searching window is better than using a single bound as in
SCOUT.
Modications to the Scout algorithm:
Use Nega-Max approach instead of mini-max.
Traverses the tree with two bounds as alpha-beta does.
A searching window.
Using a fail soft version to get a better result when the returned value
is out of the window,
TCG: NegaScout, 20071126, Tsan-sheng Hsu 13
The NegaScout Algorithm
Algorithm F4(position p, integer alpha, integer beta, integer
depth)
if depth = cut off depth then return f(p)
determine the successor positions p
1
, . . . , p
d
if d = 0, then return f(p) else
begin
m := /* the best value so far */
n := beta
for i := 1 to d do
9: t := F4(p
i
, n, max{alpha, m}, depth + 1)
10: if t > m then
11: if (n = beta or depth cut off depth 2)
12: then m := t
13: else m := F4(p
i
, beta, t, depth + 1)
14: if m beta then return(m)
15: n := max{alpha, m} + 1
end
return m
TCG: NegaScout, 20071126, Tsan-sheng Hsu 14
Search behaviors I
If the depth is enough or it is a terminal position, then stop
searching further.
Return f(p) as the value computed by an evaluation function.
Fail soft version.
For the rst child p
1
, a normal alpha beta searching window is
used.
line 9: normal alpha-beta search for the rst child
the initial value of m is , hence max{alpha, m} = alpha
m is current best value
that is, searching with the normal window (, )
TCG: NegaScout, 20071126, Tsan-sheng Hsu 15
Search behaviors II
For the second child and beyond p
i
, i > 1, rst perform a null
window search for testing whether m is the answer.
line 9: a null-window of (m, m + 1) search for the second child and
beyond.
m is best value obtained so far
ms value will be rst set at line 12 because n = beta
The null window is set at line 15.
line 11: on a smaller depth subtree, i.e., depth at least 2, NegaScout
always returns the best value.
Normally, no need to do alpha-beta or any enhancement on very small
subtrees.
The overhead is too large on small subtrees.
line 13: re-search when the null window search fails high.
The value of this subtree is at least t.
This means the best value in this subtree is more than m, the current
best value.
This subtree must be re-searched with the the window (t, beta).
line 14: the normal pruning from alpha beta.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 16
Example for NegaScout
5
4
6
7
4 4
null window[6,5]
[4,5]
TCG: NegaScout, 20071126, Tsan-sheng Hsu 17
Renements
When a subtree is re-searched, it is best to use information on
the previous search to speed up the current search.
Restart from the position that the value t is returned.
Maybe want to re-search using the normal alpha-beta procedure.
F4 runs much better with a good move ordering and
transposition tables.
Orders the moves in a best-rst list.
Reduces the number of re-searches.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 18
Performances
On a uniform random game tree.
Normally superior to alpha-beta when searching game tree with
branching factors from 20 to 60.
Shows about 10 to 20% improvements.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 19
Comments
Incooperating both Scout and alpha-beta.
Used in state-of-the-art game search engines.
The rst search, though maybe unsuccessful, can provide useful
information in the second search.
Information can be stored.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 20
References and further readings
* A. Reinefeld. An improvement of the scout tree search
algorithm. ICCA Journal, 6(4):414, 1983.
J. Pearl. Asymptotic properties of minimax trees and game-
searching procedures. Articial Intelligence, 14(2):113138,
1980.
John P. Fishburn. Another optimization of alpha-beta search.
SIGART Bull., (84):3738, 1983.
TCG: NegaScout, 20071126, Tsan-sheng Hsu 21