| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The process of visualizing potential moves done by you and your opponent to learn the result of different moves is called "reading". GNU Go does three distinct types of reading: tactical reading which typically is concerned with the life and death of individual strings, Owl reading which is concerned with the life and death of dragons, and connection reading. In this Chapter, we document the tactical reading code, which is in ‘engine/reading.c’.
What we call Tactical Reading is the analysis whether there is a direct capture of a single string, or whether there is a move to prevent such a direct capture.
If the reading module finds out that the string can get captured, this answer should (usually) be trusted. However, if it says it can be defended, this does not say as much. It is often the case that such a string has no chance to make a life, but that it cannot be captured within the horizon (and the cutoff heuristics) of the tactical reading.
The tactical reading is done by the functions in ‘engine/reading.c’. It is a minimax search that declares win for the attacker once he can physically take the string off board, whereas the defense is considered successful when the string has sufficiently many liberties. A string with five liberties is always considered alive. At higher depth within the search tree even fewer liberties cause GNU Go to give up the attack, See depthparams.
The reading code makes use of a stack onto which board positions can
be pushed. The parameter stackp is zero if GNU Go is
examining the true board position; if it is higher than zero, then
GNU Go is examining a hypothetical position obtained by playing
several moves.
The most important public reading functions are attack and
find_defense. These are wrappers for functions do_attack and
do_find_defense which are declared statically in ‘reading.c’. The
functions do_attack and do_find_defense call each other
recursively.
The function do_attack and do_find_defense are wrappers
themselves and call attack1, attack2, attack3 or
attack4 resp. defend1, defend1, defend1
or defend1 depending on the number of liberties.
These are fine-tuned to generate and try out the moves in an efficient
order. They generate a few moves themselves (mostly direct liberties
of the string), and then call helper functions called ..._moves
which suggest less obvious moves. Which of these functions get called
depends on the number of liberties and of the current search depth.
The return codes of the reading (and owl) functions and owl can
be 0, KO_B, KO_A or WIN. Each reading
function determines whether a particular player (assumed to have the
move) can solve a specific problem, typically attacking or defending
a string.
A return code of WIN means success, 0 failure, while KO_A and
KO_B are success conditioned on ko. A function returns KO_A
if the position results in ko and that the player to move
will get the first ko capture (so the opponent has to make the
first ko threat). A return code of KO_B means that the player
to move will have to make the first ko threat.
If GNU Go is compiled with the configure option
‘--enable-experimental-owl-ext’ then the owl functions also have
possible return codes of GAIN and LOSS. A code of GAIN
means that the attack (or defense) does not succeed, but that in the process
of trying to attack or defend, an opponent's worm is captured. A code
of LOSS means that the attack or defense succeeds, but that another
friendly worm dies during the attack or defense.
Depth of reading is controlled by the parameters depth
and branch_depth. The depth has a default value
DEPTH (in ‘liberty.h’), which is set to 16 in the
distribution, but it may also be set at the command line using
the ‘-D’ or ‘--depth’ option. If depth is
increased, GNU Go will be stronger and slower. GNU Go will read
moves past depth, but in doing so it makes simplifying
assumptions that can cause it to miss moves.
Specifically, when stackp > depth, GNU Go assumes that as
soon as the string can get 3 liberties it is alive. This
assumption is sufficient for reading ladders.
The branch_depth is typically set a little below depth.
Between branch_depth and depth, attacks on strings with
3 liberties are considered, but branching is inhibited, so fewer
variations are considered.
%
%Currently the reading code does not try to defend a string by
%attacking a boundary string with more than two liberties. Because
%of this restriction, it can make oversights. A symptom of this is
%two adjacent strings, each having three or four liberties, each
%classified as DEAD. To resolve such situations, a function
%small_semeai() (in ‘engine/semeai.c’) looks for such
%pairs of strings and corrects their classification.
The backfill_depth is a similar variable with a default 12. Below
this depth, GNU Go will try "backfilling" to capture stones.
For example in this situation:
.OOOOOO. on the edge of the board, O can capture X but OOXXXXXO in order to do so he has to first play at a in .aObX.XO preparation for making the atari at b. This is -------- called backfilling. |
Backfilling is only tried with stackp <= backfill_depth. The
parameter backfill_depth may be set using the ‘-B’
option.
The fourlib_depth is a parameter with a default of only 7.
Below this depth, GNU Go will try to attack strings with
four liberties. The fourlib_depth may be set using the
‘-F’ option.
The parameter ko_depth is a similar cutoff. If
stackp<ko_depth, the reading code will make experiments
involving taking a ko even if it is not legal to do so (i.e., it
is hypothesized that a remote ko threat is made and answered
before continuation). This parameter may be set using the
‘-K’ option.
int attack(int str, int *move)
Determines if the string at
strcan be attacked, and if so,*movereturns the attacking move, unless*moveiis a null pointer. (Use null pointers if you are interested in the result of the attack but not the attacking move itself.) ReturnsWIN, if the attack succeeds, 0 if it fails, andKO_AorKO_Bif the result depends on ko Return Codes.
find_defense(int str, int *move)
Attempts to find a move that will save the string at
str. It returns true if such a move is found, with*movethe location of the saving move (unless*moveis a null pointer). It is not checked that tenuki defends, so this may give an erroneous answer if!attack(str). ReturnsKO_AorKO_Bif the result depends on ko See Return Codes.
safe_move(int str, int color) :
The function
safe_move(str, color)checks whether a move atstris illegal or can immediately be captured. Ifstackp==0the result is cached. If the move only can be captured by a ko, it's considered safe. This may or may not be a good convention.
To speed up the reading process, we note that a position can be reached in several different ways. In fact, it is a very common occurrence that a previously checked position is rechecked, often within the same search but from a different branch in the recursion tree.
This wastes a lot of computing resources, so in a number of places, we store away the current position, the function we are in, and which worm is under attack or to be defended. When the search for this position is finished, we also store away the result of the search and which move made the attack or defense succeed.
All this data is stored in a hash table, sometimes also called a transposition table, where Go positions are the key and results of the reading for certain functions and groups are the data. You can increase the size of the Hash table using the ‘-M’ or ‘--memory’ option see section Invoking GNU Go: Command line options.
The hash table is created once and for all at the beginning of
the game by the function hashtable_new(). Although hash
memory is thus allocated only once in the game, the table is
reinitialized at the beginning of each move by a call to
hashtable_clear() from genmove().
| 11.2.1 Calculation of the hash value | ||
| 11.2.2 Organization of the hash table | ||
| 11.2.3 Hash Structures | Structures in ‘hash.h’ |
The hash algorithm is called Zobrist hashing, and is a standard technique for go and chess programming. The algorithm as used by us works as follows:
It is not necessary to specify the color to move (white or black)
as part of the position. The reason for this is that read results
are stored separately for the various reading functions such as
attack3, and it is implicit in the calling function which
player is to move.
These random numbers are generated once at initialization time and then used throughout the life time of the hash table.
The hash table consists of 3 parts:
Each Read Result contains:
When the hash table is created, these 3 areas are allocated using
malloc(). When the hash table is populated, all contents are taken
from the Hash nodes and the Read results. No further allocation is
done and when all nodes or results are used, the hash table is full.
Nothing is deleted from the hash table except when it is totally
emptied, at which point it can be used again as if newly initialized.
When a function wants to use the hash table, it looks up the current
position using hashtable_search(). If the position doesn't already
exist there, it can be entered using
hashtable_enter_position().
Once the function has a pointer to the hash node containing a
function, it can search for a result of a previous search using
hashnode_search(). If a result is found, it can be used, and
if not, a new result can be entered after a search using
hashnode_new_result().
Hash nodes which hash to the same position in the hash table (collisions) form a simple linked list. Read results for the same position, created by different functions and different attacked or defended strings also form a linked list.
This is deemed sufficiently efficient for now, but the representation of collisions could be changed in the future. It is also not determined what the optimum sizes for the hash table, the number of positions and the number of results are.
The basic hash structures are declared in ‘engine/hash.h’ and ‘engine/cache.c’
typedef struct hashposition_t {
Compacttype board[COMPACT_BOARD_SIZE];
int ko_pos;
} Hashposition;
|
Represents the board and optionally the location of a ko, which is an illegal move. The player whose move is next is not recorded.
typedef struct {
Hashvalue hashval;
Hashposition hashpos;
} Hash_data;
|
Represents the return value of a function (hashval) and
the board state (hashpos).
typedef struct read_result_t {
unsigned int data1;
unsigned int data2;
struct read_result_t *next;
} Read_result;
|
The data1 field packs into 32 bits the following fields:
komaster: 2 bits (EMPTY, BLACK, WHITE, or GRAY) kom_pos : 10 bits (allows MAX_BOARD up to 31) routine : 4 bits (currently 10 different choices) str1 : 10 bits stackp : 5 bits |
The data2 field packs into 32 bits the following fields:
status : 2 bits (0 free, 1 open, 2 closed) result1: 4 bits result2: 4 bits move : 10 bits str2 : 10 bits |
The komaster and (kom_pos) field are
documented in See section Ko Handling.
When a new result node is created, 'status' is set to 1 'open'. This is then set to 2 'closed' when the result is entered. The main use for this is to identify open result nodes when the hashtable is partially cleared. Another potential use for this field is to identify repeated positions in the reading, in particular local double or triple kos.
typedef struct hashnode_t {
Hash_data key;
Read_result * results;
struct hashnode_t * next;
} Hashnode;
|
The hash table consists of hash nodes. Each hash node consists of The hash value for the position it holds, the position itself and the actual information which is purpose of the table from the start.
There is also a pointer to another hash node which is used when the nodes are sorted into hash buckets (see below).
typedef struct hashtable {
size_t hashtablesize; /* Number of hash buckets */
Hashnode ** hashtable; /* Pointer to array of hashnode lists */
int num_nodes; /* Total number of hash nodes */
Hashnode * all_nodes; /* Pointer to all allocated hash nodes. */
int free_node; /* Index to next free node. */
int num_results; /* Total number of results */
Read_result * all_results; /* Pointer to all allocated results. */
int free_result; /* Index to next free result. */
} Hashtable;
|
The hash table consists of three parts:
Some calculations can be safely saved from move to move. If the opponent's move is not close to our worm or dragon, we do not have to reconsider the life or death of that group on the next move. So the result is saved in a persistent cache. Persistent caches are used for are used in the engine for several types of read results.
In this section we will discuss the persistent caching of tactical reading but the same principles apply to the other persistent caches.
Persistent caching is an important performance feature. However it can lead to mistakes and debugging problems—situations where GNU Go generates the right move during debugging but plays a wrong move during a game. If you suspect a persistent cache effect you may try loading the sgf file with the ‘--replay’ option and see if the mistake is repeated (see section Invoking GNU Go: Command line options).
The function store_persistent_cache() is called only
by attack and find_defense, never from their
static recursive counterparts do_attack and do_defend.
The function store_persistent_reading_cache() attempts to
cache the most expensive reading results. The function
search_persistent_reading_cache attempts to retrieve a
result from the cache.
If all cache entries are occupied, we try to replace the least useful one. This is indicated by the score field, which is initially the number of nodes expended by this particular reading, and later multiplied by the number of times it has been retrieved from the cache.
Once a (permanent) move is made, a number of cache entries immediately become
invalid. These are cleaned away by the function
purge_persistent_reading_cache(). To have a criterion
for when a result may be purged, the function
store_persistent_cache() computes the
reading shadow and active area. If a permanent
move is subsequently played in the active area, the cached
result is invalidated. We now explain this algorithm in detail.
The reading shadow is the concatenation of all moves in all variations, as well as locations where an illegal move has been tried.
Once the read is finished, the reading shadow is expanded to the active area which may be cached. The intention is that as long as no stones are played in the active area, the cached value may safely be used.
Here is the algorithm used to compute the active area.
This algorithm is in the function store_persistent_reading_cache().
The most expensive readings so far are stored in the persistent cache.
attack1().
The principles of ko handling are the same for tactical reading and owl reading.
We have already mentioned (see section Reading Basics) that GNU Go
uses a return code of KO_A or KO_B if the result depends on
ko. The return code of KO_B means that the position can be won
provided the player whose move calls the function can come up
with a sufficiently large ko threat. In order to verify this,
the function must simulate making a ko threat and having it
answered by taking the ko even if it is illegal. We call such an
experimental taking of the ko a conditional ko capture.
Conditional ko captures are accomplished by the function tryko().
This function is like trymove() except that
it does not require legality of the move in question.
The static reading functions, and the global functions do_attack
and do_find_defense consult parameters komaster,
kom_pos, which are declared static in ‘board.c’. These mediate ko
captures to prevent the occurrence of infinite loops. During
reading, the komaster values are pushed and popped from a stack.
Normally komaster is EMPTY but it can also be
‘BLACK’, ‘WHITE’, GRAY_BLACK, GRAY_WHITE or
WEAK_KO. The komaster is set to color when color makes a
conditional ko capture. In this case kom_pos is set to the location of
the captured ko stone.
If the opponent is komaster, the reading functions will not try to
take the ko at kom_pos. Also, the komaster is normally not
allowed to take another ko. The exception is a nested ko, characterized
by the condition that the captured ko stone is at distance 1 both
vertically and horizontally from kom_pos, which is the location
of the last stone taken by the komaster. Thus in this situation:
.OX
OX*X
OmOX
OO
|
Here if ‘m’ is the location of kom_pos, then the move at
‘*’ is allowed.
The rationale behind this rule is that in the case where there are two kos on the board, the komaster cannot win both, and by becoming komaster he has already chosen which ko he wants to win. But in the case of a nested ko, taking one ko is a precondition to taking the other one, so we allow this.
If the komaster's opponent takes a ko, then both players have taken one ko. In
this case komaster is set to GRAY_BLACK or GRAY_WHITE and
after this further ko captures are even further restricted.
If the ko at kom_pos is filled, then the komaster reverts to
EMPTY.
In detail, the komaster scheme is as follows. Color ‘O’ is to move. This scheme is known as scheme 5 since in versions of GNU Go through 3.4, several different schemes were included.
Komaster remains EMPTY if previous move was not a ko capture. Komaster is set to WEAK_KO if previous move was a ko capture and kom_pos is set to the old value of board_ko_pos.
Komaster is set to O and kom_pos to the location of the ko, where a stone was just removed.
Play at kom_pos is not allowed. Any other ko capture is allowed. If O takes another ko, komaster becomes GRAY_X.
Ko captures are not allowed. If the ko at kom_pos is filled then the komaster reverts to EMPTY.
Komaster is changed to WEAK_X and kom_pos to the old value of board_ko_pos.
To see the komaster scheme in action, consider this position from the file ‘regressions/games/life_and_death/tripod9.sgf’. We recommend studying this example by examining the variation file produced by the command:
gnugo -l tripod9.sgf --decide-dragon C3 -o vars.sgf |
In the lower left hand corner, there are kos at A2 and B4. Black is unconditionally dead because if W wins either ko there is nothing B can do.
8 . . . . . . . . 7 . . O . . . . . 6 . . O . . . . . 5 O O O . . . . . 4 O . O O . . . . 3 X O X O O O O . 2 . X X X O . . . 1 X O . . . . . . A B C D E F G H |
This is how the komaster scheme sees this. B (i.e. X) starts by taking the ko at B4. W replies by taking the ko at A1. The board looks like this:
8 . . . . . . . . 7 . . O . . . . . 6 . . O . . . . . 5 O O O . . . . . 4 O X O O . . . . 3 X . X O O O O . 2 O X X X O . . . 1 . O . . . . . . A B C D E F G H |
Now any move except the ko recapture (currently illegal) at A1 loses for B, so B retakes the ko and becomes komaster. The board looks like this:
8 . . . . . . . . komaster: BLACK 7 . . O . . . . . kom_pos: A2 6 . . O . . . . . 5 O O O . . . . . 4 O X O O . . . . 3 X . X O O O O . 2 . X X X O . . . 1 X O . . . . . . A B C D E F G H |
W takes the ko at B3 after which the komaster is GRAY and
ko recaptures are not allowed.
8 . . . . . . . . komaster: GRAY 7 . . O . . . . . kom_pos: B4 6 . . O . . . . . 5 O O O . . . . . 4 O . O O . . . . 3 X O X O O O O . 2 . X X X O . . . 1 X O . . . . . . A B C D E F G H |
Since B is not allowed any ko recaptures, there is nothing he can do and he is found dead. Thus the komaster scheme produces the correct result.
We now consider an example to show why the komaster is reset
to EMPTY if the ko is resolved in the komaster's favor. This
means that the ko is filled, or else that is becomes no longer
a ko and it is illegal for the komaster's opponent to play
there.
The position resulting under consideration is in the file ‘regressions/games/ko5.sgf’. This is the position:
. . . . . . O O 8 X X X . . . O . 7 X . X X . . O . 6 . X . X X X O O 5 X X . X . X O X 4 . O X O O O X . 3 O O X O . O X X 2 . O . X O X X . 1 F G H J K L M N |
We recommend studying this example by examining the variation file produced by the command:
gnugo -l ko5.sgf --quiet --decide-string L1 -o vars.sgf |
The correct resolution is that H1 attacks L1 unconditionally while K2
defends it with ko (code KO_A).
After Black (X) takes the ko at K3, white can do nothing but retake the ko conditionally, becoming komaster. B cannot do much, but in one variation he plays at K4 and W takes at H1. The following position results:
. . . . . . O O 8 X X X . . . O . 7 X . X X . . O . 6 . X . X X X O O 5 X X . X X X O X 4 . O X O O O X . 3 O O X O . O X X 2 . O O . O X X . 1 F G H J K L M N |
Now it is important the ‘O’ is no longer komaster. Were ‘O’ still komaster, he could capture the ko at N3 and there would be no way to finish off B.
The following alternate schemes have been proposed. It is assumed that ‘O’ is the player about to move.
kom_pos to the location of the ko, where a stone was
just removed.
kom_pos. Komaster parameters unchanged.
kom_pos to the location of the ko, where a stone was
just removed.
is_ko(kom_pos, X) returns false. In that case,
kom_pos is updated to the new ko position, i.e. the stone
captured by this move.
kom_pos. Komaster parameters unchanged.
A superstring is an extended string, where the extensions are through the following kinds of connections:
OO |
... O.O XOX X.X |
OO .. OO |
.O O. |
Like a dragon, a superstring is an amalgamation of strings, but it is a much tighter organization of stones than a dragon, and its purpose is different. Superstrings are encountered already in the tactical reading because sometimes attacking or defending an element of the superstring is the best way to attack or defend a string. This is in contrast with dragons, which are ignored during tactical reading.
The reading code searches for a path through the move tree to determine whether a string can be captured. We have a tool for investigating this with the ‘--decidestring’ option. This may be run with or without an output file.
Simply running
|
will run attack() to determine whether the string can be captured.
If it can, it will also run find_defense() to determine whether or
not it can be defended. It will give a count of the number of
variations read. The ‘-t’ is necessary, or else GNU Go will not
report its findings.
If we add ‘-o output file’ GNU Go will produce an output file with all variations considered. The variations are numbered in comments.
This file of variations is not very useful without a way of navigating the source code. This is provided with the GDB source file, listed at the end. You can source this from GDB, or just make it your GDB init file.
If you are using GDB to debug GNU Go you may find it less
confusing to compile without optimization. The optimization
sometimes changes the order in which program steps are
executed. For example, to compile ‘reading.c’ without optimization,
edit ‘engine/Makefile’ to remove the string -O2 from
the file, touch ‘engine/reading.c’ and make. Note that the
Makefile is automatically generated and may get overwritten
later.
If in the course of reading you need to analyze a result where a function gets its value by returning a cached position from the hashing code, rerun the example with the hashing turned off by the command line option ‘--hash 0’. You should get the same result. (If you do not, please send us a bug report.) Don't run ‘--hash 0’ unless you have a good reason to, since it increases the number of variations.
With the source file given at the end of this document loaded, we can now navigate the variations. It is a good idea to use cgoban with a small ‘-fontHeight’, so that the variation window takes in a big picture. (You can resize the board.)
Suppose after perusing this file, we find that variation 17 is interesting and we would like to find out exactly what is going on here.
The macro 'jt n' will jump to the n-th variation.
(gdb) set args -l [filename] -L [move number] --decidestring [location] (gdb) tbreak main (gdb) run (gdb) jt 17 |
will then jump to the location in question.
Actually the attack variations and defense variations are numbered
separately. (But find_defense() is only run if attack() succeeds,
so the defense variations may or may not exist.) It is redundant to
have to tbreak main each time. So there are two macros avar and dvar.
(gdb) avar 17 |
restarts the program, and jumps to the 17-th attack variation.
(gdb) dvar 17 |
jumps to the 17-th defense variation. Both variation sets are found in the same sgf file, though they are numbered separately.
Other commands defined in this file:
GNU Go does reading to determine if strings can be connected. The algorithms for this are in ‘readconnect.c’. As with the reading code, the connection code is not pattern based.
The connection code is invoked by the engine through the functions:
int string_connect(int str1, int str2, int *move)
Returns
WINifstr1andstr2can be connected.
int disconnect(int str1, int str2, int *move)
Returns
WINifstr1andstr2can be disconnected.
To see the connection code in action, you may try the following example.
gnugo --quiet -l connection3.sgf --decide-connection M3/N7 -o vars.sgf |
(The file ‘connection3.sgf’ is in ‘regression/games’.)
Examine the sgf file produced by this to see what kind of reading
is done by the functions string_connect() and
string_disconnect(), which are called by the function
decide_connection.
One use of the connection code is used is through the autohelper macros
oplay_connect, xplay_connect, oplay_disconnect and
xplay_disconnect which are used in the connection databases.
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This document was generated by Daniel Bump on February, 19 2009 using texi2html 1.78.