Вы находитесь на странице: 1из 204

PostgreSQL:

. . aka leopard
Creative Commons Attribution-Noncommercial 2.5
20102012

2
2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . .
2.2 . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
2.3 . . . . . . . . . . . . . . . . . .
. . . . . .
CLUSTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

(1), 2 . . . . . . . . . . . . . . . . . . . .
Web , 2
. . . . . . . . . . . . . . . . . . . . . . . .
Web , 8
. . . . . . . . . . . . . . . . . . . . . . . .
2.5 : pgtune .
2.6 . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . .
1

.
.
.
.
.
.
.
.
.
.
.
.
.

7
7
8
8
9
10
10
14
16
18
18
19
19
20

. 20
. 20
. 20
.
.
.
.
.
.
.
.

21
21
22
22
23
26
26
28


2.7

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3
3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
constraint_exclusion
3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
4.1 . . . . . . . . . . . . . . . . . . . . .
4.2 Streaming Replication ( )
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
4.3 Slony-I . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . .
4.4 Londiste . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . .
4.5 Bucardo . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
4.6 RubyRep . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . .
4.7 . . . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

33
33
34
35
35
37
39
39
41

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

42
42
44
44
45
45
52
53
53
53
53
59
61
64
64
64
65
69
71
72
72
72
72
75
75
75
76
76
78
79

5
81
5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2


5.2

5.3

5.4

5.5

PL/Proxy . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
? . . . . . . . . . . . . . . .
Postgres-XC . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .
(HA) . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
HadoopDB . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

6 PgPool-II
6.1 . . . . . . . . . . . . . . . . . . . . .
6.2 ! . . . . . . . . . . . . . . . . .
pgpool-II . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
PCP . . . . . . . . . . . . .
. . . . . . . . .
/ pgpool-II . . . . . . . . . .
6.3 . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . .
6.4 . . . . . . .
. . . . . . .
SystemDB . . . . . . . . . . . . . .
. . .
. . . . . . . . .
. . . . . . .
6.5 Master-slave . . . . . . . . . . . . . . .
Streaming Replication ( )
6.6 . . . . . . . . . . . . . .
Streaming Replication ( )
6.7 . . . . . . . . . . . . . . . . . . . .
7
7.1 .
7.2 PgBouncer .
7.3 PgPool-II vs

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

82
83
84
87
88
88
90
90
91
97
98
98
98
103
115
116

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

117
117
118
118
119
120
121
121
122
122
123
124
124
125
128
129
130
131
131
132
133
135

136
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
PgBouncer . . . . . . . . . . . . . . . . . . . . . . 137


8 PostgreSQL
8.1 . . . . . . .
8.2 Pgmemcache . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . .

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

139
139
140
140
141
142
146

9
9.1 . . . . . . . . . . . . . . . .
9.2 PostGIS . . . . . . . . . . . . . . . . .
9.3 pgSphere . . . . . . . . . . . . . . . . .
9.4 HStore . . . . . . . . . . . . . . . . . .
. . . . . . . .
. . . . . . . . . . . . . . .
9.5 PLV8 . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
9.6 Smlar . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . .
Smlar . . . . . . . . . . . . . . . . . .
:
. . . . . . . . . . . . . . . . . .
9.7 PostPic . . . . . . . . . . . . . . . . .
9.8 Fuzzystrmatch . . . . . . . . . . . . . .
9.9 Tsearch2 . . . . . . . . . . . . . . . . .
9.10 OpenFTS . . . . . . . . . . . . . . . .
9.11 PL/Proxy . . . . . . . . . . . . . . . .
9.12 Texcaller . . . . . . . . . . . . . . . . .
9.13 Pgmemcache . . . . . . . . . . . . . . .
9.14 Prefix . . . . . . . . . . . . . . . . . .
9.15 Dblink . . . . . . . . . . . . . . . . . .
9.16 Ltree . . . . . . . . . . . . . . . . . . .
9.17 . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

147
147
147
147
148
148
149
150
150
152
155
155
155
156
157
159
162
162
163
165
165
165
166
166
166
166
167
167

. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

168
168
169
170
171
172
172
173

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

10 PostgreSQL
10.1 . . . . . . . . . . . . . . . . .
10.2 SQL . . . . . . . . . . . . . . . . .
SQL . . . .
10.3 . . .
10.4 .
. . . . . . . . . . . . . . . . .
10.5
4


WAL-E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Barman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
10.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
11 PostgreSQL
11.1 . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
11.2 . . . . . . . . .
. . . . . . . . . . . . . .
11.3 . . . . . . . . .
. . . . . . . . . . . . . .
11.4 . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

12 (Performance Snippets)
12.1 . . . . . . . . . . . . . . . . . . . . . . . .
12.2 . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . . .
count . . . . . . . . . . . . . . . . . . . .
- .
. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . .

, . . . . .
LIKE . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . .

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

187
187
188
188
188
189
189
189

.
.
.
.
.
.
.
.
.
.
.
.
.

190
190
190
190
191
192
193
193
194
197
197
197
200
200
202

,

,
.

PostgreSQL.
PostgreSQL,
. ,

.

2


,
.

2.1
, ,
. ,
.

;
.
,
.
, ,
?
, ?
,
.

PostgreSQL.
,
PostgreSQL , ,
FAQ.
postgresql-performance,
.
,
,
. :
- .
7

2.1.


PostgreSQL ,

.
.
: ,
PostgreSQL,
.

( ) ,
. PostgreSQL,
, .
, , ..
PostgreSQL :

;
, ;
;
,
(, ).


PostgreSQL,
, .

.
7.1 ,
.
7.2 :
VACUUM, ;
ANALYZE,
,
;
.
7.4
( IN/NOT IN).
8.0 ,
, CHECKPOINT VACUUM .
8.1
, MIN() MAX(),
pg_autovacuum (),
.
8.2 SQL ,
.
8

2.1.
8.3 , SQL/XML
,
.
8.4 ,
, ,
EXISTS/NOT EXISTS .
9.0 , VACUUM/VACUUM FULL , .
9.1 ,
( ,
), ,

, MIN/MAX.
9.2 ,
, JSON , ,
25%, COPY.
,
8.4.


, ,
,
. ,
,
.
,
,
, ? ,
,

. ,
,
. , ,
:
.
(
).
(, ,
).

.
9

2.2.
(.
3.4).

2.2
,
.
postgresql.conf
.


: shared_buffers
PostgreSQL
. ,
,
, .
,
.
,
, .
, ,
.

,
, .

: , PostgreSQL,
PostgreSQL ,
.

,
PostgreSQL , ,
. ,
,
, .
, shared_buffers,
, ,
,
.
8
2 . ,
, , ,
,
. , ,
10

2.2.
.
, .
:
4 (512)
256512 : 1632
(20484096)
14 : 64256
(819232768)

.

ipcs
(, free vmstat).
1,2 2 ,
. ,
, . ,

. PostgreSQL
, : www.postgresql.org
,
:
Laptop, Celeron processor, 384 RAM, 25 : 12
Athlon server, 1 RAM,
10 : 200
Quad PIII server, 4 RAM, 40 , 150 ,
: 1
Quad Xeon server, 8 RAM, 200 , 300 ,
: 2
: work_mem
sort_mem, ,
,
, .
, work_mem (
).
:
( , ,
, shared_buffers)

, .

, .

11

2.2.
,
.
work_mem postgresql.conf.
1 . 1024.
24%
. -
work_mem, , ,
512 2048 . ,


work_mem 500 . ,
, ,
, . , 1
4 32128 MB.
VACUUM: maintenance_work_mem
PostgreSQL 7.x vacuum_mem.
, VACUUM, ANALYZE,
CREATE INDEX, .
,
, .
50 75%
, , 32
256 . , work_mem.
. ,
14 128512 MB.
Free Space Map: VACUUM FULL
(
PostgreSQL) :
, ,
, , ( );
( UPDATE
DELETE) 1 ( ).
, PostgreSQL VACUUM (
3.1.1).
7.2 VACUUM .
7.2, VACUUM
, SELECT,
1

12

2.2.
INSERT, UPDATE DELETE .
VACUUM FULL.

, , ,
, .
:
max_fsm_relations
,
.
VACUUM. max_fsm_relations

( ).
max_fsm_pages
,
,
.
, ,

VACUUM. ,
VACUUM VERBOSE ANALYZE
, . max_fsm_pages
,
.

FSM, VACUUM
, VACUUM FULL,
.
! 8.4 fsm ,
Free Space Map , .

temp_buffers
, .
16 .
max_prepared_transactions
(PREPARE TRANSACTION). 5.
vacuum_cost_delay
,
, ,
I/O VACUUM,
13

2.2.
. ,
vacuum_cost_delay 0.

50 200 .
vacuum_cost_page_hit vacuum_cost_page_limit.
VACUUM, .
(Jan Wieck) ,
delay 200, page_hit 6 100
VACUUM 80%,
.
max_stack_depth
,
, .
, , .
24 MB.
max_files_per_process
,
.

, Too many
open files.


PostgreSQL :
( )
,
,
.

:
. ,
:
( checkpoint_segments,
3) ,
( checkpoint_timeout, ,
300).
,
, .
: checkpoint_segments
,
1 .
1

.
checkpoint_warning ( ):

14

2.2.
- .

(checkpoint_segments).
( 16 ) .
,
,

.

12 256 ,
(warning) ,
, . ,
, (checkpoint_segments * 2 + 1) * 16 ,
, . ,
32, 1
.
,
,
.
fsync

off fsync.
,
. : ,
, !
,

.
.

commit_delay ( , 0 ) commit_siblings
(5 )

.
commit_siblings ,
commit_delay.

,
, .
, .

15

2.2.
,
.
wal_sync_method
,
. fsync=off, .
:
open_datasync open()
O_DSYNC
fdatasync fdatasync() commit
fsync_writethrough fsync() commit,

fsync fsync() commit
open_sync open()
O_SYNC
.
, .
full_page_writes
off, fsync=off.
,
on, PostgreSQL

. ,
, .
,
.

,
.
full_page_writes
,
(
checkpoint_interval).
wal_buffers
SHARED MEMORY
1 . 256512 ,
. ,
14 2561024 .




. 3 ,
:
1

16

2.2.
default_statistics_target
, ANALYZE (. 3.1.2).
,
, .

ALTER TABLE . . . SET STATISTICS.
effective_cache_size
PostgreSQL
,
1 .
1,5 , shared_buffers
32 , effective_cache_size 800 .
700 , PostgreSQL ,

merge joins.
effective_cache_size 200 ,
,
.
effective_cache_size
2/3 ;
RAM
, .
random_page_cost
,
.

3.0, 2.5 2.0.

,
.
.
, ,
(sequential scans)
(index scans), .
,
, ,
.
.
random_page_cost 2.0; , random_page_cost ,
.
1
,

17

2.3.


PostgreSQL
,
.
, ,
.
, true/false:
track_counts .

, autovacuum .
,
( autovacuum).
track_functions

.
track_activities
.
. ,

,
, .
, ,
.
,
: ,
(
autovacuum ).

2.3
,
.
, , ,
.
PostgreSQL
, ,
.
,
,
.

, ,
, noatime1 .
1

18

2.3.



, .
(
, , :
),
.
, ,
,
,
.
:
(!).
pg_clog pg_xlog,
, .
.
.
,
, ,
,
, , .

CLUSTER
CLUSTER table [ USING index ]
,
.
,
.
,
.
:

, ,
. , CLUSTER
ACCESS EXCLUSIVE ,
( )
. PostgreSQL
,
CLUSTER .

19

2.4.

2.4

.
PostgreSQL
.
RAM ;

shared_buffers = 1/8 RAM ( 1/4);


work_mem 1/20 RAM;
maintenance_work_mem 1/4 RAM;
max_fsm_relations * 1.5;
max_fsm_pages max_fsm_relations * 2000;
fsync = true;
wal_sync_method = fdatasync;
commit_delay = 10 100 ;
commit_siblings = 5 10;
effective_cache_size = 0.9 cached,
free;
random_page_cost = 2 cpu, 4 ;
cpu_tuple_cost = 0.001 cpu, 0.01 ;
cpu_index_tuple_cost = 0.0005 cpu, 0.005
;
autovacuum = on;
autovacuum_vacuum_threshold = 1800;
autovacuum_analyze_threshold = 900;


(1), 2

maintenance_work_mem = 128MB
effective_cache_size = 512MB
work_mem = 640kB
wal_buffers = 1536kB
shared_buffers = 128MB
max_connections = 500

Web , 2

maintenance_work_mem = 128MB;
checkpoint_completion_target = 0.7
effective_cache_size = 1536MB
20

2.5. : pgtune

work_mem = 4MB
wal_buffers = 4MB
checkpoint_segments = 8
shared_buffers = 512MB
max_connections = 500

Web , 8

maintenance_work_mem = 512MB
checkpoint_completion_target = 0.7
effective_cache_size = 6GB
work_mem = 16MB
wal_buffers = 4MB
checkpoint_segments = 8
shared_buffers = 2GB
max_connections = 500

2.5
: pgtune
PostgreSQL Gregory Smith
pgtune1
.
Linux
. , .
:
2.1 Pgtune
pgtune - i $PGDATA/ p o s t g r e s q l . c o n f \
- - o $PGDATA/ p o s t g r e s q l . c o n f . pgtune

Line 1

- i , -- input-config postgresql.conf, -o,


-- output-config postgresql.conf.
.
-M, --memory ,
. , pgtune
.
-T, -- type . : DW, OLTP, Web,
Mixed, Desktop.
-c, -- connections .
,
.
1

http://pgtune.projects.postgresql.org/

21

2.6.
, pgtune
PostgreSQL.
, ,
,
.

2.6

:
1. ,
. :
a) .
.
b) , .
2. .
3. .
4. .


,
.
( cron)
.
ANALYZE

.
.
VACUUM ANALYZE.
, ,
,
ANALYZE.
.
REINDEX
REINDEX
. :
;
.
22

2.6.
. , ,
. PostgreSQL
,
. ,
.
- ,
REINDEX.
: REINDEX, VACUUM FULL,
, ,
.


,
.
,
, , .
. , ,
:
, ,
. , ,
.
.
,
.
,
, , ,
, .
EXPLAIN [ANALYZE]
EXPLAIN [] , PostgreSQL
. EXPLAIN ANALYZE []
1 ,
.
, .
:
(seq scan).

(nested loop).
1

EXPLAIN ANALYZE DELETE . . .

23

2.6.
EXPLAIN ANALYZE:
?
,
.
,
. , ,
, - ,
,
. 80%
,
.
EXPLAIN ANALYZE ,
. ,
SET enable_seqscan=false;
,
, ,
.
postgresql.conf! ,
!


.

:
pg_stat_user_tables

,
,
, ,
.
pg_stat_user_indexes
,
, ,
(
, ,
).
pg_statio_user_tables
,
, ,
(. 2.1.1),
24

2.6.
, ,
TOAST.
, :
(

).
.
, , , ,
PRIMARY KEY UNIQUE.
.
,
,
.
PostgreSQL

/ , ,
. , , foo foo_name,
foo_name = ,
.
CREATE INDEX foo_name_first_idx
ON foo ((lower(substr(foo_name, 1, 1))));

SELECT * FROM foo
WHERE lower(substr(foo_name, 1, 1)) = ;
.
(partial indexes)
WHERE. , ,
scheta uplocheno boolean. ,
uplocheno = false , uplocheno = true,
.
CREATE INDEX scheta_neuplocheno ON scheta (id)
WHERE NOT uplocheno;

SELECT * FROM scheta WHERE NOT uplocheno AND ...;
, ,
WHERE, .
25

2.6.


PostrgeSQL
, PostgreSQL
, .

, , 1
. ,
,
;
.
,
: ,
.


,
,
. ,

, .
SELECT count(*) FROM < >
count() :
, ,

.

(
!)
, ,
,
.

2.2 SQL
Line 1

SELECT count ( * ) FROM f o o ;

foo,
.
, , .
:
1

RULE PostgreSQL SQL, ,


,

26

2.6.
1. , 1 ,
,
ANALYZE:
2.3 SQL
Line 1

SELECT r e l t u p l e s FROM p g _ c l a s s WHERE relname = f o o ;

2. ,
, ,
.
,
. ,

.
3. ,

(cron).
DISTINCT
DISTINCT .
GROUP BY DISTINCT. GROUP BY
, ,
DISTINCT.
2.4 DISTINCT
Line 1
5

p o s t g r e s=# s e l e c t count ( * ) from ( s e l e c t d i s t i n c t i from g )


a;
count
-- ----19125
( 1 row )

Time : 5 8 0 , 5 5 3 ms

10
-

p o s t g r e s=# s e l e c t count ( * ) from ( s e l e c t d i s t i n c t i from g )


a;
count
-- ----19125
( 1 row )

15
-

Time : 3 6 , 2 8 1 ms
1
10000 ,
50000 !

27

2.6.
2.5 GROUP BY
Line 1
5

p o s t g r e s=# s e l e c t count ( * ) from ( s e l e c t i from g group by


i) a;
count
-- ----19125
( 1 row )

Time : 2 6 , 5 6 2 ms

10
-

p o s t g r e s=# s e l e c t count ( * ) from ( s e l e c t i from g group by


i) a;
count
-- ----19125
( 1 row )

15
-

Time : 2 5 , 2 7 0 ms


pgFouine
pgFouine1 log- PostgreSQL,
log- PostgreSQL. pgFouine
,
. pgFouine PHP
-
,

GNU General Public License. ,
log- .
pgFouine PostgreSQL
log-:
syslog
2.6 pgFouine
Line 1
-

log_destination = syslog
redirect_stderr = off
silent_mode = on

, n :
1

http://pgfouine.projects.pgfoundry.org/

28

2.6.
2.7 pgFouine
log_min_duration_statement = n
- log_duration = o f f
- l o g_ s t a te m e n t = none

Line 1

log_min_duration_statement 0. ,
-1.
pgFouine .
HTML- :
2.8 pgFouine
Line 1

p g f o u i n e . php - f i l e your / l o g / f i l e . l o g > your - r e p o r t . html

10
:
2.9 pgFouine
Line 1

p g f o u i n e . php - f i l e your / l o g / f i l e . l o g - top 10 - format t e x t

,
,

pgfouine.projects.pgfoundry.org.

pgBadger
pgBadger1 , pgFouine,
Perl. ,
(
pgFouine 24.02.2010, pgBadger
12.10.2012). pgBadger :
2.10 pgBadger
t a r x z f pgbadger - 2 . x . t a r . gz
cd pgbadger - 2 . x/
- p e r l M a k e f i l e . PL
- make && sudo make i n s t a l l

Line 1
-

pgFouine PostgreSQL :
2.11 PostgreSQL
l o g g i n g _ c o l l e c t o r = on
log_min_messages = debug1
- log_min_error_statement = debug1
- log_min_duration_statement = 0
5 l o g _ l i n e _ p r e f i x = %t [%p ] : [% l - 1 ] u s e r=%u , db=%d

Line 1
-

http://dalibo.github.com/pgbadger/

29

2.6.
10

l o g _ c h e c k p o i n t s = on
l o g _ c o n n e c t i o n s = on
l o g _ d i s c o n n e c t i o n s = on
l o g _ l o c k _ w a i t s = on
log_temp_files = 0

PostgreSQL pgBadger:
2.12 pgBadger
Line 1
5
-

$ . / pgbadger
~/ p g s q l / master / pg_log / p o s t g r e s q l
[========================>] Parsed
10485768 ( 1 0 0 . 0 0 % )
[========================>] Parsed
10485828 ( 1 0 0 . 0 0 % )
[========================>] Parsed
10485851 ( 1 0 0 . 0 0 % )
[========================>] Parsed
10485848 ( 1 0 0 . 0 0 % )
[========================>] Parsed
10485839 ( 1 0 0 . 0 0 % )
[========================>] Parsed
(100.00%)

- 2 0 1 2 - 0 8 - 3 0 _132*
10485768 b y t e s o f
10485828 b y t e s o f
10485851 b y t e s o f
10485848 b y t e s o f
10485839 b y t e s o f
982536 b y t e s o f 982536

HTML ,
PostgreSQL.
dalibo.github.com/pgbadger.
pg_stat_statements
pg_stat_statements
.
, PostgreSQL,
pgFouine pgBadger. :
2.13 pg_stat_statements postgresql.conf
Line 1
-

s h a r e d _ p r e l o a d _ l i b r a r i e s = pg_stat_statements
c u s t o m _ v a r i a b l e _ c l a s s e s = pg_stat_statements #
PostgreSQL 9 . 1

pg_stat_statements . max = 10000


pg_stat_statements . t r a c k = a l l

PostgreSQL .
pg_stat_statements:
1. pg_stat_statements.max (integer) sql
, (
)
30

2.6.
2. pg_stat_statements.track (enum) SQL
.
: top (
/), all ( , )
none ( ).
3. pg_stat_statements.save (boolean)
PostgreSQL. -
.
:
2.14 pg_stat_statements
Line 1

# CREATE EXTENSION pg_stat_statements ;

:
2.15 pg_stat_statements
Line 1
5
10
15
20
-

# SELECT query , c a l l s , t o t a l _ t i m e , rows , 1 0 0 . 0 *


shared_blks_hit /
n u l l i f ( s h a r e d _ b l k s _ h i t + shared_blks_read ,
0 ) AS h i t _ p e r c e n t
FROM pg_stat_statements ORDER BY t o t a l _ t i m e DESC
LIMIT 1 0 ;
- [ RECORD 1
] -- --------------------------------------------------------------------query
| SELECT query , c a l l s , t o t a l _ t i m e , rows , ? *
shared_blks_hit /
|
n u l l i f ( shared_blks_hit +
shared_blks_read , ? ) AS h i t _ p e r c e n t
|
FROM pg_stat_statements ORDER BY
t o t a l _ t i m e DESC LIMIT ? ;
calls
| 3
total_time | 0.994
rows
| 7
hit_percent | 100.0000000000000000
- [ RECORD 2
] -- --------------------------------------------------------------------query
| insert into x ( i ) select generate_series (? ,?) ;
calls
| 2
total_time | 0.591
rows
| 110
hit_percent | 100.0000000000000000
- [ RECORD 3
] -- --------------------------------------------------------------------query
| s e l e c t * from x where i = ? ;
calls
| 2
total_time | 0.157
rows
| 6

31

2.7.
25
-

hit_percent | 100.0000000000000000
- [ RECORD 4
] -- --------------------------------------------------------------------query
| SELECT p g _ s t a t _ s t a t e m e n t s _ r e s e t ( ) ;
calls
| 1
total_time | 0.102
rows
| 1
hit_percent |

pg_stat_statements_reset:
2.16
# SELECT p g _ s t a t _ s t a t e m e n t s _ r e s e t ( ) ;
- [ RECORD 1 ] - - - - - - - - - - - - + - pg_stat_statements_reset |

Line 1
5
10
-

# SELECT query , c a l l s , t o t a l _ t i m e , rows , 1 0 0 . 0 *


shared_blks_hit /
n u l l i f ( s h a r e d _ b l k s _ h i t + shared_blks_read ,
0 ) AS h i t _ p e r c e n t
FROM pg_stat_statements ORDER BY t o t a l _ t i m e DESC
LIMIT 1 0 ;
- [ RECORD 1 ] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - query
| SELECT p g _ s t a t _ s t a t e m e n t s _ r e s e t ( ) ;
calls
| 1
total_time | 0.175
rows
| 1
hit_percent |

, PostgreSQL
9.2 contrib SQL . 9.1 SQL
, select * from table where id =
3 select * from table where id = 21 ,
.

2.7
, PostgreSQL .

,
.
.

32

- ,

.
, ,

.

3.1
(partitioning, )
(, ) .
, .
(

).
(.. ,
).
( 5. . . 10 ) 40. . . 50%
, ( ),
, .
,
( )
, (
). , ,
.
SELECT * FROM articles ORDER BY id DESC LIMIT 10
,
.
, :

33

3.2.
(, ,
)
.

(DROP TABLE ,
DELETE).

.

3.2
PostgreSQL
:
(range)

, ,
. , .
(list)
.
,
:
,
. .
,
.
,
.
,
. ,
. :
3.1
Line 1
-

CHECK ( o u t l e t I D BETWEEN 100 AND 200 )


CHECK ( o u t l e t I D BETWEEN 200 AND 300 )

,
200.
(
), .
,


.
34

3.3.
, constraint_exclusion
postgresql.conf.
,
.

3.3
. ,
,
.
. , , ()
().

, 3 .
, . ,
, , ,
(
). .
.

, :
3.2
Line 1
5
-

CREATE TABLE my_logs (


id
SERIAL PRIMARY KEY,
user_id
INT NOT NULL,
logdate
TIMESTAMP NOT NULL,
data
TEXT,
some_state
INT
);

,
.
.
my_logs,
. ():
3.3
Line 1
5
-

CREATE TABLE my_logs2010m10


CHECK ( l o g d a t e >= DATE
2010 -11 -01 )
) INHERITS ( my_logs ) ;
CREATE TABLE my_logs2010m11
CHECK ( l o g d a t e >= DATE
2010 -12 -01 )
) INHERITS ( my_logs ) ;

(
2 0 1 0 - 1 0 - 0 1 AND l o g d a t e < DATE

(
2 0 1 0 - 1 1 - 0 1 AND l o g d a t e < DATE

35

3.3.
10
-

CREATE TABLE my_logs2010m12


CHECK ( l o g d a t e >= DATE
2011 -01 -01 )
) INHERITS ( my_logs ) ;
CREATE TABLE my_logs2011m01
CHECK ( l o g d a t e >= DATE
2010 -02 -01 )
) INHERITS ( my_logs ) ;

(
2 0 1 0 - 1 2 - 0 1 AND l o g d a t e < DATE

(
2 0 1 1 - 0 1 - 0 1 AND l o g d a t e < DATE

my_logs2010m10,
my_logs2010m11 ..,
( ). CHECK
, (
, !).
logdate,
:
3.4
CREATE INDEX my_logs2010m10_logdate
( logdate ) ;
- CREATE INDEX my_logs2010m11_logdate
( logdate ) ;
- CREATE INDEX my_logs2010m12_logdate
( logdate ) ;
- CREATE INDEX my_logs2011m01_logdate
( logdate ) ;

Line 1

ON my_logs2010m10
ON my_logs2010m11
ON my_logs2010m12
ON my_logs2011m01

,
.
3.5
Line 1
5
10
15
-

CREATE OR REPLACE FUNCTION m y _ l o g s _ i n s e r t _ t r i g g e r ( )


RETURNS TRIGGER AS $$
BEGIN
IF ( NEW. l o g d a t e >= DATE 2 0 1 0 - 1 0 - 0 1 AND
NEW. l o g d a t e < DATE 2 0 1 0 - 1 1 - 0 1 ) THEN
INSERT INTO my_logs2010m10 VALUES (NEW. * ) ;
ELSIF ( NEW. l o g d a t e >= DATE 2 0 1 0 - 1 1 - 0 1 AND
NEW. l o g d a t e < DATE 2 0 1 0 - 1 2 - 0 1 ) THEN
INSERT INTO my_logs2010m11 VALUES (NEW. * ) ;
ELSIF ( NEW. l o g d a t e >= DATE 2 0 1 0 - 1 2 - 0 1 AND
NEW. l o g d a t e < DATE 2 0 1 1 - 0 1 - 0 1 ) THEN
INSERT INTO my_logs2010m12 VALUES (NEW. * ) ;
ELSIF ( NEW. l o g d a t e >= DATE 2 0 1 1 - 0 1 - 0 1 AND
NEW. l o g d a t e < DATE 2 0 1 1 - 0 2 - 0 1 ) THEN
INSERT INTO my_logs2011m01 VALUES (NEW. * ) ;
ELSE

36

3.3.
20
-

RAISE EXCEPTION Date out o f r a n g e .


my_logs_insert_trigger ( ) function ! ;
END IF ;
RETURN NULL;
END;
$$
LANGUAGE p l p g s q l ;

Fix t h e

: logdate,
.
.
:
3.6
CREATE TRIGGER i n s e r t _ m y _ l o g s _ t r i g g e r
BEFORE INSERT ON my_logs
FOR EACH ROW EXECUTE PROCEDURE m y _ l o g s _ i n s e r t _ t r i g g e r ( ) ;

Line 1
-

my_logs:
3.7
INSERT INTO my_logs ( user_id , l o g d a t e , data , some_state )
VALUES( 1 , 2 0 1 0 - 1 0 - 3 0 , 3 0 . 1 0 . 2 0 1 0 data , 1 ) ;
- INSERT INTO my_logs ( user_id , l o g d a t e , data , some_state )
VALUES( 2 , 2 0 1 0 - 1 1 - 1 0 , 1 0 . 1 1 . 2 0 1 0 data2 , 1 ) ;
- INSERT INTO my_logs ( user_id , l o g d a t e , data , some_state )
VALUES( 1 , 2 0 1 0 - 1 2 - 1 5 , 1 5 . 1 2 . 2 0 1 0 data3 , 1 ) ;

Line 1

:
3.8
Line 1
-

p a r t i t i o n i n g _ t e s t=# SELECT * FROM ONLY my_logs ;


i d | u s e r _ i d | l o g d a t e | data | some_state
- - - -+ - - - - - - - - -+ - - - - - - - - -+ - - - - - -+ - - - - - - - - - - - ( 0 rows )

.
:
3.9
Line 1
-

p a r t i t i o n i n g _ t e s t=# SELECT * FROM my_logs ;


id | user_id |
logdate
|
data
|
some_state
- - - -+ - - - - - - - - -+ - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - -

37

3.3.
-

1 |

1 | 2 0 1 0 - 1 0 - 3 0 0 0 : 0 0 : 0 0 | 3 0 . 1 0 . 2 0 1 0 data

1
5

2 |

3 |

2 | 2 0 1 0 - 1 1 - 1 0 0 0 : 0 0 : 0 0 | 1 0 . 1 1 . 2 0 1 0 data2 |
1
1 | 2 0 1 0 - 1 2 - 1 5 0 0 : 0 0 : 0 0 | 1 5 . 1 2 . 2 0 1 0 data3 |
1

( 3 rows )

.
:

3.10
Line 1
5

p a r t i t i o n i n g _ t e s t=# S e l e c t * from my_logs2010m10 ;


id | user_id |
logdate
|
data
|
some_state
- - - -+ - - - - - - - - -+ - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - 1 |
1 | 2 0 1 0 - 1 0 - 3 0 0 0 : 0 0 : 0 0 | 3 0 . 1 0 . 2 0 1 0 data |
1
( 1 row )

10
-

p a r t i t i o n i n g _ t e s t=# S e l e c t * from my_logs2010m11 ;


id | user_id |
logdate
|
data
|
some_state
- - - -+ - - - - - - - - -+ - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - 2 |
2 | 2 0 1 0 - 1 1 - 1 0 0 0 : 0 0 : 0 0 | 1 0 . 1 1 . 2 0 1 0 data2 |
1
( 1 row )

! .
my_logs :
3.11
Line 1
5

p a r t i t i o n i n g _ t e s t=# SELECT * FROM my_logs WHERE u s e r _ i d = 2 ;


id | user_id |
logdate
|
data
|
some_state
- - - -+ - - - - - - - - -+ - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - 2 |
2 | 2 0 1 0 - 1 1 - 1 0 0 0 : 0 0 : 0 0 | 1 0 . 1 1 . 2 0 1 0 data2 |
1
( 1 row )

10
-

p a r t i t i o n i n g _ t e s t=# SELECT * FROM my_logs WHERE data LIKE


%0.1% ;
id | user_id |
logdate
|
data
|
some_state
- - - -+ - - - - - - - - -+ - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - 1 |
1 | 2 0 1 0 - 1 0 - 3 0 0 0 : 0 0 : 0 0 | 3 0 . 1 0 . 2 0 1 0 data |
1
2 |
2 | 2 0 1 0 - 1 1 - 1 0 0 0 : 0 0 : 0 0 | 1 0 . 1 1 . 2 0 1 0 data2 |
1

38

3.3.
-

( 2 rows )



.

. ,
2008 , 10 .
:
3.12
Line 1

DROP TABLE my_logs2008m10 ;

DROP TABLE ,
DELETE.

, ,
, ,
:
3.13
Line 1

ALTER TABLE my_logs2008m10 NO INHERIT my_logs ;

,
.

constraint_exclusion
constraint_exclusion ,
.
, :
3.14 constraint_exclusion OFF
Line 1
-

p a r t i t i o n i n g _ t e s t=# SET c o n s t r a i n t _ e x c l u s i o n = o f f ;
p a r t i t i o n i n g _ t e s t=# EXPLAIN SELECT * FROM my_logs WHERE
logdate > 2010 -12 -01 ;

5
10

QUERY PLAN
-- ------------------------------------------------------------------------R e s u l t ( c o s t = 6 . 8 1 . . 1 0 4 . 6 6 rows =1650 width =52)
-> Append ( c o s t = 6 . 8 1 . . 1 0 4 . 6 6 rows =1650 width =52)
-> Bitmap Heap Scan on my_logs ( c o s t = 6 . 8 1 . . 2 0 . 9 3
rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on my_logs_logdate
( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330 width =0)

39

3.3.
-

15
-

20
-

25
-

Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1


0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Heap Scan on my_logs2010m10 my_logs
( c o s t = 6 . 8 1 . . 2 0 . 9 3 rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on
my_logs2010m10_logdate ( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330
width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Heap Scan on my_logs2010m11 my_logs
( c o s t = 6 . 8 1 . . 2 0 . 9 3 rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on
my_logs2010m11_logdate ( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330
width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Heap Scan on my_logs2010m12 my_logs
( c o s t = 6 . 8 1 . . 2 0 . 9 3 rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on
my_logs2010m12_logdate ( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330
width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Heap Scan on my_logs2011m01 my_logs
( c o s t = 6 . 8 1 . . 2 0 . 9 3 rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on
my_logs2011m01_logdate ( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330
width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
( 2 2 rows )

EXPLAIN,
, ,
logdate > 2010-12-01 ,
, .
constraint_exclusion:
3.15 constraint_exclusion ON
Line 1

p a r t i t i o n i n g _ t e s t=# SET c o n s t r a i n t _ e x c l u s i o n = on ;

40

3.4.
5
10
-

15
-

SET
p a r t i t i o n i n g _ t e s t=# EXPLAIN SELECT * FROM my_logs WHERE
logdate > 2010 -12 -01 ;
QUERY PLAN
-- ------------------------------------------------------------------------R e s u l t ( c o s t = 6 . 8 1 . . 4 1 . 8 7 rows =660 width =52)
-> Append ( c o s t = 6 . 8 1 . . 4 1 . 8 7 rows =660 width =52)
-> Bitmap Heap Scan on my_logs ( c o s t = 6 . 8 1 . . 2 0 . 9 3
rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on my_logs_logdate
( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330 width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Heap Scan on my_logs2010m12 my_logs
( c o s t = 6 . 8 1 . . 2 0 . 9 3 rows =330 width =52)
Recheck Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
-> Bitmap Index Scan on
my_logs2010m12_logdate ( c o s t = 0 . 0 0 . . 6 . 7 3 rows =330
width =0)
Index Cond : ( l o g d a t e > 2 0 1 0 - 1 2 - 0 1
0 0 : 0 0 : 0 0 : : timestamp w i t h o u t time zone )
( 1 0 rows )

, ,
, .
constraint_exclusion , ,
CHECK ,
, .
8.4 PostgreSQL constraint_exclusion on,
off partition.
( )
constraint_exclusion on, off, partition,
CHECK .

3.4

.
,
. ,
, (
) 50%
.

41

,
.
,
.

4.1
(.
replication)
(,
). ,

. , ,
.
.
, ,

. ,
.

(, ).

,
,
- ( , ,
).

,
. ,
, ,
(
,
).

42

4.1.

,
.
,
,

. ,
(
). ,
, , , ,
.
(, ,
).
,
, (
) .
, ,
.
, ,

X, B ,
, Y X.
Y, Y (, -
), B Y
, , ,
. ,
Y , X .

. ,
, ,
(
) .

: ,

. , ,
.

, .

,
.
,
,
,
.
,
43

4.2. Streaming Replication ( )


.
PostgreSQL ,
, .
(, ).
:
Slony-I Master-Slave ,
(cascading) (failover).
Slony-I
PostgreSQL INSERT/
DELETE/UPDATE
.
PGCluster Multi-Master .
, .
pgpool-I/II PostgreSQL (
II ). :
( ,
stand-by );
online-;
pooling ;
;

SELECT-

postgresql-;

.
Bucardo ,
Multi-Master Master-Slave ,
.
Londiste Master-Slave .
Skytools1 . , Slony-I.
Mammoth Replicator Multi-Master .
Postgres-R Multi-Master .
RubyRep Ruby, Multi-Master
, PostgreSQL MySQL.
, , ,
PostgreSQL.

4.2 Streaming Replication ( )

(Streaming Replication, SR)


wall xlog
1

http://pgfoundry.org/projects/skytools/

44

4.2. Streaming Replication ( )


.
PostgreSQL 9 ( !).
, , ,

, PostgreSQL.
:

PostgreSQL



-

:

PostgreSQL
(
, ,
,
, Hot Standby)

PostgreSQL 9 .
9.0.1 . ,
, Linux.

masterdb(192.168.0.10)
slavedb(192.168.0.20).


ssh. postgres . ,
:
4.1 userssh
$sudo groupadd u s e r s s h
$sudo u s e r a d d -m - g u s e r s s h - d /home/ u s e r s s h - s / b i n / bash \
- - c " user ssh allow " userssh

Line 1
-

(
postgres):

45

4.2. Streaming Replication ( )


4.2 postgres
Line 1

su p o s t g r e s

RSA-
:
4.3 RSA-
Line 1
5
-

p o s t g r e s @ l o c a l h o s t ~ $ ssh - keygen - t r s a -P ""


G e n e r a t i n g p u b l i c / p r i v a t e r s a key p a i r .
Enter f i l e i n which t o s a v e t h e key
( / var / l i b / p o s t g r e s q l / . s s h / i d _ r s a ) :
Created d i r e c t o r y / var / l i b / p o s t g r e s q l / . s s h .
Your i d e n t i f i c a t i o n has been saved i n
/ var / l i b / p o s t g r e s q l / . s s h / i d _ r s a .
Your p u b l i c key has been saved i n
/ var / l i b / p o s t g r e s q l / . s s h / i d _ r s a . pub .
The key f i n g e r p r i n t i s :
1 6 : 0 8 : 2 7 : 9 7 : 2 1 : 3 9 : b5 : 7 b : 8 6 : e1 : 4 6 : 9 7 : b f : 1 2 : 3 d : 7 6
postgres@localhost

:
4.4
Line 1

c a t $HOME/ . s s h / i d _ r s a . pub >> $HOME/ . s s h / a u t h o r i z e d _ k e y s

,
:

4.5 ssh
Line 1

ssh l o c a l h o s t

sshd:
4.6 sshd
Line 1

/ e t c / i n i t . d/ ss hd s t a r t

$HOME/.ssh
slavedb.
ssh.
pg_hba.conf ,
(trust) ( replication):
4.7 pg_hba.conf
Line 1

host

replication

all

192.168.0.20/32

trust

192.168.0.10/32

trust

4.8 pg_hba.conf
Line 1

host

replication

all

postgresql .
46

4.2. Streaming Replication ( )



masterdb. postgresql.conf
:
4.9
# To e n a b l e read - o n l y q u e r i e s on a standby s e r v e r ,
w a l _ l e v e l must be s e t t o
- # " hot_standby " . But you can c h o o s e " a r c h i v e " i f you n e v e r
connect to the
- # s e r v e r i n standby mode .
- w a l _ l e v e l = hot_standby

Line 1

# S e t t h e maximum number o f c o n c u r r e n t c o n n e c t i o n s from t h e


standby s e r v e r s .
- max_wal_senders = 5
-

10
15
-

# To p r e v e n t t h e primary s e r v e r from removing t h e WAL


segments r e q u i r e d f o r
# t h e standby s e r v e r b e f o r e s h i p p i n g them , s e t t h e minimum
number o f segments
# r e t a i n e d i n t h e pg_xlog d i r e c t o r y . At l e a s t
wal_keep_segments s h o u l d be
# l a r g e r than t h e number o f segments g e n e r a t e d between t h e
beginning of
# o n l i n e - backup and t h e s t a r t u p o f s t r e a m i n g r e p l i c a t i o n .
I f you e n a b l e WAL
# a r c h i v i n g t o an a r c h i v e d i r e c t o r y a c c e s s i b l e from t h e
standby , t h i s may
# not be n e c e s s a r y .
wal_keep_segments = 32

20
-

# Enable WAL a r c h i v i n g on t h e primary t o an a r c h i v e


d i r e c t o r y a c c e s s i b l e from
# t h e standby . I f wal_keep_segments i s a h i g h enough number
t o r e t a i n t h e WAL
# segments r e q u i r e d f o r t h e standby s e r v e r , t h i s may not be
necessary .
archive_mode
= on
archive_command = cp %p / path_to / a r c h i v e/% f

:
wal_level = hot_standby WAL
archive, ,
( archive,
).
max_wal_senders = 5 .
47

4.2. Streaming Replication ( )


wal_keep_segments = 32 c
WAL pg_xlog .
archive_mode = on WAL
archive_command .
/path_to/archive/.
- . 9.1
synchronous_standby_names,
. application_name,
recovery.conf:
4.10 recovery.conf
restore_command = cp /tmp/% f %p
# e . g . cp
/mnt/ s e r v e r / a r c h i v e d i r /% f %p
- standby_mode = on
- primary_conninfo = h o s t=l o c a l h o s t p o r t =59121
u s e r=r e p l i c a t i o n password=r e p l i c a t i o n
application_name=n e w c l u s t e r
# e.g.
h o s t=l o c a l h o s t p o r t =5432
- t r i g g e r _ f i l e = /tmp/ t r i g _ f _ n e w c l u s t e r

Line 1

PostgreSQL .
slavedb.

slavedb
masterdb. .
masterdb . :
4.11
Line 1

p s q l - c "SELECT pg_start_backup ( l a b e l , t r u e ) "

.
:
4.12
Line 1
-

r s y n c -C - a - - d e l e t e - e s s h - - e x c l u d e p o s t g r e s q l . c o n f
- - exclude postmaster . pid \
- - e x c l u d e p o s t m a s t e r . o p t s - - e x c l u d e pg_log - - e x c l u d e
pg_xlog \
- - e x c l u d e r e c o v e r y . c o n f master_db_datadir /
s l a v e d b _ h o s t : slave_db_datadir /

master_db_datadir postgresql
masterdb
48

4.2. Streaming Replication ( )


slave_db_datadir postgresql slavedb
slavedb_host slavedb( - 192.168.1.20)
,
. :
4.13
Line 1

p s q l - c "SELECT pg_stop_backup ( ) "

postgresql.conf,
( ).
:
4.14
Line 1

hot_standby = on

! wal_level = archive,
(hot_standby = off).
slavedb PostgreSQL
recovery.conf :
4.15 recovery.conf
# S p e c i f i e s whether t o s t a r t t h e s e r v e r a s a standby . In
streaming r e p l i c a t i o n ,
- # t h i s parameter must t o be s e t t o on .
- standby_mode
= on

Line 1

# S p e c i f i e s a c o n n e c t i o n s t r i n g which i s used f o r t h e
standby s e r v e r t o c o n n e c t
- # with t h e primary .
- primary_conninfo
= h o s t = 1 9 2 . 1 6 8 . 0 . 1 0 p o r t =5432
u s e r=p o s t g r e s

# S p e c i f i e s a t r i g g e r f i l e whose p r e s e n c e s h o u l d c a u s e
streaming r e p l i c a t i o n to
10 # end ( i . e . , f a i l o v e r ) .
- t r i g g e r _ f i l e = / path_to / t r i g g e r
-

15
-

# S p e c i f i e s a command t o l o a d a r c h i v e segments from t h e WAL


archive . I f
# wal_keep_segments i s a h i g h enough number t o r e t a i n t h e
WAL segments
# r e q u i r e d f o r t h e standby s e r v e r , t h i s may not be
n e c e s s a r y . But
# a l a r g e workload can c a u s e segments t o be r e c y c l e d b e f o r e
t h e standby
# i s f u l l y s y n c h r o n i z e d , r e q u i r i n g you t o s t a r t a g a i n from
a new b a s e backup .

49

4.2. Streaming Replication ( )


-

restore_command = s c p masterdb_host : / path_to / a r c h i v e/% f


"%p"

standby_mode=on
primary_conninfo
trigger_file -,
.
restore_command , WAL
. scp masterdb (masterdb_host
- masterdb).
PostgreSQL slavedb.


:
4.16
Line 1
5

$ p s q l - c "SELECT p g _ c u r r e n t _ x l o g _ l o c a t i o n ( ) "
- h192 . 1 6 8 . 0 . 1 0 ( masterdb )
pg_current_xlog_location
-- -----------------------0/2000000
( 1 row )

10
-

$ psql - c " s e l e c t pg_last_xlog_receive_location () "


- h192 . 1 6 8 . 0 . 2 0 ( s l a v e d b )
pg_last_xlog_receive_location
-- ----------------------------0/2000000
( 1 row )

15
-

$ psql - c " s e l e c t pg_last_xlog_replay_location () "


- h192 . 1 6 8 . 0 . 2 0 ( s l a v e d b )
pg_last_xlog_replay_location
-- ---------------------------0/2000000
( 1 row )

9.1 view
. master slaves:
4.17
Line 1

# SELECT * from p g _ s t a t _ r e p l i c a t i o n ;

50

4.2. Streaming Replication ( )


-

procpid | usesysid |
usename
| application_name |
client_addr | client_hostname | client_port |
backend_start
|
state
| sent_location |
write_location | flush_location | replay_location |
sync_priority | sync_state

- - - - - - - - -+ - - - - - - - - - -+ - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - -+ - - - 17135 |
16671 | r e p l i c a t i o n | n e w c l u s t e r
|
127.0.0.1
|
|
43745 | 2 0 1 1 - 0 5 - 2 2
1 8 : 1 3 : 0 4 . 1 9 2 8 3 + 0 2 | s t r e a m i n g | 1/30008750
|
1/30008750
| 1/30008750
| 1/30008750
|
1 | sync

9.1 view pg_stat_database_conflicts,



:
4.18
Line 1
-

# SELECT * from p g _ s t a t _ d a t a b a s e _ c o n f l i c t s ;
d a t i d | datname | c o n f l _ t a b l e s p a c e | c o n f l _ l o c k |
confl_snapshot | confl_bufferpin | confl_deadlock

5
-

- - - - - - -+ - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - -+ - - - - - - - - - - - - - - - -+ - - 1 | template1 |
0 |
0 |
0 |
0 |
0
11979 | t e m p l a t e 0 |
0 |
0 |
0 |
0 |
0
11987 | p o s t g r e s |
0 |
0 |
0 |
0 |
0
16384 | marc
|
0 |
0 |
1 |
0 |
0

ps:
4.19
Line 1
-

[ masterdb ] $ ps - e f | g r e p s e n d e r
p o s t g r e s 6879 6831 0 1 0 : 3 1 ?
00:00:00 postgres :
wal s e n d e r p r o c e s s p o s t g r e s 1 2 7 . 0 . 0 . 1 ( 4 4 6 6 3 ) s t r e a m i n g
0/2000000

[ s l a v e d b ] $ ps - e f | g r e p r e c e i v e r
p o s t g r e s 6878 6872 1 1 0 : 3 1 ?
00:00:01 postgres :
wal r e c e i v e r p r o c e s s
s t r e a m i n g 0/2000000

. :
4.20
Line 1

$ p s q l test_db

51

4.2. Streaming Replication ( )


5
-

test_db=# c r e a t e t a b l e t e s t 3 ( i d
key , name v a r c h a r ( 2 0 ) ) ;
NOTICE : CREATE TABLE / PRIMARY
i n d e x " test3_pkey " f o r t a b l e
CREATE TABLE
test_db=# i n s e r t i n t o t e s t 3 ( id ,
INSERT 0 1
test_db=#

i n t not n u l l primary
KEY w i l l c r e a t e i m p l i c i t
" test3 "
name ) v a l u e s ( 1 , t e s t 1 ) ;

:
4.21
Line 1
5
-

$ p s q l test_db
test_db=# s e l e c t * from t e s t 3 ;
i d | name
- - - -+ - - - - - - 1 | test1
( 1 row )

,
.



- (trigger_file) ,
.

- (trigger_file) . 9.1
pg_xlog_replay_pause() pg_xlog_replay_resume()
.

.

,
.

PostgreSQL .

52

4.3. Slony-I

, ,
. PostgreSQL
.

4.3 Slony-I

Slony ,
PostgreSQL .
Slony Postgre INSERT/
DELETE/UPDATE .
Slony
, slony
slonik. slonik-,
slon .
, slon , .
slonik-e
slonik stdin.

slonik-a ,
, ,
slonik syntax error,
. . .

Ubuntu :
4.22
Line 1

sudo a p t i t u d e i n s t a l l s l o n y 1 - b i n


customers
(
,

).

: customers
master_host: customers_master.com
slave_host_1: customers_slave.com
cluster name ( ): customers_rep
53

4.3. Slony-I
master-
Postgres,
Slony. , ,
slony.
4.23 master-
Line 1
-

pgsql@customers_master$ c r e a t e u s e r - a - d s l o n y
pgsql@customers_master$ p s q l - d t e m p l a t e 1 - c " a l t e r \
u s e r s l o n y with password slony_user_password ; "


slony, slon.
, ( slon)
.
slave-
,
Internet ( ),
PostgreSQL
-, . , :
4.24 slave-
Line 1
-

anyuser@customers_slave$ p s q l - d c u s t o m e r s \
- h customers_master . com -U s l o n y

- ( ,
). - ,
firewall-a, pg_hba.conf, $PGDATA.
slave- PostgreSQL.
, Postgres up and ready,
- ,
(
postmaster):
4.25 slave-
Line 1
5
-

p g s q l @ c u s t o m e r s _ s l a v e $ rm - r f $PGDATA
p g s q l @ c u s t o m e r s _ s l a v e $ mkdir $PGDATA
p g s q l @ c u s t o m e r s _ s l a v e $ i n i t d b -E UTF8 -D $PGDATA
pgsql@customers_slave$ c r e a t e u s e r - a -d slony
pgsql@customers_slave$ psql -d template1 - c " a l t e r \
u s e r s l o n y with password slony_user_password ; "

postmaster.
!

. !

54

4.3. Slony-I
4.26 slave-
Line 1
-

p g s q l @ c u s t o m e r s _ s l a v e $ c r e a t e u s e r - a - d customers_owner
pgsql@customers_slave$ psql -d template1 - c " a l t e r \
u s e r customers_owner with password
customers_owner_password ; "

customers_master,
-h customers_slave,
slave.
slave, master, Slony.
plpgsql slave
slony.

(slony_user_password). :
4.27 plpgsql slave
Line 1
-

slony@customers_master$ c r e a t e d b -O customers_owner \
- h c u s t o m e r s _ s l a v e . com c u s t o m e r s
slony@customers_master$ c r e a t e l a n g - d c u s t o m e r s \
- h c u s t o m e r s _ s l a v e . com p l p g s q l

! , replication set
primary key. -
, primary
key ALTER TABLE ADD PRIMARY KEY.
primary key ,
serial (ALTER TABLE ADD COLUMN),
. table add
key slonik-a.
. slave:
4.28 plpgsql slave
Line 1
-

slony@customers_master$ pg_dump - s c u s t o m e r s | \
p s q l -U s l o n y - h c u s t o m e r s _ s l a v e . com c u s t o m e r s

pg_dump -s .
pg_dump -s customers , psql
-U slony -h customers_slave.com customers
(slony_user_pass). : -
Slony ( make install),
sl_*, . , :

( 5)
:-) slave
:
55

4.3. Slony-I
4.29 plpgsql slave
Line 1
5
-

s l o n i k <<EOF
c l u s t e r name = c u s t o m e r s _ s l a v e ;
node Y admin c o n n i n f o = dbname=c u s t o m e r s
h o s t=customers_master . com
p o r t =5432 u s e r=s l o n y password=slony_user_pass ;
u n i n s t a l l node ( i d = Y) ;
echo okay ;
EOF

Y . . : , cluster
name - , T1 (default).
uninstall.
( ),
uninstall ( master slave).

PgSQL
, - ,
.
- :
4.30
Line 1

#!/ b i n / sh

CLUSTER=customers_rep

5
-

DBNAME1=c u s t o m e r s
DBNAME2=c u s t o m e r s

HOST1=customers_master . com
- HOST2=c u s t o m e r s _ s l a v e . com
-

10

PORT1=5432
- PORT2=5432
-

SLONY_USER=s l o n y

15
20
-

s l o n i k <<EOF
c l u s t e r name = $CLUSTER ;
node 1 admin c o n n i n f o = dbname=$DBNAME1 h o s t=$HOST1
p o r t=$PORT1
u s e r=s l o n y password=slony_user_password ;
node 2 admin c o n n i n f o = dbname=$DBNAME2 h o s t=$HOST2
p o r t=$PORT2 u s e r=s l o n y password=slony_user_password ;

56

4.3. Slony-I
-

i n i t c l u s t e r ( i d = 1 , comment = Customers DB
replication cluster ) ;

25

echo C r e a t e s e t ;

c r e a t e s e t ( i d = 1 , o r i g i n = 1 , comment = Customers
- DB r e p l i c a t i o n s e t ) ;
30

echo Adding t a b l e s t o t h e s u b s c r i p t i o n s e t ;

echo Adding t a b l e p u b l i c . c u s t o m e r s _ s a l e s . . . ;
- s e t add t a b l e ( s e t i d = 1 , o r i g i n = 1 , i d = 4 , f u l l
qualified
- name = p u b l i c . c u s t o m e r s _ s a l e s , comment = Table
public . customers_sales ) ;
35 echo done ;
-

40
-

echo Adding t a b l e p u b l i c . customers_something . . . ;


s e t add t a b l e ( s e t i d = 1 , o r i g i n = 1 , i d = 5 , f u l l
qualified
name = p u b l i c . customers_something ,
comment = Table p u b l i c . customers_something ) ;
echo done ;

45
50

echo done adding ;


s t o r e node ( i d = 2 , comment = Node 2 , $HOST2 ) ;
echo s t o r e d node ;
s t o r e path ( s e r v e r = 1 , c l i e n t = 2 , c o n n i n f o =
dbname=$DBNAME1 h o s t=$HOST1
p o r t=$PORT1 u s e r=s l o n y password=slony_user_password ) ;
echo s t o r e d path ;
s t o r e path ( s e r v e r = 2 , c l i e n t = 1 , c o n n i n f o =
dbname=$DBNAME2 h o s t=$HOST2
p o r t=$PORT2 u s e r=s l o n y password=slony_user_password ) ;

store l i s t e n ( origin = 1 , provider = 1 , receiver = 2 ) ;


store l i s t e n ( origin = 2 , provider = 2 , receiver = 1 ) ;
- EOF
-

, ,
. : ,
, id
, primary key.
: replication set .
set.
:
. unsubscribe subscribe .
57

4.3. Slony-I
slave- replication set
:
4.31 slave- replication set
Line 1

#!/ b i n / sh

CLUSTER=customers_rep

DBNAME1=c u s t o m e r s
- DBNAME2=c u s t o m e r s

5
-

HOST1=customers_master . com
- HOST2=c u s t o m e r s _ s l a v e . com
-

10

PORT1=5432
- PORT2=5432
-

SLONY_USER=s l o n y

15
20
-

s l o n i k <<EOF
c l u s t e r name = $CLUSTER ;
node 1 admin c o n n i n f o = dbname=$DBNAME1 h o s t=$HOST1
p o r t=$PORT1 u s e r=s l o n y password=slony_user_password ;
node 2 admin c o n n i n f o = dbname=$DBNAME2 h o s t=$HOST2
p o r t=$PORT2 u s e r=s l o n y password=slony_user_password ;

echo s u b s c r i b i n g ;
s u b s c r i b e s e t ( id = 1 , provider = 1 , r e c e i v e r = 2 , forward
= no ) ;

25
-

EOF


, .
4.32
Line 1
-

slony@customers_master$ s l o n customers_rep \
"dbname=c u s t o m e r s u s e r=s l o n y "

4.33
Line 1
-

s l o n y @ c u s t o m e r s _ s l a v e $ s l o n customers_rep \
"dbname=c u s t o m e r s u s e r=s l o n y "

58

4.3. Slony-I
.
COPY, slave DB
.
slave-
10- . slon
, .



4.3.1 4.3.2.
id = 3. customers_slave3.com,
- PgSQL.
( 4.3.2) :
4.34
Line 1
5
-

s l o n i k <<EOF
c l u s t e r name = c u s t o m e r s _ s l a v e ;
node 3 admin c o n n i n f o = dbname=c u s t o m e r s
h o s t=c u s t o m e r s _ s l a v e 3 . com
p o r t =5432 u s e r=s l o n y password=slony_user_pass ;
u n i n s t a l l node ( i d = 3 ) ;
echo okay ;
EOF

, ,
.
.

:
4.35
Line 1

#!/ b i n / sh

CLUSTER=customers_rep

DBNAME1=c u s t o m e r s
- DBNAME3=c u s t o m e r s

5
-

HOST1=customers_master . com
- HOST3=c u s t o m e r s _ s l a v e 3 . com
-

10
-

PORT1=5432
PORT2=5432

SLONY_USER=s l o n y

15

59

4.3. Slony-I
20
-

s l o n i k <<EOF
c l u s t e r name = $CLUSTER ;
node 1 admin c o n n i n f o = dbname=$DBNAME1 h o s t=$HOST1
p o r t=$PORT1 u s e r=s l o n y password=slony_user_pass ;
node 3 admin c o n n i n f o = dbname=$DBNAME3
h o s t=$HOST3 p o r t=$PORT2 u s e r=s l o n y
password=slony_user_pass ;

echo done adding ;

25
30
-

s t o r e node ( i d = 3 , comment = Node 3 , $HOST3 ) ;


echo s o r e d node ;
s t o r e path ( s e r v e r = 1 , c l i e n t = 3 , c o n n i n f o =
dbname=$DBNAME1
h o s t=$HOST1 p o r t=$PORT1 u s e r=s l o n y
password=slony_user_pass ) ;
echo s t o r e d path ;
s t o r e path ( s e r v e r = 3 , c l i e n t = 1 , c o n n i n f o =
dbname=$DBNAME3
h o s t=$HOST3 p o r t=$PORT2 u s e r=s l o n y
password=slony_user_pass ) ;

35

echo a g a i n ;
store l i s t e n ( origin = 1 , provider = 1 , receiver = 3 ) ;
store l i s t e n ( origin = 3 , provider = 3 , receiver = 1 ) ;

EOF

id 3, 2 .
3 replication set:
4.36
Line 1

#!/ b i n / sh

CLUSTER=customers_rep

DBNAME1=c u s t o m e r s
- DBNAME3=c u s t o m e r s

5
-

HOST1=customers_master . com
- HOST3=c u s t o m e r s _ s l a v e 3 . com
-

10

PORT1=5432
- PORT2=5432
-

SLONY_USER=s l o n y

15
-

s l o n i k <<EOF

60

4.3. Slony-I
c l u s t e r name = $CLUSTER ;
node 1 admin c o n n i n f o = dbname=$DBNAME1 h o s t=$HOST1
- p o r t=$PORT1 u s e r=s l o n y password=slony_user_pass ;
20 node 3 admin c o n n i n f o = dbname=$DBNAME3 h o s t=$HOST3
- p o r t=$PORT2 u s e r=s l o n y password=slony_user_pass ;
-

echo s u b s c r i b i n g ;
s u b s c r i b e s e t ( id = 1 , provider = 1 , r e c e i v e r = 3 , forward
= no ) ;

25
-

EOF

slon , .
slon .
4.37
Line 1
-

s l o n y @ c u s t o m e r s _ s l a v e 3 $ s l o n customers_rep \
"dbname=c u s t o m e r s u s e r=s l o n y "



,
: ,
:
4.38
Line 1
5
10
15
-

%s l o n customers_rep "dbname=c u s t o m e r s u s e r=s l o n y _ u s e r "


CONFIG main : s l o n v e r s i o n 1 . 0 . 5 s t a r t i n g up
CONFIG main : l o c a l node i d = 3
CONFIG main : l o a d i n g c u r r e n t c l u s t e r c o n f i g u r a t i o n
CONFIG s t o r e N o d e : no_id=1 no_comment= CustomersDB
replication cluster
CONFIG s t o r e N o d e : no_id=2 no_comment= Node 2 ,
node2 . example . com
CONFIG s t o r e N o d e : no_id=4 no_comment= Node 4 ,
node4 . example . com
CONFIG s t o r e P a t h : pa_server=1 p a _ c l i e n t=3
pa_conninfo="dbname=c u s t o m e r s
h o s t=mainhost . com p o r t =5432 u s e r=s l o n y _ u s e r
password=slony_user_pass " pa_connretry=10
CONFIG s t o r e L i s t e n : l i _ o r i g i n =1 l i _ r e c e i v e r =3
l i _ p r o v i d e r =1
CONFIG s t o r e S e t : s e t _ i d=1 s e t _ o r i g i n =1
set_comment= CustomersDB r e p l i c a t i o n s e t

61

4.3. Slony-I
20
25
30
35
40
45
50
-

WARN remoteWorker_wakeup : node 1 - no worker t h r e a d


CONFIG s t o r e S u b s c r i b e : sub_set=1 sub_provider=1
sub_forward= f
WARN remoteWorker_wakeup : node 1 - no worker t h r e a d
CONFIG e n a b l e S u b s c r i p t i o n : sub_set=1
WARN remoteWorker_wakeup : node 1 - no worker t h r e a d
CONFIG main : c o n f i g u r a t i o n c o m p l e t e - s t a r t i n g t h r e a d s
CONFIG enableNode : no_id=1
CONFIG enableNode : no_id=2
CONFIG enableNode : no_id=4
ERROR remoteWorkerThread_1 : " b e g i n t r a n s a c t i o n ; s e t
transaction isolation level
s e r i a l i z a b l e ; l o c k t a b l e " _customers_rep " . s l _ c o n f i g _ l o c k ;
select
" _customers_rep " . e n a b l e S u b s c r i p t i o n ( 1 , 1 , 4 ) ;
n o t i f y " _customers_rep_Event " ; n o t i f y
" _customers_rep_Confirm " ;
i n s e r t i n t o " _customers_rep " . s l _ e v e n t ( e v _ o r i g i n , ev_seqno ,
ev_timestamp , ev_minxid , ev_maxxid , ev_xip ,
ev_type , ev_data1 , ev_data2 , ev_data3 , ev_data4 ) v a l u e s
( 1 , 219440 ,
2005 -05 -05 18:52:42.708351 , 52501283 , 52501292 ,
5 2 5 0 1 2 8 3 , ENABLE_SUBSCRIPTION ,
1 , 1 , 4 , f ) ; i n s e r t i n t o " _customers_rep " .
s l _ c o n f i r m ( c o n _ o r i g i n , con_received ,
con_seqno , con_timestamp ) v a l u e s ( 1 , 3 , 2 1 9 4 4 0 ,
CURRENT_TIMESTAMP) ; commit t r a n s a c t i o n ; "
PGRES_FATAL_ERROR ERROR: i n s e r t o r update on t a b l e
" s l _ s u b s c r i b e " v i o l a t e s f o r e i g n key
c o n s t r a i n t " s l _ s u b s c r i b e - sl_path - r e f "
DETAIL : Key ( sub_provider , s u b _ r e c e i v e r ) =(1 ,4)
i s not p r e s e n t i n t a b l e " sl_path " .
INFO remoteListenThread_1 : d i s c o n n e c t i n g from
dbname=c u s t o m e r s h o s t=mainhost . com
p o r t =5432 u s e r=s l o n y _ u s e r password=slony_user_pass
%

_< >.sl_path;,
_customers_rep.sl_path
. , id 4, (1,4)
sl_path .
, Slony.
.
,
( ,
(1,4)):
4.39

62

4.3. Slony-I
Line 1
-

slon y _ u s e r @ m a s t e r h o s t $ p s q l - d c u s t o m e r s - h
_every_one_of_slaves -U s l o n y
c u s t o m e r s=# i n s e r t i n t o _customers_rep . sl_path
v a l u e s ( 1 , 4 , dbname=c u s t o m e r s h o s t=mainhost . com
p o r t =5432 u s e r=s l o n y _ u s e r
password=slony_user_password , 10 ) ;

,
.
_< >,
_customers_rep.


master-, SELECT-
. pg_stat_activity :
4.40
Line 1
5
10
-

s e l e c t e v _ o r i g i n , ev_seqno , ev_timestamp , ev_minxid ,


ev_maxxid , ev_xip ,
ev_type , ev_data1 , ev_data2 , ev_data3 , ev_data4 , ev_data5 ,
ev_data6 ,
ev_data7 , ev_data8 from " _customers_rep " . s l _ e v e n t e where
( e . e v _ o r i g i n = 2 and e . ev_seqno > 336996 ) o r
( e . e v _ o r i g i n = 3 and e . ev_seqno > 1712871 ) o r
( e . e v _ o r i g i n = 4 and e . ev_seqno > 721285 ) o r
( e . e v _ o r i g i n = 5 and e . ev_seqno > 807715 ) o r
( e . e v _ o r i g i n = 1 and e . ev_seqno > 3544763 ) o r
( e . e v _ o r i g i n = 6 and e . ev_seqno > 2529445 ) o r
( e . e v _ o r i g i n = 7 and e . ev_seqno > 2512532 ) o r
( e . e v _ o r i g i n = 8 and e . ev_seqno > 2500418 ) o r
( e . e v _ o r i g i n = 10 and e . ev_seqno > 1692318 )
o r d e r by e . e v _ o r i g i n , e . ev_seqno ;

_customers_rep ,
.
sl_event - ,
.
:
4.41
Line 1
-

d e l e t e from _customers_rep . s l _ e v e n t where


ev_timestamp<NOW( ) - 1 DAY : : i n t e r v a l ;

.
_customers_rep.sl_log_*
, -
63

4.4. Londiste
, _customers_rep.sl_log_1
.

4.4 Londiste

Londiste ,
python.
:
.
-
, Slony-I. Londiste
PgQ (
,

,
PostgreSQL). :

,
(failover)
(switchover) ( 3 )

, Linux,
Ubuntu Server. ,
( Windows) ,
PostgreSQL Windows, ,
.
Londiste Skytools,
. , Debian Ubuntu skytools
:
4.42
Line 1

sudo a p t i t u d e i n s t a l l s k y t o o l s


.
2.1.11. , :

64

4.4. Londiste
4.43
Line 1
5
10
15
-

$wget
h t t p : / / pgfoundry . o r g / f r s / download . php /2561/ s k y t o o l s - 2 . 1 . 1 1 . t a r . gz
$ t a r z x v f s k y t o o l s - 2 . 1 . 1 1 . t a r . gz
$cd s k y t o o l s - 2 . 1 . 1 1 /
# deb
$sudo a p t i t u d e i n s t a l l b u i l d - e s s e n t i a l a u t o c o n f \
automake a u t o t o o l s - dev dh - make \
debhelper devscripts fakeroot x u t i l s l i n t i a n pbuilder \
python - dev yada
# p o s t g r e s q l 8 . 4 . x
$sudo a p t i t u d e i n s t a l l p o s t g r e s q l - s e r v e r - dev - 8 . 4
# python - psycopg L o n d i s t e
$sudo a p t i t u d e i n s t a l l python - psycopg2
# deb
# p o s t g r e s q l 8 . 4 . x ( 8 . 3 . x "make deb83 " )
$sudo make deb84
$cd . . /
# s k y t o o l s
$dpkg - i s k y t o o l s - modules - 8 . 4 _2 . 1 . 1 1 _i386 . deb
s k y t o o l s _ 2 . 1 . 1 1 _i386 . deb

Skytools
4.44
./ configure
make
- make i n s t a l l

Line 1
-

,
4.45
$ l o n d i s t e . py -V
- Skytools version 2.1.11
- $pgqadm . py -V
- Skytools version 2.1.11

Line 1

,
.

:
host1 ;
host2 ;

65

4.4. Londiste
ticker-
Londiste ticker ,
. , ,
, .

ticker- (
/etc/skytools/db1-ticker.ini):
4.46 ticker-
[ pgqadm ]
- #
- job_name = db1 - t i c k e r

Line 1

5
-

#
db = dbname=P h o s t=h o s t 1

#
# ( . . )
10 maint_delay = 600
-

#
# ( )
- loop_delay = 0 . 1
-

15

# l o g p i d
l o g f i l e = / var / l o g /%(job_name ) s . l o g
- p i d f i l e = / var / p i d /%(job_name ) s . p i d
-

(SQL)
ticker .
pgqadm.py :
4.47 ticker-
Line 1
-

pgqadm . py / e t c / s k y t o o l s /db1 - t i c k e r . i n i i n s t a l l
pgqadm . py / e t c / s k y t o o l s /db1 - t i c k e r . i n i t i c k e r - d

, (/var/log/skytools/db1-tickers.log)
. (
).
ticker,
:
4.48 ticker-
Line 1

pgqadm . py / e t c / s k y t o o l s /db1 - t i c k e r . i n i t i c k e r - s

ticker:
4.49 ticker-
Line 1

pgqadm . py / e t c / s k y t o o l s /db1 - t i c k e r . i n i t i c k e r - k

66

4.4. Londiste

Londiste .
slave
, .


( /etc/skytools/db1-londiste.ini):
4.50
[ londiste ]
#
- job_name = db1 - l o n d i s t e

Line 1
-

#
- provider_db = dbname=db1 p o r t =5432 h o s t=h o s t 1
- #
- s u b s c r i b e r _ d b = dbname=db1 h o s t=h o s t 2

10
15

#
# SQL - , . .
# .
# ! ,
# pgq_queue_name = db1 - l o n d i s t e - queue

# l o g p i d
l o g f i l e = / var / l o g /%(job_name ) s . l o g
- p i d f i l e = / var / run/%(job_name ) s . p i d
-

20

#
- l o g _ s i z e = 5242880
- log_count = 3
-

Londiste
SQL
.
:
4.51 Londiste
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i p r o v i d e r i n s t a l l

67

4.4. Londiste
4.52 Londiste
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i s u b s c r i b e r
install

.
Londiste
:
4.53
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i r e p l a y - d

, , ..
,
.
(/var/log/db1-londistes.log).


:
4.54
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i p r o v i d e r add
-- all

:
4.55
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i s u b s c r i b e r add
-- all

- all,
,
, .
(sequence)
. :
4.56
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i p r o v i d e r add - s e q
-- all

:
4.57
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i s u b s c r i b e r
add - s e q - - a l l

68

4.4. Londiste

all.

, .
Londiste
bulk copy , ( COPY)
,
.
:
4.58
Line 1

l e s s / var / l o g /db1 - l o n d i s t e . l o g

, .

, "ok".
4.59
Line 1

l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i s u b s c r i b e r t a b l e s

5
10

Table S t a t e
public . table1
public . table2
public . table3
public . table4
public . table5
public . table6
...

ok
ok
in - copy
-


( ):
4.60
Line 1
5
-

(
while [ $ (
python l o n d i s t e . py / e t c / s k y t o o l s /db1 - l o n d i s t e . i n i
subscriber tables |
t a i l - n+2 | awk { p r i n t $2 } | g r e p - v ok | wc - l ) - ne 0 ] ;
do s l e e p 6 0 ; done ; echo | m a i l - s R e p l i c a t i o n done EOM
user@domain . com
) &



:
69

4.4. Londiste
4.61
Line 1

l o n d i s t e . py <i n i > p r o v i d e r t a b l e s | x a r g s l o n d i s t e . py <i n i >


s u b s c r i b e r add



.
4.62
Line 1
-

SELECT queue_name , consumer_name , l a g , l a s t _ s e e n


FROM pgq . get_consumer_info ( ) ;

lag ,
last_seen .
, 60 .

Londiste
, .
PGQ,
, API:
4.63
Line 1

SELECT pgq . u n r e g i s t e r _ c o n s u m e r ( queue_name ,


consumer_name ) ;

pgqadm.py:
4.64
Line 1

pgqadm . py < t i c k e r . i n i > u n r e g i s t e r queue_name consumer_name


:
1.
2.
3.
4.


BEGIN;

SELECT
londiste.provider_refresh_trigger(queue_name,
tablename);
5. COMMIT;

70

4.4. Londiste

1. BEGIN;
2.
3. SELECT
londiste.provider_refresh_trigger(queue_name,
tablename);
4. COMMIT;
5. lag, londiste
6.
, ,
.


Londiste lag
, ,
ticker. UPDATE
DELETE ,
. . .
,
pgq.subscription sub_last_tick sub_next_tick.
4.65
Line 1
5
-

SELECT count ( * )
FROM pgq . event_1 ,
(SELECT t i c k _ s n a p s h o t
FROM pgq . t i c k
WHERE t i c k _ i d BETWEEN 5715138 AND 5715139
) as t ( snapshots )
WHERE t x i d _ v i s i b l e _ i n _ s n a p s h o t ( ev_txid , s n a p s h o t s ) ;

, 5 400 .
.
Londiste, .
Londiste .
INI ticker- :
4.66
Line 1

pgq_lazy_fetch = 500

Londiste 500
. .

71

4.5. Bucardo

4.5 Bucardo

Bucardo master-master master-slave


PostgreSQL, Perl.
,
.

Ubuntu Server.
DBIx::Safe Perl .
4.67
Line 1

sudo a p t i t u d e i n s t a l l l i b d b i x - s a f e - p e r l

1 :
4.68
t a r x v f z DBIx - Safe - 1 . 2 . 5 . t a r . gz
- cd DBIx - Safe - 1 . 2 . 5
- p e r l M a k e f i l e . PL
- make && make t e s t && sudo make i n s t a l l

Line 1

Bucardo. 2 :
4.69
Line 1
5

t a r x v f z Bucardo - 4 . 4 . 0 . t a r . gz
cd Bucardo - 4 . 4 . 0
p e r l M a k e f i l e . PL
make
sudo make i n s t a l l

Bucardo pl/perlu
PostgreSQL.
4.70
Line 1

sudo a p t i t u d e i n s t a l l p o s t g r e s q l - p l p e r l - 8 . 4

Bucardo
:
1
2

http://search.cpan.org/CPAN/authors/id/T/TU/TURNSTEP/
http://bucardo.org/wiki/Bucardo#Obtaining_Bucardo

72

4.5. Bucardo
4.71 Bucardo
Line 1

bu ca r d o _ c t l i n s t a l l

Bucardo PostgreSQL,
:
4.72 Bucardo
This w i l l i n s t a l l t h e bucardo d a t a b a s e i n t o an e x i s t i n g
Postgres c l u s t e r .
- P o s t g r e s must have been c o m p i l e d with P e r l support ,
- and you must c o n n e c t a s a s u p e r u s e r

Line 1

5
-

We w i l l c r e a t e a new s u p e r u s e r named bucardo ,


and make i t t h e owner o f a new d a t a b a s e named bucardo

10
-

Current c o n n e c t i o n s e t t i n g s :
1 . Host :
<none>
2 . Port :
5432
3 . User :
postgres
4 . Database :
postgres
5 . PID d i r e c t o r y : / var / run / bucardo

,
Bucardo bucardo bucardo.
Unix socket,
pg_hda.conf.

,
Bucardo. master_db slave_db.
:
4.73
Line 1
-

bu ca r d o _ c t l add db master_db name=master


bu ca r d o _ c t l add a l l t a b l e s herd=a l l _ t a b l e s
bu ca r d o _ c t l add a l l s e q u e n c e s herd=a l l _ t a b l e s

master (
, master_db slave_db
Bucardo ).
,
all_tables.
slave_db:
4.74
Line 1

bu ca r d o _ c t l add db slave_db name=r e p l i c a p o r t =6543


h o s t=s l a v e _ h o s t

73

4.5. Bucardo
replica Bucardo.


. (master-slave):
4.75
Line 1

bu ca r d o _ c t l add sync d e l t a type=p u s h d e l t a s o u r c e=a l l _ t a b l e s


t a r g e t d b=r e p l i c a

Bucardo PostgreSQL.
:
type
. 3 :
Fullcopy. .
Pushdelta. Master-slave .
Swap.
Master-master .

Bucardo
.
goat
( )
standard_conflict (
):
* source source
(master_db ).
* target target
(slave_db ).
* skip .

.
* random ,
.
* latest ,
.
* abort .
source
.
targetdb
, .
master-master:
4.76
Line 1

bu ca r d o _ c t l add sync d e l t a type=swap s o u r c e=a l l _ t a b l e s


t a r g e t d b=r e p l i c a

74

4.6. RubyRep
/
:
4.77
Line 1

bu ca r d o _ c t l s t a r t

:
4.78
Line 1

bu ca r d o _ c t l s t o p



:
4.79
Line 1

bu ca r d o _ c t l show a l l


4.80
Line 1

bu ca r d o _ c t l s e t name=v a l u e

:
4.81
Line 1

bu ca r d o _ c t l s e t s y s l o g _ f a c i l i t y=LOG_LOCAL3


4.82
Line 1

bu ca r d o _ c t l r e l o a d _ c o n f i g

bucardo.org/wiki/Bucardo_ctl

4.6 RubyRep

RubyRep
, ruby. :
. master-master,
master-slave , PostgreSQL MySQL.
:
75

4.6. RubyRep


:
MySQL

RubyRep : Ruby
JRuby. JRuby
.
JRuby
Java ( 1.6).
1. JRuby rubyrep c Rubyforge.
2.
3.
Ruby
1. Ruby, Rubygems.
2. .
Mysql:
4.83
Line 1

sudo gem i n s t a l l mysql

PostgreSQL:
4.84
Line 1

sudo gem i n s t a l l p o s t g r e s

3. rubyrep:
4.85
Line 1

sudo gem i n s t a l l rubyrep


:
4.86
Line 1

rubyrep g e n e r a t e myrubyrep . c o n f

76

4.6. RubyRep
generate
myrubyrep.conf:

4.87
Line 1
5
-

RR : : I n i t i a l i z e r : : run do | c o n f i g |
config . l e f t = {
: a d a p t e r => p o s t g r e s q l , # o r mysql
: d a t a b a s e => SCOTT ,
: username => s c o t t ,
: password => t i g e r ,
: host
=> 1 7 2 . 1 6 . 1 . 1
}

10
15
-

config . right
: a d a p t e r =>
: d a t a b a s e =>
: username =>
: password =>
: host
=>
}

= {
postgresql ,
SCOTT ,
scott ,
tiger ,
172.16.1.2

c o n f i g . i n c l u d e _ t a b l e s dept
- c o n f i g . i n c l u d e _ t a b l e s /^ e / # r e g e x p matches a l l t a b l e s
s t a r t i n g with e
20 # c o n f i g . i n c l u d e _ t a b l e s / . / # r e g e x p matches a l l t a b l e s
- end
-

. left
right. config.include_tables
( RegEx).

:
4.88
Line 1

rubyrep s c a n - c myrubyrep . c o n f

:
4.89
Line 1
-

dept 100% . . . . . . . . . . . . . . . . . . . . . . . . .
emp 100% . . . . . . . . . . . . . . . . . . . . . . . . .

0
1

dept , emp
.

77

4.6. RubyRep

:
4.90
Line 1

rubyrep sync - c myrubyrep . c o n f


:
4.91
Line 1

rubyrep sync - c myrubyrep . c o n f dept /^ e /


.

.

:
4.92
Line 1

rubyrep r e p l i c a t e - c myrubyrep . c o n f

(
) .

, .
, rubyrep.
, .
:
4.93
Line 1

rubyrep u n i n s t a l l - c myrubyrep . c o n f



rubyrep Ruby :
4.94
Line 1
5
-

$rubyrep r e p l i c a t e - c myrubyrep . c o n f
V e r i f y i n g RubyRep t a b l e s
Checking f o r and removing rubyrep t r i g g e r s from
unconfigured tables
V e r i f y i n g rubyrep t r i g g e r s o f c o n f i g u r e d t a b l e s
Starting replication
E x c e p t i o n caught : Thread#j o i n : d e a d l o c k 0 xb76ee1ac - mutual
j o i n (0 xb758cfac )

78

4.7.
Ruby. :
1. rubyrep JRuby ( )
2. rubyrep :
4.95
Line 1

-- -

/ L i b r a r y /Ruby/Gems / 1 . 8 / gems/ rubyrep - 1 . 1 . 2 / l i b / rubyrep /


r e p l i c a t i o n _ r u n n e r . rb 2 0 1 0 - 0 7 - 1 6 1 5 : 1 7 : 1 6 . 0 0 0 0 0 0 0 0 0
-0400
- +++ . / r e p l i c a t i o n _ r u n n e r . rb 2 0 1 0 - 0 7 - 1 6
1 7 : 3 8 : 0 3 . 0 0 0 0 0 0 0 0 0 -0400
- @@ - 2 , 6 +2 ,12 @@
-

5
10
-

require optparse
r e q u i r e thread
+r e q u i r e monitor
+
+c l a s s Monitor
+ a l i a s l o c k mon_enter
+ a l i a s u n l o c k mon_exit
+end

15
20
25
-

module RR
# This c l a s s implements t h e f u n c t i o n a l i t y o f t h e
r e p l i c a t e command .
@@ - 9 4 , 7 +100 ,7 @@
# I n i t i a l i z e s t h e w a i t e r t h r e a d used f o r
r e p l i c a t i o n pauses
# and p r o c e s s i n g
# t h e p r o c e s s TERM s i g n a l .
def init_waiter
- @termination_mutex = Mutex . new
+ @termination_mutex = Monitor . new
@termination_mutex . l o c k
@waiter_thread | | = Thread . new
{ @termination_mutex . l o c k ;
s e l f . termination_requested = true }
%w(TERM INT) . each do | s i g n a l |

4.7
,
PostgreSQL.
,
, ..
79

4.7.
PostgreSQL.
.

, 9.0 PostgreSQL. Slony-I
,
,
, (failover)
(switchover). Londiste
, . Bucardo
master-master, master-slave ,
, (failover)
(switchover).
RubyRep,
master-master , ,

( ).

80

,
.

5.1
.

.
.
, .
.
,
(, 90%
). , (
!) ,
().
,
, , ..
.
,
, .
..
. ? .
,
. ,
10 , 10 -
.. .. - ,
,
.
, ID
(user_id)
81

5.2. PL/Proxy
. ..
:
,
user_id


:
-
=. , ,
.
,
.
=
()
. , ,
( ,
).

.
,
.. .
, ,
,
(, ,
).
. , ,
, ID , 100 .

.
PostgreSQL
:

Postgres-XC
PL/Proxy
HadoopDB (Shared-nothing clustering)
Greenplum Database

5.2 PL/Proxy
PL/Proxy -
.
,
, ,
82

5.2. PL/Proxy
(,
, , -
).
PL/Proxy ?
.

, ,
26 . ,
-, :
, , -
.

.
PL/Proxy
OLTP . failover-
, -,
.
:
autocommit-

SELECT;

-
-;

-, PgBouncer
( , )
-

1. PL/Proxy1 .
2. PL/Proxy make make install.
PL/Proxy .
Ubuntu Server PostgreSQL
8.4:
5.1
Line 1

sudo a p t i t u d e i n s t a l l p o s t g r e s q l - 8 . 4 - p l p r o x y
1

http://pgfoundry.org/projects/plproxy

83

5.2. PL/Proxy

3 PostgreSQL. 2
node1 node2, ,
proxy. pl/proxy
.
plproxytest, users. !
node1 node2.
.
plproxytest( ):
5.2
CREATE DATABASE p l p r o x y t e s t
WITH OWNER = p o s t g r e s
ENCODING = UTF8 ;

Line 1
-

users:
5.3
Line 1
5
-

CREATE TABLE p u b l i c . u s e r s
(
username c h a r a c t e r v a r y i n g ( 2 5 5 ) ,
email character varying (255)
)
WITH (OIDS=FALSE) ;
ALTER TABLE p u b l i c . u s e r s OWNER TO p o s t g r e s ;

users:
5.4
Line 1
5
-

CREATE OR REPLACE FUNCTION p u b l i c . i n s e r t _ u s e r ( i_username


text ,
i_emailaddress
text )
RETURNS i n t e g e r AS
$BODY$
INSERT INTO p u b l i c . u s e r s ( username , e m a i l ) VALUES ( $1 , $2 ) ;
SELECT 1 ;
$BODY$
LANGUAGE s q l VOLATILE ;
ALTER FUNCTION p u b l i c . i n s e r t _ u s e r ( t e x t , t e x t ) OWNER TO
postgres ;

. proxy.
, (proxy)
:
5.5
Line 1

CREATE DATABASE p l p r o x y t e s t

84

5.2. PL/Proxy
-

WITH OWNER = p o s t g r e s
ENCODING = UTF8 ;


pl/proxy:
5.6
Line 1
5
10

CREATE OR REPLACE FUNCTION p u b l i c . p l p r o x y _ c a l l _ h a n d l e r ( )


RETURNS l a n g u a g e _ h a n d l e r AS
$ l i b d i r / plproxy , plproxy_call_handler
LANGUAGE c VOLATILE
COST 1 ;
ALTER FUNCTION p u b l i c . p l p r o x y _ c a l l _ h a n d l e r ( )
OWNER TO p o s t g r e s ;
- - language
CREATE LANGUAGE p l p r o x y HANDLER p l p r o x y _ c a l l _ h a n d l e r ;
CREATE LANGUAGE p l p g s q l ;

,
3 pl/proxy
. .
kay-value:
5.7
Line 1
5
10
15
-

CREATE OR REPLACE FUNCTION p u b l i c . g e t _ c l u s t e r _ c o n f i g


( IN cluster_name t e x t ,
OUT " key " t e x t , OUT v a l t e x t )
RETURNS SETOF r e c o r d AS
$BODY$
BEGIN
- - l e t s u s e same c o n f i g f o r a l l c l u s t e r s
key := c o n n e c t i o n _ l i f e t i m e ;
v a l := 3 0 * 6 0 ; - - 30m
RETURN NEXT;
RETURN;
END;
$BODY$
LANGUAGE p l p g s q l VOLATILE
COST 100
ROWS 1 0 0 0 ;
ALTER FUNCTION p u b l i c . g e t _ c l u s t e r _ c o n f i g ( t e x t )
OWNER TO p o s t g r e s ;

.
DSN :
5.8
CREATE OR REPLACE FUNCTION
- p u b l i c . g e t _ c l u s t e r _ p a r t i t i o n s ( cluster_name t e x t )

Line 1

85

5.2. PL/Proxy
5
10
15
-

RETURNS SETOF t e x t AS
$BODY$
BEGIN
IF cluster_name = u s e r c l u s t e r THEN
RETURN NEXT dbname=p l p r o x y t e s t h o s t=node1
u s e r=p o s t g r e s ;
RETURN NEXT dbname=p l p r o x y t e s t h o s t=node2
u s e r=p o s t g r e s ;
RETURN;
END IF ;
RAISE EXCEPTION Unknown c l u s t e r ;
END;
$BODY$
LANGUAGE p l p g s q l VOLATILE
COST 100
ROWS 1 0 0 0 ;
ALTER FUNCTION p u b l i c . g e t _ c l u s t e r _ p a r t i t i o n s ( t e x t )
OWNER TO p o s t g r e s ;

:
5.9
Line 1
5
10
15

CREATE OR REPLACE FUNCTION


p u b l i c . g e t _ c l u s t e r _ v e r s i o n ( cluster_name t e x t )
RETURNS i n t e g e r AS
$BODY$
BEGIN
IF cluster_name = u s e r c l u s t e r THEN
RETURN 1 ;
END IF ;
RAISE EXCEPTION Unknown c l u s t e r ;
END;
$BODY$
LANGUAGE p l p g s q l VOLATILE
COST 1 0 0 ;
ALTER FUNCTION p u b l i c . g e t _ c l u s t e r _ v e r s i o n ( t e x t )
OWNER TO p o s t g r e s ;


:
5.10
Line 1
5
-

CREATE OR REPLACE FUNCTION


p u b l i c . i n s e r t _ u s e r ( i_username t e x t , i _ e m a i l a d d r e s s t e x t )
RETURNS i n t e g e r AS
$BODY$
CLUSTER u s e r c l u s t e r ;
RUN ON h a s h t e x t ( i_username ) ;

86

5.2. PL/Proxy
$BODY$
LANGUAGE p l p r o x y VOLATILE
COST 1 0 0 ;
10 ALTER FUNCTION p u b l i c . i n s e r t _ u s e r ( t e x t , t e x t )
- OWNER TO p o s t g r e s ;
-

. proxy :
5.11
SELECT i n s e r t _ u s e r ( Sven , sven@somewhere . com ) ;
- SELECT i n s e r t _ u s e r ( Marko , marko@somewhere . com ) ;
- SELECT i n s e r t _ u s e r ( S t e v e , steve@somewhere . com ) ;

Line 1

.
:

5.12
Line 1
5
10
-

CREATE OR REPLACE FUNCTION


p u b l i c . get_user_email ( i_username t e x t )
RETURNS SETOF t e x t AS
$BODY$
CLUSTER u s e r c l u s t e r ;
RUN ON h a s h t e x t ( i_username ) ;
SELECT e m a i l FROM p u b l i c . u s e r s
WHERE username = i_username ;
$BODY$
LANGUAGE p l p r o x y VOLATILE
COST 100
ROWS 1 0 0 0 ;
ALTER FUNCTION p u b l i c . get_user_email ( t e x t )
OWNER TO p o s t g r e s ;

:
5.13
Line 1

s e l e c t p l p r o x y . get_user_email ( S t e v e ) ;

,
, users .

?
pl/proxy
. ,
. 16 .
- .
?
Highload++ 2008,

87

5.3. Postgres-XC
Skype,
opensource.

,
.
,
,
get_cluster_partitions.

5.3 Postgres-XC
Postgres-XC - ,

. Postgres-XC
,
:
,
GTM
(Global Transaction Manager).
,
Postgres-XC 64%, .. 10

6.4 , (
).

PostgreSQL,

,
( PostgreSQL API).
(GTM),
, ,
(
, ).
Postgres-XC MultiMaster,
.
, -

( ).

. 5.1 Postgres-XC
:

88

5.3. Postgres-XC

. 5.1: Postgres-XC
1. (GTM)
Postgres-XC,

(
).
, .
,
.
2. (coordinators)
().

( ). ,
(datanodes)
SQL PostgreSQL .

, ,
.
, ,
..
3. (datanodes)
89

5.3. Postgres-XC
.
Datanodes
.

Postgres-XC
. Ubuntu 12.10 postgres-xc :
5.14 Postgres-XC
Line 1

sudo apt - g e t i n s t a l l p o s t g r e s - xc p o s t g r e s - xc - c l i e n t
p o s t g r e s - xc - c o n t r i b p o s t g r e s - xc - s e r v e r - dev

-
.


Postgres-XC :

. 5.2:

1. (distributed tables, . 5.2):



(hash, round-robin, modulo).

.
.
.
2. (replicated tables, . 5.3):
()
.
90

5.3. Postgres-XC

. 5.3:

( )
.
,
,
.

.


Postgres-XC
PostgreSQL.
(- 5432).
.
5.15
CREATE TABLE
users_with_hash ( i d SERIAL , type INT , . . . )
- DISTRIBUTE by HASH( i d ) TO NODE dn1 , dn2 ;

Line 1
-

CREATE TABLE
- users_with_modulo ( i d SERIAL , type INT , . . . )
- DISTRIBUTE by MODULO( i d ) TO NODE dn1 , dn2 ;

CREATE TABLE
u s e r s _ w i t h _ r r o b i n ( i d SERIAL , type INT , . . . )
- DISTRIBUTE by ROUND ROBIN TO NODE dn1 , dn2 ;
-

10

5.15 3 :
1. users_with_hash
91

5.3. Postgres-XC
( id)
. 15 :
5.16
#
$ psql
- # SELECT id , type from users_with_hash ORDER BY i d ;
id
| type
5 - - - - - - -+ - - - - - - 1 |
946
2 |
153
3 |
484
4 |
422
10
5 |
785
6 |
906
7 |
973
8 |
699
9 |
434
15
10 |
986
11 |
135
12 | 1012
13 |
395
14 |
667
20
15 |
324

Line 1
-

#
$ p s q l - p15432
- # SELECT id , type from users_with_hash ORDER BY i d ;
25
i d | type
- - - - - - -+ - - - - - - 1 |
946
2 |
153
5 |
785
30
6 |
906
8 |
699
9 |
434
12 | 1012
13 |
395
35
15 |
324
-

#
$ p s q l - p15433
- # SELECT id , type from users_with_hash ORDER BY i d ;
40
id
| type
- - - - - - - -+ - - - - - - 3 |
484
4 |
422
-

92

5.3. Postgres-XC
45
-

7
10
11
14

|
|
|
|

973
986
135
667

2. users_with_modulo
( id)
. 15 :
5.17
#
$ psql
- # SELECT id , type from users_with_modulo ORDER BY i d ;
id
| type
5 - - - - - - -+ - - - - - - 1 |
883
2 |
719
3 |
29
4 |
638
10
5 |
363
6 |
946
7 |
440
8 |
331
9 |
884
15
10 |
199
11 |
78
12 |
791
13 |
345
14 |
476
20
15 |
860

Line 1
-

#
- $ p s q l - p15432
- # SELECT id , type from users_with_modulo ORDER BY i d ;
25
id
| type
- - - - - - - -+ - - - - - - 2 |
719
4 |
638
6 |
946
30
8 |
331
10 |
199
12 |
791
14 |
476
-

#
- $ p s q l - p15433
- # SELECT id , type from users_with_modulo ORDER BY i d ;
i d | type

35

93

5.3. Postgres-XC
40
45
-

- - - - - -+ - - - - - - 1 |
883
3 |
29
5 |
363
7 |
440
9 |
884
11 |
78
13 |
345
15 |
860

3.
users_with_rrobin

(round-robin) .
15 :

5.18
#
$ psql
- # SELECT id , type from u s e r s _ w i t h _ r r o b i n ORDER BY i d ;
id
| type
5 - - - - - - -+ - - - - - - 1 |
890
2 |
198
3 |
815
4 |
446
10
5 |
61
6 |
337
7 |
948
8 |
446
9 |
796
15
10 |
422
11 |
242
12 |
795
13 |
314
14 |
240
20
15 |
733

Line 1
-

#
- $ p s q l - p15432
- # SELECT id , type from u s e r s _ w i t h _ r r o b i n ORDER BY i d ;
25
id
| type
- - - - - - - -+ - - - - - - 2 |
198
4 |
446
6 |
337
30
8 |
446
10 |
422
12 |
795
14 |
240
-

94

5.3. Postgres-XC
-

#
$ p s q l - p15433
- # SELECT id , type from u s e r s _ w i t h _ r r o b i n ORDER BY i d ;
i d | type
- - - - - - -+ - - - - - - 40
1 |
890
3 |
815
5 |
61
7 |
948
9 |
796
45
11 |
242
13 |
314
15 |
733

35
-

:
5.19
CREATE TABLE
- u s e r s _ r e p l i c a t e d ( i d SERIAL , type INT , . . . )
- DISTRIBUTE by REPLICATION TO NODE dn1 , dn2 ;

Line 1

:
5.20
Line 1
5
10
15
-

# SELECT id , type from u s e r s _ r e p l i c a t e d


id
| type
- - - - - - -+ - - - - - - 1 |
75
2 |
262
3 |
458
4 |
779
5 |
357
6 |
51
7 |
249
8 |
444
9 |
890
10 |
810
11 |
809
12 |
166
13 |
605
14 |
401
15 |
58

ORDER BY i d ;

.
:
5.21

95

5.3. Postgres-XC
Line 1
5
10
-

# EXPLAIN VERBOSE SELECT * from users_with_modulo ORDER BY


id ;
QUERY PLAN
-- ------------------------------------------------------------------------S o r t ( c o s t = 4 9 . 8 3 . . 5 2 . 3 3 rows =1000 width =8)
Output : id , type
S o r t Key : users_with_modulo . i d
-> R e s u l t ( c o s t = 0 . 0 0 . . 0 . 0 0 rows =1000 width =8)
Output : id , type
-> Data Node Scan on users_with_modulo
( c o s t = 0 . 0 0 . . 0 . 0 0 rows =1000 width =8)
Output : id , type
Node/ s : dn1 , dn2
Remote query : SELECT id , type FROM ONLY
users_with_modulo WHERE t r u e
( 9 rows )

5.21
, .
:
5.22
Line 1

# EXPLAIN VERBOSE SELECT sum ( i d ) from users_with_modulo


GROUP BY type ;

5
10
-

QUERY PLAN
-- ------------------------------------------------------------------------HashAggregate ( c o s t = 5 . 0 0 . . 5 . 0 1 rows=1 width =8)
Output : pg_catalog . sum ( ( sum ( users_with_modulo . i d ) ) ) ,
users_with_modulo . type
-> M a t e r i a l i z e ( c o s t = 0 . 0 0 . . 0 . 0 0 rows=0 width =0)
Output : ( sum ( users_with_modulo . i d ) ) ,
users_with_modulo . type
-> Data Node Scan on "__REMOTE_GROUP_QUERY__"
( c o s t = 0 . 0 0 . . 0 . 0 0 rows =1000 width =8)
Output : sum ( users_with_modulo . i d ) ,
users_with_modulo . type
Node/ s : dn1 , dn2
Remote query : SELECT sum ( group_1 . i d ) ,
group_1 . type FROM (SELECT id , type FROM ONLY
users_with_modulo WHERE t r u e ) group_1 GROUP BY 2
( 8 rows )

JOIN , JOIN

. JOIN

( 5.23).
96

5.3. Postgres-XC
5.23
Line 1

5
10
15

# EXPLAIN VERBOSE SELECT * from users_with_modulo ,


users_with_hash WHERE users_with_modulo . i d =
users_with_hash . i d ;
QUERY PLAN
-- ------------------------------------------------------------------------Nested Loop ( c o s t = 0 . 0 0 . . 0 . 0 1 rows=1 width =16)
Output : users_with_modulo . id , users_with_modulo . type ,
users_with_hash . id , users_with_hash . type
J o i n F i l t e r : ( users_with_modulo . i d = users_with_hash . i d )
-> Data Node Scan on users_with_modulo
( c o s t = 0 . 0 0 . . 0 . 0 0 rows =1000 width =8)
Output : users_with_modulo . id ,
users_with_modulo . type
Node/ s : dn1 , dn2
Remote query : SELECT id , type FROM ONLY
users_with_modulo WHERE t r u e
-> Data Node Scan on users_with_hash ( c o s t = 0 . 0 0 . . 0 . 0 0
rows =1000 width =8)
Output : users_with_hash . id , users_with_hash . type
Node/ s : dn1 , dn2
Remote query : SELECT id , type FROM ONLY
users_with_hash WHERE t r u e
( 1 1 rows )

:
5.24
Line 1
5
-

# EXPLAIN VERBOSE SELECT * from u s e r s _ r e p l i c a t e d ;


QUERY PLAN
-- ------------------------------------------------------------------------Data Node Scan on "__REMOTE_FQS_QUERY__" ( c o s t = 0 . 0 0 . . 0 . 0 0
rows=0 width =0)
Output : u s e r s _ r e p l i c a t e d . id , u s e r s _ r e p l i c a t e d . type
Node/ s : dn1
Remote query : SELECT id , type FROM u s e r s _ r e p l i c a t e d
( 4 rows )


, ( ).

(HA)
Postgres-XC .
CAP1 .

1

http://en.wikipedia.org/wiki/CAP_theorem

97

5.4. HadoopDB
,
.
Postgres-XC
PostgreSQL (streaming)
hot-standby .

,
. GTM
, GTM-standby
.
.

1. Postgres-XC PostgreSQL 9.1 (9.2 )


2.
( )
3. UNIQUE
4. foreign keys

5. ( )
6. INSERT . . . RETURNING ( )
7.
( )

Postgres-XC
PostgreSQL. ,
(
) ,
PostgreSQL.

5.4 HadoopDB
Hadoop ,
.


, :
:
Hadoop
,
;
:
, .
;
98

5.4. HadoopDB
:
,
;
:
, .

;
:

, Java,
,
JVM.
HDFS

Hadoop Distributed File System.
,
:
,
, ;

;

;
;
:
;
:
,
;
HDFS
Namenode
.
.

, .
.
Datanode

.


. Rack,
, , .

99

5.4. HadoopDB

. 5.4: HDFS


,
. .
HDFS
:
.

( , ; -
64 mb) , Datanode.


,
. ,
- ,
/trash

.
, .
- ,
Datanode Namenode
.

Namenode ,
-
100

5.4. HadoopDB
.
,

.

, TCP/IP.
Namenode ClientProtocol,
DatanodeProtocol,
Remote Procedure Call (RPC).
,
DFSShell,
DFSAdmin, , -.
API : Java
API, C pipeline, WebDAV .
MapReduce
, Hadoop framework
,
. Job ()
, , :
Map

( -)
-.
.
Reduce
map
.

,
.
, ,
( ).
JobTracker,
TaskTracker.
JobTracker
,
, .
, framework,
map reduce,
.
JobTracker
.

, , ,
.
101

5.4. HadoopDB
HBase
Hadoop ,
.
Google
BigTable.
HBase
.
, .
HQL ( Hadoop
Query Language),
SQL.
.
HBase
,
--,
.
,
, ,
.
- ,
. ,
.
HQL , SQL,

help;,
. SELECT, INSERT, UPDATE, DROP ,
.
HBase Shell,
HBase API
: Java, Jython, REST Thrift.
HadoopDB
HadoopDB Yale Brown

, MapReduce,
. MapReduce
,
,
. SQL,
MapReduce,
. MapReduce
,
.

102

5.4. HadoopDB


Ubuntu Server .
Hadoop
, Hadoop,
,
:
ssh
,
[hadoop]:
5.25
$sudo groupadd hadoop
- $sudo u s e r a d d -m - g hadoop - d /home/ hadoop - s / b i n / bash
\
- - c "Hadoop s o f t w a r e owner " hadoop

Line 1

:
5.26 hadoop
Line 1

su hadoop

RSA-
:
5.27 RSA-
Line 1
5
-

h a d o o p @ l o c a l h o s t ~ $ ssh - keygen - t r s a -P ""


G e n e r a t i n g p u b l i c / p r i v a t e r s a key p a i r .
Enter f i l e i n which t o s a v e t h e key
( / home/ hadoop / . s s h / i d _ r s a ) :
Your i d e n t i f i c a t i o n has been saved i n
/home/ hadoop / . s s h / i d _ r s a .
Your p u b l i c key has been saved i n
/home/ hadoop / . s s h / i d _ r s a . pub .
The key f i n g e r p r i n t i s :
7b : 5 c : c f : 7 9 : 6 b : 9 3 : d6 : d6 : 8 d : 4 1 : e3 : a6 : 9 d : 0 4 : f 9 : 8 5
hadoop@localhost

:
5.28
Line 1

c a t $HOME/ . s s h / i d _ r s a . pub >> $HOME/ . s s h / a u t h o r i z e d _ k e y s

,
:
103

5.4. HadoopDB
5.29 ssh
Line 1

ssh l o c a l h o s t

sshd:
5.30 sshd
Line 1

/ e t c / i n i t . d/ ss hd s t a r t

JVM
1.5.0 .
5.31 JVM
Line 1

sudo a p t i t u d e i n s t a l l openjdk - 6 - j d k

Hadoop:
5.32 Hadoop
Line 1
5
-

cd / opt
sudo wget h t t p : / /www. g t l i b . g a t e c h . edu /pub/ apache / hadoop
/ c o r e / hadoop - 0 . 2 0 . 2 / hadoop - 0 . 2 0 . 2 . t a r . gz
sudo t a r z x v f hadoop - 0 . 2 0 . 2 . t a r . gz
sudo l n - s / opt / hadoop - 0 . 2 0 . 2 / opt / hadoop
sudo chown -R hadoop : hadoop / opt / hadoop / opt / hadoop - 0 . 2 0 . 2
sudo mkdir - p / opt / hadoop - data /tmp - b a s e
sudo chown -R hadoop : hadoop / opt / hadoop - data /

/opt/hadoop/conf/hadoop-env.sh
:
5.33
Line 1
5
-

export
export
export
export
export
export

JAVA_HOME=/u s r / l i b /jvm/ java - 6 - openjdk


HADOOP_HOME=/opt / hadoop
HADOOP_CONF=$HADOOP_HOME/ c o n f
HADOOP_PATH=$HADOOP_HOME/ b i n
HIVE_HOME=/opt / h i v e
HIVE_PATH=$HIVE_HOME/ b i n

e x p o r t PATH=$HIVE_PATH:$HADOOP_PATH:$PATH

/opt/hadoop/conf/hadoop-site.xml:
5.34 hadoop
Line 1
5

<c o n f i g u r a t i o n>
<p r o p e r t y>
<name>hadoop . tmp . d i r</name>
<v a l u e>/ opt / hadoop - data /tmp - b a s e</ v a l u e>
<d e s c r i p t i o n>A b a s e f o r o t h e r temporary
d i r e c t o r i e s</ d e s c r i p t i o n>

104

5.4. HadoopDB
-

</ p r o p e r t y>

10
-

<p r o p e r t y>
<name> f s . d e f a u l t . name</name>
<v a l u e>l o c a l h o s t : 5 4 3 1 1</ v a l u e>
<d e s c r i p t i o n>
The name o f t h e d e f a u l t f i l e system .
</ d e s c r i p t i o n>
</ p r o p e r t y>

15
20
-

<p r o p e r t y>
<name>hadoopdb . c o n f i g . f i l e </name>
<v a l u e>HadoopDB . xml</ v a l u e>
<d e s c r i p t i o n>The name o f t h e HadoopDB
c l u s t e r c o n f i g u r a t i o n f i l e </ d e s c r i p t i o n>
</ p r o p e r t y>
</ c o n f i g u r a t i o n>

/opt/hadoop/conf/mapred-site.xml:
5.35 mapreduce
Line 1
5
10

<c o n f i g u r a t i o n>
<p r o p e r t y>
<name>mapred . j o b . t r a c k e r</name>
<v a l u e>l o c a l h o s t : 5 4 3 1 0</ v a l u e>
<d e s c r i p t i o n>
The h o s t and p o r t t h a t t h e
MapReduce j o b t r a c k e r r u n s a t .
</ d e s c r i p t i o n>
</ p r o p e r t y>
</ c o n f i g u r a t i o n>

/opt/hadoop/conf/hdfs-site.xml:
5.36 hdfs
Line 1
5
-

<c o n f i g u r a t i o n>
<p r o p e r t y>
<name>d f s . r e p l i c a t i o n</name>
<v a l u e>1</ v a l u e>
<d e s c r i p t i o n>
Default block r e p l i c a t i o n .
</ d e s c r i p t i o n>
</ p r o p e r t y>
</ c o n f i g u r a t i o n>

Namenode:
5.37 Namenode
Line 1

$ hadoop namenode - format

105

5.4. HadoopDB
5
10
15
20
25
-

10/05/07 1 4 : 2 4 : 1 2 INFO namenode . NameNode : STARTUP_MSG:


/* ***********************************************************
STARTUP_MSG: S t a r t i n g NameNode
STARTUP_MSG:
h o s t = hadoop1 / 1 2 7 . 0 . 1 . 1
STARTUP_MSG:
a r g s = [ - format ]
STARTUP_MSG:
version = 0.20.2
STARTUP_MSG:
b u i l d = h t t p s : / / svn . apache . o r g / r e p o s
/ a s f / hadoop /common/ b r a n c h e s / branch - 0 . 2 0 - r
9 1 1 7 0 7 ; c o m p i l e d by c h r i s d o on F r i Feb 19 0 8 : 0 7 : 3 4 UTC
2010
*********************************************************** */
10/05/07 1 4 : 2 4 : 1 2 INFO namenode . FSNamesystem :
fsOwner=hadoop , hadoop
10/05/07 1 4 : 2 4 : 1 2 INFO namenode . FSNamesystem :
s u p e r g r o u p=s u p e r g r o u p
10/05/07 1 4 : 2 4 : 1 2 INFO namenode . FSNamesystem :
i s P e r m i s s i o n E n a b l e d=t r u e
10/05/07 1 4 : 2 4 : 1 2 INFO common . S t o r a g e :
Image f i l e o f s i z e 96 saved i n 0 s e c o n d s .
10/05/07 1 4 : 2 4 : 1 2 INFO common . S t o r a g e :
S t o r a g e d i r e c t o r y / opt / hadoop - data /tmp - b a s e / d f s /name has
been
s u c c e s s f u l l y formatted .
10/05/07 1 4 : 2 4 : 1 2 INFO namenode . NameNode :
SHUTDOWN_MSG:
/* ***********************************************************
SHUTDOWN_MSG: S h u t t i n g down NameNode a t hadoop1 / 1 2 7 . 0 . 1 . 1
*********************************************************** */

. Hadoop:
5.38 Hadoop
Line 1
5
10
-

$ s t a r t - a l l . sh
s t a r t i n g namenode , l o g g i n g t o / opt / hadoop / b i n / . .
/ l o g s / hadoop - hadoop - namenode - hadoop1 . out
l o c a l h o s t : s t a r t i n g datanode , l o g g i n g t o
/ opt / hadoop / b i n / . . / l o g s / hadoop - hadoop - datanode - hadoop1 . out
l o c a l h o s t : s t a r t i n g secondarynamenode , l o g g i n g t o
/ opt / hadoop / b i n / . . / l o g s / hadoop - hadoop - secondarynamenode - hadoop1 . out
s t a r t i n g jobtracker , l o g g i n g to
/ opt / hadoop / b i n / . . / l o g s / hadoop - hadoop - j o b t r a c k e r - hadoop1 . out
l o c a l h o s t : s t a r t i n g tasktracker , l o g g i n g to
/ opt / hadoop / b i n / . . / l o g s / hadoop - hadoop - t a s k t r a c k e r - hadoop1 . out

Hadoop stop-all.sh.

106

5.4. HadoopDB
HadoopDB Hive
HaddopDB1
$HADOOP_HOME/lib:

hadoopdb.jar

5.39 HadoopDB
Line 1

$cp hadoopdb . j a r $HADOOP_HOME/ l i b

PostgreSQL JDBC . 2
$HADOOP_HOME/lib.
Hive HadoopDB SQL .

HDFS Hive:
5.40 HadoopDB
hadoop
hadoop
- hadoop
- hadoop

Line 1
-

fs
fs
fs
fs

- mkdir
- mkdir
- chmod
- chmod

/tmp
/ u s e r / h i v e / warehouse
g+w /tmp
g+w / u s e r / h i v e / warehouse

HadoopDB SMS_dist. :
5.41 HadoopDB
t a r z x v f SMS_dist . t a r . gz
- sudo mv d i s t / opt / h i v e
- sudo chown -R hadoop : hadoop h i v e

Line 1

Hadoop,
Hive :
5.42 HadoopDB
$ hive
- Hive h i s t o r y f i l e =/tmp/ hadoop /
- hive_job_log_hadoop_201005081717_1990651345 . t x t
- hive >

Line 1

5
-

hive > q u i t ;

. :
5.43
Line 1
-

svn co h t t p : / / g r a f f i t i . c s . brown . edu / svn / benchmarks /


cd benchmarks / datagen / t e r a g e n

benchmarks/datagen/teragen/teragen.pl :
1
2

http://sourceforge.net/projects/hadoopdb/files/
http://jdbc.postgresql.org/download.html

107

5.4. HadoopDB
5.44
Line 1
-

use s t r i c t ;
use warnings ;

my $CUR_HOSTNAME = hostname - s ;
5 chomp ($CUR_HOSTNAME) ;
10
-

my
my
my
my
my
my
my

$NUM_OF_RECORDS_1TB
= 10000000000;
$NUM_OF_RECORDS_535MB = 1 0 0 ;
$BASE_OUTPUT_DIR
= "/ data " ;
$PATTERN_STRING
= "XYZ" ;
$PATTERN_FREQUENCY = 1 0 8 2 9 9 ;
$TERAGEN_JAR
= " teragen . jar " ;
$HADOOP_COMMAND
= $ENV{ HADOOP_HOME } . "/ b i n / hadoop " ;

15
20
-

my %f i l e s = ( " 535MB" => 1 ,


);
system ( "$HADOOP_COMMAND f s - rmr $BASE_OUTPUT_DIR" ) ;
f o r e a c h my $ t a r g e t ( k e y s %f i l e s ) {
my $output_dir = $BASE_OUTPUT_DIR. " / S o r t G r e p $ t a r g e t " ;
my $num_of_maps = $ f i l e s { $ t a r g e t } ;
my $num_of_records = ( $ t a r g e t eq " 535MB" ?
$NUM_OF_RECORDS_535MB : $NUM_OF_RECORDS_1TB) ;
p r i n t " G e n e r a t i n g $num_of_maps f i l e s i n $output_dir \ n" ;

25
30
35
-

##
## EXEC: hadoop j a r t e r a g e n . j a r 10000000000
## / data / SortGrep / XYZ 108299 100
##
my @args = ( $num_of_records ,
$output_dir ,
$PATTERN_STRING,
$PATTERN_FREQUENCY,
$num_of_maps ) ;
my $cmd = "$HADOOP_COMMAND j a r $TERAGEN_JAR " . j o i n ( " " ,
@args ) ;
p r i n t "$cmd\n" ;
system ( $cmd ) == 0 | | d i e ( "ERROR: $ ! " ) ;
} # FOR
exit (0) ;

Perl ,
HDFS.
, HDFS.

. , ,
HDFS, :
108

5.4. HadoopDB
5.45
Line 1
5
-

$hadoop f s - g e t / data /SortGrep535MB/ part - 0 0 0 0 0 my_file


$psql
p s q l > CREATE DATABASE g r e p 0 ;
p s q l > USE g r e p 0 ;
p s q l > CREATE TABLE g r e p (
->
key1 c h a r a c t e r v a r y i n g ( 2 5 5 ) ,
->
f i e l d character varying (255)
-> ) ;
COPY g r e p FROM my_file WITH DELIMITER | ;

HadoopDB. HadoopDB
Catalog.properties. :
5.46
Line 1
5
10
15
20
25
30
-

#P r o p e r t i e s f o r C a t a l o g G e n e r a t i o n
##################################
n o d e s _ f i l e=machines . t x t
r e l a t i o n s _ u n c h u n k e d=grep , E n t i r e R a n k i n g s
r e l a t i o n s _ c h u n k e d=Rankings , U s e r V i s i t s
c a t a l o g _ f i l e=HadoopDB . xml
##
#DB Connection Parameters
##
p o r t =5432
username=p o s t g r e s
password=password
d r i v e r=com . p o s t g r e s q l . D r i v e r
u r l _ p r e f i x=j d b c \ : p o s t g r e s q l \ : / /
##
#Chunking p r o p e r t i e s
##
chunks_per_node=0
unchunked_db_prefix=g r e p
chunked_db_prefix=cdb
##
#R e p l i c a t i o n P r o p e r t i e s
##
dump_script_prefix=/r o o t /dump_
r e p l i c a t i o n _ s c r i p t _ p r e f i x =/r o o t / l o a d _ r e p l i c a _
dump_file_u_prefix=/mnt/dump_udb
dump_file_c_prefix=/mnt/dump_cdb
##
#C l u s t e r Connection
##
ssh_key=i d _ r s a

109

5.4. HadoopDB
machines.txt localhost (
). HadoopDB HDFS:
5.47
j a v a - cp $HADOOP_HOME/ l i b / hadoopdb . j a r \
> edu . y a l e . c s . hadoopdb . c a t a l o g . S i m p l e C a t a l o g G e n e r a t o r \
- > Catalog . p r o p e r t i e s
- hadoop d f s - put HadoopDB . xml HadoopDB . xml

Line 1
-

:
5.48
Line 1

j a v a - cp hadoopdb . j a r
edu . y a l e . c s . hadoopdb . c a t a l o g . SimpleRandomReplicationFactorTwo
Catalog . p r o p e r t i e s

HadoopDB.xml,
.
HDFS:
5.49
Line 1
-

hadoop d f s - rmr HadoopDB . xml


hadoop d f s - put HadoopDB . xml HadoopDB . xml

true

hadoopdb.config.replication

HADOOP_HOME/conf/hadoop-site.xml.
HadoopDB.
, HDFS:
5.50
j a v a - cp $CLASSPATH : hadoopdb . j a r \
> edu . y a l e . c s . hadoopdb . benchmark . GrepTaskDB \
- > - p a t t e r n %wo% - output p a d r a i g - hadoop . c o n f i g . f i l e
HadoopDB . xml

Line 1
-

:
5.51
Line 1
5

$ j a v a - cp $CLASSPATH : hadoopdb . j a r
edu . y a l e . c s . hadoopdb . benchmark . GrepTaskDB \
> - p a t t e r n %wo% - output p a d r a i g - hadoop . c o n f i g . f i l e
HadoopDB . xml
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 edu . y a l e . c s . hadoopdb . e x e c . DBJobBase
initConf
INFO : SELECT key1 , f i e l d FROM g r e p WHERE f i e l d LIKE
%%wo%% ;
14.08.2010 19:08:48
o r g . apache . hadoop . m e t r i c s . jvm . JvmMetrics i n i t

110

5.4. HadoopDB
10
-

15
-

20
-

25
30
-

INFO : I n i t i a l i z i n g JVM M e t r i c s with processName=JobTracker ,


s e s s i o n I d=
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . J o b C l i e n t
configureCommandLineOptions
WARNING: Use G e n e r i c O p t i o n s P a r s e r f o r p a r s i n g t h e arguments .
A p p l i c a t i o n s s h o u l d implement Tool f o r t h e same .
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : Running j o b : job_local_0001
14.08.2010 19:08:48
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
getConnection
INFO : Data l o c a l i t y f a i l e d f o r l e o - p g s q l
14.08.2010 19:08:48
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
getConnection
INFO : Task from l e o - p g s q l i s c o n n e c t i n g t o chunk 0 on h o s t
l o c a l h o s t with
db u r l j d b c : p o s t g r e s q l : / / l o c a l h o s t : 5 4 3 4 / g r e p 0
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . MapTask
runOldMapper
INFO : numReduceTasks : 0
14.08.2010 19:08:48
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
close
INFO : DB t i m e s (ms) : c o n n e c t i o n = 1 0 4 , query e x e c u t i o n =
2 0 , row r e t r i e v a l = 79
14.08.2010 19:08:48
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
close
INFO : Rows r e t r i e v e d = 3
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . Task done
INFO : Task : attempt_local_0001_m_000000_0 i s done . And i s i n
t h e p r o c e s s o f commiting
14.08.2010 19:08:48
o r g . apache . hadoop . mapred . LocalJobRunner$Job s t a t u s U p d a t e
INFO :
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . Task commit
INFO : Task attempt_local_0001_m_000000_0 i s a l l o w e d t o
commit now
14.08.2010 19:08:48
o r g . apache . hadoop . mapred . FileOutputCommitter commitTask
INFO : Saved output o f t a s k attempt_local_0001_m_000000_0
t o f i l e : / home/ l e o / p a d r a i g
14.08.2010 19:08:48
o r g . apache . hadoop . mapred . LocalJobRunner$Job s t a t u s U p d a t e
INFO :

111

5.4. HadoopDB
35
40
45
50
55
-

1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 8 o r g . apache . hadoop . mapred . Task sendDone


INFO : Task attempt_local_0001_m_000000_0 done .
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : map 100% r e d u c e 0%
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : Job c o m p l e t e : job_local_0001
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO : Counters : 6
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FileSystemCounters
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FILE_BYTES_READ=141370
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FILE_BYTES_WRITTEN=153336
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map- Reduce Framework
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map i n p u t r e c o r d s =3
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
S p i l l e d Records=0
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map i n p u t b y t e s=3
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map output r e c o r d s =3
1 4 . 0 8 . 2 0 1 0 1 9 : 0 8 : 4 9 edu . y a l e . c s . hadoopdb . e x e c . DBJobBase run
INFO :
JOB TIME : 1828 ms .

HDFS, padraig:
5.52
$ cd p a d r a i g
$ c a t part - 0 0 0 0 0
- some data

Line 1
-

PostgreSQL:
5.53
Line 1
5
-

p s q l > s e l e c t * from g r e p where f i e l d l i k e %wo% ;


+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - - -+
| key1
| field
|
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - - - - - - -+
some data

1 rows i n s e t ( 0 . 0 0 s e c )

112

5.4. HadoopDB
10

psql>

. .
. PostgreSQL:
5.54
Line 1
-

p s q l > INSERT i n t o g r e p ( key1 ,


Maybe ) ;
p s q l > INSERT i n t o g r e p ( key1 ,
Maybewqe ) ;
p s q l > INSERT i n t o g r e p ( key1 ,
Maybewqesad ) ;
p s q l > INSERT i n t o g r e p ( key1 ,
string ! ) ;

f i e l d ) VALUES( I am l i v e ! ,
f i e l d ) VALUES( I am l i v e ! ,
f i e l d ) VALUES( I am l i v e ! ,
f i e l d ) VALUES( : ) , May c o o l

HadoopDB:
5.55
Line 1

5
-

10
-

$ j a v a - cp $CLASSPATH : hadoopdb . j a r
edu . y a l e . c s . hadoopdb . benchmark . GrepTaskDB - p a t t e r n %May%
- output p a d r a i g - hadoopdb . c o n f i g . f i l e
/ opt / hadoop / c o n f /HadoopDB . xml
padraig
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 5 edu . y a l e . c s . hadoopdb . e x e c . DBJobBase
initConf
INFO : SELECT key1 , f i e l d FROM g r e p WHERE f i e l d LIKE
%%May%% ;
01.11.2010 23:14:46
o r g . apache . hadoop . m e t r i c s . jvm . JvmMetrics i n i t
INFO : I n i t i a l i z i n g JVM M e t r i c s with processName=JobTracker ,
s e s s i o n I d=
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 6 o r g . apache . hadoop . mapred . J o b C l i e n t
configureCommandLineOptions
WARNING: Use G e n e r i c O p t i o n s P a r s e r f o r p a r s i n g t h e
arguments . A p p l i c a t i o n s s h o u l d implement Tool f o r t h e
same .
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 6 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : Running j o b : job_local_0001
01.11.2010 23:14:46
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
getConnection
INFO : Data l o c a l i t y f a i l e d f o r l e o - p g s q l
01.11.2010 23:14:46
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
getConnection

113

5.4. HadoopDB
-

15
-

20
25
30
35
40
-

INFO : Task from l e o - p g s q l i s c o n n e c t i n g t o chunk 0 on h o s t


l o c a l h o s t with db u r l
jdbc : p o s t g r e s q l :// l o c a l h o s t :5434/ grep0
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . MapTask
runOldMapper
INFO : numReduceTasks : 0
01.11.2010 23:14:47
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
close
INFO : DB t i m e s (ms) : c o n n e c t i o n = 1 8 1 , query e x e c u t i o n =
2 2 , row r e t r i e v a l = 96
01.11.2010 23:14:47
edu . y a l e . c s . hadoopdb . c o n n e c t o r . AbstractDBRecordReader
close
INFO : Rows r e t r i e v e d = 4
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Task done
INFO : Task : attempt_local_0001_m_000000_0 i s done . And i s i n
t h e p r o c e s s o f commiting
01.11.2010 23:14:47
o r g . apache . hadoop . mapred . LocalJobRunner$Job s t a t u s U p d a t e
INFO :
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Task commit
INFO : Task attempt_local_0001_m_000000_0 i s a l l o w e d t o
commit now
01.11.2010 23:14:47
o r g . apache . hadoop . mapred . FileOutputCommitter commitTask
INFO : Saved output o f t a s k attempt_local_0001_m_000000_0
t o f i l e : / home/ hadoop / p a d r a i g
01.11.2010 23:14:47
o r g . apache . hadoop . mapred . LocalJobRunner$Job s t a t u s U p d a t e
INFO :
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Task sendDone
INFO : Task attempt_local_0001_m_000000_0 done .
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : map 100% r e d u c e 0%
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . J o b C l i e n t
monitorAndPrintJob
INFO : Job c o m p l e t e : job_local_0001
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO : Counters : 6
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FileSystemCounters
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FILE_BYTES_READ=141345
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
FILE_BYTES_WRITTEN=153291

114

5.4. HadoopDB
45
50
55
-

0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g


INFO :
Map- Reduce Framework
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map i n p u t r e c o r d s =4
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
S p i l l e d Records=0
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map i n p u t b y t e s=4
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 o r g . apache . hadoop . mapred . Counters l o g
INFO :
Map output r e c o r d s =4
0 1 . 1 1 . 2 0 1 0 2 3 : 1 4 : 4 7 edu . y a l e . c s . hadoopdb . e x e c . DBJobBase run
INFO :
JOB TIME : 2332 ms .

May.
. :
5.56
Line 1
5
-

$ cd p a d r a i g
$ c a t part - 0 0 0 0 0
I am l i v e !
Maybe
I am l i v e !
Maybewqe
I am l i v e !
Maybewqesad
:)
May c o o l s t r i n g !

PostgreSQL
. HadoopDB
PostgreSQL,
PostgreSQL, shared-nothing .
HadoopDB
hadoopdb.sourceforge.net.

, Hive,
HadoopDB. ,
c Hadoop.
Hadoop HaddopDB.
HadoopDB Hadoop.
,

.
HadoopDB
,

, Hadoop.
HadoopDB, ,
, ,
115

5.5.
PostgreSQL , , PostgreSQL
. , Hadoop Hive
.
HadoopDB
Hadoop ,
,
,
, MapReduce. HadoopDB
Hadoop
( ) HadoopDB

.

5.5
.
PostgreSQL ,
, .
, ,
.

116

6
PgPool-II

.

6.1
pgpool-II , PostgreSQL
PostgreSQL. :

pgpool-II PostgreSQL

(.. , ,
).
.

pgpool-II PostgreSQL.

2
,
.

,
SELECT .
pgpool-II
PostgreSQL
SELECT ,
.

PostgreSQL.

.
117

6.2. !


PostgreSQL,
.

, ,
. pgpool-II
,

.


,

.
.
pgpool-II PostgreSQL
. ,
() pgpool-II PostgreSQL,
() pgpool-II .
pgpool-II , ,
, ,
pgpool-II .

pgpool.projects.pgfoundry.org.

6.2 !

pgpool-II
.

pgpool-II
pgpool-II .
,
,
.
6.1 pgpool-II
./ configure
make
- make i n s t a l l

Line 1
-

configure
.
118

6.2. !
configure -,
, , . pgpool-II -
/usr/local.
make , make install
.
.
: pgpool-II libpq
PostgreSQL 7.4 (3 ).
configure ,
libpq 3 .
6.2 pgpool-II
Line 1

c o n f i g u r e : e r r o r : l i b p q i s not i n s t a l l e d o r l i b p q i s o l d

3 , -
, libpq, ,
configure.
configure libpq
/usr/local/pgsql. PostgreSQL
/usr/local/pgsql with-pgsql
with-pgsql-includedir with-pgsql-libdir
configure.
Linux pgpool-II
. Ubuntu Linux, , :
6.3 pgpool-II
Line 1

sudo a p t i t u d e i n s t a l l p g p o o l 2


pgpool-II pgpool.conf.
: = .
pgpool-II pgpool.conf.sample.
pgpool.conf,
.
6.4
Line 1

cp / u s r / l o c a l / e t c / p g p o o l . c o n f . sample
/ usr / l o c a l / etc / pgpool . conf

pgpool-II localhost 9999.


,
listen_addresses *.
6.5
Line 1
-

listen_addresses = localhost
p o r t = 9999

119

6.2. !
- .
Ubuntu Linux /etc/pgpool.conf.

PCP
pgpool-II
, pgpool-II ..
. PCP,
.

PostgreSQL.
pcp.conf.
, (:). .
md5.
6.6 PCP
Line 1

postgres : e8a48653851e28c69d0506508fb27fc5

pgpool-II

pcp.conf.sample. pcp.conf
.
6.7 PCP
Line 1

$ cp / u s r / l o c a l / e t c / pcp . c o n f . sample / u s r / l o c a l / e t c / pcp . c o n f

Ubuntu Linux /etc/pcp.conf.


md5
pg_md5,
pgpool-II. pg_md5
md5 .
, postgres
pg_md5 md5 .
6.8 PCP
$ / u s r / b i n /pg_md5 p o s t g r e s
- e8a48653851e28c69d0506508fb27fc5

Line 1

PCP , pgpool.conf
pcp_port.
-
pcp_port 9898 .
6.9 PCP
Line 1

pcp_port = 9898

120

6.2. !


PostgreSQL
pgpool-II. pgpool-II
.
,
. ,

pgpool-II.

pgpool-II 5432, 5433,
5434 . pgpool-II
pgpool.conf .
6.10
Line 1
5
-

backend_hostname0 = l o c a l h o s t
backend_port0 = 5432
backend_weight0 = 1
backend_hostname1 = l o c a l h o s t
backend_port1 = 5433
backend_weight1 = 1
backend_hostname2 = l o c a l h o s t
backend_port2 = 5434
backend_weight2 = 1

backend_hostname, backend_port, backend_weight


,
.

0 (.. 0, 1, 2).
backend_weight 1,
SELECT .

/ pgpool-II
pgpool-II .
6.11
Line 1

pgpool

, ,
pgpool .
pgpool, -n
pgpool. pgpool-II -
.
6.12
Line 1

pgpool -n &

121

6.3.
,
,
, .
6.13
Line 1

p g p o o l - n - d > /tmp/ p g p o o l . l o g 2>&1 &

-d .

/tmp/pgpool.log.
, ,
. , , cronolog.
6.14
Line 1
-

p g p o o l - n 2>&1 | / u s r / s b i n / c r o n o l o g
- - h a r d l i n k =/var / l o g / p g s q l / p g p o o l . l o g
/ var / l o g / p g s q l/%Y-%m-%d - p g p o o l . l o g &

pgpool-II, .
6.15
Line 1

pgpool stop

- , pgpool-II
.
pgpool-II ,
.
6.16
Line 1

p g p o o l -m f a s t s t o p

6.3

.
,
6.2. !,
.

pgbench.



true replication_mode pgpool.conf.

122

6.3.
6.17
Line 1

replication_mode = true

replication_mode true, pgpool-II


.
load_balance_mode true, pgpool-II
SELECT .
6.18
Line 1

load_balance_mode = t r u e

replication_mode
load_balance_mode.


, pgpool.conf, pgpool-II
.

/ pgpool-II.
pgpool.conf pgpool-II,
.
,
. bench_replication.
. createdb
pgpool-II .
6.19
Line 1

c r e a t e d b - p 9999 b e n c h _ r e p l i c a t i o n

pgbench -i.
-i

.
6.20
Line 1

pgbench - i - p 9999 b e n c h _ r e p l i c a t i o n


, pgbench -i.
,
.

branches
1
tellers
10
accounts
100000
history
0

shell.
123

6.4.
branches, tellers, accounts history
(5432, 5433, 5434).
6.21
f o r p o r t i n 5432 5433 5 4 3 4 ; do
>
echo $ p o r t
- >
f o r table_name i n b r a n c h e s t e l l e r s a c c o u n t s h i s t o r y ;
do
- >
echo $table_name
5 >
p s q l - c "SELECT count ( * ) FROM $table_name " - p \
- >
$port bench_replication
- >
done
- > done

Line 1
-

6.4

.
( partitioning . ).
,
.
pgpool-II
,
(System Database) ( SystemDB).
SystemDB

,

. SystemDB
dblink.
,
6.2. !,
.
pgbench.



parallel_mode true
pgpool.conf.
6.22
Line 1

parallel_mode = true

parallel_mode true
.
pgpool-II

124

6.4.
SystemDB
.
SystemDB dblink
pgpool-II. ,
listen_addresses pgpool-II .
6.23
Line 1

listen_addresses = *

:
,
, ,
. , -

, bench_replication, 6.3.
.
6.24
replication_mode = true
- load_balance_mode = f a l s e

Line 1

6.25
replication_mode = f a l s e
- load_balance_mode = t r u e

Line 1

parallel_mode
load_balance_mode true, listen_addresses *,
replication_mode false.

SystemDB
,
. , dblink
,
. dist_def . ,
,
pgpool-II
.
SystemDB 5432.
SystemDB
6.26 SystemDB
system_db_hostname = l o c a l h o s t
- system_db_port = 5432
- system_db_dbname = p g p o o l
- system_db_schema = p g p o o l _ c a t a l o g

Line 1

125

6.4.
5
-

system_db_user = p g p o o l
system_db_password =

,
- pgpool.conf.

pgpool pgpool
pgpool.
6.27 SystemDB
Line 1
-

c r e a t e u s e r - p 5432 p g p o o l
c r e a t e d b - p 5432 -O p g p o o l p g p o o l

dblink
dblink pgpool. dblink
contrib
PostgreSQL.
dblink
.
6.28 dblink
USE_PGXS=1 make -C c o n t r i b / d b l i n k
- USE_PGXS=1 make -C c o n t r i b / d b l i n k i n s t a l l

Line 1

dblink
dblink pgpool. PostgreSQL
/usr/local/pgsql, dblink.sql ( )
/usr/local/pgsql/share/contrib.
dblink.
6.29 dblink
Line 1

p s q l - f / u s r / l o c a l / p g s q l / s h a r e / c o n t r i b / d b l i n k . s q l - p 5432
pgpool

dist_def
dist_def,
.
pgpool-II , system_db.sql
/usr/local/share/system_db.sql (

- /usr/local). system_db.sql
, dist_def.
dist_def.
6.30 dist_def

126

6.4.
Line 1

$ p s q l - f / u s r / l o c a l / s h a r e / system_db . s q l - p 5432 -U p g p o o l
pgpool

system_db.sql,
dist_def,
pgpool_catalog.
system_db_schema ,
, system_db.sql.
dist_def .
.
6.31 dist_def
Line 1
5
10
-

CREATE TABLE p g p o o l _ c a t a l o g . d i s t _ d e f (
dbname t e x t , - -
schema_name t e x t , - -
table_name t e x t , - -
col_name t e x t NOT NULL CHECK ( col_name = ANY
( col_list ) ) ,
- -
c o l _ l i s t t e x t [ ] NOT NULL, - -
t y p e _ l i s t t e x t [ ] NOT NULL, - -
d i s t _ d e f _ f u n c t e x t NOT NULL,
- -
PRIMARY KEY ( dbname , schema_name , table_name )
);

, dist_def, .
(col_name, dist_def_func)
- (dbname, schema_name, table_name,
col_list, type_list)

.
col_name. dist_def_func
, col_name
,
, .
- .

, -,
.
replicate_def
,
SQL, dist_def
, ,
,
127

6.4.
replicate_def. replicate_def
system_db.sql dist_def.
replicate_def .
6.32 replicate_def
Line 1
5
-

CREATE TABLE p g p o o l _ c a t a l o g . r e p l i c a t e _ d e f (
dbname t e x t , - -
schema_name t e x t , - -
table_name t e x t , - -
c o l _ l i s t t e x t [ ] NOT NULL, - -
t y p e _ l i s t t e x t [ ] NOT NULL, - -
PRIMARY KEY ( dbname , schema_name , table_name )
);



, pgbench, .
pgbench -i -s 3 (..
3).
bench_parallel.
sample pgpool-II
dist_def_pgbench.sql.
pgbench.
pgpool-II.
6.33
Line 1

p s q l - f sample / dist_def_pgbench . s q l - p 5432 p g p o o l

dist_def_pgbench.sql.
dist_def_pgbench.sql
dist_def. accounts.
- aid.
6.34
Line 1
5
10

INSERT INTO p g p o o l _ c a t a l o g . d i s t _ d e f VALUES (


bench_parallel ,
public ,
accounts ,
aid ,
ARRAY[ a i d , b i d , a b a l a n c e , f i l l e r ] ,
ARRAY[ i n t e g e r , i n t e g e r , i n t e g e r ,
character (84) ] ,
pgpool_catalog . dist_def_accounts
);

128

6.4.

accounts. ,
.
SQL (, PL/pgSQL, PL/Tcl,
..).
accounts
3, aid 1 300000.

.
SQL- .
6.35
Line 1
5
-

CREATE OR REPLACE FUNCTION


p g p o o l _ c a t a l o g . d i s t _ d e f _ b r a n c h e s ( anyelement )
RETURNS i n t e g e r AS $$
SELECT CASE WHEN $1 > 0 AND $1 <= 1 THEN 0
WHEN $1 > 1 AND $1 <= 2 THEN 1
ELSE 2
END;
$$ LANGUAGE s q l ;



.
pgbench
branches tellers.
, accounts
, branches tellers.
6.36
Line 1
5
-

INSERT INTO p g p o o l _ c a t a l o g . r e p l i c a t e _ d e f VALUES (


bench_parallel ,
public ,
branches ,
ARRAY[ b i d , b b a l a n c e , f i l l e r ] ,
ARRAY[ i n t e g e r , i n t e g e r , c h a r a c t e r ( 8 8 ) ]
);

10
15

INSERT INTO p g p o o l _ c a t a l o g . r e p l i c a t e _ d e f VALUES (


bench_parallel ,
public ,
tellers ,
ARRAY[ t i d , b i d , t b a l a n c e , f i l l e r ] ,
ARRAY[ i n t e g e r , i n t e g e r , i n t e g e r , c h a r a c t e r ( 8 4 ) ]
);

129

6.4.
Replicate_def_pgbench.sql
sample. psql ,
, , .
6.37
Line 1

p s q l - f sample / r e p l i c a t e _ d e f _ p g b e n c h . s q l - p 5432 p g p o o l


, pgpool.conf, pgpool-II
.

/ pgpool-II.
pgpool.conf pgpool-II,
.
, .
bench_parallel.
. createdb pgpool-II
.
6.38
Line 1

c r e a t e d b - p 9999 b e n c h _ p a r a l l e l

pgbench -i -s 3.
-i
.
-s .
6.39
Line 1

pgbench - i - s 3 - p 9999 b e n c h _ p a r a l l e l


.

SELECT pgpool-II
.
bench_parallel .

branches
3
tellers
30
accounts
300000
history
0

pgpool-II shell.

accounts 5432, 5433, 5434 9999.
6.40

130

6.5. Master-slave
f o r p o r t i n 5432 5433 5434 i 9 9 9 9 ; do
>
echo $ p o r t
- >
p s q l - c "SELECT min ( a i d ) , max( a i d ) FROM a c c o u n t s " \
- >
-p $port bench_parallel
5 > done

Line 1
-

6.5 Master-slave
pgpool-II
( Slony-I, Londiste).

.
master_slave_mode
load_balance_mode true. pgpool-II
INSERT/UPDATE/DELETE Master DB (1 ),
SELECT , .
, DDL DML
. SELECT ,
/*NO LOAD BALANCE*/
SELECT.
Master/Slave replication_mode false,
master_slave_mode true.

Streaming Replication ( )
master-slave ,
,
pgpool-II. PostgreSQL,
pgpool-II (
), (
). , ,
( recovery.conf,
trigger_file), PostgreSQL
. :
6.41 PostgreSQL
Line 1
5
-

#! / b i n / sh
# F a i l o v e r command f o r s t r e m i n g r e p l i c a t i o n .
# This s c r i p t assumes t h a t DB node 0 i s primary , and 1 i s
standby .
#
# I f standby g o e s down , d o e s n o t h i n g . I f primary g o e s down ,
create a
# t r i g g e r f i l e s o t h a t standby t a k e o v e r primary node .
#
# Arguments : $1 : f a i l e d node i d . $2 : new master hostname .
$3 : path t o

131

6.6.
-

# trigger f i l e .

10

f a i l e d _ n o d e=$1
- new_master=$2
- t r i g g e r _ f i l e=$3
-

# Do n o t h i n g i f standby g o e s down .
i f [ $ f a i l e d _ n o d e = 1 ] ; then
exit 0;
- fi

15
-

20
-

# Create t r i g g e r f i l e .
/ u s r / b i n / s s h -T $new_master / b i n / touch $ t r i g g e r _ f i l e

exit 0;

: ,
.
failover_stream.sh pgpool.conf :
6.42
Line 1

failover_command = / p a th _ t o_ s c r ip t / f a i l o v e r _ s t r e a m . sh %d
%H /tmp/ t r i g g e r _ f i l e

/tmp/trigger_file , recovery.conf.
, ,
.

6.6
pgpool-II, ,
pgpool.
. ,
.
.
pgpool,
.
:

CHECKPOINT;
;
, ;
CHECKPOINT;
;
postmaster ( pgpool_remote_start);
;

132

6.6.

:
backend_data_directory
PostgreSQL .
recovery_user
PostgreSQL.
recovery_password
PostgreSQL.
recovery_1st_stage_command

.

- .
,
recovery_1st_stage_command = some_script, pgpool-II
$PGDATA/some_script.
, pgpool-II

recovery_1st_stage.
recovery_2nd_stage_command

.

- .
,
recovery_2st_stage_command = some_script, pgpool-II
$PGDATA/some_script.
, pgpool-II

recovery_2st_stage. , pgpool-II ,
.

Streaming Replication ( )
master-slave ,

PostgreSQL.
,
.
PostgreSQL pgpool-II (
).
:
recovery_user. postgres.
6.43 recovery_user
Line 1

recovery_user = postgres

133

6.6.

recovery_password
.

recovery_user

6.44 recovery_password
Line 1

recovery_password = some_password

recovery_1st_stage_command.
basebackup.sh
($PGDATA), .
:
6.45 basebackup.sh
Line 1
5
-

#! / b i n / sh
# Recovery s c r i p t f o r s t r e a m i n g r e p l i c a t i o n .
# This s c r i p t assumes t h a t DB node 0 i s primary , and 1
i s standby .
#
d a t a d i r=$1
d e s t h o s t=$2
d e s t d i r=$3

p s q l - c "SELECT pg_start_backup ( Streaming


Replication , true )" postgres

10
-

r s y n c -C - a - - d e l e t e - e s s h - - e x c l u d e p o s t g r e s q l . c o n f
- - exclude postmaster . pid \
- - e x c l u d e p o s t m a s t e r . o p t s - - e x c l u d e pg_log - - e x c l u d e
pg_xlog \
- - exclude recovery . conf $datadir / $desthost : $destdir /

15

s s h -T l o c a l h o s t mv $ d e s t d i r / r e c o v e r y . done
$destdir / recovery . conf

p s q l - c "SELECT pg_stop_backup ( ) " p o s t g r e s

,
rsync .
SSH , recovery_user
.

:
6.46 recovery_1st_stage_command
Line 1

recovery_1st_stage_command = basebackup . sh

recovery_2nd_stage_command .

,
134

6.7.
,
basebackup.sh, WAL .
C SQL
.
6.47 C SQL
$
$
- $
- $

Line 1
-

cd pgpool - I I - x . x . x/ s q l / pgpool - r e c o v e r y
make
make i n s t a l l
p s q l - f pgpool - r e c o v e r y . s q l t e m p l a t e 1

. pcp_recovery_node
.

6.7
PgPool-II ,
PostgreSQL.

135

7


,
.
.
?

7.1
(
) ,
,
PostgreSQL.
Windows, .
-,
.
, :
PgBouncer
Pgpool

7.2 PgBouncer
PostgreSQL Skype.
.
Session Pooling. .
;

.
Transaction Pooling.
. PgBouncer ,
, .
136

7.3. PgPool-II vs PgBouncer


Statement Pooling. .
.
,
.
(prepared statements) .
PgBouncer :
( 2 );
;
.
:
7.1 PgBouncer
Line 1

pgbouncer [ - d ] [ - R ] [ - v ] [ - u u s e r ] <pgbouncer . i n i >

:
7.2 PgBouncer
Line 1
5
10

[ databases ]
t e m p l a t e 1 = h o s t = 1 2 7 . 0 . 0 . 1 p o r t =5432 dbname=t e m p l a t e 1
[ pgbouncer ]
l i s t e n _ p o r t = 6543
listen_addr = 127.0.0.1
auth_type = md5
auth_file = u s e r l i s t . txt
l o g f i l e = pgbouncer . l o g
p i d f i l e = pgbouncer . p i d
admin_users = someuser

userlist.txt
:someuser same_password_as_in_server
pgbouncer:

7.3 PgBouncer
Line 1

p s q l - h 1 2 7 . 0 . 0 . 1 - p 6543 pgbouncer


SHOW.

7.3 PgPool-II vs PgBouncer


. PgBouncer
, PgPool-II. ,
PgPool-II ( ),
PgBouncer.

137

7.3. PgPool-II vs PgBouncer


PgBouncer , PgPool-II
PgBouncer
PgBouncer (
)
PgBouncer PgPool-II .

138

8
PostgreSQL
- ,
- .

8.1
,
,
.
SELECT
PostgreSQL.

, , ,
.
SQL ,
, , . PostgreSQL
. ? -,
. ?
(MVCC MultiVersion
Concurrency Control)
,
, ,

.
, ,
.
, ,
. -,
, , ,
.
( ),

.
139

8.2. Pgmemcache
, ,
.
PostgreSQL:
Pgmemcache ( memcached)
Pgpool-II (query cache)

8.2 Pgmemcache
Memcached1 ,

-.


.

-.
.
,

memcached .
Pgmemcache2 PostgreSQL API libmemcached
memcached.

PostgreSQL , ,
memcached.
,
.

2.0.4
pgmemcache3 . Pgmemcache
PostgreSQL 8.4 ( 9.0 ),
Ubuntu Server 10.10. Pgmemcache ,
PostgreSQL PGXS ( ,
Linux PGXS). memcached
libmemcached 0.38.
,
Pgmemcache:

:
1

http://memcached.org/
http://pgfoundry.org/projects/pgmemcache/
3
http://pgfoundry.org/frs/download.php/2672/pgmemcache_2.0.4.tar.bz2
2

140

8.2. Pgmemcache
8.1
Line 1
-

$ make
$ sudo make i n s t a l l

deb ( Debian, Ubuntu)


, Debian Ubuntu,
deb
PostgreSQL:
8.2 deb
$ sudo apt - g e t i n s t a l l libmemcached - dev
p o s t g r e s q l - s e r v e r - dev - 8 . 4 l i b p q - dev d e v s c r i p t s yada
f l e x bison
- $ make deb84
- # deb
- $ sudo dpkg - i . . / p o s t g r e s q l - pgmemcache - 8 . 4 * . deb

Line 1

2.0.4 yada deb


:
8.3 deb
Cannot r e c o g n i z e s o u r c e name i n d e b i a n / c h a n g e l o g a t
/ u s r / b i n / yada l i n e 1 4 5 , <CHANGELOG> l i n e 1 .
- make : *** [ deb84 ] 9

Line 1


debian/changelog , :
8.4 deb
$PostgreSQL : pgmemcache/ d e b i a n / c h a n g e l o g , v 1 . 2
2010/05/05 1 9 : 5 6 : 5 0 ormod Exp $ < - - - -
- pgmemcache ( 2 . 0 . 4 ) u n s t a b l e ; urgency=low

Line 1

* v2 . 0 . 4

, deb
.

Pgmemcache
( Pgmemcache)
:
8.5
Line 1
-

% p s q l [ mydbname ] [ p g u s e r ]
[ mydbname]=# BEGIN ;

141

8.2. Pgmemcache
[ mydbname]=# \ i
/ u s r / l o c a l / p o s t g r e s q l / s h a r e / c o n t r i b /pgmemcache . s q l
- # Debian : \ i
/ u s r / s h a r e / p o s t g r e s q l / 8 . 4 / c o n t r i b /pgmemcache . s q l
5 [ mydbname]=# COMMIT;
-

memcached

memcache_server_add . .
memcached
PostgreSQL. ,
postgresql.conf :
pgmemcache shared_preload_libraries (
pgmemcache PostgreSQL)
pgmemcache custom_variable_classes (
pgmemcache)
pgmemcache.default_servers, host:port
(port - ) . :
8.6 default_servers
Line 1

pgmemcache . d e f a u l t _ s e r v e r s = 1 2 7 . 0 . 0 . 1 ,
1 9 2 . 1 6 8 . 0 . 2 0 : 1 1 2 1 1 # memcached

pgmemcache
pgmemcache.default_behavior.
libmemcached. :
8.7 pgmemcache
Line 1

pgmemcache . d e f a u l t _ b e h a v i o r= BINARY_PROTOCOL: 1

PostgreSQL
memcached.

pgmemcache,
memcached :
.
memcached :
8.8 memcache_stats
pgmemcache=# SELECT memcache_stats ( ) ;
memcache_stats
- -- -------------------------

Line 1
5
-

Server : 1 2 7 . 0 . 0 . 1 (11211)
p i d : 1116

142

8.2. Pgmemcache
Table 8.1: pgmemcache

10
-

memcache_server_add(hostname:port::TEXT)
memcache_server_add(hostname::TEXT)

memcached
. ,
11211.

memcache_add(key::TEXT, value::TEXT,
expire::TIMESTAMPTZ)
memcache_add(key::TEXT, value::TEXT,
expire::INTERVAL)
memcache_add(key::TEXT, value::TEXT)

newval = memcache_decr(key::TEXT,
decrement::INT4)
newval = memcache_decr(key::TEXT)


,
( ).
.

memcache_delete(key::TEXT,
hold_timer::INTERVAL)
memcache_delete(key::TEXT)

.

,

.

memcache_flush_all()

memcached
.

value = memcache_get(key::TEXT)

. NULL,
,
.

memcache_get_multi(keys::TEXT[])
memcache_get_multi(keys::BYTEA[])

.

=.

newval = memcache_incr(key::TEXT,
increment::INT4)
newval = memcache_incr(key::TEXT)


,
( ).
.

memcache_replace(key::TEXT, value::TEXT,
expire::TIMESTAMPTZ)
memcache_replace(key::TEXT, value::TEXT,
expire::INTERVAL)
memcache_replace(key::TEXT, value::TEXT)

memcache_set(key::TEXT, value::TEXT,
expire::TIMESTAMPTZ)
memcache_set(key::TEXT, value::TEXT,
expire::INTERVAL)
memcache_set(key::TEXT, value::TEXT)

.

.

stats = memcache_stats()

memcached.

uptime : 70
time : 1289598098
version : 1.4.5
p o i n t e r _ s i z e : 32
rusage_user : 0.0
rusage_system : 0 . 2 4 0 0 1
cur r _ it e m s : 0
total_items : 0

143

8.2. Pgmemcache
15
20
25
-

bytes : 0
curr_connections : 5
total_connections : 7
connection_structures : 6
cmd_get : 0
cmd_set : 0
get_hits : 0
get_misses : 0
evictions : 0
bytes_read : 20
b y t e s _ w r i t t e n : 782
limit_maxbytes : 67108864
threads : 4

( 1 row )

memcached :
8.9
pgmemcache=# SELECT memcache_add ( some_key , t e s t _ v a l u e ) ;
memcache_add
- -- -----------t
5 ( 1 row )

Line 1
-

pgmemcache=# SELECT memcache_get ( some_key ) ;


memcache_get
- -- -----------10
test_value
- ( 1 row )
-

memcached (
):
8.10
pgmemcache=# SELECT memcache_add ( some_seq , 10 ) ;
memcache_add
- -- -----------t
5 ( 1 row )

Line 1
-

pgmemcache=# SELECT memcache_incr ( some_seq ) ;


memcache_incr
- -- ------------10
11
- ( 1 row )
-

pgmemcache=# SELECT memcache_incr ( some_seq ) ;

144

8.2. Pgmemcache
15
-

memcache_incr
-- ------------12
( 1 row )

20
-

pgmemcache=# SELECT memcache_incr ( some_seq , 1 0 ) ;


memcache_incr
-- ------------22
( 1 row )

pgmemcache=# SELECT memcache_decr ( some_seq ) ;


memcache_decr
- -- ------------21
- ( 1 row )

25
-

30
35

pgmemcache=# SELECT memcache_decr ( some_seq ) ;


memcache_decr
-- ------------20
( 1 row )

40
-

pgmemcache=# SELECT memcache_decr ( some_seq , 6 ) ;


memcache_decr
-- ------------14
( 1 row )

pgmemcache , ,
.
,
memcached ( ),
, .
:
8.11
Line 1
5
-

CREATE OR REPLACE FUNCTION auth_passwd_upd ( ) RETURNS


TRIGGER AS $$
BEGIN
IF OLD. passwd != NEW. passwd THEN
PERFORM memcache_set ( user_id_ | |
NEW. u s e r _ i d | | _password , NEW. passwd ) ;
END IF ;
RETURN NEW;
END;
$$ LANGUAGE p l p g s q l ;

145

8.2. Pgmemcache
:
8.12
Line 1

CREATE TRIGGER auth_passwd_upd_trg AFTER UPDATE ON passwd


FOR EACH ROW EXECUTE PROCEDURE auth_passwd_upd ( ) ;

(!!!)
.
:
8.13
Line 1
5
-

CREATE OR REPLACE FUNCTION auth_passwd_upd ( ) RETURNS


TRIGGER AS $$
BEGIN
IF OLD. passwd != NEW. passwd THEN
PERFORM memcache_delete ( user_id_ | |
NEW. u s e r _ i d | | _password ) ;
END IF ;
RETURN NEW;
END; $$ LANGUAGE p l p g s q l ;

:
8.14
Line 1

CREATE TRIGGER auth_passwd_del_trg AFTER DELETE ON passwd


FOR EACH ROW EXECUTE PROCEDURE auth_passwd_upd ( ) ;

, memcached
( ) .

PostgreSQL Pgmemcache
memcached ,
.
memcached,
SQL ,
PostgreSQL.

146

9.1
PostgreSQL
.
.

9.2 PostGIS
: Open Source
: www.postgis.org
PostGIS
PostgreSQL. PostGIS PostgreSQL

(), , ESRI SDE
Oracle. PostGIS OpenGIS
SQL .

9.3 pgSphere
: Open Source
: pgsphere.projects.postgresql.org
pgSphere PostgreSQL ,
.
( PostGIS)
.

147

9.4. HStore

9.4 HStore
: Open Source
HStore ,
/ PostgreSQL (
). ,
, ,
- .
.


:
9.1 hstore
Line 1

# CREATE EXTENSION h s t o r e ;

:
9.2 hstore
Line 1
5

# SELECT a=>1,a=>2 : : h s t o r e ;
hstore
-- -------"a"=>"1 "
( 1 row )

9.2 hstore .
:
9.3 hstore
Line 1
5
10
15
-

CREATE TABLE p r o d u c t s (
i d s e r i a l PRIMARY KEY,
name varchar ,
attributes hstore
);
INSERT INTO p r o d u c t s ( name , a t t r i b u t e s )
VALUES (
Geek Love : A Novel ,
author
=> " K a t h e r i n e Dunn" ,
pa g e s
=> 3 6 8 ,
c a t e g o r y => f i c t i o n
),
(
L e i c a M9 ,
m a n u f a c t u r e r => L e i c a ,
type
=> camera ,
megapixels
=> 1 8 ,
sensor
=> " f u l l - frame 35mm"

148

9.4. HStore
20
25
-

),
( MacBook Air 11 ,
m a n u f a c t u r e r => Apple ,
type
=> computer ,
ram
=> 4GB,
storage
=> 256GB,
processor
=> " 1 . 8 ghz I n t e l i 7 d u e l c o r e " ,
weight
=> 2 . 3 8 l b s
);

:
9.4
Line 1
5

# SELECT name , a t t r i b u t e s -> p a g e s a s page FROM p r o d u c t s


WHERE a t t r i b u t e s ? p a g e s ;
name
| page
- - - - - - - - - - - - - - - - - - - -+ - - - - - Geek Love : A Novel | 368
( 1 row )

:
9.5
Line 1
5

# SELECT name , a t t r i b u t e s -> m a n u f a c t u r e r a s m a n u f a c t u r e r


FROM p r o d u c t s WHERE a t t r i b u t e s -> type = computer ;
name
| manufacturer
- - - - - - - - - - - - - - - -+ - - - - - - - - - - - - - MacBook Air 11 | Apple
( 1 row )

:
9.6
CREATE INDEX p r o d u c t s _ h s t o r e _ i n d e x ON p r o d u c t s USING GIST
( attributes ) ;
- CREATE INDEX p r o d u c t s _ h s t o r e _ i n d e x ON p r o d u c t s USING GIN
( attributes ) ;

Line 1

c :
9.7
Line 1
-

CREATE INDEX product_manufacturer


ON p r o d u c t s ( ( p r o d u c t s . a t t r i b u t e s -> m a n u f a c t u r e r ) ) ;

HStore
PostgreSQL.
149

9.5. PLV8

9.5 PLV8
: Open Source
: code.google.com/p/plv8js
PLV8 , PostgreSQL
V8 JavaScript.

PostgreSQL JavaScript ,
SQL.


V81 JavaScript
.
.
plpgsql:
9.8 plpgsql
Line 1
5
-

CREATE OR REPLACE FUNCTION


p s q l f i b ( n i n t ) RETURNS i n t AS $$
BEGIN
IF n < 2 THEN
RETURN n ;
END IF ;
RETURN p s q l f i b ( n - 1 ) + p s q l f i b ( n - 2 ) ;
END;
$$ LANGUAGE p l p g s q l IMMUTABLE STRICT ;

:
9.9 plpgsql
Line 1
5
10
-

SELECT n , p s q l f i b ( n ) FROM g e n e r a t e _ s e r i e s ( 0 , 3 0 , 5 ) a s n ;
n | psqlfib
- - - -+ - - - - - - - - 0 |
0
5 |
5
10 |
55
15 |
610
20 |
6765
25 |
75025
30 | 832040
( 7 rows )

Time : 1 6 0 0 3 , 2 5 7 ms

, PLV8:
1

http://en.wikipedia.org/wiki/V8_(JavaScript_engine)

150

9.5. PLV8
9.10 plv8
CREATE OR REPLACE FUNCTION
- f i b ( n i n t ) RETURNS i n t a s $$

Line 1
5
-

function fib (n) {


r e t u r n n<2 ? n : f i b ( n - 1 ) + f i b ( n - 2 )
}
return f i b (n)

$$ LANGUAGE p l v 8 IMMUTABLE STRICT ;

:
9.11 plv8
Line 1
5
10
-

SELECT n , f i b ( n ) FROM g e n e r a t e _ s e r i e s ( 0 , 3 0 , 5 ) a s n ;
n |
fib
- - - -+ - - - - - - - 0 |
0
5 |
5
10 |
55
15 |
610
20 |
6765
25 | 75025
30 | 832040
( 7 rows )

Time : 5 9 , 2 5 4 ms

PLV8 270 (16003.257/59.254)


plpgsql. PLV8
:
9.12 plv8
Line 1
5
10

CREATE OR REPLACE FUNCTION


f i b 1 ( n i n t ) RETURNS i n t a s $$
var memo = { 0 : 0 , 1 : 1 } ;
function fib (n) {
i f ( ! ( n i n memo) )
memo [ n ] = f i b ( n - 1 ) + f i b ( n - 2 )
r e t u r n memo [ n ]
}
return f i b (n) ;
$$ LANGUAGE p l v 8 IMMUTABLE STRICT ;

:
9.13 plv8
Line 1

SELECT n , f i b 1 ( n ) FROM g e n e r a t e _ s e r i e s ( 0 , 3 0 , 5 ) a s n ;

151

9.5. PLV8
5
10
-

n |
fib1
- - - -+ - - - - - - - 0 |
0
5 |
5
10 |
55
15 |
610
20 |
6765
25 | 75025
30 | 832040
( 7 rows )

Time : 0 , 7 6 6 ms

PLV8 20892 (16003.257/0.766)


, plpgsql.

PLV8
PostgreSQL .

hstore,
:

( )

JSON (MongoDB, CouchDB, Couchbase ..).
PostgreSQL 9.2 JSON.
PostgreSQL 9.1 PLV8 DOMAIN:
9.14 JSON
Line 1
5
-

CREATE OR REPLACE FUNCTION


valid_json ( json text )
RETURNS BOOLEAN AS $$
try {
JSON . p a r s e ( j s o n ) ; r e t u r n t r u e ;
} catch ( e ) {
return f a l s e ;
}
$$ LANGUAGE p l v 8 IMMUTABLE STRICT ;

10
-

CREATE DOMAIN j s o n AS TEXT


CHECK( v a l i d _ j s o n (VALUE) ) ;

valid_json JSON .
:
152

9.5. PLV8
9.15 JSON
Line 1
5
10
-

$ INSERT INTO members


VALUES( not good j s o n ) ;
ERROR: v a l u e f o r domain j s o n
v i o l a t e s check c o n s t r a i n t " json_check "
$ INSERT INTO members
VALUES( {" good " : " j s o n " , " i s " : t r u e } ) ;
INSERT 0 1
$ s e l e c t * from members ;
profile
-- ---------------------------{" good " : " j s o n " , " i s " : t r u e }
( 1 row )

JSON PLV8
. :
9.16 JSON
Line 1
5
-

$ CREATE TABLE members ( i d SERIAL , p r o f i l e j s o n ) ;


$ SELECT count ( * ) FROM members ;
count
-- ------1000000
( 1 row )

Time : 2 0 1 . 1 0 9 ms

profile JSON:
9.17 JSON
Line 1

5
10
15
-

+
"name" : " L i t z y S a t t e r f i e l d " ,
+
" age " : 2 4 ,
+
"siblings": 2,
+
" faculty " : false ,
+
" numbers " : [
+
{
+
" type " :
" work " ,
+
"number" : " 6 8 4 . 5 7 3 . 3 7 8 3 x368 "+
},
+
{
+
" type " :
"home" ,
+
"number" : " 6 2 5 . 1 1 2 . 6 0 8 1 "
+
}
+
]
+

JSON (
):
153

9.5. PLV8
9.18 JSON
Line 1
5

CREATE OR REPLACE FUNCTION get_numeric ( json_raw j s o n , key


text )
RETURNS numeric AS $$
var o = JSON . p a r s e ( json_raw ) ;
r e t u r n o [ key ] ;
$$ LANGUAGE p l v 8 IMMUTABLE STRICT ;


age, siblings :
9.19 JSON
$ SELECT * FROM members WHERE get_numeric ( p r o f i l e , age ) =
36;
- Time : 9 3 4 0 . 1 4 2 ms
- $ SELECT * FROM members WHERE get_numeric ( p r o f i l e ,
siblings ) = 1;
- Time : 1 4 3 2 0 . 0 3 2 ms

Line 1

, .
, :

9.20
CREATE INDEX member_age ON members ( get_numeric ( p r o f i l e ,
age ) ) ;
- CREATE INDEX member_siblings ON members
( get_numeric ( p r o f i l e , s i b l i n g s ) ) ;

Line 1

JSON :
9.21 JSON
Line 1
5
-

$ SELECT * FROM members WHERE get_numeric ( p r o f i l e , age ) =


36;
Time : 5 7 . 4 2 9 ms
$ SELECT * FROM members WHERE get_numeric ( p r o f i l e ,
siblings ) = 1;
Time : 6 5 . 1 3 6 ms
$ SELECT count ( * ) from members where get_numeric ( p r o f i l e ,
age ) = 26 and get_numeric ( p r o f i l e , s i b l i n g s ) = 1 ;
Time : 1 0 6 . 4 9 2 ms


PostgreSQL.
PLV8 JavaScript
PostgreSQL. Mustache1 :
9.22 Mustache
1

http://mustache.github.com/

154

9.6. Smlar
Line 1
5

CREATE OR REPLACE FUNCTION mustache ( t e m p l a t e t e x t , view


json )
RETURNS t e x t a s $$
// . . . 400 l i n e s o f mustache . . . . j s
r e t u r n Mustache . r e n d e r ( template , JSON . p a r s e ( view ) )
$$ LANGUAGE p l v 8 IMMUTABLE STRICT ;
9.23

Line 1
5
-

$ SELECT mustache (
h e l l o {{# t h i n g s } } { { . } } {{/ t h i n g s } } : )
{{#data }}{{ key }}{{/ data }} ,
{" t h i n g s " : [ " world " , " from " , " p o s t g r e s q l " ] , " data " :
{" key " : "and me"}}
);
mustache
-- ------------------------------------h e l l o world from p o s t g r e s q l : ) and me
( 1 row )

10

Time : 0 . 8 3 7 ms

PLV8.
Mustache PostgreSQL .

PLV8 PostgreSQL
V8 JavaScript, JavaScript
, JSON
.

9.6 Smlar
: Open Source
: sigaev.ru

( ),
- ( ), (
, ) ..
PostgreSQL
. ,
, .

.
, ,
155

9.6. Smlar
- , , ..
,
, ( 1 , n-grams2 ).
.
?


.
, :
,


-

:
(, ) =

( + )

(9.1)

(9.2)


(, ) =
:

: * log

3 :
(, ) =

(9.3)

:
: * log

:

: ,
1

http://en.wikipedia.org/wiki/Fingerprint
http://en.wikipedia.org/wiki/N-gram
3
http://en.wikipedia.org/wiki/Law_of_cosines
2

156

9.6. Smlar
.
-
.
TF/IDF1
:
< ,< , = *
(, ) =
2
2
<
< *

(9.4)

:
= log (

+ 1)

(9.5)

:
= *

(9.6)

! smlar ,
( ) . ,
TF/IDF
, .

Smlar
.
PostgreSQL smlar,
(
)
GIST GIN.
(PostgreSQL ):
9.24 smlar
g i t c l o n e g i t : / / s i g a e v . ru / s m l a r
- cd s m l a r
- USE_PGXS=1 make && make i n s t a l l

Line 1

PostgreSQL 9.2 ,
PostgreSQL 9.1
. smlar_guc.c 214 :
9.25 9.1
Line 1

s e t _ c o n f i g _ o p t i o n ( " s m l ar . t h r e s h o l d " , buf , PGC_USERSET,


PGC_S_SESSION ,GUC_ACTION_SET, t r u e , 0 ) ;

( ):
1

http://en.wikipedia.org/wiki/Tf*idf

157

9.6. Smlar
9.26 9.1
Line 1

s e t _ c o n f i g _ o p t i o n ( " s m l ar . t h r e s h o l d " , buf , PGC_USERSET,


PGC_S_SESSION ,GUC_ACTION_SET, t r u e ) ;

:
9.27 smlar
$ psql
psql ( 9 . 2 . 1 )
- Type " h e l p " f o r h e l p .

Line 1
-

t e s t=# CREATE EXTENSION s m l a r ;


- CREATE EXTENSION

5
10
-

t e s t=# SELECT s m l a r ( { 1 , 4 , 6 } : : i n t [ ] ,
sm l a r
-- -------0.666667
( 1 row )

{5 ,4 ,6} : : int [ ] ) ;

t e s t=# SELECT s m l a r ( { 1 , 4 , 6 } : : i n t [ ] ,
N. i / s q r t (N. a * N. b ) ) ;
sm l a r
-- -------0.666667
( 1 row )

{5 ,4 ,6} : : int [ ] ,

15
-

,
. , :
float4 smlar(anyarray, anyarray)
. .
float4 smlar(anyarray, anyarray, bool useIntersect)
.
:
9.28
Line 1

CREATE TYPE type_name AS ( element_name anytype ,


weight_name f l o a t 4 ) ;

useIntersect

float4 smlar( anyarray a, anyarray b, text formula )
,
. :
N.i
()

158

9.6. Smlar
N.a
N.b
anyarray % anyarray ,
, .
PostgreSQL:
9.29 Smlar
Line 1
-

custom_variable_classes = smlar
s m l a r . t h r e s h o l d = 0 . 8 # 0 1


smlar:
9.30 Smlar
Line 1
-

custom_variable_classes = smlar
s m l a r . t h r e s h o l d = 0 . 8 # 0 1
s m l a r . type = c o s i n e #
: c o s i n e , t f i d f , o v e r l a p
s m l a r . s t a t t a b l e = s t a t #
t f i d f

README .
GiST GIN %.

:
.
,
( , ,
). , ,
. ,
?
.
. , ,
, .
:
(
), 15X15 (. 9.1).
(
0.299 * + 0.587 * +
0.114 * ).
, .

(. 9.2).
159

9.6. Smlar

. 9.1:
(
+ ).
.

. 9.2:
, ,
:
160

9.6. Smlar
9.31
Line 1
5
-

CREATE TABLE images (


i d s e r i a l PRIMARY KEY,
name v a r c h a r ( 5 0 ) ,
img_path v a r c h a r ( 2 5 0 ) ,
image_array i n t e g e r [ ]
);

GIN GIST :
9.32 GIN GIST
CREATE INDEX image_array_gin ON images USING
GIN( image_array _int4_sml_ops ) ;
- CREATE INDEX image_array_gist ON images USING
GIST ( image_array _int4_sml_ops ) ;

Line 1

:
9.33
Line 1
5

t e s t=# SELECT count ( * ) from images ;


count
-- ------1000000
( 1 row )

t e s t=# EXPLAIN ANALYZE SELECT count ( * ) FROM images WHERE


images . image_array %
{1010259 ,1011253 ,... ,2423253 ,2424252} : : int [ ] ;

10
-

Bitmap Heap Scan on images ( c o s t = 2 8 6 . 6 4 . . 3 9 6 9 . 4 5 rows =986


width =4) ( a c t u a l time = 5 0 4 . 3 1 2 . . 2 0 4 7 . 5 3 3 rows =200000
l o o p s =1)
Recheck Cond : ( image_array %
{1010259 ,1011253 ,... ,2423253 ,2424252} : : integer [ ] )
-> Bitmap Index Scan on image_array_gist
( c o s t = 0 . 0 0 . . 2 8 6 . 3 9 rows =986 width =0) ( a c t u a l
time = 4 4 6 . 1 0 9 . . 4 4 6 . 1 0 9 rows =200000 l o o p s =1)
Index Cond : ( image_array %
{1010259 ,1011253 ,... ,2423253 ,2424252} : : integer [ ] )
T o t a l runtime : 2 1 5 2 . 4 1 1 ms
( 5 rows )

1010259,...,2424252::int[] ,
. smlar.threshold
% (
).

:
161

9.7. PostPic
9.34
Line 1

t e s t=# EXPLAIN ANALYZE SELECT s m l a r ( images . image_array ,


{ 1 0 1 0 2 5 9 , . . . , 2 4 2 4 2 5 2 } : : i n t [ ] ) a s s i m i l a r i t y FROM
images WHERE images . image_array % { 1 0 1 0 2 5 9 , 1 0 1 1 2 5 3 ,
. . . , 2 4 2 3 2 5 3 , 2 4 2 4 2 5 2 } : : i n t [ ] ORDER BY s i m i l a r i t y DESC;

5
-

10
-

S o r t ( c o s t = 4 0 2 0 . 9 4 . . 4 0 2 3 . 4 1 rows =986 width =924) ( a c t u a l


time = 2 8 8 8 . 4 7 2 . . 2 9 0 1 . 9 7 7 rows =200000 l o o p s =1)
S o r t Key : ( s m l a r ( image_array ,
{... ,2424252} : : integer [ ] ) )
S o r t Method : q u i c k s o r t Memory : 15520kB
-> Bitmap Heap Scan on images ( c o s t = 2 8 6 . 6 4 . . 3 9 7 1 . 9 1
rows =986 width =924) ( a c t u a l time = 4 7 4 . 4 3 6 . . 2 7 2 9 . 6 3 8
rows =200000 l o o p s =1)
Recheck Cond : ( image_array %
{... ,2424252} : : integer [ ] )
-> Bitmap Index Scan on image_array_gist
( c o s t = 0 . 0 0 . . 2 8 6 . 3 9 rows =986 width =0) ( a c t u a l
time = 4 2 1 . 1 4 0 . . 4 2 1 . 1 4 0 rows =200000 l o o p s =1)
Index Cond : ( image_array %
{... ,2424252} : : integer [ ] )
T o t a l runtime : 2 9 1 2 . 2 0 7 ms
( 8 rows )

1 . P.S.
PostgreSQL ,
, (
).

Smlar ,
, , , , ,
, , .

9.7 PostPic
: Open Source
: github.com/drotiro/postpic
PostPic PostgreSQL,
, PostGIS
. image,
(,
, ..) (, ,
).
162

9.8. Fuzzystrmatch

9.8 Fuzzystrmatch
: Open Source
Fuzzystrmatch
. soundex

. difference soundex
, . soundex
,
: 0 , 4 ( ,
similarity):
9.35 soundex
Line 1
5
-

# CREATE EXTENSION f u z z y s t r m a t c h ;
CREATE EXTENSION
# SELECT soundex ( h e l l o world ! ) ;
soundex
-- ------H464
( 1 row )

10
-

# SELECT soundex ( Anne ) , soundex ( Ann ) ,


d i f f e r e n c e ( Anne , Ann ) ;
soundex | soundex | d i f f e r e n c e
- - - - - - - - -+ - - - - - - - - -+ - - - - - - - - - - - A500
| A500
|
4
( 1 row )

15
-

# SELECT soundex ( Anne ) , soundex ( Andrew ) ,


d i f f e r e n c e ( Anne , Andrew ) ;
soundex | soundex | d i f f e r e n c e
- - - - - - - - -+ - - - - - - - - -+ - - - - - - - - - - - A500
| A536
|
2
( 1 row )

20
25

# SELECT soundex ( Anne ) , soundex ( Margaret ) ,


d i f f e r e n c e ( Anne , Margaret ) ;
soundex | soundex | d i f f e r e n c e
- - - - - - - - -+ - - - - - - - - -+ - - - - - - - - - - - A500
| M626
|
0
( 1 row )

# CREATE TABLE s (nm t e x t ) ;


CREATE TABLE
- # INSERT INTO s VALUES ( john ) , ( j o a n ) , ( wobbly ) ,
( jack ) ;
30 INSERT 0 4
-

163

9.8. Fuzzystrmatch
35
-

# SELECT * FROM s WHERE soundex (nm) = soundex ( john ) ;


nm
-- ---john
joan
( 2 rows )

40
-

# SELECT * FROM s WHERE d i f f e r e n c e ( s . nm, john ) > 2 ;


nm
-- ---john
joan
jack
( 3 rows )

levenshtein 1
. levenshtein_less_equal levenshtein
:
9.36 levenshtein
Line 1
5

# SELECT l e v e n s h t e i n ( GUMBO , GAMBOL ) ;


levenshtein
-- ----------2
( 1 row )

# SELECT l e v e n s h t e i n ( GUMBO , GAMBOL , 2 , 1 , 1 ) ;


levenshtein
- -- ----------10
3
- ( 1 row )
-

15
-

# SELECT l e v e n s h t e i n _ l e s s _ e q u a l ( e x t e n s i v e , e x h a u s t i v e ,
2) ;
levenshtein_less_equal
-- ---------------------3
( 1 row )

20
-

t e s t=# SELECT l e v e n s h t e i n _ l e s s _ e q u a l ( e x t e n s i v e ,
exhaustive , 4) ;
levenshtein_less_equal
-- ---------------------4
( 1 row )
1

http://en.wikipedia.org/wiki/Levenshtein_distance

164

9.9. Tsearch2
metaphone, soundex,
: , ,
.
metaphone . dmetaphone
:
9.37 metaphone
Line 1
5
10

# SELECT metaphone ( GUMBO , 4 ) ;


metaphone
-- --------KM
( 1 row )
# SELECT dmetaphone ( p o s t g r e s q l ) ;
dmetaphone
-- ---------PSTK
( 1 row )

# SELECT dmetaphone_alt ( p o s t g r e s q l ) ;
dmetaphone_alt
- -- -------------15
PSTK
- ( 1 row )
-

9.9 Tsearch2
: Open Source
Tsearch2 .
PostgreSQL 8.3.

9.10 OpenFTS
: Open Source
: openfts.sourceforge.net
OpenFTS (Open Source Full Text Search engine)
PostgreSQL ,

.
,
.

9.11 PL/Proxy
: Open Source
165

9.12. Texcaller
: pgfoundry.org/projects/plproxy
PL/Proxy -
.
5.2 .

9.12 Texcaller
: Open Source
: www.profv.de/texcaller
Texcaller TeX,
. C,
, , TeX.
TeX NULL,
. , ,
NOTICEs.

9.13 Pgmemcache
: Open Source
: pgfoundry.org/projects/pgmemcache
Pgmemcache PostgreSQL API libmemcached
memcached.

PostgreSQL , ,
memcached. 8.2 .

9.14 Prefix
: Open Source
: pgfoundry.org/projects/prefix
Prefix (prefix @> text). Prefix
,
/
.

9.15 Dblink
: Open Source
Dblink ,
SQL,
.

166

9.16. Ltree

9.16 Ltree
: Open Source
Ltree ,
,
. Materialized Path (
, ).

9.17
PostgreSQL
. PostgreSQL
, , ,
.

167

10
PostgreSQL

,
, ,


-

, .

10.1
.
, ,
, - .
PostgreSQL .
! , ,
, PostgreSQL
.
, ,
(DELETE, DROP),
( ).

PostgreSQL:
SQL ;
;
;
.

168

10.2. SQL

10.2 SQL
SQL.

, . PostgreSQL
pg_dump.
pg_dump:
10.1 pg_dump
Line 1

pg_dump dbname > o u t f i l e

:
10.2
Line 1

p s q l dbname < i n f i l e

dbname
. ,
, (
, ).
,
, :
10.3
Line 1

p s q l - - s e t ON_ERROR_STOP=on dbname < i n f i l e

,
:
10.4
Line 1

pg_dump - h h o s t 1 dbname | p s q l - h h o s t 2 dbname

ANALYZE,
.
, , ,
?
PostgreSQL pg_dumpall. pg_dumpall
PostgreSQL:
10.5 PostgreSQL
Line 1

pg_dumpall > o u t f i l e

10.6 PostgreSQL
Line 1

psql - f i n f i l e postgres

169

10.2. SQL

SQL

,
pg_dump.
, pg_dump
.
Unix, .
:
.
, GZIP:
10.7 PostgreSQL
Line 1

pg_dump dbname | g z i p > f i l e n a m e . gz

:
10.8 PostgreSQL
Line 1

g u n z i p - c f i l e n a m e . gz | p s q l dbname

10.9 PostgreSQL
Line 1

c a t f i l e n a m e . gz | g u n z i p | p s q l dbname

split.
split
,
. , 1 :
10.10 PostgreSQL
Line 1

pg_dump dbname | s p l i t - b 1m - f i l e n a m e

:
10.11 PostgreSQL
Line 1

c a t f i l e n a m e * | p s q l dbname

pg_dump
PostgreSQL Zlib,
.
GZIP,
:
10.12 PostgreSQL
Line 1

pg_dump - Fc dbname > f i l e n a m e

170

10.3.
psql ,
pg_restore:
10.13 PostgreSQL
Line 1

p g _ r e s t o r e - d dbname f i l e n a m e

, split
.

10.3

, PostgreSQL
. :
10.14 PostgreSQL
Line 1

t a r - c f backup . t a r / u s r / l o c a l / p g s q l / data

,

, , , SQL :
PostgreSQL , ,
(PostgreSQL
, ).
,
PostgreSQL.

.
, (snapshot)
( PostgreSQL). PostgreSQL
. , , ,
,
.

PostgreSQL , ,
WAL.
, ( WAL
). , PostgreSQL
,

(!!!).
,
.
rsync.
rsync PostgreSQL (PostgreSQL
171

10.4.
). PostgreSQL
rsync. rsync ,
,
,
.
.

10.4
PostgreSQL (Write Ahead
Log, WAL) pg_xlog ,
.
.
PostgreSQL: ,
.
,
:
WAL .
,
,
WAL .
,
, :
.

( ,
).

.
WAL
, ,
PostgreSQL
( ).
,
. ,
WAL .

.
WAL
pg_xlog. postgresql.conf:

172

10.5.
10.15
archive_mode = on # e n a b l e a r c h i v i n g
- archive_command = cp - v %p / data / p g s q l / a r c h i v e s/% f
- a r c h i v e _ t i m e o u t = 300 # t i m e o u t t o c l o s e b u f f e r s

Line 1

( )
. rsync.
, ,
.
10.16 WAL
r s y n c - avz - - d e l e t e prod1 : / data / p g s q l / a r c h i v e s / \
- / data / p g s q l / a r c h i v e s / > / dev / n u l l

Line 1

, pg_xlog
PostgreSQL ( ).
PostgreSQL recovery.conf
:
10.17 recovery.conf
Line 1

restore_command = cp / data / p g s q l / a r c h i v e s/% f "%p"

PostgreSQL
, (,
,
).
http://www.postgresql.org/docs/9.0/static/continuous-archiving.html.

10.5


.
, (
,
). ,
( S31 ).

,

.
1

http://aws.amazon.com/s3/

173

10.5.

WAL-E
: github.com/heroku/WAL-E
WAL-E PostgreSQL
WAL-logs S3 pg_start_backup
pg_stop_backup. Python
Heroku, .

WAL-E : lzop, psql, mbuffer, python 2.6+


python (gevent >= 0.13, boto >= 2.0).
daemontools.
Ubuntu :
10.18 WAL-E
Line 1
-

# PostgreSQL
a p t i t u d e i n s t a l l g i t - c o r e python - dev python - s e t u p t o o l s
b u i l d - e s s e n t i a l l i b e v e n t - dev l z o p m b u f f e r daemontools
daemontools - run

WAL-E:
10.19 WAL-E
g i t c l o n e g i t : / / g i t h u b . com/ heroku /WAL-E . g i t
cd WAL-E
- python s e t u p . py b u i l d
- sudo python s e t u p . py i n s t a l l

Line 1
-

WAL-E.

, WAL-E AWS S3,
Access Key ID Secret Access Key (
Amazon AWS).
S3:
10.20 S3
AWS_SECRET_ACCESS_KEY= . . . wal - e
\
- k AWS_ACCESS_KEY_ID
\
- - s3 - p r e f i x=s 3 : / / some - bucket / d i r e c t o r y / o r / whatever \
backup - push / var / l i b / p o s t g r e s q l / 9 . 2 / main

Line 1
-

s3-prefix , S3 (bucket)
, .
WAL- S3:

174

10.5.
10.21 WAL- S3
AWS_SECRET_ACCESS_KEY= . . . wal - e
\
- k AWS_ACCESS_KEY_ID
\
- - s3 - p r e f i x=s 3 : / / some - bucket / d i r e c t o r y / o r / whatever \
wal - push
/ var / l i b / p o s t g r e s q l / 9 . 2 / main/ pg_xlog /WAL_SEGMENT_LONG_HEX

Line 1
-


envdir ( daemontools). envdir
:
10.22 WAL-E envdir
$
$
- $
- $

mkdir - p / e t c / wal - e . d/ env


echo " s e c r e t - key " > / e t c / wal - e . d/ env /AWS_SECRET_ACCESS_KEY
echo " a c c e s s - key " > / e t c / wal - e . d/ env /AWS_ACCESS_KEY_ID
echo s 3 : / / some - bucket / d i r e c t o r y / o r / whatever >
/ e t c / wal - e . d/ env /WALE_S3_PREFIX
5 $ chown -R r o o t : p o s t g r e s / e t c / wal - e . d

Line 1
-


WAL-E
:
10.23 WAL-E envdir
Line 1
-

$ e n v d i r / e t c / wal - e . d/ env wal - e backup - push . . .


$ e n v d i r / e t c / wal - e . d/ env wal - e wal - push . . .

PostgreSQL WAL- S3 c
WAL-E. postgresql.conf:
10.24 PostgreSQL
w a l _ l e v e l = hot_standby # a r c h i v e , PostgreSQL < 9 . 0
- archive_mode = on
- archive_command = e n v d i r / e t c / wal - e . d/ env
/ u s r / l o c a l / b i n / wal - e wal - push %p
- a r c h i v e _ t i m e o u t = 60

Line 1

WAL-E ( which
wal-e), PostgreSQL .
PostgreSQL. :
10.25 PostgreSQL
Line 1
-

2 0 1 2 - 1 1 - 0 7 1 4 : 5 2 : 1 9 UTC LOG: d a t a b a s e system was s h u t down


a t 2 0 1 2 - 1 1 - 0 7 1 4 : 5 1 : 4 0 UTC
2 0 1 2 - 1 1 - 0 7 1 4 : 5 2 : 1 9 UTC LOG: d a t a b a s e system i s ready t o
accept connections
2 0 1 2 - 1 1 - 0 7 1 4 : 5 2 : 1 9 UTC LOG: autovacuum l a u n c h e r s t a r t e d
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 1 9 . 7 8 4 + 0 0 p i d =7653 wal_e . worker . s3_worker
INFO
MSG: b e g i n a r c h i v i n g a f i l e

175

10.5.
5

10

15
-

DETAIL : Uploading
" pg_xlog /000000010000000000000001 " t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 . l z o " .
2 0 1 2 - 1 1 - 0 7 1 4 : 5 2 : 1 9 UTC LOG: i n c o m p l e t e s t a r t u p p a c k e t
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 2 8 . 2 3 4 + 0 0 p i d =7653 wal_e . worker . s3_worker
INFO
MSG: completed a r c h i v i n g t o a f i l e
DETAIL : A r c h i v i n g t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 . l z o "
c o m p l e t e a t 2 1 5 8 3 . 3 KiB/ s .
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 2 8 . 3 4 1 + 0 0 p i d =7697 wal_e . worker . s3_worker
INFO
MSG: b e g i n a r c h i v i n g a f i l e
DETAIL : Uploading
" pg_xlog / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 . 0 0 0 0 0 0 2 0 . backup " t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 . 0 0 0 0 0 0 2 0 . b
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 3 4 . 0 2 7 + 0 0 p i d =7697 wal_e . worker . s3_worker
INFO
MSG: completed a r c h i v i n g t o a f i l e
DETAIL : A r c h i v i n g t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 . 0 0 0 0 0 0 2 0 . b
c o m p l e t e a t 00KiB/ s .
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 3 4 . 1 8 7 + 0 0 p i d =7711 wal_e . worker . s3_worker
INFO
MSG: b e g i n a r c h i v i n g a f i l e
DETAIL : Uploading
" pg_xlog /000000010000000000000002 " t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 . l z o " .
2 0 1 2 - 1 1 - 0 7 T14 : 5 2 : 4 0 . 2 3 2 + 0 0 p i d =7711 wal_e . worker . s3_worker
INFO
MSG: completed a r c h i v i n g t o a f i l e
DETAIL : A r c h i v i n g t o
" s 3 : / / c l e v e r d b - pg - backups /pg/ wal_005 / 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 . l z o "
c o m p l e t e a t 2 4 6 6 . 6 7 KiB/ s .

,
.
,
:
10.26 S3
Line 1
-

$ e n v d i r / e t c / wal - e . d/ env wal - e backup - push


/ var / l i b / p o s t g r e s q l / 9 . 2 / main
2 0 1 2 - 1 1 - 0 7 T14 : 4 9 : 2 6 . 1 7 4 + 0 0 p i d =7493
wal_e . o p e r a t o r . s 3 _ o p e r a t o r INFO
MSG: s t a r t upload
p o s t g r e s v e r s i o n metadata
DETAIL : Uploading t o
s 3 : / / c l e v e r d b - pg - backups /pg/ basebackups_005 / base_000000010000000000000006
2 0 1 2 - 1 1 - 0 7 T14 : 4 9 : 3 2 . 7 8 3 + 0 0 p i d =7493
wal_e . o p e r a t o r . s 3 _ o p e r a t o r INFO
MSG: p o s t g r e s
v e r s i o n metadata upload c o m p l e t e
2 0 1 2 - 1 1 - 0 7 T14 : 4 9 : 3 2 . 8 5 9 + 0 0 p i d =7493 wal_e . worker . s3_worker
INFO
MSG: b e g i n n i n g volume c o m p r e s s i o n

176

10.5.
DETAIL : B u i l d i n g volume 0 .
...
- HINT :
Check t h a t your archive_command i s e x e c u t i n g
p r o p e r l y . pg_stop_backup can be c a n c e l e d s a f e l y , but
t h e d a t a b a s e backup w i l l not be u s a b l e w i t h o u t a l l t h e
WAL segments .
- NOTICE :
pg_stop_backup complete , a l l r e q u i r e d WAL segments
have been a r c h i v e d
-

. 10.1: S3

. 10.2: S3
(,
crontab). 10.1-10.3 S3.
lzop1 .

gzip, ( 25/
5% ).
mbuffer ( cluster-read-rate-limit
, ).
.
backup-fetch :
10.27 S3
1

http://en.wikipedia.org/wiki/Lzop

177

10.5.

. 10.3: WAL- S3
Line 1

$ sudo - u p o s t g r e s bash - c " e n v d i r / e t c / wal - e . d/ env wal - e


- - s3 - p r e f i x=s 3 : / / some - bucket / d i r e c t o r y / o r / whatever
backup - f e t c h / var / l i b / p o s t g r e s q l / 9 . 2 / main LATEST"

LATEST
(PostgreSQL ).

:
10.28
Line 1

$ sudo - u p o s t g r e s bash - c " e n v d i r / e t c / wal - e . d/ env wal - e


- - s3 - p r e f i x=s 3 : / / some - bucket / d i r e c t o r y / o r / whatever
backup - f e t c h / var / l i b / p o s t g r e s q l / 9 . 2 / main
base_LONGWALNUMBER_POSITION_NUMBER"


backup-list:
10.29
$ e n v d i r / e t c / wal - e . d/ env wal - e backup - l i s t
name
last_modified
expanded_size_bytes
wal_segment_backup_start
wal_segment_offset_backup_start wal_segment_backup_stop
wal_segment_offset_backup_stop
- base_000000010000000000000008_00000032
2 0 1 2 - 1 1 - 0 7 T14 : 0 0 : 0 7 . 0 0 0 Z
000000010000000000000008
00000032
- base_00000001000000000000000C_00000032
2 0 1 2 - 1 1 - 0 8 T15 : 0 0 : 0 8 . 0 0 0 Z
00000001000000000000000C
00000032

Line 1
-

178

10.5.

WAL- (
). recovery.conf:
10.30 recovery.conf
Line 1

restore_command = e n v d i r / e t c / wal - e . d/ env


/ u s r / l o c a l / b i n / wal - e wal - f e t c h "% f " "%p"

PostgreSQL.
.
( )
delete:
10.31
#
base_00000004000002DF000000A6_03626144
- $ e n v d i r / e t c / wal - e . d/ env wal - e d e l e t e - - c o n f i r m b e f o r e
base_00000004000002DF000000A6_03626144
- #
- $ e n v d i r / e t c / wal - e . d/ env wal - e d e l e t e - - c o n f i r m e v e r y t h i n g

Line 1

confirm ,
, (dry run).

WAL-E PostgreSQL
Amazon S3.

Barman
: www.pgbarman.org
Barman, WAL-E,
PostgreSQL
. Barman ,
,
PostgreSQL .

PostgreSQL (
) pghost.
(
brhost).
SSH ( , ).
authorized_keys .

179

10.5.
10.32 SSH
# PostgreSQL ( p g h o s t )
- s s h barman@brhost
- # ( b r h o s t )
- ssh postgres@pghost

Line 1

barman. barman
python : python 2.6+, rsync >= 3.0.4
python (argh, psycopg2, python-dateutil < 2.0 ( python 3.0
), distribute). Ubuntu
:
10.33 barman
Line 1

a p t i t u d e i n s t a l l python - dev python - argh python - psycopg2


python - d a t e u t i l r s y n c python - s e t u p t o o l s

barman:
10.34 barman
#
- #
- #
- #

Line 1

t a r - x z f barman - 1 . 1 . 1 . t a r . gz
cd barman - 1 . 1 . 1 /
python s e t u p . py b u i l d
sudo python s e t u p . py i n s t a l l

PostgreSQL. , barman
,
PostgreSQL:
10.35 postgresql.conf
Line 1

listen_adress = *
10.36 pg_hba.conf

Line 1

host

all

all

b r h o s t /32

trust

PostgreSQL.
PostgreSQL:
10.37
Line 1
5

$ p s q l - c SELECT v e r s i o n ( ) -U p o s t g r e s - h p g h o s t
version
-- ------------------------------------------------------------------------PostgreSQL 9 . 2 . 1 on x86_64 - unknown - l i n u x - gnu , c o m p i l e d by
g c c ( Ubuntu/ L i n a r o 4 . 7 . 2 - 2 ubuntu1 ) 4 . 7 . 2 , 64 - b i t
( 1 row )

180

10.5.
10.38
# sudo mkdir - p / s r v /barman
- # sudo chown barman : barman / s r v /barman

Line 1

barman /etc/barman.conf:
10.39 barman.conf
[ barman ]
; Main d i r e c t o r y
- barman_home = / s r v /barman

Line 1
5
-

; Log l o c a t i o n
l o g _ f i l e = / var / l o g /barman/barman . l o g

; D e f a u l t c o m p r e s s i o n l e v e l : p o s s i b l e v a l u e s a r e None
( d e f a u l t ) , bzip2 , g z i p o r custom
compression = gzip

10
-

; main PostgreSQL S e r v e r c o n f i g u r a t i o n
[ main ]
; Human r e a d a b l e d e s c r i p t i o n
d e s c r i p t i o n = "Main PostgreSQL Database "

15

; SSH o p t i o n s
- ssh_command = s s h p o s t g r e s @ p g h o s t
20

; PostgreSQL c o n n e c t i o n s t r i n g
c o n n i n f o = h o s t=p g h o s t u s e r=p o s t g r e s

main ( barman PostgreSQL )


PostgreSQL .
:
10.40 barman
Line 1
5
10
-

# barman show - s e r v e r main


S e r v e r main :
active : true
d e s c r i p t i o n : Main PostgreSQL Database
ssh_command : s s h p o s t g r e s @ p g h o s t
c o n n i n f o : h o s t=p g h o s t u s e r=p o s t g r e s
b a c k u p _ d i r e c t o r y : / s r v /barman/main
b a s e b a c k u p s _ d i r e c t o r y : / s r v /barman/main/ b a s e
w a l s _ d i r e c t o r y : / s r v /barman/main/ w a l s
i n c o m i n g _ w a l s _ d i r e c t o r y : / s r v /barman/main/ incoming
l o c k _ f i l e : / s r v /barman/main/main . l o c k
compression : gzip
c u s t o m _ c o m p r e s s i o n _ f i l t e r : None
c u s t o m _ d e c o m p r e s s i o n _ f i l t e r : None

181

10.5.
15
20
25
-

r e t e n t i o n _ p o l i c y : None
w a l _ r e t e n t i o n _ p o l i c y : None
pre_backup_script : None
post_backup_script : None
c u r r e n t _ x l o g : None
last_shipped_wal : None
archive_command : None
server_txt_version : 9 . 2 . 1
d a t a _ d i r e c t o r y : / var / l i b / p o s t g r e s q l / 9 . 2 / main
archive_mode : o f f
config_file :
/ e t c / p o s t g r e s q l / 9 . 2 / main/ p o s t g r e s q l . c o n f
h b a _ f i l e : / e t c / p o s t g r e s q l / 9 . 2 / main/pg_hba . c o n f
i d e n t _ f i l e : / e t c / p o s t g r e s q l / 9 . 2 / main/ pg_ident . c o n f

30
35
-

# barman check main


S e r v e r main :
s s h : OK
PostgreSQL : OK
archive_mode : FAILED ( p l e a s e s e t i t t o on )
archive_command : FAILED ( p l e a s e s e t i t a c c o r d i n g l y
t o documentation )
d i r e c t o r i e s : OK
c o m p r e s s i o n s e t t i n g s : OK

, PostgreSQL .
PostgreSQL :
10.41 PostgreSQL
w a l _ l e v e l = hot_standby # a r c h i v e PostgreSQL < 9 . 0
archive_mode = on
- archive_command = r s y n c - a %p
barman@brhost :INCOMING_WALS_DIRECTORY/% f

Line 1
-

INCOMING_WALS_DIRECTORY
WAL-.

barman show-server main( 10.40,
/srv/barman/main/incoming).

PostgreSQL. :
10.42
Line 1
5
-

# barman check main


S e r v e r main :
s s h : OK
PostgreSQL : OK
archive_mode : OK
archive_command : OK
d i r e c t o r i e s : OK

182

10.5.
c o m p r e s s i o n s e t t i n g s : OK

.
, barman.conf .

:
10.43
Line 1
-

# barman l i s t - s e r v e r
main - Main PostgreSQL Database

PostgreSQL (
):
10.44
Line 1
5
-

# barman backup main


S t a r t i n g backup f o r s e r v e r main i n
/ s r v /barman/main/ b a s e /20121109 T090806
Backup s t a r t a t x l o g l o c a t i o n : 0/3000020
(000000010000000000000003 , 00000020)
Copying f i l e s .
Copy done .
Asking PostgreSQL s e r v e r t o f i n a l i z e t h e backup .
Backup end a t x l o g l o c a t i o n : 0/30000D8
( 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 , 000000D8)
Backup completed

( cron).
:
10.45
# barman l i s t - backup main
main 20121110 T091608 - F r i Nov 10 0 9 : 2 0 : 5 8 2012 - S i z e : 1 . 0
GiB - WAL S i z e : 4 4 6 . 0 KiB
- main 20121109 T090806 - F r i Nov
9 0 9 : 0 8 : 1 0 2012 - S i z e :
2 3 . 0 MiB - WAL S i z e : 4 7 7 . 0 MiB

Line 1
-

:
10.46
Line 1
5
-

# barman show - backup main 20121110 T091608


Backup 20121109 T091608 :
S e r v e r Name
: main
Status :
: DONE
PostgreSQL V e r s i o n : 90201
PGDATA d i r e c t o r y
: / var / l i b / p o s t g r e s q l / 9 . 2 / main

183

10.5.
10
15
-

Base backup i n f o r m a t i o n :
Disk u s a g e
: 1 . 0 GiB
Timeline
: 1
Begin WAL
: 00000001000000000000008C
End WAL
: 000000010000000000000092
WAL number
: 7
Begin time
: 2012 -11 -10 0 9 : 1 6 : 0 8 . 8 5 6 8 8 4
End time
: 2012 -11 -10 0 9 : 2 0 : 5 8 . 4 7 8 5 3 1
Begin O f f s e t
: 32
End O f f s e t
: 3576096
Begin XLOG
: 0/8 C000020
End XLOG
: 0/92369120

20
-

WAL i n f o r m a t i o n :
No o f f i l e s
Disk u s a g e
Last a v a i l a b l e

: 1
: 4 4 6 . 0 KiB
: 000000010000000000000093

25
-

Catalog information :
P r e v i o u s Backup : 20121109 T090806
Next Backup
: - ( t h i s i s t h e l a t e s t b a s e backup )

WAL-,
cron:
10.47 WAL-
Line 1
5
-

# barman c r o n
P r o c e s s i n g x l o g segments f o r main
000000010000000000000001
000000010000000000000002
000000010000000000000003
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 . 0 0 0 0 0 0 2 0 . backup
000000010000000000000004
000000010000000000000005
000000010000000000000006

cron.
, WAL- (
- ). Barman
WAL- gzip, bzip2 (
custom_compression_filter
custom_decompression_filter ).
recover:
10.48
Line 1

# barman r e c o v e r - - remote - ssh - command " s s h p o s t g r e s @ p g h o s t "


main 20121109 T090806 / var / l i b / p o s t g r e s q l / 9 . 2 / main

184

10.5.
5
-

S t a r t i n g remote r e s t o r e f o r s e r v e r main u s i n g backup


20121109 T090806
D e s t i n a t i o n d i r e c t o r y : / var / l i b / p o s t g r e s q l / 9 . 2 / main
Copying t h e b a s e backup .
Copying r e q u i r e d wal segments .
The archive_command was s e t t o f a l s e t o p r e v e n t data
losses .

Your PostgreSQL s e r v e r has been s u c c e s s f u l l y p r e p a r e d f o r


recovery !

10
-

P l e a s e r e v i e w network and a r c h i v e r e l a t e d s e t t i n g s i n t h e
PostgreSQL
c o n f i g u r a t i o n f i l e b e f o r e s t a r t i n g the j u s t recovered
instance .

WARNING: B e f o r e s t a r t i n g up t h e r e c o v e r e d PostgreSQL s e r v e r ,
p l e a s e review a l s o the s e t t i n g s of the f o l l o w i n g
configuration
15 o p t i o n s a s they might i n t e r f e r e with your c u r r e n t r e c o v e r y
attempt :
-

20

d a t a _ d i r e c t o r y = / var / l i b / p o s t g r e s q l / 9 . 1 / main
# u s e data i n a n o t h e r d i r e c t o r y
external_pid_file =
/ var / run / p o s t g r e s q l / 9 . 1 - main . p i d
# write
an e x t r a PID f i l e
h b a _ f i l e = / e t c / p o s t g r e s q l / 9 . 1 / main/pg_hba . c o n f
#
host - based a u t h e n t i c a t i o n f i l e
ident_file =
/ e t c / p o s t g r e s q l / 9 . 1 / main/ pg_ident . c o n f
# ident
configuration f i l e

Barman
SSH ( remote-ssh-command).
barman PITR1 :
target-time ( ) target-xid
(id ).

Barman
PostgreSQL . ,
PostgreSQL .
1

http://en.wikipedia.org/wiki/Point-in-time_recovery

185

10.6.

10.6
, ,
, .
,
PostgreSQL (,
).

186

11

PostgreSQL
,
(),

.
-
,
, .

11.1
,

. -
( ,
).

, SQL ,
, PostgreSQL ,
.
, .
: ?

( ..),
, ,
. , ,
.

187

11.2.


, - - ,
. ,
:
;
;
,
, ( Twitter Facebook
).
, .

11.2
,
, .
, ..
,
, .


PgPool-II v.3 + PostgreSQL v.9 Streaming Replication
,
. :



(failover)


(
pg_current_xlog_location pg_last_xlog_receive_location).
pgpool-II

PgPool-II v.3 + PostgreSQL Slony
, Slony. :
(failover)


.
pgpool-II
Postgresql 9
Postgres-XC 5.3 .
188

11.3.

11.3
,
( Google Analytics).
(
).



.
PgQ , PostgreSQL.
Skype.
Londiste
( 4.4). :

PostgreSQL
,

PgQ ,
,
(batches)
API SQL

RabbitMQ.
RabbitMQ ,
(Message Oriented
Middleware) AMQP (Advanced Message Queuing
Protocol).
RabbitMQ Mozilla Public License.
RabbitMQ Open Telecom Platform,

Erlang.
Postgres-XC 5.3 .

11.4

PostgreSQL.
, .
,
/
.

189

12

(Performance Snippets)



.
. -
.
, ,
.
(House M.D.),
1 1

12.1
PostgreSQL,
,
( ).
,
PostgreSQL.
,
PostgreSQL ,
.

12.2

(
).

190

12.2.
snippets/biggest_relations.sql
Line 1
5
-

SELECT nspname | | . | | relname AS " r e l a t i o n " ,


p g _ s i z e _ p r e t t y ( p g _ r e l a t i o n _ s i z e (C . o i d ) ) AS " s i z e "
FROM p g _ c l a s s C
LEFT JOIN pg_namespace N ON (N. o i d = C . r e l n a m e s p a c e )
WHERE nspname NOT IN ( pg_catalog , information_schema )
ORDER BY p g _ r e l a t i o n _ s i z e (C. o i d ) DESC
LIMIT 2 0 ;

:
12.1 .
Line 1
5
-

relation
|
size
- - - - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - public . accounts
| 326 MB
p u b l i c . accounts_pkey
| 44 MB
public . history
| 592 kB
public . tellers_pkey
| 16 kB
p u b l i c . branches_pkey
| 16 kB
public . t e l l e r s
| 16 kB
public . branches
| 8192 b y t e s



.
snippets/biggest_tables.sql
Line 1
5
-

SELECT nspname | | . | | relname AS " r e l a t i o n " ,


p g _ s i z e _ p r e t t y ( p g _ t o t a l _ r e l a t i o n _ s i z e (C . o i d ) ) AS
" total_size "
FROM p g _ c l a s s C
LEFT JOIN pg_namespace N ON (N. o i d = C . r e l n a m e s p a c e )
WHERE nspname NOT IN ( pg_catalog , information_schema )
AND C. r e l k i n d <> i
AND nspname ! ~ ^pg_toast
ORDER BY p g _ t o t a l _ r e l a t i o n _ s i z e (C. o i d ) DESC
LIMIT 2 0 ;

:
12.2 .
Line 1
5

relation
| total_size
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+ - - - - - - - - - - - public . actions
| 4249 MB
p u b l i c . p r o d u c t _ h i s t o r y _ r e c o r d s | 197 MB
p u b l i c . product_updates
| 52 MB

191

12.2.
-

p u b l i c . import_products
public . products
public . v i s i t s

| 34 MB
| 29 MB
| 25 MB

count

. ,
count.
snippets/count_estimate.sql
Line 1
5
10

CREATE LANGUAGE p l p g s q l ;
CREATE FUNCTION c ou n t _ es t i ma t e ( query t e x t ) RETURNS i n t e g e r
AS $$
DECLARE
rec
record ;
rows i n t e g e r ;
BEGIN
FOR r e c IN EXECUTE EXPLAIN | | query LOOP
rows := s u b s t r i n g ( r e c . "QUERY PLAN" FROM
rows = ( [ [ : d i g i t : ] ] + ) ) ;
EXIT WHEN rows IS NOT NULL;
END LOOP;

RETURN rows ;
END;
- $$ LANGUAGE p l p g s q l VOLATILE STRICT ;
-

:
12.3 count.
CREATE TABLE f o o ( r d o u b l e p r e c i s i o n ) ;
- INSERT INTO f o o SELECT random ( ) FROM g e n e r a t e _ s e r i e s ( 1 ,
1000) ;
- ANALYZE f o o ;

Line 1

5
-

# SELECT count ( * ) FROM f o o WHERE r < 0 . 1 ;


count
-- ----92
( 1 row )

10

# SELECT c o u nt _ e st i m a te ( SELECT * FROM f o o WHERE r < 0 . 1 ) ;


coun t _ es t i ma t e
- -- -------------94
15 ( 1 row )
-

192

12.2.

-
-
( DEFAULT).
snippets/default_value.sql
Line 1
5
-

CREATE OR REPLACE FUNCTION r e t _ d e f ( t e x t , t e x t , t e x t ) RETURNS


t e x t AS $$
SELECT
COLUMNS. column_default : : t e x t
FROM
information_schema .COLUMNS
WHERE table_name = $2
AND table_schema = $1
AND column_name = $3
$$ LANGUAGE s q l IMMUTABLE;

:
12.4 - .
Line 1

# SELECT r e t _ d e f ( schema , t a b l e , column ) ;

5
-

SELECT r e t _ d e f ( p u b l i c , i m a g e _ f i l e s , i d ) ;
ret_def
-- --------------------------------------nextval ( image_files_id_seq : : r e g c l a s s )
( 1 row )

SELECT r e t _ d e f ( p u b l i c , schema_migrations , v e r s i o n ) ;
ret_def
- -- -------

10
-

( 1 row )


(random)
( ).
snippets/random_from_range.sql
CREATE OR REPLACE FUNCTION random ( numeric , numeric )
RETURNS numeric AS
- $$
SELECT ( $1 + ( $2 - $1 ) * random ( ) ) : : numeric ;
5 $$ LANGUAGE s q l VOLATILE ;

Line 1
-

193

12.2.
12.5 .
Line 1
5

SELECT random ( 1 , 1 0 ) : : i n t , random ( 1 , 1 0 ) ;


random |
random
- - - - - - - -+ - - - - - - - - - - - - - - - - - 6 | 5.11675184825435
( 1 row )

10
-

SELECT random ( 1 , 1 0 ) : : i n t , random ( 1 , 1 0 ) ;


random |
random
- - - - - - - -+ - - - - - - - - - - - - - - - - - 7 | 1.37060070643201
( 1 row )


1
, .

, ,
, .
IBM
1960 .

,
.
SQL. ,
.
snippets/luhn_algorithm.sql
Line 1
5
10

CREATE OR REPLACE FUNCTION l u h n _ v e r i f y ( i n t 8 ) RETURNS


BOOLEAN AS $$
- - Take t h e sum o f t h e
- - doubled d i g i t s and t h e even - numbered undoubled d i g i t s ,
and s e e i f
- - t h e sum i s e v e n l y d i v i s i b l e by z e r o .
SELECT
- - Doubled d i g i t s might i n t u r n be two d i g i t s . In
that case ,
- - we must add each d i g i t i n d i v i d u a l l y r a t h e r than
adding t h e
- - doubled d i g i t v a l u e t o t h e sum . I e i f t h e
o r i g i n a l d i g i t was
- - 6 t h e doubled r e s u l t was 1 2 and we must add
1+2 t o t h e
- - sum r a t h e r than 1 2 .
1

http://en.wikipedia.org/wiki/Luhn_algorithm

194

12.2.
15
20
25

MOD(SUM( d o u b l e d _ d i g i t / INT8 10 + d o u b l e d _ d i g i t
% INT8 10 ) , 1 0 ) = 0
FROM
- - Double odd - numbered d i g i t s ( c o u n t i n g l e f t with
- - l e a s t s i g n i f i c a n t a s z e r o ) . I f t h e doubled d i g i t s end up
- - having v a l u e s
- - > 10 ( i e they r e two d i g i t s ) , add t h e i r d i g i t s t o g e t h e r .
(SELECT
- - E x t r a c t d i g i t n c o u n t i n g l e f t from l e a s t
significant
- - as zero
MOD( ( $1 : : i n t 8 / (10^ n ) : : i n t 8 ) , 1 0 : : i n t 8 )
- - Double odd - numbered d i g i t s
* (MOD( n , 2 ) + 1 )
AS d o u b l e d _ d i g i t
FROM g e n e r a t e _ s e r i e s ( 0 , CEIL (LOG( $1 ) ) : : INTEGER 1 ) AS n
) AS d o u b l e d _ d i g i t s ;

$$ LANGUAGE SQL
IMMUTABLE
- STRICT ;
-

30
35
40
45
-

COMMENT ON FUNCTION l u h n _ v e r i f y ( i n t 8 ) IS Return t r u e i f f


the l a s t d i g i t of the
i n p u t i s a c o r r e c t check d i g i t f o r t h e r e s t o f t h e i n p u t
a c c o r d i n g t o Luhn s
algorithm . ;
CREATE OR REPLACE FUNCTION l u h n _ g e n e r a t e _ c h e c k d i g i t ( i n t 8 )
RETURNS i n t 8 AS $$
SELECT
- - Add t h e d i g i t s , d o u b l i n g even - numbered d i g i t s
( counting l e f t
- - with l e a s t - s i g n i f i c a n t a s z e r o ) . S u b t r a c t t h e
remainder of
- - d i v i d i n g t h e sum by 10 from 1 0 , and t a k e t h e
remainder
- - o f d i v i d i n g t h a t by 10 i n t u r n .
( ( INT8 10 - SUM( d o u b l e d _ d i g i t / INT8 10 +
d o u b l e d _ d i g i t % INT8 10 ) %
INT8 10 ) % INT8 10 ) : : INT8
FROM (SELECT
- - E x t r a c t d i g i t n c o u n t i n g l e f t from l e a s t
significant\
- - as zero
MOD( ( $1 : : i n t 8 / (10^ n ) : : i n t 8 ) , 1 0 : : i n t 8 )
- - d o u b l e even - numbered d i g i t s

195

12.2.
50

* ( 2 - MOD( n , 2 ) )
AS d o u b l e d _ d i g i t
FROM g e n e r a t e _ s e r i e s ( 0 , CEIL (LOG( $1 ) ) : : INTEGER 1 ) AS n
) AS d o u b l e d _ d i g i t s ;

$$ LANGUAGE SQL
IMMUTABLE
- STRICT ;
-

55
60
-

COMMENT ON FUNCTION l u h n _ g e n e r a t e _ c h e c k d i g i t ( i n t 8 ) IS For


the input
value , g e n e r a t e a check d i g i t a c c o r d i n g t o Luhn s
algorithm ;
CREATE OR REPLACE FUNCTION l u h n _ g e n e r a t e ( i n t 8 ) RETURNS i n t 8
AS $$
SELECT 10 * $1 + l u h n _ g e n e r a t e _ c h e c k d i g i t ( $1 ) ;
$$ LANGUAGE SQL
IMMUTABLE
STRICT ;

65
70
-

COMMENT ON FUNCTION l u h n _ g e n e r a t e ( i n t 8 ) IS Append a check


d i g i t generated
a c c o r d i n g t o Luhn s a l g o r i t h m t o t h e i n p u t v a l u e . The
i n p u t v a l u e must be no
g r e a t e r than ( maxbigint / 1 0 ) . ;
CREATE OR REPLACE FUNCTION l u h n _ s t r i p ( i n t 8 ) RETURNS i n t 8 AS
$$
SELECT $1 / 1 0 ;
$$ LANGUAGE SQL
IMMUTABLE
STRICT ;

COMMENT ON FUNCTION l u h n _ s t r i p ( i n t 8 ) IS S t r i p t h e l e a s t
s i g n i f i c a n t d i g i t from
- t h e i n p u t v a l u e . I n t e n d e d f o r u s e when s t r i p p i n g t h e check
d i g i t from a number
75 i n c l u d i n g a Luhn s a l g o r i t h m check d i g i t . ;
-

:
12.6 .
Line 1
5

S e l e c t luhn_verify (49927398716) ;
luhn_verify
-- ----------t
( 1 row )

196

12.2.
10
-

S e l e c t luhn_verify (49927398714) ;
luhn_verify
-- ----------f
( 1 row )



IN.
, .
:
: (2,6,4,10,25,7,9)
.. 2 2 2 6 6 4 4
snippets/order_like_in.sql
SELECT f o o . * FROM f o o
- JOIN (SELECT i d . val , row_number ( ) o v e r ( ) FROM
(VALUES( 3 ) , ( 2 ) , ( 6 ) , ( 1 ) , ( 4 ) ) AS
- i d ( v a l ) ) AS i d
- ON ( f o o . c a t a l o g _ i d = i d . v a l ) ORDER BY row_number ;

Line 1

VALUES(3),(2),(6),(1),(4)
foo ,
foo.catalog_id (
foo.catalog_id IN(3,2,6,1,4))

,
, (. quine) (
),
.
snippets/quine.sql
Line 1
-

s e l e c t a | | from ( s e l e c t | | q u o t e _ l i t e r a l ( a ) | | b | | ,
| | q u o t e _ l i t e r a l ( b ) | | : : t e x t a s b ) a s q u i n e from
( s e l e c t s e l e c t a | | from ( s e l e c t | | q u o t e _ l i t e r a l ( a )
| | b | | , | | q u o t e _ l i t e r a l (b) | | : : text as b) as
quine : : text as a , : : text as a : : text as b) as quine ;

LIKE
web .
LIKE some%, some ,
197

12.2.
. , ( )
.
LIKE bla%
text_pattern_ops ( varchar_pattern_ops varchar).
snippets/speed_like.sql
Line 1
5
10
-

p r e f i x _ t e s t=# c r e a t e t a b l e t a g s (
p r e f i x _ t e s t (# t a g
t e x t primary key ,
p r e f i x _ t e s t (# name
t e x t not n u l l ,
p r e f i x _ t e s t (# shortname t e x t ,
p r e f i x _ t e s t (# s t a t u s
char d e f a u l t S ,
p r e f i x _ t e s t (#
p r e f i x _ t e s t (# check ( s t a t u s i n ( S , R ) )
p r e f i x _ t e s t (# ) ;
NOTICE : CREATE TABLE / PRIMARY KEY w i l l c r e a t e i m p l i c i t
i n d e x " tags_pkey " f o r t a b l e " t a g s "
CREATE TABLE
p r e f i x _ t e s t=# CREATE INDEX i_tag ON t a g s USING
b t r e e ( l o w e r ( t a g ) text_pattern_ops ) ;
CREATE INDEX

15
20
-

p r e f i x _ t e s t=# c r e a t e t a b l e i n v a l i d _ t a g s (
p r e f i x _ t e s t (# t a g
t e x t primary key ,
p r e f i x _ t e s t (# name
t e x t not n u l l ,
p r e f i x _ t e s t (# shortname t e x t ,
p r e f i x _ t e s t (# s t a t u s
char d e f a u l t S ,
p r e f i x _ t e s t (#
p r e f i x _ t e s t (# check ( s t a t u s i n ( S , R ) )
p r e f i x _ t e s t (# ) ;
NOTICE : CREATE TABLE / PRIMARY KEY w i l l c r e a t e i m p l i c i t
index " invalid_tags_pkey " f o r t a b l e " invalid_tags "
CREATE TABLE

25
30
-

p r e f i x _ t e s t=# s e l e c t count ( * ) from t a g s ;


count
-- ----11966
( 1 row )

35
-

p r e f i x _ t e s t=# s e l e c t count ( * ) from i n v a l i d _ t a g s ;


count
-- ----11966
( 1 row )

198

12.2.
40
45

# EXPLAIN ANALYZE s e l e c t * from i n v a l i d _ t a g s where


l o w e r ( t a g ) LIKE l o w e r ( 0146% ) ;
QUERY PLAN
-- ------------------------------------------------------------------------Seq Scan on i n v a l i d _ t a g s ( c o s t = 0 . 0 0 . . 2 6 5 . 4 9 rows=60
width =26) ( a c t u a l time = 0 . 3 5 9 . . 2 0 . 6 9 5 rows=1 l o o p s =1)
F i l t e r : ( l o w e r ( t a g ) ~~ 0146% : : t e x t )
T o t a l runtime : 2 0 . 8 0 3 ms
( 3 rows )

50
-

# EXPLAIN ANALYZE s e l e c t * from i n v a l i d _ t a g s where


l o w e r ( t a g ) LIKE l o w e r ( 0146% ) ;
QUERY PLAN
-- ------------------------------------------------------------------------Seq Scan on i n v a l i d _ t a g s ( c o s t = 0 . 0 0 . . 2 6 5 . 4 9 rows=60
width =26) ( a c t u a l time = 0 . 5 4 9 . . 1 9 . 5 0 3 rows=1 l o o p s =1)
F i l t e r : ( l o w e r ( t a g ) ~~ 0146% : : t e x t )
T o t a l runtime : 1 9 . 5 5 0 ms
( 3 rows )

55
60
-

# EXPLAIN ANALYZE s e l e c t * from t a g s where l o w e r ( t a g ) LIKE


l o w e r ( 0146% ) ;
QUERY
PLAN
-- ------------------------------------------------------------------------Bitmap Heap Scan on t a g s ( c o s t = 5 . 4 9 . . 9 7 . 7 5 rows =121
width =26) ( a c t u a l time = 0 . 0 5 4 . . 0 . 0 5 7 rows=1 l o o p s =1)
F i l t e r : ( l o w e r ( t a g ) ~~ 0146% : : t e x t )
-> Bitmap Index Scan on i_tag ( c o s t = 0 . 0 0 . . 5 . 4 6 rows =120
width =0) ( a c t u a l time = 0 . 0 3 2 . . 0 . 0 3 2 rows=1 l o o p s =1)
Index Cond : ( ( l o w e r ( t a g ) ~>=~ 0146 : : t e x t ) AND
( l o w e r ( t a g ) ~<~ 0147 : : t e x t ) )
T o t a l runtime : 0 . 1 1 9 ms
( 5 rows )

65
70
-

# EXPLAIN ANALYZE s e l e c t * from t a g s where l o w e r ( t a g ) LIKE


l o w e r ( 0146% ) ;
QUERY
PLAN
-- ------------------------------------------------------------------------Bitmap Heap Scan on t a g s ( c o s t = 5 . 4 9 . . 9 7 . 7 5 rows =121
width =26) ( a c t u a l time = 0 . 0 2 5 . . 0 . 0 2 5 rows=1 l o o p s =1)
F i l t e r : ( l o w e r ( t a g ) ~~ 0146% : : t e x t )
-> Bitmap Index Scan on i_tag ( c o s t = 0 . 0 0 . . 5 . 4 6 rows =120
width =0) ( a c t u a l time = 0 . 0 1 6 . . 0 . 0 1 6 rows=1 l o o p s =1)
Index Cond : ( ( l o w e r ( t a g ) ~>=~ 0146 : : t e x t ) AND
( l o w e r ( t a g ) ~<~ 0147 : : t e x t ) )

199

12.2.
-

T o t a l runtime : 0 . 0 5 0 ms
( 5 rows )


,
( , ).
snippets/duplicate_indexes.sql
Line 1
5

SELECT p g _ s i z e _ p r e t t y ( sum ( p g _ r e l a t i o n _ s i z e ( i d x ) ) : : b i g i n t )
AS s i z e ,
( array_agg ( i d x ) ) [ 1 ] AS idx1 , ( array_agg ( i d x ) ) [ 2 ] AS
idx2 ,
( array_agg ( i d x ) ) [ 3 ] AS idx3 , ( array_agg ( i d x ) ) [ 4 ] AS
idx4
FROM (
SELECT i n d e x r e l i d : : r e g c l a s s AS idx , ( i n d r e l i d : : t e x t
| | E \n | | i n d c l a s s : : t e x t | | E \n | | i n d k e y : : t e x t | | E \n | |

c o a l e s c e ( i n d e x p r s : : t e x t , ) | | E \n | |
c o a l e s c e ( i n d p r e d : : t e x t , ) ) AS KEY
FROM pg_index ) sub
- GROUP BY KEY HAVING count ( * )>1
- ORDER BY sum ( p g _ r e l a t i o n _ s i z e ( i d x ) ) DESC;


snippets/indexes_statustic.sql
SELECT
t . tablename ,
indexname ,
c . r e l t u p l e s AS num_rows ,

Line 1
-

p g _ s i z e _ p r e t t y ( p g _ r e l a t i o n _ s i z e ( quote_ident ( t . tablename ) : : t e x t ) )
AS t a b l e _ s i z e ,
-

10
-

p g _ s i z e _ p r e t t y ( p g _ r e l a t i o n _ s i z e ( quote_ident ( i n d e x r e l n a m e ) : : t e x t ) )
AS i n d e x _ s i z e ,
CASE WHEN x . i s _ u n i q u e = 1 THEN Y
ELSE N
END AS UNIQUE,
idx_scan AS number_of_scans ,
idx_tup_read AS t u p l e s _ r e a d ,
idx_tup_fetch AS t u p l e s _ f e t c h e d
FROM p g _ t a b l e s t

200

12.2.
15
20
-

25
-

LEFT OUTER JOIN p g _ c l a s s c ON t . tablename=c . relname


LEFT OUTER JOIN
(SELECT i n d r e l i d ,
max(CAST( i n d i s u n i q u e AS i n t e g e r ) ) AS i s _ u n i q u e
FROM pg_index
GROUP BY i n d r e l i d ) x
ON c . o i d = x . i n d r e l i d
LEFT OUTER JOIN
( SELECT c . relname AS ctablename , i p g . relname AS
indexname , x . i n d n a t t s AS number_of_columns , idx_scan ,
idx_tup_read , idx_tup_fetch , i n d e x r e l n a m e FROM pg_index x
JOIN p g _ c l a s s c ON c . o i d = x . i n d r e l i d
JOIN p g _ c l a s s i p g ON i p g . o i d = x . i n d e x r e l i d
JOIN p g _ s t a t _ a l l _ i n d e x e s p s a i ON x . i n d e x r e l i d =
psai . indexrelid )
AS f o o
ON t . tablename = f o o . ctablename
WHERE t . schemaname= p u b l i c
ORDER BY 1 , 2 ;

201


[1]

(Sad
Spirit)
borz_off@cs.msu.su
PostgreSQL:

http://www.phpclub.ru/detail/store/pdf/postgresql-performance.pdf
[2] Eugene
Kuzin
eugene@kuzin.net

PostgreSQL

Slony-I
http://www.kuzin.net/work/sloniki-privet.html
[3] Sergey Konoplev gray.ru@gmail.com Londiste
http://gray-hemp.blogspot.com/2010/04/londiste.html
[4] Dmitry
Stasyuk

pgpool-II
http://undenied.ru/2009/03/04/uchebnoe-rukovodstvo-po-pgpool-ii/
[5]

dmitry.chirkin@gmail.com

PostgreSQL

PL/Proxy
http://habrahabr.ru/blogs/postgresql/45475/
[6]

wordpress@insight-it.ru
http://www.insight-it.ru/masshtabiruemost/hadoop/

Hadoop

[7] Padraig
OSullivan
Up
and
Running
with
HadoopDB
http://posulliv.github.com/2010/05/10/hadoopdb-mysql.html
[8] PostgreSQL:
Skype http://postgresmen.ru/articles/view/25
[9] Streaming Replication. http://wiki.postgresql.org/wiki/Streaming_Replication

[10] Den Golotyuk , , -


? http://highload.com.ua/index.php/2009/05/06/--
[11] Postgres-XC

A
PostgreSQL
Clustering
Solution
http://www.linuxforu.com/2012/01/postgres-xc-database-clustering-solution/

202