Chapter 11

Streaming Replication
The synchronous streaming replication was implemented in version 9.1. It is a so-called single-master-multi-slaves
type replication, and those two terms – master and slave(s) – are usually referred to
as primary and standby(s) respectively in PostgreSQL.

This native replication feature is based on the log shipping, one of the general replication techniques, in which a
primary server continues to send WAL data and then, each standby server replays the received data immediately.

This chapter covers the following topics focusing on how streaming replication works:

 How Streaming Replication starts up

 How the data are transferred between primary and standby servers
 How primary server manages multiple standby servers
 How primary server detects failures of standby servers

Though the first replication feature, only for asynchronous replication, had implemented in version 9.0, it had replaced with the new
implementation (currently in use) for synchronous replication in 9.1.

11.1 Starting the streaming replication

In Streaming Replication, three kinds of processes work cooperatively. A walsender process on the primary server
sends WAL data to standby server; and then, a walreceiver and a startupprocesses on standby server receives and
replays these data. A walsender and a walreceiver communicate using a single TCP connection.

In this section, we will explore the start-up sequence of the streaming replication to understand how those processes
are started and how the connection between them is established. Figure 11.1 shows the startup sequence diagram of
streaming replication:
Figure 11.1: SR startup sequence
 (1) Start primary and standby servers.
 (2) The standby server starts a startup process.
 (3) The standby server starts a walreceiver process.
 (4) The walreceiver sends a connection request to the primary server. If the primary server is not running,
the walreceiver sends these requests periodically.
 (5) When the primary server receives a connection request, it starts a walsender process and a TCP
connection is established between the walsender and walreceiver.
 (6) The walreceiver sends the latest LSN of standby's database cluster. In general, this phase is known
as handshaking in the field of information technology.
 (7) If the standby's latest LSN is less than the primary's latest LSN (Standby's LSN < Primary's LSN), the
walsender sends WAL data from the former LSN to the latter LSN. Such WAL data are provided by WAL
segments stored in the primary's pg_xlog subdirectory (in version 10 or later, pg_wal subdirectory). Then,
the standby server replays the received WAL data. In this phase, the standby catches up with the primary,
so it is called catch-up.
 (8) Streaming Replication begins to work.
Each walsender process keeps a state appropriate for the working phase of connected walreceiver or any application
(Note that it is not the state of walreceiver or application connected to the walsender.) The following are the possible
states of it:

 start-up – From starting the walsender to the end of handshaking. See Figure 11.1 (5)–(6).
 catch-up – During the catch-up phase. See Figure 11.1 (7).
 streaming – While Streaming Replication is working. See Figure 11.1 (8).
 backup – During sending the files of the whole database cluster for backup tools such
as pg_basebackup utility.

The pg_stat_replication view shows the state of all running walsenders. An example is shown below:
testdb=# SELECT application_name,state FROM pg_stat_replication;
application_name | state
------------------+-----------
standby1 | streaming
standby2 | streaming
pg_basebackup | backup
(3 rows)
As shown in the above result, two walsenders are running to send WAL data for the connected standby servers, and
another one is running to send all files of the database cluster for pg_basebackup utility.

What will happen if a standby server restarts after a long time in the stopped condition?

In version 9.3 or earlier, if the primary's WAL segments required by the standby server have already been recycled, the standby cannot
catch up with the primary server. There is no reliable solution for this problem, but only to set a large value to the configuration
parameter wal_keep_segments to reduce the possibility of the occurrence. It's a stopgap solution.

In version 9.4 or later, this problem can be prevented by using replication slot. The replication slot is a feature that expands the flexibility
of the WAL data sending, mainly for the logical replication, which also provides the solution to this problem – the WAL segment files
which contain unsent data under the pg_xlog (or pg_wal if version 10 or later) can be kept in the replication slot by pausing recycling
process. Refer the official document for detail.

11.2 How to conduct streaming replication

Streaming replication has two aspects: log shipping and database synchronization. Log shipping is obviously one of
those aspects since the streaming replication is based on it – the primary server sends WAL data to the connected
standby servers whenever the writing of them occurs. Database synchronization is required for synchronous
replication – the primary server communicates with each multiple-standby server to synchronize its database
clusters.

To accurately understand how streaming replication works, we should explore how one primary server manages
multiple standby servers. To make it rather simple, the special case (i.e. single-primary single-standby system) is
described in this section, while the general case (single-primary multi-standbys system) will be described in the next
section.

11.2.1 Communication between a primary and a synchronous standby

Assume that the standby server is in the synchronous replication mode, but the configuration parameter hot-
standby is disabled and wal_level is 'archive'. The main parameter of the primary server is shown below:

synchronous_standby_names = 'standby1'
hot_standby = off
wal_level = archive
Additionally, among the three triggers to write the WAL data mentioned in Section 9.5, we focus on the
transaction commits here.

Suppose that one backend process on the primary server issues a simple INSERT statement in autocommit mode.
The backend starts a transaction, issues an INSERT statement, and then commits the transaction immediately. Let's
explore further how this commit action will be completed. See the following sequence diagram in Figure 11.2:
 Figure 11.2: Streaming Replication's communication sequence diagram

 (1) The backend process writes and flushes WAL data to a WAL segment file by executing the
functions XLogInsert() and XLogFlush().
 (2) The walsender process sends the WAL data written into the WAL segment to the walreceiver process.
 (3) After sending the WAL data, the backend process continues to wait for an ACK response from the
standby server. More precisely, the backend process gets a latch by executing the internal
function SyncRepWaitForLSN(), and waits for it to be released.
 (4) The walreceiver on standby server writes the received WAL data into the standby's WAL segment
using the write() system call, and returns an ACK response to the walsender.
 (5) The walreceiver flushes the WAL data to the WAL segment using the system call such as fsync(),
returns another ACK response to the walsender, and informs the startup process about WAL data
updated.
 (6) The startup process replays the WAL data, which has been written to the WAL segment.
 (7) The walsender releases the latch of the backend process on receiving the ACK response from the
walreceiver, and then, backend process's commit or abort action will be completed. The timing for latch-
release depends on the parameter synchronous_commit. It is 'on' (default), the latch is released when the
ACK of step (5) received, whereas it is 'remote_write', when the ACK of step (4) received.

If the configuration parameter wal_level is 'hot_standby' or 'logical', PostgreSQL writes a WAL record regarding the hot standby feature,
following the records of a commit or abort action. (In this example, PostgreSQL does not write that record because it’s 'archive'.)

Each ACK response informs the primary server of the internal information of standby server. It contains
four items below:

 LSN location where the latest WAL data has been written.
 LSN location where the latest WAL data has been flushed.
 LSN location where the latest WAL data has been replayed in the startup process.
 The timestamp when this response has be sent.
 Walreceiver returns ACK responses not only when WAL data have been written and flushed, but also periodically
as the heartbeat of standby server. The primary server therefore always grasps the status of all connected standby
servers.

By issuing the queries as shown below, the LSN related information of the connected standby servers can be
displayed.

testdb=# SELECT application_name AS host,

write_location AS write_LSN, flush_location AS flush_LSN,
replay_location AS replay_LSN FROM pg_stat_replication;

host | write_lsn | flush_lsn | replay_lsn

----------+-----------+-----------+------------
standby1 | 0/5000280 | 0/5000280 | 0/5000280
standby2 | 0/5000280 | 0/5000280 | 0/5000280
(2 rows)

The heartbeat's interval is set to the parameter wal_receiver_status_interval, which is 10 seconds by default.

11.2.2 Behavior when a failure occurs

In this subsection, descriptions are made on how the primary server behaves when the synchronous standby server
has failed, and how to deal with the situation.

Even if the synchronous standby server has failed and is no longer able to return an ACK response, the primary
server continues to wait for responses forever. So, the running transactions cannot commit and subsequent query
processing cannot be started. In other words, all of the primary server operations are practically stopped. (Streaming
replication does not support a function to revert automatically to asynchronous-mode by timing out.)

There are two ways to avoid such situation. One of them is to use multiple standby servers to increase the system
availability, and the other is to switch from synchronous to asynchronous mode by performing the following steps
manually.

 (1) Set empty string to the parameter synchronous_standby_names.

synchronous_standby_names = ''

 (2) Execute the pg_ctl command with reload option.

postgres> pg_ctl -D $PGDATA reload
The above procedure does not affect the connected clients. The primary server continues the transaction processing
as well as all sessions between clients and the respective backend processes are kept.

11.3 Managing multiple standby servers

In this section, the way streaming replication works with multiple standby servers is described.

11.3.1 sync_priority and sync_state

Primary server gives sync_priority and sync_state to all managed standby servers, and treats each standby server
depending on its respective values. (The primary server gives those values even if it manages just one standby
server; haven’t mentioned this in the previous section.)
sync_priority indicates the priority of standby server in synchronous-mode and is a fixed value. The smaller value
shows the higher priority, while 0 is the special value that means "in asynchronous-mode". Priorities of standby
servers are given in the order listed in the primary's configuration parameter synchronous_standby_names. For
example, in the following configuration, priorities of standby1 and standby2 are 1 and 2, respectively.

synchronous_standby_names = 'standby1, standby2'

(Standby servers not listed on this parameter are in asynchronous-mode, and their priority is 0.)

sync_state is the state of the standby server. It is variable according to the running status of all standby servers and
the individual priority. The followings are the possible states:

 Sync is the state of synchronous-standby server of the highest priority among all working standbys
(except asynchronous-servers).
 Potential is the state of spare synchronous-standby server of the second or lower priority among the
all working standbys (except asynchronous-servers). If the synchronous-standby has failed, it will be
replaced with the highest priority standby within the potential ones.
 Async is the state of asynchronous-standby server, and this state is fixed. The primary server treats
asynchronous-standbys in the same way as potential standbys except that their sync_statenever
be 'sync' or 'potential'.

The priority and the state of the standby servers can be shown by issuing the following query:

testdb=# SELECT application_name AS host,

sync_priority, sync_state FROM pg_stat_replication;
host | sync_priority | sync_state
----------+---------------+------------
standby1 | 1 | sync
standby2 | 2 | potential
(2 rows)

Several developers are recently trying to implement "multiple synchronous-standby". See here for details.

11.3.2 How the primary manages multiple standbys

The primary server waits for ACK responses from the synchronous standby server alone. In other words, the
primary server confirms only synchronous standby's writing and flushing of WAL data. Streaming replication,
therefore, ensures that only synchronous standby is in the consistent and synchronous state with the primary.

Figure 11.3 shows the case in which the ACK response of potential standby has been returned earlier than that of the
primary standby. There, the primary server does not complete the commit action of the current transaction, and
continues to wait for the primary's ACK response. And then, when the primary's response is received, the backend
process releases the latch and completes the current transaction processing.
Figure 11.3: Managing multiple standby servers
The sync_state of standby1 and standby2 are 'sync' and 'potential' respectively. (1) In spite of receiving an ACK response from the
potential standby server, the primary's backend process continues to wait for an ACK response from the synchronous-standby
server. (2) The primary's backend process releases the latch, completes the current transaction processing.

In the opposite case (i.e. the primary's ACK response has been returned earlier than the potential's one), the primary
server immediately completes the commit action of the current transaction without ensuring if the potential standby
writes and flushes WAL data or not.

11.3.3 Behavior when a failure occurs

Once again, see how the primary server behaves when the standby server has failed.

When either a potential or an asynchronous standby server has failed, the primary server terminates the walsender
process connected to the failed standby and continues all processing. In other words, transaction processing of the
primary server would not be affected by the failure of either type of standby server.

When a synchronous standby server has failed, the primary server terminates the walsender process connected to the
failed standby, and replaces synchronous standby with the highest priority potential standby. See Figure 11.4. In
contrast to the failure described above, query processing on the primary server will be paused from the point of
failure to the replacement of synchronous standby. (Therefore, failure detection of standby server is a very important
function to increase availability of replication system. Failure detection will be described in the next section.)
Figure 11.4: Replacing of synchronous standby server
In any case, if one or more standby server shall be running in syncrhonous-mode, the primary server keeps only one
synchronous standby server at all times, and the synchronous standby server is always in a consistent and
synchronous state with the primary.

11.4 Detecting failures of standby servers

Streaming replication uses two common failure detection procedures that will not require any special hardware at
all.

1. Failure detection of standby server process

When a connection drop between walsender and walreceiver has been detected, the primary
server immediately determines that the standby server or walreceiver process is faulty. When a low level
network function returns an error by failing to write or to read the socket interface of walreceiver, the primary
also immediately determines its failure.

2. Failure detection of hardware and networks

If a walreceiver returns nothing within the time set for the parameter wal_sender_timeout (default 60 seconds),
the primary server determines that the standby server is faulty. In contrast to the failure described above, it
takes a certain amount of time – maximum is wal_sender_timeout seconds – to confirm the standby's death on
the primary server even if a standby server is no longer able to send any response by some failures (e.g. standby
server's hardware failure, network failure, and so on).
Depending on the types of failures, it can usually be detected immediately after a failure occurs, while sometimes
there might be a time lag between the occurrence of failure and the detection of it. In particular, if a latter type of
failure occurs in synchronous standby server, all transaction processing on the primary server will be stopped until
detecting the failure of standby, even though multiple potential standby servers may have been working.

The parameter wal_sender_timeout was called as replication_timeout in version 9.2 or earlier.

Process architecture
PostgreSQL is a client/server type relational database management system with the multi-process architecture and
runs on a single host.

A collection of multiple processes cooperatively managing one database cluster is usually referred to as
a 'PostgreSQL server', and it contains the following types of processes:

 A Postgres Server process is a parent of all processes related to database cluster management.

 Each backend process handles all queries and statements issued by a connected client.

 Various background processes perform processes of each feature (e.g., VACUUM and CHECKPOINT
processes) for database management.

 In the replication associated processes, they perform the streaming replication.

 In the background worker process supported from version 9.3, it can perform any processing
implemented by users.

In the following subsections, the details of the first three types of processes are described.
Figure 2.1: An example of the process architecture in PostgreSQL.

This figure shows processes of a PostgreSQL server: a postgres server process, two backend processes, seven background processes, and
two client processes. The database cluster, the shared memory, and two client processes are also illustrated.

2.1.1 postgres server process

As already described above, a postgres server process is a parent of all in a PostgreSQL server. In the earlier
versions, it was called ‘postmaster’.

By executing the pg_ctl utility with start option, a postgres server process starts up. Then, it allocates a shared
memory area in memory, starts various background processes, starts replication associated processes and
background worker processes if necessary, and waits for connection requests from clients. Whenever receiving a
connection request from a client, it starts a backend process. (And then, the started backend process handles all
queries issued by the connected client.)

A postgres server process listens to one network port, the default port is 5432. Although more than one PostgreSQL
server can be run on the same host, each server should be set to listen to different port number in each other, e.g.,
5432, 5433, etc.

2.1.2 Backend processes

A backend process, which is also called postgres, is started by the postgres server process and handles all queries
issued by one connected client. It communicates with the client by a single TCP connection, and terminates when
the client gets disconnected.

As it is allowed to operate only one database, you have to specify a database you want to use explicitly when
connecting to a PostgreSQL server.

PostgreSQL allows multiple clients to connect simultaneously; the configuration

parameter max_connections controls the maximum number of the clients (default is 100).

If many clients such as WEB applications frequently repeat the connection and disconnection with a PostgreSQL
server, it increases both costs of establishing connections and of creating backend processes because PostgreSQL
has not implemented a native connection pooling feature. Such circumstance has a negative effect on the
performance of database server. To deal with such a case, a pooling middleware (either pgbouncer or pgpool-II) is
usually used.

2.1.3 Background processes

Table 2.1 shows a list of background processes. In contrast to the postgres server and the backend process, it is
impossible to explain each of the functions simply, because these functions depend on the individual specific
features and PostgreSQL internals. Thus, in this chapter, only introductions are made. Details will be described in
the following chapters.

Table 2.1: background processes.

Process Description reference

background In this process, dirty pages on the shared buffer pool are written to a Section 8.6
writer persistent storage (e.g., HDD, SSD) on a regular basis gradually. (In version
9.1 or earlier, it was also responsible for checkpoint process.)

checkpointer In this process in version 9.2 or later, checkpoint process is performed. Section
8.6, Section
9.7

autovacuum The autovacuum-worker processes are invoked for vacuum process Section 6.5
launcher periodically. (More precisely, it requests to create the autovacuum workers to
the postgres server.)
WAL writer This process writes and flushes periodically the WAL data on the WAL Section 9.9
buffer to persistent storage.

statistics In this process, statistics information such as for pg_stat_activity and for
pg_stat_database, etc. is collected.
collector

logging collector This process writes error messages into log files.
(logger)

Archiver In this process, archiving logging is executed. Section 9.10

The actual processes of a PostgreSQL server is shown here. In the following example, one postgres server process (pid is
9687), two backend processes (pids are 9697 and 9717) and the several background processes listed in Table 2.1 are
running. See also Figure 2.1.

postgres> pstree -p 9687

-+= 00001 root /sbin/launchd
\-+- 09687 postgres /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
|--= 09688 postgres postgres: logger process
|--= 09690 postgres postgres: checkpointer process
|--= 09691 postgres postgres: writer process
|--= 09692 postgres postgres: wal writer process
|--= 09693 postgres postgres: autovacuum launcher process
|--= 09694 postgres postgres: archiver process
|--= 09695 postgres postgres: stats collector process
|--= 09697 postgres postgres: postgres sampledb 192.168.1.100(54924) idle
\--= 09717 postgres postgres: postgres sampledb 192.168.1.100(54964) idle in transaction

2.2 Memory architecture

Memory architecture in PostgreSQL can be classified into two broad categories:

 Local memory area – allocated by each backend process for its own use.

Shared memory area – used by all processes of a PostgreSQL server.

In the following subsections, those are briefly descibed.

Figure 2.2: Memory architecture in PostgreSQL

2.2.1 Local memory area
Each backend process allocates a local memory area for query processing; each area is divided into several sub-areas
– whose sizes are either fixed or variable. Table 2.2 shows a list of the major sub-areas. The details will be described
in the following chapters.

Table 2.2: Local memory area

sub-area description

work_mem Executor uses this area for sorting tuples by ORDER BY and DISTINCT operations, and for joining tables by m

maintenance_work_mem Some kinds of maintenance operations (e.g., VACUUM, REINDEX) use this area.

temp_buffers Executor uses this area for storing temporary tables.

2.2.2 Shared memory area

A shared memory area is allocated by a PostgreSQL server when it starts up. This area is also divided into several
fix sized sub-areas. Table 2.3 shows a list of the major sub-areas. The details will be described in the following
chapters.
 Table 2.3: Shared memory area

sub-area description

shared buffer PostgreSQL loads pages within tables and indexes from a persistent storage to here, and operates them directly.
pool

WAL buffer To ensure that no data has been lost by server failures, PostgreSQL supports the WAL mechanism. WAL data (also referred
WAL buffer is a buffering area of the WAL data before writing to a persistent storage.

commit log Commit Log(CLOG) keeps the states of all transactions (e.g., in_progress,committed,aborted) for Concurrency Control (CC)

In addition to them, PostgreSQL allocates several areas as shown below:

 Sub-areas for the various access control mechanisms. (e.g., semaphores, lightweight locks, shared
and exclusive locks, etc)
 Sub-areas for the various background processes, such as checkpointer and autovacuum.
 Sub-areas for transaction processing such as save-point and two-phase-commit.

and others.

Buffer Manager
A buffer manager manages data transfers between shared memory and persistent storage and can have a significant
impact on the performance of the DBMS. The PostgreSQL buffer manager works very efficiently.

In this chapter, the PostgreSQL buffer manager is described. The first section provides an overview and
the subsequent sections describe the following topics:

 Buffer manager structure

 Buffer manager locks
 How the buffer manager works
 Ring buffer
 Flushing of dirty pages

Figure 8.1: Relations between buffer manager, storage, and backend processes
PostgreSQL Administration
The PostgreSQL administration covers the most important PostgreSQL database server
administration activities. We will discuss roles and groups administration, databases
creation, tablespaces management, databases backup and restore.

PostgreSQL database and schema management

In this section, we will show you how to manage databases in PostgreSQL including
creating databases, modifying existing database’s features and deleting databases. In
addition, the schema management is covered.

 Create a new database – create a new database using CREATE DATABASE

statement.
 Modify an existing database – modify the features of an existing database using the
ALTER DATABASE statement.
 Drop a database – removes a database permanently using DROP DATABASE
statement.
 Copy a PostgreSQL database – copy a database within a database server or from a
server to another.
 Get database object sizes – introduces you various handy function to get the size of a
database, a table, indexes, etc.
PostgreSQL CREATE DATABASE
Summary: in this tutorial, you will learn how to create new databases with various options
by using the PostgreSQL CREATE DATABASE statement.

Introduction to PostgreSQL CREATE DATABASE statement

To create a new PostgreSQL database, you use CREATE DATABASE statement as
shown below:
CREATE DATABASE db_name
OWNER = role_name
TEMPLATE = template ENCODING = encoding
LC_COLLATE = collate
LC_CTYPE = ctype TABLESPACE = tablespace_name
CONNECTION LIMIT = max_concurrent_connection

The CREATE DATABASE statement provides you with various options when creating a
new database. Let’s examine those options in more detail:

 db_name: is the name of the new database that you want to create. The database
name must be unique in the PostgreSQL database server. If you try to create a new
database that has the same name as an existing database, PostgreSQL will issue an
error.
 role_name: is the role name of the user who will own the new database. PostgreSQL
uses user’s role name who executes the CREATE DATABASE statement as the
default role name.
 template: is the name of the database template from which the new database
creates. PostgreSQL allows you to create a database based on a template database.
The template1 is the default template database.
 encoding: specifies the character set encoding for the new database. By default, it is
the encoding of the template database.
 collate: specifies a collation for the new database. The collation specifies the sort
order of strings that affect the result of the ORDER BY clause in the SELECT
statement. The template database’s collation is the default collation for the new
database if you don’t specify it explicitly in the LC_COLLATE parameter.
 ctype: specifies the character classification for the new database. The ctype affects
the categorization e.g., digit, lower and upper. The default is the character
classification of the template database.
 tablespace_name: specifies the tablespace name for the new database. The default
is the template database’s tablespace.
 max_concurrent_connection: specifies the maximum concurrent connections to the
new database. The default is -1 i.e., unlimited. This feature is very useful in the
shared hosting environments where you can configure the maximum concurrent
connections for a particular database.
Besides the CREATE DATABASE statement, you can also use the createdb program to
create a new database. The createdb program uses CREATE DATABASE statement
behind the scenes.

PostgreSQL create database examples

The simplest way to create a new database is to use all default settings and only specify the
database name as the following query:

postgres# CREATE DATABASE testdb1;

PostgreSQL created a new database named testdb1 that has default parameters from the default
template database i.e., template1.
The following statement creates a new database name hrdb with the following parameters:

 Encoding: utf-8.
 Owner: hr, with the assumption that the hr user exists in the database server.
 Maximum concurrent connections: 25.
postgres# CREATE DATABASE hrdb WITH ENCODING='UTF8' OWNER=hr CONNECTION LIMIT=25;

PostgreSQL create database using pgAdmin example

Follow the steps below to create a new database via pgAdmin:

First, log in to the PostgreSQL via pgAdmin.

Second, from the Databases, right mouse click and chose the New Databases… menu
item. A new window will appear.

Third, enter the new database name, owner and configure parameters. After that click OK
button to create the new database.
In this tutorial, you have learned how to create new databases by using PostgreSQL
CREATE DATABASE statement.

PostgreSQL ALTER DATABASE

Summary: in this tutorial, you will learn how to modify existing databases by
using PostgreSQL ALTER DATABASE statement.

Introduction to PostgreSQL ALTER DATABASE statement

Once you created a database, you can change its features by using the ALTER
DATABASE statement as shown following:
postgres# ALTER DATABASE target_database action;

You specify the database name that you want to change after the ALTER DATABASE.
PostgreSQL allows you to perform various actions on the existing database. Let’s examine
each action in more detail.
Rename database
To rename the database, you use ALTER DATABASE RENAME TO statement as follows:
postgres# ALTER DATABASE target_database RENAME TO new_database;

To rename a database, you have to connect to another database e.g., Postgres.

Change owner
To change the owner of the database, you use ALTER DATABASE OWNER TO as the
following statement:
postgres# ALTER DATABASE target_database OWNER TO new_owner;

Only the superuser or owner of the database can change the database’s owner. The
database owner must also have the CREATEDB privilege to rename the database.
Change tablespace
To change the default tablespace of the database, you use ALTER DATABASE SET
TABLESPACE as follows:
postgres# ALTER DATABASE target_database SET TABLESPACE new_tablespace;

The statement moves tables and indexes from the legacy tablespace to the new one.
Change session defaults for run-time configuration variables

Whenever you connect to a database, PostgreSQL loads the configuration variables

presented in the postgresql.conf file and uses these variables by default.
To override these settings for a particular database, you use ALTER DATABASE SET
statement as follows:
postgres# ALTER DATABASE target_database SET configuration_parameter = value;

In the subsequent sessions, PostgreSQL will override the settings in the postgresql.conf
file. Note that only a supperuser or the database owner can change the default session
variables for a database.

PostgreSQL ALTER DATABASE example

First, let’s log in as the postgres user and create a new database named testdb2 for the
demonstration.
postgres# CREATE DATABASE testdb2;

Second, use the following statement to rename the testdb2 database to testhrdb.
postgres# ALTER DATABASE testdb2 TO testhrdb;

Third, execute the following statement to change the owner of the testhrdb database
from postgres to hr, with the assumption that the hr role already exists.
postgres# ALTER DATABASE testhrdb OWNER TO hr;

If the hr role does not exist, create it by using the following statement:
postgres# CREATE ROLE hr VALID UNTIL ‘inifinty’;
Fourth, change the default tablespace of the testhrdb from pg_default to hr_default, with
the assumption that the hr_default tablespace already exists.
postgres# ALTER DATABASE testhrdb SET TABLESPACE hr_default;

If the hr_default tablespace does not exist, you can create it by using the following
statement:
postgres# CREATE TABLESPACE hr_default OWNER hr LOCATION ‘c:\\pgdata\\hr’;

Fifth, to set escape_string_warning configuration variable to off, you can use the following
statement:
postgres# ALTER DATABASE testhrdb SET escape_string_warning TO off;

In this tutorial, we have shown you how to change the existing database’s features and
configuration parameters by using the PostgreSQL ALTER DATABASE statement.

PostgreSQL DROP DATABASE

Summary: in this tutorial, you will learn how to delete existing database by using
PostgreSQL DROP DATABASE statement.

Introduction to PostgreSQL DROP DATABASE statement

Once a database is no longer needed, you can delete it by using the DROP DATABASE
statement. The following illustrates the syntax of the DROP DATABASE statement:
# DROP DATABASE [IF EXISTS] dbname;
To delete a database:

 Specify the name of the database that you want to delete after the DROP
DATABASE clause.
 Use IF EXISTS to prevent an error from removing a non-existent database.
PostgreSQL will issue a notice instead.
The DROP DATABASE statement deletes catalog entries and data directory permanently.
This action cannot be undone so you have to use it with caution.
Only the database owner can execute the DROP DATABASE statement. In addition, you
cannot execute the DROP DATABASE statement if there is any active connection to the
database. You have to connect to another database e.g., postgresql to execute the DROP
DATABASE statement.
PostgreSQL also provides a utility program named dropdb that allows you to remove a
database. The dropdb program executes the DROP DATABASE statement behind the
scenes.
Delete a database that has active connections
To delete the database that still has active connections, you can follow the steps below:
First, find the activities that are taken place against the target database, you can query the
pg_stat_activity view as the following query:
postgres# SELECT * FROM pg_stat_activity WHERE datname = ‘target_database’;

Second, terminate the active connections by issuing the following query:

postgres# SELECT pg_terminate_backend (pg_stat_activity.pid) FROM pg_stat_activity
WHERE datname = ‘target_database’;

Notice that if you use PostgreSQL version 9.1 or earlier, use the procpid column instead of
the pid column because PostgreSQL changed procid column to pid column since version
9.2.

Third, execute the DROP DATABASE statement:

Postgres# DROP DATABASE target_database;

PostgreSQL DROP DATABASE examples

For sample we create two db’s demonstration, if don’t exist by executing the following
statements:

postgres# CREATE DATABASE hrdb;

postgres# CREATE DATABASE testdb1;

Delete database that has no active connection example

To remove the hrdb database, use the hrdb owner to connect to a database other than
hrdb database e.g., postgres and issue the following statement:
postgres# DROP DATABASE hrdb;

PostgreSQL deleted the hrdb database.

Delete database that has active connections example

The following statement deletes the testdb1 database:

postgres# DROP DATABASE testdb1;

However, PostgreSQL issued an error as follows:

Error: database “testdb1” is being accessed by other users
SQL state: 55006
Detail: There is 1 other session using the database.

To delete the testdb1 database, you need to follow the steps as described in the above
section.

First, query the pg_stat_activity view to find what activities are taking place against the
testdb1 database:
postgres# select * from pg_stat_activity where datname = ‘testdb1’;

The testdb1 database has 1 connection from localhost therefore it is safe to terminate this
connection and remove the database.

Second, terminate the connection to the testdb1 database by using the following statement:
postgres# select pg_teminate_backend (pg_stat_activity) where pg_stat_activity.datname
= ‘testdb1’;

Third, issue the DROP DATABASE command to remove the testdb1 database:
postgres# DROP DATABASE testdb1;

PostgreSQL deleted the testdb1 permanently.

In this tutorial, you have learned how to use the PostgreSQL DROP DATABASE statement
to delete a database. In addition, you also learned how to delete a database that has active
connections.

PostgreSQL Copy Database Made Easy

Summary: in this tutorial, you will learn how to copy a PostgreSQL database on the same
server or from a server to another.

PostgreSQL copy database within the same server

Sometimes, you want to copy a PostgreSQL database within a database server for testing
purposes. PostgreSQL makes it so easy to do it via the CREATE DATABASE statement as
follows:
postgres# CREATE DATABASE targetdb WITH TEMPLATE sourcedb;

This statement copies the sourcedb to the targetdb. For example, to copy the dvdrental
sample database to the dvdrental_test database, you use the following statement:
postgres# CREATE DATABASE dvdrental_test WITH TEMPLATE dvdrental;

Depending on the size of the source database, it may take a while to complete copying.

PostgreSQL copy database from a server to another

There are several ways to copy a database between PostgreSQL database servers. If the
size of the source database is big and the connection between the database servers is
slow, you can dump the source database to a file, copy the file to the remote server, and
restore it.

Here is the command of each step:

1. First, dump the source database to a file.

#pg_dump –U Postgres –O sourcedb sourcedb.sql
2. Second, copy the dump file to the remote server.
3. Third, create a new database in the remote server:
postgres# CREATE DATABASE targetdb;
4. Fourth, restore the dump file on the remote server:
#pg_dump –U Postgres –d targetdb -f sourcedb.sql
For example, to copy the dvdrental database from the local server to the remote server,
you do it as follows:

1. First, dump the dvdrental database into a dump file e.g., dvdrental.sql:
#pg_dump –U Postgres –O dvdrental dvdrental.sql
2. Second, copy the dump file to the remote server.
3. Third, create the dvdrental database on the remote server:
postgres# CREATE DATABASE dvdrental;
4. Fourth, restore the dvdrental.sql dump file in the remote server:
#pg_dump –U Postgres –d dvdrental -f dvdrental.sql

In case the connection between servers are fast and the size of the database is not big, you
can use the following command:
#pg_dump –C –h local –U localuser sourcedb | psql –h remote –U remoteuser targetdb

For example, to copy the dvdrental database from the localhost server to the remote
server, you do it as follows:
#pg_dump –C –h localhost –U postgres dvdrental | psql –h remote –U postgres dvdrental

In this tutorial, you have learned how to copy a PostgreSQL database within a database
server, or from a database server to anther.
How to Get Table, Database, Indexes, Tablespace, and Value
Size in PostgreSQL
Summary: This tutorial shows you how to get PostgreSQL database, table, indexes,
tablespace, and value size using various handy functions.

PostgreSQL table size

To get the size of a specific table, you use the pg_relation_size() function. For example,
you can get the size of the actor table in the dvdrental sample database as follows:
postgres# select pg_relation_size(‘actor’);

The pg_relation_size() function returns the size of a specific table in bytes:

postgres# select pg_relation_size(‘actor’);
-----------------------------------------
16384
To make the result more human readable, you use the pg_size_pretty() function. The
pg_size_pretty() function takes the result of another function and format it using bytes, kB,
MB, GB or TB as appropriate. For example:
postgres# select pg_relation_size(‘actor’);

The following is the output in kB

The pg_relation_size() function returns the size of the table only, not included indexes or
additional objects.
To get the total size of a table, you use the pg_total_relation_size() function. For example,
to get the total size of the actor table, you use the following statement:
postgres# select pg_size_pretty(pg_total_relation_size (‘actor’));

You can use the pg_total_relation_size() function to find the size of biggest tables
including indexes.
For example, the following query returns top 5 biggest tables in the dvdrental database:
postgres# SELECT relname AS “relation”, pg_size_pretty (pg_total_relation_size (C.oid)) A
S “total_size” from pg_class C Left JOIN pg_namespace N ON (N.oid = C.relnamespace) W
HERE nspname NOT IN ( ‘pg_catalog’, ‘information_schema’) AND C.relkind <> ‘I’ And nsp
name !~ '^pg_toast' ORDER BY pg_total_relation_size (C .oid) DESC LIMIT 5;

Here is the output:

relation | total_size

--------------+------------

rental | 2472 kB

payment | 2232 kB

film | 688 kB

film_actor | 536 kB

inventory | 464 kB

(5 rows)

PostgreSQL database size

To get the size of the whole database, you use the pg_database_size() function. For
example, the following statement returns the size of the dvdrental database:
postgres# select pg_size_pretty(
1 SELECT

2 pg_size_pretty (

3 pg_database_size ('dvdrental')

4 );

The statement returns the following result:

1 pg_size_pretty

2 ----------------
3 15 MB

4 (1 row)

To get the size of each database in the current database server, you use the following
statement:

1 SELECT

2 pg_database.datname,

3 pg_size_pretty(pg_database_size(pg_database.datname)) AS size

4 FROM pg_database;

1 datname | size

2 ----------------+---------

3 postgres | 7055 kB

4 template1 | 7055 kB

5 template0 | 6945 kB

6 dvdrental | 15 MB

PostgreSQL indexes size

To get total size of all indexes attached to a table, you use
the pg_indexes_size() function.
The pg_indexes_size() function accepts the OID or table name as the argument and
returns the total disk space used by all indexes attached of that table.
For example, to get the total size of all indexes attached to the film table, you use the
following statement:
1 SELECT

2 pg_size_pretty (pg_indexes_size('actor'));

Here is the output:

1 pg_size_pretty

2 ----------------

3 32 kB

4 (1 row)

PostgreSQL tablespace size

To get the size of a tablespace, you use the pg_tablespace_size() function.
The pg_tablespace_size() function accepts a tablespace name and returns the size in
bytes.
The following statement returns the size of the pg_default tablespace:
1 SELECT

2 pg_size_pretty (

3 pg_tablespace_size ('pg_default')

4 );

The statement returns the following output:

1 pg_size_pretty

2 ----------------

3 43 MB

4 (1 row)

PostgreSQL value size

To find how much space that needs to store a specific value, you use
the pg_column_size() function, for examples:
1 dvdrental=# select pg_column_size(5::smallint);

2 pg_column_size

3 ----------------

4 2

5 (1 row)

8 dvdrental=# select pg_column_size(5::int);

9 pg_column_size

10 ----------------

11 4

12 (1 row)

13

14
15 dvdrental=# select pg_column_size(5::bigint);

16 pg_column_size

17 ----------------

18 8

19 (1 row)

In this tutorial, you have learned various handy functions to get the size of a database, a
table, indexes, a tablespace, and a value.

PostgreSQL roles administration

In PostgreSQL, a role is an account. A role that has login right is called a user. A role may
be a member of other roles. A role that contains other roles is known as a group. In this
section, you will learn how to manage roles and groups effectively.

 PostgreSQL Roles Management: introduces you to PostgreSQL roles concept and

shows you how to create user roles and group roles by using PostgreSQL CREATE
ROLE statement.

PostgreSQL Roles Management

Summary: in this tutorial, we will introduce you to PostgreSQL roles concept and show you
how to create user roles and group roles.

PostgreSQL uses the roles concept to manage database access permissions. A role can be
a user or a group, depending on how you setup the role. A role that has login right is called
user. A role may be a member of other roles, which are known as groups.
Creating PostgreSQL roles
From version 8.1, PostgreSQL uses the roles concept to incorporate the users and groups
concepts. To create a new role, you use the CREATE ROLE statement as follows:
1 CREATE ROLE role_name;

To get all available roles in the cluster, you query from the pg_roles system catalog as the
following statement:

1 SELECT

2 rolname

3 FROM

4 pg_roles;

If you use the psql tool, you can use the \du command to list all existing roles.

Role attributes

The attributes of a database role define role’s privileges including login, superuser,
database creation, role creation, password, etc.

The following statement creates a role that has login privilege, password, and valid date.

1 CREATE ROLE doe WITH PASSWORD 'pgSecpas1970' VALID UNTIL '2020-01-01';

The following statement creates a role that has superuser status, which means this role can
bypass all authorization checks:

1 CREATE ROLE bigboss SUPERUSER;

Notice that you must be a superuser in order to create another superuser.

If you want a role to have database creation privilege, you use the following statement:

1 CREATE ROLE admin CREATEDB;

Use the following statement to create a role that has creation privilege:

1 CREATE ROLE security CREATEROLE;

Role membership

It is easier to manage roles as a group so that you can grant or revoke privileges from a
group as a whole. In PostgreSQL, you create a role that represents a group, and then grant
membership in the group role to individual user roles.

By convention, a group role does not have LOGIN privilege.

To create a group role, you use the CREATE ROLE statement as follows:
1 CREATE ROLE group_role;

For example, the following statement creates sales group role:

1 CREATE ROLE sales;

Now, you can add a user role to a group role by using the GRANT statement:
1 GRANT group_role to user_role;

For example, to add the doe user role to the sales group role, you use the following
statement:

1 GRANT sales TO doe;

To remove a user role from a group role, you use REVOKE statement:
1 REVOKE group_role FROM user_role;

For example, to remove doe user role from the sales group role, you use the following
statement:

1 REVOKE sales FROM doe;

Notice that PostgreSQL does not allow you to have circular membership loops, in which a
role is the member of another role and vice versa.

Group and user role inheritance

A user role can use privileges of the group role in the following ways:

 First, a user role can use the SET ROLE statement to temporarily become the group
role, which means the user role use privileges of the group role rather than the
original privileges. In addition, any database objects created in the session are owned
by the group role, instead of the user role.
 Second, a user role that has the INHERIT attribute will automatically have the
privileges of the group roles of which it is a member, including all privileges inherited
by the group roles.
See the following example:
1 CREATE ROLE doe LOGIN INHERIT;

2 CREATE ROLE sales NOINHERIT;

3 CREATE ROLE marketing NOINHERIT;

4 GRANT sales to doe;

5 GRANT marketing to sales;

If you connect to PostgreSQL as doe, you will have privileges of doe plus privileges granted
to sales, because doe user role has the INHERIT attribute. However, you do not have
privileges of marketing because the NOINHERIT attribute is defined for the sales user role.
After executing the following statement:

1 SET ROLE sales;

You will have only privileges granted to sales, not the ones that granted to doe.

And after executing the following statement:

1 SET ROLE marketing;

You only have privileges granted to marketing, not the ones that granted to admin and doe.

To restore the original privilege, you can use the following statement:

1 RESET ROLE;

Notice that only privileges on the database object are inheritable. The LOGIN, SUPERUSER,
CREATEROLE, and CREATEDB are the special role that cannot be inherited as an ordinary
privilege.
Removing roles
You can use the DROP ROLE statement to remove a group role or user role.
1 DROP ROLE role_name;

Before removing a role, you must reassign or remove all objects it owns and revoke its
privileges.

If you remove a group role, PostgreSQL revokes all memberships in a group automatically.
The user roles of the group are not affected.
PostgreSQL Backup and restore Databases
This section shows you how to use various PostgreSQL backup and restore tools
including pg_dump , pg_dumpall , psql , pg_restore and pgAdmin to backup and restore
databases.

 PostgreSQL Backup – introduces you to practical ways to backup your databases by

using PostgreSQL backup tool including pg_dump and pg_dumpall .
 PostgreSQL Restore – shows you various ways to restore PostgreSQL databases by
using psql and pg_restore tools.

Backing Up Databases Using PostgreSQL

Backup Tools
Summary: in this tutorial, we will show you how to backup databases using PostgreSQL
backup tools including pg_dump and pg_dumpall .
Backing up databases is one of the most critical tasks in database administration. Before
backing up the databases, you should consider the following points:

 Full / partial databases

 Both data and structures, or only structures
 Point In Time recovery
 Restore performance
PostgreSQL provides pg_dump and pg_dumpall tools to help you backup databases easily
and effectively.
For ones who want to see the command to backup databases quickly, here it is:

1 >pg_dump -U username -W -F t database_name > c:\backup_file.tar

In this following section, we will show you practical ways to backup one database, all
databases, and only database objects.

How to backup one database

To backup one database, you can use the pg_dump tool. The pg_dump dumps out the
content of all database objects into a single file.
First, navigate to PostgreSQL BIN folder:

1 C:\>cd C:\Program Files\PostgreSQL\9.2\bin

Second, execute the pg_dump program and use the following options to backup
the dvdrental database to the dvdrental.tar file in the c:\pgbackup\ folder.
1 >pg_dump -U postgres -W -F t dvdrental > c:\pgbackup\dvdrental.tar
Let’s examine the options in more detail.

-U postgres : specifies the user to connect to PostgreSQL database server. We used

postgres in this example.
-W : forces pg_dump to prompt for the password before connecting to the PostgreSQL
database server. After you hit enter, pg_dump will prompt for the password of postgres
user.
-F : specifies the output file format that can be one of the following:

 c: custom-format archive file format

 d: directory-format archive
 t:tar
 p: plain text SQL script file).
Because we want the output file to be a tar-format archive file, we use -F t in this
example.
dvdrental : is the name of the database that we want to back
> c:\pgbackup\dvdrental.tar is the output backup file path.
How to backup all databases
To backup all databases, you can run the individual pg_dump command above
sequentially, or parallel if you want to speed up the backup process.

 First, from the psql, use the command \list to list all available databases in your
cluster
 Second, back up each individual database using the pg_dump program as described
in the above section.
Besides the pg_dump program, PostgreSQL also provides you with the pg_dumpall tool
that allows you to backup all databases at once. However, it is not recommended to use this
tool because of the following reasons:

 The pg_dumpall program exports all databases, one after another, into a single
script file, which prevents you from performing the parallel restore. If you backup all
databases this way, the restore process will take more time.
 The processing of dumping all databases takes longer than each individual one so
you do not know which dump of each database relates to a specific point in time.
If you have a good reason to use the pg_dumpall to backup all databases, the following is
the command:
1 >pg_dumpall -U postgres > c:\pgbackup\all.sql
The options of the pg_dumpall program are similar to the options of the pg_dump
program. We omit the -W option to avoid typing the password for each individual database,
which is quite tedious.
How to backup database object definitions
Sometimes, you want to backup only database object definitions so that you can restore the
schema only. This is helpful in the test phase, which you do not want keep the old test data
populated during the testing period.

To backup all objects in all databases, including roles, tablespaces, databases, schemas,
tables, indexes, triggers, functions, constraints, views, ownerships and privileges, you use
the following command:

1 >pg_dumpall --schema-only > c:\pgdump\definitiononly.sql

If you want to backup role definition only, use the following command:

1 >pg_dumpall --roles-only > c:\pgdump\allroles.sql

If you want to backup tablespaces definition, use the following command:

1 >pg_dumpall --tablespaces-only > c:\pgdump\allroles.sql

How to backup using pgAdmin

The pgAdmin provides an intuitive user interface that allows you to backup a database
using pg_dump tool.

For example to backup the dvdrental database to a dvdrental.tar in

the c:\pgbackup\dvdrental.tar file, you can follow the following steps:
First, right mouse click on the dvdrental database, and choose the Backup... menu item.
Second, enter the output file name and choose the file format.
pgAdmin backup tool provides various dump options as follows:
In the objects tab, you can select which objects to backup:
Third, click OK button to start performing a backup. The messages tag provides you with
detailed messages of the backup process.
In this tutorial, we have shown you some practical way to backup PostgreSQL databases by
using pg_dump and pg_dumpall tools.
Further Reading
 http://www.postgresql.org/docs/8.4/static/app-pgdump.html – How to use
the pg_dump tool.
Related Tutorials
 PostgreSQL Restore Database

PostgreSQL Restore Database

Summary: in this tutorial, you will learn how to restore a database by using PostgreSQL
restore tools including pg_restore and psql .
Before restoring a database, you need to terminate all connections to that database and
prepare the backup file. In PostgreSQL, you can restore a database in two ways:

 Using psql to restore plain SQL script file generated

by pg_dump and pg_dumpall tools.
  Using pg_restore to restore tar file and directory format created by
the pg_dump tool
How to restore databases using psql
The psql utility allows you to restore SQL script file generated by
the pg_dump , pg_dumpall or any other tools that generates compatible backed up files. By
using the psql tool, you have to execute the entire script.
To restore a full backup and ignore any error occurred during the restoration process, you
use the following command:

1 >psql -U username -f backupfile.sql

If you want to stop restoring database in case an error occurs, you use the following
command:

1 >psql -U username --set ON_ERROR_STOP=on -f backupfile

Notice that we have added an additional option --set ON_ERROR_STOP=on
If you backup specific database objects in a particular database, you can restore them using
the following command:

1 >psql -U username -d database_name -f objects.sql

How to restore databases using pg_restore

Besides psql tool, you can use pg_restore program to restore databases backed up by
the pg_dump or pg_dumpall tools. With pg_restore program, you have various options for
restoration databases, for example:

 The pg_restore allows you to perform parallel restores using the -j option to
specify the number of threads for restoration. Each thread restores a separate table
simultaneously, which speeds up the process dramatically. Currently,
the pg_restore support this option for the only custom file format.
 The pg_restore enables you to restore specific database objects in a backup file
that contains the full database.
 The pg_restore can take a database backed up in the older version and restore it in
the newer version.
Let’s create a new database named newdvdrental for practicing with the pg_restore tool.
1 CREATE DATABASE newdvdrental;
You can restore the dvdrental database in tar file format generated by the pg_dump tool
in the PostgreSQL backup database tutorial using the following command:
1 >pg_restore --dbname=newdvdrental --verbose c:\pgbackup\dvdrental.tar
If you restore the database, which is the same as the one that you backed up, you can use
the following command:

1 >pg_restore --dbname=dvdrental --create --verbose c:\pgbackup\dvdrental.tar

As PostgreSQL 9.2, you can use the --section option to restore table structure only. This
allows you to use the new database as the template for creating other databases.
First, you can create a new database named dvdrental_tpl .
1 CREATE DATABASE dvdrental_tpl;
Second, we can restore table structure only from the dvdrental.tar backup file by using the
following command:

1 >pg_restore --dbname=dvdrental_tpl --section=pre-data c:\pgbackup\dvdrental.tar

PostgreSQL restore databases using pgAdmin tool

If you want to run the pg_restore via an intuitive user interface instead of the command
line, you can use the pgAdmin restore tool.
The following example demonstrates how to restore the dvdrental database from
the dvdrental.tar file
First, drop the existing dvdrental database:
1 DROP DATABASE dvdrental;
Second, create an empty dvdrental database:
1 CREATE DATABASE dvdrental;
Third, choose the dvdrental database, right mouse click and choose
the Restore... menu item. A dialog that provides restore options displays.

Fourth, choose appropriate options such as backed up file, user, restore options, and click
the Restore button to start restoring the database.
pgAdmin displays log messages in the Messages tab. If the restoration completed
successfully, click the Done button to finish.
You can verify the restoration by checking the dvdrental database:

In this tutorial, we have shown you practical ways to restore databases by using
PostgreSQL restore tools.

Further Reading
 http://www.postgresql.org/docs/9.2/static/app-pgrestore.html – pg_restore tool
documentation
Related Tutorials
 Backing Up Databases Using PostgreSQL Backup Tools

Tablespace management
PostgreSQL tablespaces allow you to control how data stored in the file system. The
tablespaces are very useful in many cases such as managing large tables and improving
database performance. In this section, we will show you how to manage tablespaces in
PostgreSQL effectively.

 PostgreSQL Creating Tablespace – introduces you to PostgreSQL tablespace and

shows you how to create tablespaces by using CREATE TABLESPACE statement.
 PostgreSQL Changing Tablespace – shows you how to rename, change owner and
set parameter for a tablespace by using ALTER TABLESPACE statement.
 PostgreSQL Deleting Tablespaces – learns how to delete tablespaces by using DROP
TABLESPACE statement.

PostgreSQL Creating Tablespaces

Summary: in this tutorial, you will learn how to create tablespaces by using
the PostgreSQL CREATE TABLESPACE statement.

Introduction to PostgreSQL tablespace

A tablespace is a location on disk where PostgreSQL stores data files containing database
objects e.g., indexes., tables, etc. PostgreSQL uses a tablespace to map a logical name to
a physical location on disk.

PostgreSQL comes with two default tablespaces:

 pg_default tablespace stores all user data.

 pg_global tablespace stores all global data.
The tablespaces allow you to control the disk layout of PostgreSQL. There are two main
advantages of using tablespaces:

 First, if a partition on which the cluster was initialized is out of space, you can create a
new tablespace on a different partition and use it until you reconfigure the system.
 Second, you can use the statistics of database objects’ usages to optimize the
performance of databases. For example, you can place the frequent access indexes
 or tables on devices that perform very fast e.g., solid state devices, and put the tables
containing archive data which is rarely used on a slower devices.

PostgreSQL CREATE TABLESPACE statement

To create new tablespaces, you use CREATE TABLESPACE statement as follows:
1 CREATE TABLESPACE tablespace_name

2 OWNER user_name

3 LOCATION directory_path;

The name of the tablespace should not begin with pg_, because these names are reversed
for the system tablespaces.

By default, the user who executed the CREATE TABLESPACE is the owner of the tablespace.
The statement also allows assign the ownership of tablespace to another user specified in
the ONWER clause.
The directory_path is the absolute path to an empty directory used for the tablespace.
PostgreSQL system user must own this directory in order to read and write data into it.

Once a tablespace is created, you can specify it in the CREATE DATABASE, CREATE
TABLE and CREATE INDEX statements to store data files of the objects in the tablespace.

PostgreSQL create tablespace examples

The following statement creates a new tablespace named dvdrental that has the physical
location is c:/data/dvdrental.

1 CREATE TABLESPACE dvdrental LOCATION 'c:\data\dvdrental';

Notice that we used the Unix-style slashes for the directory path.

To create new tablespace in pgAdmin, follow the steps below:

First, from the Tablespaces, right mouse click and choose New Tablespaces… menu item.
A new Window will display.
Second, enter the name and owner of the tablespace in the properties tab, as well as the
location in the Definition tab.
Click on the SQL tab, you will see the CREATE TABLESPACE command in detail.
Third, click OK button to create the tablespace.

Creating tablespace in UNIX

When you create a tablespace in UNIX systems, a permission error may occur even 777
permission granted to the tablespace directory. To fix this issue, you need to change to
owner of the data directory to postgres user by using the chwon command as the following:
1 # chown postgres /usr/data/tablespace_dir
It changes the owner of the /usr/data/tablespace_dir directory to postgres user. Once
the postgres user takes over data directory ownership, it will remove all other accesses
e.g., 700.
In this tutorial, we have shown you how to create new tablespaces by using the PostgreSQL
CREATE TABLE statement.

PostgreSQL ALTER TABLESPACE

Summary: in this tutorial, you will learn how to change the definition of a tablespace by
using the PostgreSQL ALTER TABLESPACE statement.

Introduction to ALTER TABLESPACE statement

Once a tablespace is created, you can change its definition by using the ALTER
TABLESPACE as shown below:
1 ALTER TABLESPACE action;

PostgreSQL provides some actions such as renaming tablespace name, changing the
owner and setting tablespace’s parameters.

To change the name of the tablespace, use the following statement:

1 ALTER TABLESPACE tablespace_name RENAME TO new_name;

To change the owner of the tablespace, use the following statement:

1 ALTER TABLESPACE tablespace_name OWNER TO new_owner;

You can also change the tablespace’s parameters including seq_page_cost and
random_page_cost, which specify the cost of reading pages from tables in the tablespace.

1 ALTER TABLESPACE tablespace_name SET parameter = value;

To execute the ALTER TABLESPACE statement, you must be a superuser or the owner of the
tablespace.
Currently, PostgreSQL does not support change the location of the tablespace.

Notice that ALTER TABLESPACE statement is a PostgreSQL extension.

PostgreSQL ALTER TABLESPACE examples
The following statement renames dvdrental tablespace to dvdrental_raid :
1 ALTER TABLESPACE dvdrental RENAME TO dvdrental_raid;
To change the owner of the dvdrental_raid from postgres to hr , use the following
statement:
1 ALTER TABLESPACE dvdrental_raid OWNER to hr;

In this tutorial, you have learned how to change the tablespace’s definition by using the
PostgreSQL ALTER TABLESPACE statement.

Deleting Tablespaces Using PostgreSQL

DROP TABLESPACE Statement
Summary: in this tutorial, you will learn how to remove a tablespace by using
the PostgreSQL DROP TABLESPACE statement.

Introduction to DROP TABLESPACE statement

To remove a tablespace, you use DROP TABLESPACE statement as follows:
1 DROP TABLESPACE IF EXISTS tablespace_name;

You need to specify the name of the tablespace after the DROP TABLESPACE clause. The IF
EXISTS helps you avoid the error of removing a non-existent tablespace.
Only tablespace owner or the superuser can delete the tablespace.

Before deleting the tablespace, make sure that it is empty, which means there are no
database objects inside it.

PostgreSQL DROP TABLESPACE example

First, create a new tablespace named demo and maps it to the c:\data\demo directory .
1 CREATE TABLESPACE demo LOCATION 'c:/data/demo';

Second, create a new database named dbdemo and set its tablespace to demo :
1 CREATE DATABASE dbdemo TABLESPACE = demo;

Third, create a new table named test in the dbdemo and set it tablespace to demo :
1 CREATE TABLE test (

2 ID serial PRIMARY KEY,

3 title VARCHAR (255) NOT NULL

4 ) TABLESPACE demo;

You can get all objects in the demo tablespace by using the following query:
1 SELECT
2 ts.spcname,

3 cl.relname

4 FROM

5 pg_class cl

6 JOIN pg_tablespace ts ON cl.reltablespace = ts.oid

7 WHERE

8 ts.spcname = 'demo';

Fourth, try to delete the demo tablespace:

1 DROP TABLESPACE demo;

We got an error message:

1 [Err] ERROR: tablespace "demo" is not empty

Because the demo tablespace is not empty, we could not delete it.

Fifth, login to the postgres database and delete the dbdemo database:
1 DROP DATABASE dbdemo;

Instead of deleting the database, you can move it to the another tablespace
e.g., pg_default by using the ALTER TABLE statement as follows:
1 ALTER DATABASE dbdemo

2 SET TABLESPACE = pg_default;

Sixth, delete the demo tablespace again:

1 DROP TABLESPACE demo;

It worked. The demo tablespace was deleted.

In this tutorial, we have shown you how step by step how to delete tablespace by using the
PostgreSQL DROP TABLESPACE statement.

PostgreSQL tips
  PostgreSQL Reset Password – shows you how to reset forgotten password of
the postgres user.
 psql Commands – gives you the most common psql command to help you query data
from PostgreSQL faster and more effective.
 PostgreSQL Describe Table – gets information on a particular table.
 PostgreSQL Show Databases – lists all databases in the current database server
 PostgreSQL Show Tables – shows all tables in the current database.

Reset Forgotten Password For postgres User

Summary: in this tutorial, we will show you step by step how to reset forgotten password of
postgres user in PostgreSQL.

For some reasons, after installing PostgreSQL, you may forget the password for
the postgres user. In this case, you need to know how to reset the password.
PostgreSQL uses the pg_hba.conf configuration file that is stored in the database data
directory to control the client authentication. HBA means host-based authentication. To
reset the password for the postgres user, you need to modify some parameters in this
configuration file.
Step 1. Backup the pg_dba.conf file by copying it to a different location or just rename it
to pg_dba_bk.conf
Step 2. Edit the pg_dba.conf file by adding the following line as the first line after the
comment lines. The comment line starts with the # sign.
1 local all all trust

If your PostgreSQL installation does not support local, which indicates UNIX sockets, for
example, if you install PostgreSQL on Windows OS. If you use local in this case, you cannot
start PostgreSQL service. In this situation, you need to use the following entry in
the pg_hba.conf file:
1 host all postgres 127.0.0.1/32 trust

This step ensures that you can log into PostgreSQL database server without using the
password.

Step 3. Restart PostgreSQL server e.g., in Linux, you use the following command:

1 sudo /etc/init.d/postgresql restart

Step 4. Connect to PostgreSQL database server and change the password of

the postgres user.
1 ALTER USER postgres with password 'very_secure_password';

Step 5. Restore the pg_db.conf file and restart the server, and connect to the PostgreSQL
database server with new password.
1 sudo /etc/init.d/postgresql restart

In this tutorial, we have shown you how to reset the forgotten password of
the postgres user.

17 Practical psql Commands That You Don’t

Want To Miss
Summary: in this tutorial, we give you a list of common psql commands that helps you
query data from PostgreSQL database server faster and more effective.

Connect to PostgreSQL database

The following command connects to a database under a specific user. After
pressing Enter PostgreSQL will ask for the password of the user.
1 psql -d database -U user -W
For example, to connect to dvdrental database under postgres user, you use the
following command:
1 C:\Program Files\PostgreSQL\9.5\bin>psql -d dvdrental -U postgres -W
2 Password for user postgres:
3 dvdrental=#
If you want to connect to a database that resides on another host, you add the -h option as
follows:

1 psql -h host -d database -U user -W

In case you want to use SSL mode for the connection, just specify it in the command as the
following command:

1 psql -U user -h host "dbname=db sslmode=require"

Switch connection to a new database

Once you are connected to a database, you can switch the connection to a new database
under a user specified by user . The previous connection will be closed. If you omit
the user parameter, the current user is assumed.
1 \c dbname username
The following command connects to dvdrental database under postgres user:
1 postgres=# \c dvdrental
2 You are now connected to database "dvdrental" as user "postgres".
3 dvdrental=#

List available databases

To list all databases in the current PostgreSQL database server, you use \l command:
1 \l

List available tables

To list all tables in the current database, you use \dt command:
1 \dt
Note that this command shows only table in the current connected database.

Describe a table
To describe a table such as a column, type, modifiers of columns, etc., you use the
following command:

1 \d table_name

List available schema

To list all schema of the currently connected database, you use the \dn command.
1 \dn

List available functions

To list available functions in the current database, you use the \df command.
1 \df

List available views

To list available views in the current database, you use the \dv command.
1 \dv

List users and their roles

To list all users and their assign roles, you use \du command:
1 \du

Execute the previous command

To retrieve the current version of PostgreSQL server, you use the version() function as
follows:
1 SELECT version();
Now, you want to save time typing the previous command again, you can use \g command
to execute the previous command:
1 \g
psql executes the previous command again, which is the SELECT statement,.

Command history
To display command history, you use the \s command.
1 \s
If you want to save the command history to a file, you need to specify the file name followed
the \s command as follows:
1 \s filename
Execute psql commands from a file
In case you want to execute psql commands from a file, you use \i command as follows:
1 \i filename

Get help on psql commands

To know all available psql commands, you use the \? command.
1 \?
To get help on specific PostgreSQL statement, you use the \h command.
For example, if you want to know detailed information on ALTER TABLE statement, you use
the following command:

1 \h ALTER TABLE

Turn on query execution time

To turn on query execution time, you use the \timing command.
1 dvdrental=# \timing
2 Timing is on.
3 dvdrental=# select count(*) from film;
4 count
5 -------
6 1000
7 (1 row)
8
9 Time: 1.495 ms
10 dvdrental=#
You use the same command \timing to turn it off.
1 dvdrental=# \timing
2 Timing is off.
3 dvdrental=#

Edit command in your own editor

It is very handy if you can type the command in your favorite editor. To do this in psql,
you \e command. After issuing the command, psql will open the text editor defined by your
EDITOR environment variable and place the most recent command that you entered in psql
into the editor.
After you type the command in the editor, save it, and close the editor, psql will execute the
command and return the result.

It is more useful when you edit a function in the editor.

1 \ef [function name]

Switch output options
psql supports some types of output format and allows you to customize how the output is
formatted on fly.

 \a command switches from aligned to non-aligned column output.

 \H command formats the output to HTML format.
Quit psql
To quit psql, you use \q command and press e nter to exit psql.
1 \q
In this tutorial, we have shown you how to use psql commands to perform various
commonly used tasks.

Related Tutorials
 PostgreSQL Describe Table
 PostgreSQL List Users

PostgreSQL Describe Table

This tutorial shows you how to use the psql tool and information_schema to describe
table in PostgreSQL.
If you are using MySQL, you use the DESCRIBE statement to find the information on the
columns of a particular table. PostgreSQL does not provide the DESCRIBE statement.
However, you can query the information on columns of a table in a couple of ways.
PostgreSQL DESCRIBE TABLE using psql
First, connect to PostgreSQL server, the database dvdrental .
Second, issue the command \d table_name or \d+ table_name to find the information on
columns of a table. The following example queries information on columns of
the city table.
1 Server [localhost]:
2 Database [postgres]: dvdrental
3 Port [5432]:
4 Username [postgres]:
5 psql (9.4.2)
6
7 dvdrental=# \d city
8 Table "public.city"
9 Column | Type | Modifiers
10 -------------+-----------------------------+--------------------------------------------------------
11 city_id | integer | not null default nextval('city_city_id_seq'::regclass)
12 city | character varying(50) | not null
13 country_id | smallint | not null
14 last_update | timestamp without time zone | not null default now()
15 Indexes:
16 "city_pkey" PRIMARY KEY, btree (city_id)
17 "idx_fk_country_id" btree (country_id)
18 Foreign-key constraints:
19 "fk_city" FOREIGN KEY (country_id) REFERENCES country(country_id)
20 Referenced by:
21 TABLE "address" CONSTRAINT "fk_address_city" FOREIGN KEY (city_id) REFERENCES city(city_id)
22 Triggers:
23 last_updated BEFORE UPDATE ON city FOR EACH ROW EXECUTE PROCEDURE last_updated()
24
25
26 dvdrental=#
The command issued a lot of information on the columns of the city table. In addition, it
also returned indexes, foreign key constraints, and triggers.
If you just want to know a list of columns of a table, you can use the second way.

PostgreSQL DESCRIBE TABLE using information_schema

In this way, you just use the SELECT statement to query the column_names of
the columns table in the information_schema database.
For example, following query returns all column names of the city table:
1 SELECT
2 COLUMN_NAME
3 FROM
4 information_schema.COLUMNS
5 WHERE
6 TABLE_NAME = 'city';
In this tutorial, we have shown you how to query information on columns of a particular table
using the psql tool and information_schema .
Related Tutorials
 PostgreSQL Show Tables
 PostgreSQL Show Databases
 17 Practical psql Commands That You Don't Want To Miss

PostgreSQL Show Databases

In this tutorial, we will show you how to show databases in a PostgreSQL database server.

In MySQL, you can show all databases in the database server using SHOW
DATABASES statement. PostgreSQL does not provide this statement directly but offers you
something similar. PostgreSQL provides you with two ways to show databases.
PostgreSQL listing databases using psql tool
If you are using psql tool to connect to PostgreSQL database server, you can issue
the \l command to shows all databases in the current server as follows:
1 \l
For example, you can connect to the dvdrental sample database, and list all databases in
that server as follows:

1 Server [localhost]:
2 Database [postgres]: dvdrental
3 Port [5432]:
4 Username [postgres]:
5 psql (9.4.2)
6
7 dvdrental=# \l
8 List of databases
9 Name | Owner | Encoding | Collate | Ctype | Access privileges
10 -----------+----------+----------+----------------------------+----------------------------+-----------------------
11 dvdrental | postgres | UTF8 | English_United States.1252 | English_United States.1252 |
12 postgres | postgres | UTF8 | English_United States.1252 | English_United States.1252 |
13 template0 | postgres | UTF8 | English_United States.1252 | English_United States.1252 | =c/postgres +
14 | | | | | postgres=CTc/postgres
15 template1 | postgres | UTF8 | English_United States.1252 | English_United States.1252 | =c/postgres +
16 | | | | | postgres=CTc/postgres
17 (4 rows)
18
19
20 dvdrental=#

PostgreSQL listing databases using SELECT statement

Besides using the \l command, you can use the SELECT statement to query database
names from the pg_database catalog that stores data about all available databases.
1 SELECT
2 datname
3 FROM
4 pg_database;

It shows that we have four databases in the current database server.

Related Tutorials
 PostgreSQL Show Tables
 PostgreSQL List Users
 PostgreSQL Describe Table

PostgreSQL Show Tables

Summary: this tutorial shows you different ways to show tables in PostgreSQL
using psql tool and pg_catalog schema.
If you are coming from MySQL, you may miss the SHOW TABLES statement that displays all
tables in a specific database. PostgreSQL does not provide the SHOW TABLES statement
directly but give you something similar.
PostgreSQL show tables using psql
If you are using psql, you can use the following command to show tables in the current
database.

1 \dt
For example, you can connect to the dvdrental database and show all tables as follows:

1 Server [localhost]:
2 Database [postgres]: dvdrental
3 Port [5432]:
4 Username [postgres]:
5 psql (9.4.2)
6
7 dvdrental=# \dt
8 List of relations
9 Schema | Name | Type | Owner
10 --------+---------------+-------+----------
11 public | actor | table | postgres
12 public | address | table | postgres
13 public | category | table | postgres
14 public | city | table | postgres
15 public | country | table | postgres
16 public | customer | table | postgres
17 public | film | table | postgres
18 public | film_actor | table | postgres
19 public | film_category | table | postgres
20 public | inventory | table | postgres
21 public | language | table | postgres
22 public | payment | table | postgres
23 public | persons | table | postgres
24 public | rental | table | postgres
25 public | staff | table | postgres
26 public | store | table | postgres
27 (16 rows)
28
29
30 dvdrental=#

PostgreSQL show tables using pg_catalog schema

Another way to show tables in PostgreSQL is to use SELECT statement to query data from
the PostgreSQL catalog as follows:

1 SELECT
2 *
3 FROM
4 pg_catalog.pg_tables
5 WHERE
6 schemaname != 'pg_catalog'
7 AND schemaname != 'information_schema';

We used condition in the WHERE clause to filter system tables. If you omit
the WHERE clause, you will get many tables that are the system tables, which you may not
want to.
Related Tutorials
  PostgreSQL Show Databases
 PostgreSQL List Users
 PostgreSQL Describe Table

PostgreSQL Architecture
================

Template0: it’s a standard system database; this database contains the same data as the initial
contents of template1, that is, only the standard objects predefined by your version

of PostgreSQL. template0 should never be changed after the database cluster has been

initialized. By instructing CREATE DATABASE to copy template0 instead

of template1, you can create a "virgin" user database that contains none of the site-local

additions in template1. This is particularly handy when restoring a pg_dump dump: the
dump script should be restored in a virgin database to ensure that one recreates the correct contents
of the dumped database, without conflicting with objects that might have been added
to template1 later on.
Template1: CREATE DATABASE actually works by copying an existing database. By
default, it copies the standard system database named template1. Thus that database is

the "template" from which new databases are made. If you add objects to template1, these
objects will be copied into subsequently created user databases. This behavior allows site-local
modifications to the standard set of objects in databases. For example, if you install the procedural
language PL/Perl in template1, it will automatically be available in user databases without any
extra action being taken when those databases are created.

Working Process:
===========
When you start PostgreSQL, The Postmaster starts first and allocates the shared memory. It
also accepts connections and spins off a backend for each new connection. So each backend
(server process) gets its pointers to shared memory from the postmaster. It is pretty
disastrous if the postmaster dies with backends still running, so we have it do as little as
possible, so that there isn't as much which can crash it. Postgres does have a pool of shared
memory; however, it does not have a library or dictionary cache stored in that memory.
This means that statements do need to be parsed and planned every time they are entered.
If parse/plan overhead is an issue, we suggest the use of prepared statements. While Oracle
is able to avoid the repeated parse/plan overhead, it must still do enough analysis of the
query to determine whether the information is present in the library cache, which also
consumes some time and CPU resources. The parser is quite lightweight, so we feel that the
overhead of parsing the query each time is acceptable.

1. Shared Memory:
----------------------
SHARED BUFFERS -- The biggest chunk of shared memory is shared_buffers. When pages
from a table or index are read from the OS, they are read into shared_buffers, and the
backends reference the pages and their contents right there in shared memory. An
exception is temporary tables, where (since only the creating backend can reference the
temp table) data is accessed in temp_buffer space as much as possible(temp_buffers is
separate. It is not in shared memory). It's faster to access process-local memory like that
because you don't need to worry about pinning or locking the data, since you are not
sharing it.
WAL BUFFERS -- are for buffering data to be written to the WAL files.
CLOG BUFFERS -- are one of the SLRU-style buffers oriented toward circular "rings" of data,
like which transaction numbers have been committed or rolled back.
LOCK SPACE -- Memory structures in shared memory are generally protected by
"lightweight" locks, which are in shared memory. Tables are protected by "heavyweight"
locks which are also in shared memory (and themselves protected by lightweight locks). Of
course, lightweight locks are protected by spinlocks. It gets very complicated and fussy. :-)
OTHER BUFFERS -- are probably mostly SLRU buffers besides CLOG (which was the first
user of the SLRU system). SLRU is good for data where you mostly want to use recently
accessed data and then you are done with it relatively quickly.
The opposite of shared memory is process-local memory -- only the one process that
allocates it can access it. Each SLRU system has a separate subdirectory. Shared memory is
memory that all of the backend server processes can directly access. To prevent chaos,
access to shared memory must follow some rules which tends to make it a little slower, like
locking areas of memory a process will be using. Process-local memory is allocated by one
backend server process, and the other backend server processes can't see it or use it, so it's
faster to access, and no worries about another process trashing it while you're using it.
Getting back to the shared memory uses I hadn't talked about, CLOG buffers and SLRU
buffers like multixact, notify, subtrans, serial, etc. use buffers in memory for recently
accessed data, but spill to disk in those subdirectories beyond a limited number of buffers.

2. Utility Processes:
-----------------------
With a default configuration I see the postmaster, the checkpointer process, the writer
process, the wal writer process, the autovacuum launcher process, and the stats collector
process. I think you will see more processes running if you turn on archiving or streaming
replication. You might also get a process for writing the server log, depending on
configuration. As their name say, WRITER -- process is responsible to write the dirty buffers
to data files, CHECKPOINTER -- process is for checkpoint, WAL WRITER -- is for writing the
dirty buffers in WAL buffers to WAL files, AUTOVACUUM LAUNCHER -- process lauches
autovacuum when require(depends on your autovacuum settings in postgresql.conf file) and
STATS COLLECTOR -- process to collect the statistics of objects in the database require by
Optimizer to improve the performance.

The checkpointer process is responsible for creating safe points from which a recovery can
begin; the background writer tries to keep some pages available for re-use so that
processes running queries don't need to wait for page writes in order to have free spots to
use in shared buffers. Both checkpointer and writer processes writes to the same files,
however the checkpointer writes all data that was dirty as of a certain time (the start of
the checkpoint) regardless of how often it was used since dirtied, and the background writer
writes data that hasn't been used recently, regardless of when it was first dirtied. Neither
knows or cares whether the data being written was committed, rolled back, or still in
progress.

3. Directory Structure:
--------------------------
All the data needed for a database cluster is stored within the cluster's data directory,
commonly referred to as PGDATA. You can get the detailed description at below link:
http://www.enterprisedb.com/docs/en/9.2/pg/storage-file-layout.html

I see the diagram left out the one I would like to add: pg_serial. pg_serial is used to track
summarized information about committed serializable transactions which might still become
part of a serialization failure rolling back some not-yet-committed transaction to protect
data integrity.
The catalog cache is information from the system tables which describes the tables,
indexes, views, etc. in the database. If you had to re-read that from the system tables each
time, it would be slow. Even shared memory would be clumsy for that, so each backend
process has its own cache of system catalog data for fast lookup. When anything changes,
all backends are sent a signal to update or reload their cache data. When pages are read or
written, they go through the OS cache, which is not directly under PostgreSQL control. The
optimizer needs to keep track of a lot of information while it parses and plans a query,
which is why that is shown. A plan has execution nodes, some of which may need to use
memory; that is where work_mem comes in -- a sort or hash table (as examples) will try
not to exceed work_mem *for that node*. It is significant that one query might use quite a
few nodes which each allocate memory up to work_mem. But since most queries are
simpler and might not use any work_mem allocations, people often do their calculations
based on an expected maximum of one allocation per backend (i.e., per connection). But
that could be off by quite a bit if all connections might be running queries with five nodes
allocating memory.

It is worth noting that if there is enough RAM on the machine to have a good-sized OS
cache, a PostgreSQL page read will often just be a copy from system cache to pg
shared_buffers, and a page write will often just be a copy from pg shared_buffers to the
system cache. The fsync of tables which is part of the checkpoint process is when they are
actually written from the OS to the storage system. But even there a server may have a
battery-backed RAM cache, so the OS write to storage is often just a copy in RAM.... unless
there is so much writing that the RAID controller's cache fills, at which point writes suddenly
become hundreds of times slower than they were.

Other interesting dynamics: pg will try to minimize disk writes by hanging onto dirty buffers
(ones which have logically been updated) before writing them to the OS. But buffers may
need to be written so they can be freed so that a new read or write has a buffer to use. If a
request to read a page or write to a new buffer can't find an idle page, the query might
need to write a buffer dirtied by some other backend before it can do its read (or whatever).
The background writer can help with this. It tries to watch how fast new pages are being
requested and write out dirty pages at a rate which will stay ahead of demand.