When you run PostgreSQL workloads in production you must have a backup and restore implementation. Even for development instances, which are like production for the developers, a well-tested backup and restore procedure sometimes must be in place. Community PostgreSQL comes with pg_basebackup to help you with creating a full consistent backup of your PostgreSQL cluster. If you want to be able to do point in time recovery (PITR) you need to archive the WAL segments which can later be used to roll forward from a base backup. WAL segments go to the pg_wal directory, but if you have closer look into this directory you will see more than just the WAL segments in case you have enabled archiving.
IF you look at a freshly initialized PostgreSQL cluster the content of pg_wal looks like this:
postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal total 16396 drwx------ 3 postgres postgres 4096 Jul 4 11:47 . drwx------ 20 postgres postgres 4096 Jul 4 11:47 .. -rw------- 1 postgres postgres 16777216 Jul 4 11:52 000000010000000000000001 drwx------ 2 postgres postgres 4096 Jul 4 11:47 archive_status
There is one WAL segment and a directory called “archived_status” which is empty right now:
postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/ total 8 drwx------ 2 postgres postgres 4096 Jul 4 11:47 . drwx------ 3 postgres postgres 4096 Jul 4 11:47 ..
To see what’s going on in this directory we need to enable archiving and tell PostgreSQL what do when a WAL segment needs to be archived:
postgres@debian11:/home/postgres/ [pgdev] mkdir /tmp/archived_wal postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_mode='on'" postgres ALTER SYSTEM postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='test ! -f /tmp/archived_wal/%f && cp %p /tmp/archived_wal/%f'" postgres ALTER SYSTEM
Of course you should not archive to /tmp and you also should not use cp, as this does not guarantee that the archived segment really is written to disk. For the scope of this post it is fine like that.
Changing the archive command can be done online, but enabling or disabling archiving requires a restart:
postgres@debian11:/home/postgres/ [pgdev] pg_ctl restart
Let’s see what happens if we force the current segment to be archived:
postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_switch_wal()" postgres pg_switch_wal --------------- 0/167AF68 (1 row) postgres@debian11:/home/postgres/ [pgdev] ls -la /tmp/archived_wal/ total 16392 drwxr-xr-x 2 postgres postgres 4096 Jul 4 12:08 . drwxrwxrwt 10 root root 4096 Jul 4 12:07 .. -rw------- 1 postgres postgres 16777216 Jul 4 12:08 000000010000000000000001
As expected we see the archived segment in the directory we specified in the archive command. In addition PostgreSQL generated a “.done” file in the archive_status directory in pg_wal:
postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/ total 8 drwx------ 2 postgres postgres 4096 Jul 4 12:08 . drwx------ 3 postgres postgres 4096 Jul 4 12:08 .. -rw------- 1 postgres postgres 0 Jul 4 12:08 000000010000000000000001.done
So, whenever a segment was successfully archived you get the corresponding “.done” file. When do you see “.ready” files then? The answer is simple: Exactly when the opposite happens: Archiving of a segment failed for whatever reason:
postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='/bin/false'" postgres ALTER SYSTEM postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_reload_conf()"; pg_reload_conf ---------------- t (1 row) postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_switch_wal()" postgres pg_switch_wal --------------- 0/2000160 (1 row) postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/ total 8 drwx------ 2 postgres postgres 4096 Jul 4 12:13 . drwx------ 3 postgres postgres 4096 Jul 4 12:12 .. -rw------- 1 postgres postgres 0 Jul 4 12:13 000000010000000000000002.ready
This file will be there until archiving succeeds again:
postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='test ! -f /tmp/archived_wal/%f && cp %p /tmp/archived_wal/%f'" postgres ALTER SYSTEM postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_reload_conf()"; pg_reload_conf ---------------- t (1 row) postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/ total 8 drwx------ 2 postgres postgres 4096 Jul 4 12:15 . drwx------ 3 postgres postgres 4096 Jul 4 12:12 .. -rw------- 1 postgres postgres 0 Jul 4 12:13 000000010000000000000002.done
When you see many “.ready” files this can either mean your archive command is currently failing or load on the system is so high, that archiving can not keep up.
Cet article What are these *.ready and *.done files for in PostgreSQL? est apparu en premier sur Blog dbi services.