Troubleshooting Postmaster Startup Failures There are several common reasons for the postmaster to fail to start up. Check the postmaster's log file, or start it by hand (without redirecting standard output or standard error) to see what complaint messages appear. Some of the possible error messages are reasonably self-explanatory, but here are some that are not: FATAL: StreamServerPort: bind() failed: Address already in use Is another postmaster already running on that port? This usually means just what it suggests: you accidentally started a second postmaster on the same port where one is already running. However, if the kernel error message is not "Address already in use" or some variant of that wording, there may be a different problem. For example, trying to start a postmaster on a reserved port number may draw something like $ postmaster -i -p 666 FATAL: StreamServerPort: bind() failed: Permission denied Is another postmaster already running on that port? IpcMemoryCreate: shmget failed (Invalid argument) key=5440001, size=83918612, permission=600 FATAL 1: ShmemCreate: cannot create region A message like this probably means that your kernel's limit on the size of shared memory areas is smaller than the buffer area that Postgres is trying to create. (Or it could mean that you don't have SysV-style shared memory support configured into your kernel at all.) As a temporary workaround, you can try starting the postmaster with a smaller-than-normal number of buffers (-B switch). You will eventually want to reconfigure your kernel to increase the allowed shared memory size, however. You may see this message when trying to start multiple postmasters on the same machine, if their total space requests exceed the kernel limit. IpcSemaphoreCreate: semget failed (No space left on device) key=5440026, num=16, permission=600 A message like this does not mean that you've run out of disk space; it means that your kernel's limit on the number of SysV semaphores is smaller than the number Postgres wants to create. As above, you may be able to work around the problem by starting the postmaster with a reduced number of backend processes (-N switch), but you'll eventually want to increase the kernel limit. Client Connection Problems Once you have a running postmaster, trying to connect to it with client applications can fail for a variety of reasons. The sample error messages shown here are for clients based on recent versions of libpq --- clients based on other interface libraries may produce other messages with more or less information. connectDB() -- connect() failed: Connection refused Is the postmaster running (with -i) at 'server.joe.com' and accepting connections on TCP/IP port '5432'? This is the generic "I couldn't find a postmaster to talk to" failure. It looks like the above when TCP/IP communication is attempted, or like this when attempting Unix-socket communication to a local postmaster: connectDB() -- connect() failed: No such file or directory Is the postmaster running at 'localhost' and accepting connections on Unix socket '5432'? The last line is useful in verifying that the client is trying to connect where it is supposed to. If there is in fact no postmaster running there, the kernel error message will typically be either "Connection refused" or "No such file or directory", as illustrated. (It is particularly important to realize that "Connection refused" in this context does not mean that the postmaster got your connection request and rejected it --- that case will produce a different message, as shown below.) Other error messages such as "Connection timed out" may indicate more fundamental problems, like lack of network connectivity. No pg_hba.conf entry for host 123.123.123.123, user joeblow, database testdb This is what you are most likely to get if you succeed in contacting a postmaster, but it doesn't want to talk to you. As the message suggests, the postmaster refused the connection request because it found no authorizing entry in its pg_hba.conf configuration file. Password authentication failed for user 'joeblow' Messages like this indicate that you contacted the postmaster, and it's willing to talk to you, but not until you pass the authorization method specified in the pg_hba.conf file. Check the password you're providing, or check your Kerberos or IDENT software if the complaint mentions one of those authentication types. FATAL 1: SetUserId: user 'joeblow' is not in 'pg_shadow' This is another variant of authentication failure: no Postgres create_user command has been executed for the given username. FATAL 1: Database testdb does not exist in pg_database There's no database by that name under the control of this postmaster. Note that if you don't specify a database name, it defaults to your Postgres username, which may or may not be the right thing. NOTICE: Unrecognized variable client_encoding This isn't an error; in fact, it's quite harmless. You'll see this message at startup if you use a client compiled with MULTIBYTE support to connect to a server compiled without it. (The client is trying to tell the server what character set encoding it wants, but the server has no idea what it's talking about.) If the message bothers you, use a client compiled with the same options as the server. Debugging Messages The postmaster occasionally prints out messages which are often helpful during troubleshooting. If you wish to view debugging messages from the postmaster, you can start it with the -d option and redirect the output to the log file: % postmaster -d > pm.log 2>&1 & If you do not wish to see these messages, you can type % postmaster -S and the postmaster will be "S"ilent. No ampersand ("&") is required in this case, since the postmaster automatically detaches from the terminal when -S is specified. pg_options Contributed by Massimo Dal Zotto The optional file data/pg_options contains runtime options used by the backend to control trace messages and other backend tunable parameters. What makes this file interesting is the fact that it is re-read by a backend when it receives a SIGHUP signal, making thus possible to change run-time options on the fly without needing to restart Postgres. The options specified in this file may be debugging flags used by the trace package (backend/utils/misc/trace.c) or numeric parameters which can be used by the backend to control its behaviour. New options and parameters must be defined in backend/utils/misc/trace.c and backend/include/utils/trace.h. pg_options can also be specified with the switch of Postgres: postgres options -T "verbose=2,query,hostlookup-" The functions used for printing errors and debug messages can now make use of the syslog(2) facility. Message printed to stdout or stderr are prefixed by a timestamp containing also the backend pid: #timestamp #pid #message 980127.17:52:14.173 [29271] StartTransactionCommand 980127.17:52:14.174 [29271] ProcessUtility: drop table t; 980127.17:52:14.186 [29271] SIIncNumEntries: table is 70% full 980127.17:52:14.186 [29286] Async_NotifyHandler 980127.17:52:14.186 [29286] Waking up sleeping backend process 980127.19:52:14.292 [29286] Async_NotifyFrontEnd 980127.19:52:14.413 [29286] Async_NotifyFrontEnd done 980127.19:52:14.466 [29286] Async_NotifyHandler done This format improves readability of the logs and allows people to understand exactly which backend is doing what and at which time. It also makes easier to write simple awk or perl scripts which monitor the log to detect database errors or problem, or to compute transaction time statistics. Messages printed to syslog use the log facility LOG_LOCAL0. The use of syslog can be controlled with the syslog pg_option. Unfortunately many functions call directly printf() to print their messages to stdout or stderr and this output can't be redirected to syslog or have timestamps in it. It would be advisable that all calls to printf would be replaced with the PRINTF macro and output to stderr be changed to use EPRINTF instead so that we can control all output in a uniform way. The format of the pg_options file is as follows: # comment option=integer_value # set value for option option # set option = 1 option+ # set option = 1 option- # set option = 0 Note that keyword can also be an abbreviation of the option name defined in backend/utils/misc/trace.c. Refer to for a complete list of option keywords and possible values.