Thomas Lockhart Tom Ivar Helbekkmo 1999-05-27 The purpose of documentation is to make Postgres easier to learn, use, and extend.. The documentation set should describe the Postgres system, language, and interfaces. It should be able to answer common questions and to allow a user to find those answers on his own without resorting to mailing list support. Postgres has four primary documentation formats: Plain text for pre-installation information. HTML, for on-line browsing and reference. Hardcopy (Postscript or PDF), for in-depth reading and reference. man pages, for quick reference. File Description ./COPYRIGHTCopyright notice ./INSTALL Installation instructions (text from sgml->rtf->text) ./README Introductory info ./register.txt Registration message during make ./doc/bug.template Bug report template ./doc/postgres.tar.gz Integrated docs (HTML) ./doc/programmer.ps.gz Programmer's Guide (Postscript) ./doc/programmer.tar.gz Programmer's Guide (HTML) ./doc/reference.ps.gz Reference Manual (Postscript) ./doc/reference.tar.gz Reference Manual (HTML) ./doc/tutorial.ps.gz Introduction (Postscript) ./doc/tutorial.tar.gz Introduction (HTML) ./doc/user.ps.gz User's Guide (Postscript) ./doc/user.tar.gz User's Guide (HTML) There are man pages available, as well as a large number of plain-text README-type files throughout the Postgres source tree. Packaged documentation is available in both HTML and Postscript formats. These are available as part of the standard Postgres installation. We discuss here working with the documentation sources and generating documentation packages. The documentation sources are written using SGML markup of plain text files. The purpose of DocBook SGML is to allow an author to specify the structure and content of a technical document (using the DocBook DTD), and to have a document style define how that content is rendered into a final form (e.g. using Norm Walsh's Modular Style Sheets). See Introduction to DocBook for a nice "quickstart" summary of DocBook features. DocBook Elements provides a powerful cross-reference for features of DocBook. This documentation set is constructed using several tools, including James Clark's jade and Norm Walsh's Modular DocBook Stylesheets. Currently, hardcopy is produced by importing Rich Text Format (RTF) output from jade into ApplixWare for minor formatting fixups, then exporting as a Postscript file. TeX is a supported format for jade output, but is not used at this time for several reasons, including the inability to make minor format fixes before committing to hardcopy and generally inadequate table support in the TeX stylesheets. Documentation sources include plain text files, man pages, and html. However, most new Postgres documentation will be written using the Standard Generalized Markup Language (SGML) DocBook Document Type Definition (DTD). Much of the existing documentation has been or will be converted to SGML. The purpose of SGML is to allow an author to specify the structure and content of a document (e.g. using the DocBook DTD), and to have the document style define how that content is rendered into a final form (e.g. using Norm Walsh's stylesheets). Documentation has accumulated from several sources. As we integrate and assimilate existing documentation into a coherent documentation set, the older versions will become obsolete and will be removed from the distribution. However, this will not happen immediately, and will not happen to all documents at the same time. To ease the transition, and to help guide developers and writers, we have defined a transition roadmap. There are currently five separate documents written in DocBook. Each document has a container source document which defines the DocBook environment and other document source files. These primary source files are located in doc/src/sgml/, along with many of the other source files used for the documentation. The primary source files are: postgres.sgml This is the integrated document, including all other documents as parts. Output is generated in HTML since the browser interface makes it easy to move around all of the documentation by just clicking. The other documents are available in both HTML and hardcopy. tutorial.sgml The introductory tutorial, with examples. Does not include programming topics, and is intended to help a reader unfamiliar with SQL. This is the "getting started" document. user.sgml The User's Guide. Includes information on data types and user-level interfaces. This is the place to put information on "why". reference.sgml The Reference Manual. Includes Postgres SQL syntax. This is the place to put information on "how". programming.sgml The Programmer's Guide. Includes information on Postgres extensibility and on the programming interfaces. admin.sgml The Administrator's Guide. Include installation and release notes. DocBook has a rich set of tags and constructs, and a suprisingly large percentage are directly and obviously useful for well-formed documentation. The Postgres documentation set has only recently been adapted to SGML, and in the near future several sections of the set will be selected and maintained as prototypical examples of DocBook usage. Also, a short summary of DocBook tags will be included below. The current Postgres documentation set was written using a plain text editor (or emacs/psgml; see below) with the content marked up using SGML DocBook tags. SGML and DocBook do not suffer from an oversupply of open-source authoring tools. The most common toolset is the emacs/xemacs editing package with the psgml feature extension. On some systems (e.g. RedHat Linux) these tools are provided in a typical full installation. emacs (and xemacs) have an SGML major mode. When properly configured, this will allow you to use emacs to insert tags and check markup consistancy. Put the following in your ~/.emacs environment file (adjusting the path names to be appropriate for your system): ; ********** for SGML mode (psgml) (setq sgml-catalog-files "/usr/lib/sgml/CATALOG") (setq sgml-local-catalogs "/usr/lib/sgml/CATALOG") (autoload 'sgml-mode "psgml" "Major mode to edit SGML files." t ) and add an entry in the same file for SGML into the (existing) definition for auto-mode-alist: (setq auto-mode-alist '(("\\.sgml$" . sgml-mode) )) Each SGML source file has the following block at the end of the file: !-- Keep this comment at the end of the file Local variables: mode: sgml sgml-omittag:t sgml-shorttag:t sgml-minimize-attributes:nil sgml-always-quote-attributes:t sgml-indent-step:1 sgml-indent-data:t sgml-parent-document:nil sgml-default-dtd-file:"./reference.ced" sgml-exposed-tags:nil sgml-local-catalogs:("/usr/lib/sgml/catalog") sgml-local-ecat-files:nil End: -- The Postgres distribution includes a parsed DTD definitions file reference.ced. You may find that When using emacs/psgml, a comfortable way of working with these separate files of book parts is to insert a proper DOCTYPE declaration while you're editing them. If you are working on this source, for instance, it's an appendix chapter, so you would specify the document as an "appendix" instance of a DocBook document by making the first line look like this: !doctype appendix PUBLIC "-//Davenport//DTD DocBook V3.0//EN" This means that anything and everything that reads SGML will get it right, and I can verify the document with "nsgmls -s docguide.sgml". GNU make is used to build documentation from the DocBook sources. There are a few environment definitions which may need to be set or modified for your installation. The Makefile looks for doc/../src/Makefile and (implicitly) for doc/../src/Makefile.custom to obtain environment information. On my system, the src/Makefile.custom looks like # Makefile.custom # Thomas Lockhart 1998-03-01 POSTGRESDIR= /opt/postgres/current CFLAGS+= -m486 YFLAGS+= -v # documentation HSTYLE= /home/lockhart/SGML/db143.d/docbook/html PSTYLE= /home/lockhart/SGML/db143.d/docbook/print where HSTYLE and PSTYLE determine the path to docbook.dsl for HTML and hardcopy (print) stylesheets, respectively. These stylesheet file names are for Norm Walsh's Modular Style Sheets; if other stylesheets are used then one can define HDSL and PDSL as the full path and file name for the stylesheet, as is done above for HSTYLE and PSTYLE. On many systems, these stylesheets will be found in packages installed in /usr/lib/sgml/, /usr/share/lib/sgml/, or /usr/local/lib/sgml/. HTML documentation packages can be generated from the SGML source by typing % cd doc/src % make tutorial.tar.gz % make user.tar.gz % make admin.tar.gz % make programmer.tar.gz % make postgres.tar.gz % make install These packages can be installed from the main documentation directory by typing % cd doc % make install We use the docbook2man utility to convert DocBook REFENTRY pages to *roff output suitable for man pages. At the time of writing, the utility required patching to successfully run on the Postgres markup, and we added a small amount of new functionality to allow setting the man page section in the output file name. docbook2man is written in perl, and requires the CPAN package SGMLSpm to run. Also, it requires nsgmls to be available, which is included in the jade distribution. After installing these packages, then simply run $ cd doc/src $ make man which will result in a tar file being generated in the doc/src directory. Install the docbook2man package, available at http://shell.ipoline.com/~elmert/comp/docbook2X/ Install the SGMLSpm perl module, available from CPAN mirrors. Install nsgmls if not already available from your jade installation. The hardcopy Postscript documentation is generated by converting the SGML source code to RTF, then importing into ApplixWare-4.4.1. After a little cleanup (see the following section) the output is "printed" to a postscript file. INSTALL and HISTORY are updated for each release. For historical reasons, these files are in plain text, but are derived from the newer SGML sources. Both INSTALL and HISTORY are generated from existing SGML sources. They are extracted from the same intermediate RTF file. Generate RTF by typing: % cd doc/src/sgml % make installation.rtf Import installation.rtf into Applix Words. Set the page width and margins. Adjust the page width in File.PageSetup to 10 inches. Select all text. Adjust the right margin using the ruler to 9.5 inches. This will give a maximum column width of 79 characters, within the 80 columns upper limit goal. Lop off the parts of the document which are not needed. For INSTALL, remove all release notes from the end of the text, except for those from the current release. For HISTORY, remove all text up to the release notes, preserving and modifying the title and ToC. Export the result as "ASCII Layout". Using emacs or vi, clean up the tabular information in INSTALL. Remove the "mailto" URLs for the porting contributors to shrink the column heights. Several areas are addressed while generating Postscript hardcopy, including RTF repair, ToC generation, and page break adjustments. jade, an integral part of the hardcopy procedure, omits specifying a default style for body text. In the past, this undiagnosed problem led to a long process of Table of Contents (ToC) generation. However, with great help from the ApplixWare folks the symptom was diagnosed and a workaround is available. Generate the RTF input by typing (for example): % cd doc/src/sgml % make tutorial.rtf Repair the RTF file to correctly specify all styles, in particular the default style. The field can be added using vi or the following small sed procedure: #!/bin/sh # fixrtf.sh # Utility to repair slight damage in RTF files generated by jade # Thomas Lockhart <lockhart@alumni.caltech.edu> # for i in $* ; do mv $i $i.orig cat $i.orig | sed 's#\\stylesheet#\\stylesheet{\\s0 Normal;}#' > $i done exit where the script is adding {\s0 Normal;} as the zero-th style in the document. According to ApplixWare, the RTF standard would prohibit adding an implicit zero-th style, though M$Word happens to handle this case. Open a new document in Applix Words and then import the RTF file. Generate a new ToC using ApplixWare. Select the existing ToC lines, from the beginning of the first character on the first line to the last character of the last line. Build a new ToC using Tools.BookBuilding.CreateToC. Select the first three levels of headers for inclusion in the ToC. This will replace the existing lines imported in the RTF with a native ApplixWare ToC. Adjust the ToC formatting by using Format.Style, selecting each of the three ToC styles, and adjusting the indents for First and Left. Use the following values: Style First Indent (inches) Left Indent (inches) TOC-Heading 1 0.6 0.6 TOC-Heading 2 1.0 1.0 TOC-Heading 3 1.4 1.4 Work through the document to: Adjust page breaks. Adjust table column widths. Insert figures into the document. Center each figure on the page using the centering margins button on the ApplixWare toolbar. Not all documents have figures. You can grep the SGML source files for the string "graphic" to identify those parts of the documentation which may have figures. A few figures are replicated in various parts of the documentation. Replace the right-justified page numbers in the Examples and Figures portions of the ToC with correct values. This only takes a few minutes per document. If a bibliography is present, remove the short form reference title from each entry. The DocBook stylesheets from Norm Walsh seem to print these out, even though this is a subset of the information immediately following. Save the document as native Applix Words format to allow easier last minute editing later. "Print" the document to a file in Postscript format. Compress the Postscript file using gzip. Place the compressed file into the doc directory. We have documented experience with three installation methods for the various tools that are needed to process the documentation. One is installation from RPMs on Linux, the second is installation from FreeBSD port, and the last is a general installation from original distributions of the individual tools. These will be described below. There may be some other packaged distributions for these tools. Please report package status to the docs mailing list and we will include that information here. The simplest installation for a RedHat-compatible Linux system uses the RPM set developed by Mark Galassi at Cygnus. It should also be possible to install from sources, as described in a subsequent section. Install RPMs for Jade and related packages. Install Norm Walsh's latest style sheets. Depending on the age of the RPMs, the latest style sheets may be substantially improved from those contained in the RPMs. Update your src/Makefile.custom to include HSTYLE and PSTYLE definitions pointing to the style sheets. There is a full set of ports of the documentation tools available on FreeBSD. In fact, postgresql.org, on which documentation is automatically updated every evening, is a FreeBSD machine. To build the documentation on FreeBSD a number of ports will need to be installed. % cd /usr/ports/devel/gmake && make install % cd /usr/ports/textproc/docproj && make install % cd /usr/ports/textproc/docbook && make install % cd /usr/ports/textproc/dsssl-docbook-modular && make install Set environment variables to access the jade toolset. This was not required for the FreeBSD machine at postgresql.org, so you may not have to do this. export SMGL_ROOT=/usr/local/share/sgml SGML_CATALOG_FILES=/usr/local/share/sgml/jade/catalog SGML_CATALOG_FILES=/usr/local/share/sgml/html/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/iso8879/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/transpec/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/docbook/catalog:$SGML_CATALOG_FILES export SGML_CATALOG_FILES (this is sh/bash syntax; adjust accordingly for csh/tcsh). Make needs some special arguments, or these need to be added to your Makefile.custom: HSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/html/ PSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/print/ Of course you'll need to use gmake rather than just plain 'make' to build. There is a full set of packages of the documentation tools available for Debian. Install jade, docbook, and unzip: apt-get install jade apt-get install docbook apt-get install docbook-stylesheets Install the latest style sheets. Verify that unzip is installed, or install the package: apt-get install unzip Grab the latest stylesheet zipballs from http://www.nwalsh.com/docbook/dsssl and unzip it somewhere (possibly /usr/share). Edit src/Makefile.custom to add appropriate HSTYLE and PSTYLE definitions: HSTYLE= /usr/share/docbook/html PSTYLE= /usr/share/docbook/print This is a brief run-through of the process of obtaining and installing the software you'll need to edit DocBook source with Emacs and process it with Norman Walsh's DSSSL style sheets to create HTML and RTF. The easiest way to obtain the SGML and DocBook tools may be to get sgmltools from sgmltools. sgmltools requires the GNU version of m4. To confirm that you have the correct version of m4 available, try gnum4 --version If you install GNU m4, install it with the name gnum4 and sgmltools will find it. After the install, you will have sgmltools, jade, and Norm Walsh's DocBook style sheets. The instructions below are for installing these tools separately. What you need: A working installation of GCC 2.7.2 A working installation of Emacs 19.19 or later An unzip program for Unix to unpack things What you must fetch: James Clark's Jade (version 1.1 in file jade1_1.zip was current at the time of writing) DocBook version 3.0 Norman Walsh's Modular Stylesheets (version 1.19 was originally used to produce these documents) Lennart Staflin's PSGML (version 1.0.1 in psgml-1.0.1.tar.gz was available at the time of writing) Important URLs: The Jade web page The DocBook web page The Modular Stylesheets web page The PSGML web page Steve Pepper's Whirlwind Guide Robin Cover's database of SGML software Read the installation instructions at the above listed URL. Unzip the distribution kit in a suitable place. The command to do this will be something like unzip -aU jade1_1.zip Jade is not built using GNU autoconf, so you'll need to edit a Makefile yourself. Since James Clark has been good enough to prepare his kit for it, it is a good idea to make a build directory (named for your machine architecture, perhaps) under the main directory of the Jade distribution, copy the file Makefile from the main directory into it, edit it there, and then run make there. However, the Makefile does need to be edited. There is a file called Makefile.jade in the main directory, which is intended to be used with make -f Makefile.jade when building Jade (as opposed to just SP, the SGML parser kit that Jade is built upon). We suggest that you don't do that, though, since there is more that you need to change than what is in Makefile.jade, so you'd have to edit one of them anyway. Go through the Makefile, reading James' instructions and editing as needed. There are various variables that need to be set. Here is a collected summary of the most important ones, with typical values: prefix = /usr/local XDEFINES = -DSGML_CATALOG_FILES_DEFAULT=\"/usr/local/share/sgml/catalog\" XLIBS = -lm RANLIB = ranlib srcdir = .. XLIBDIRS = grove spgrove style XPROGDIRS = jade Note the specification of where to find the default catalog of SGML support files -- you may want to change that to something more suitable for your own installation. If your system doesn't need the above settings for the math library and the ranlib command, leave them as they are in the Makefile. Type make to build Jade and the various SP tools. Once the software is built, make install will do the obvious. You'll want to place the files that make up the DocBook DTD kit in the directory you built Jade to expect them in, which, if you followed our suggestion above, is /usr/local/share/sgml/. In addition to the actual DocBook files, you'll need to have a catalog file in place, for the mapping of document type specifications and external entity references to actual files in that directory. You'll also want the ISO character set mappings, and probably one or more versions of HTML. One way to install the various DTD and support files and set up the catalog file, is to collect them all into the above mentioned directory, use a single file named CATALOG to describe them all, and then create the file catalog as a catalog pointer to the former, by giving it the single line of content: CATALOG /usr/local/share/sgml/CATALOG The CATALOG file should then contain three types of lines. The first is the (optional) SGML declaration, thus: SGMLDECL docbook.dcl Next, the various references to DTD and entity files must be resolved. For the DocBook files, these lines look like this: PUBLIC "-//Davenport//DTD DocBook V3.0//EN" docbook.dtd PUBLIC "-//USA-DOD//DTD Table Model 951010//EN" cals-tbl.dtd PUBLIC "-//Davenport//ELEMENTS DocBook Information Pool V3.0//EN" dbpool.mod PUBLIC "-//Davenport//ELEMENTS DocBook Document Hierarchy V3.0//EN" dbhier.mod PUBLIC "-//Davenport//ENTITIES DocBook Additional General Entities V3.0//EN" dbgenent.mod Of course, a file containing these comes with the DocBook kit. Note that the last item on each of these lines is a file name, given here without a path. You can put the files in subdirectories of your main SGML directory if you like, of course, and modify the reference in the CATALOG file. DocBook also references the ISO character set entities, so you need to fetch and install these (they are available from several sources, and are easily found by way of the URLs listed above), along with catalog entries for all of them, such as: PUBLIC "ISO 8879-1986//ENTITIES Added Latin 1//EN" ISO/ISOlat1 Note how the file name here contains a directory name, showing that we've placed the ISO entity files in a subdirectory named ISO. Again, proper catalog entries should accompany the entity kit you fetch. Read the installation instructions at the above listed URL. To install Norman's style sheets, simply unzip the distribution kit in a suitable place. A good place to dot this would be /usr/local/share, which places the kit in a directory tree under /usr/local/share/docbook. The command will be something like unzip -aU db119.zip One way to test the installation is to build the HTML and RTF forms of the PostgreSQL User's Guide. To build the HTML files, go to the SGML source directory, doc/src/sgml, and say jade -t sgml -d /usr/local/share/docbook/html/docbook.dsl -D ../graphics postgres.sgml book1.htm is the top level node of the output.. To generate the RTF output, ready for importing into your favorite word processing system and printing, type: jade -t rtf -d /usr/local/share/docbook/print/docbook.dsl -D ../graphics postgres.sgml Read the installation instructions at the above listed URL. Unpack the distribution file, run configure, make and make install to put the byte-compiled files and info library in place. Then add the following lines to your /usr/local/share/emacs/site-lisp/site-start.el file to make Emacs properly load PSGML when needed: (setq load-path (cons "/usr/local/share/emacs/site-lisp/psgml" load-path)) (autoload 'sgml-mode "psgml" "Major mode to edit SGML files." t) If you want to use PSGML when editing HTML too, also add this: (setq auto-mode-alist (cons '("\\.s?html?\\'" . sgml-mode) auto-mode-alist)) There is one important thing to note with PSGML: its author assumed that your main SGML DTD directory would be /usr/local/lib/sgml. If, as in the examples in this chapter, you use /usr/local/share/sgml, you have to compensate for this. You can set the SGML_CATALOG_FILES environment variable. You can customize your PSGML installation (its manual tells you how). You can even edit the source file psgml.el before compiling and installing PSGML, changing the hard-coded paths to match your own default. If you want to, you can also install JadeTeX to use TeX as a formatting backend for Jade. Note that this is still quite unpolished software, and will generate printed output that is inferior to what you get from the RTF backend. Still, it works all right, especially for simpler documents that don't use tables, and as both JadeTeX and the style sheets are under continuous improvement, it will certainly get better over time. To install and use JadeTeX, you will need a working installation of TeX and LaTeX2e, including the supported tools and graphics packages, Babel, AMS fonts and AMS-LaTeX, the PSNFSS extension and companion kit of "the 35 fonts", the dvips program for generating PostScript, the macro packages fancyhdr, hyperref, minitoc, url and ot2enc, and of course JadeTeX itself. All of these can be found on your friendly neighborhood CTAN site. JadeTeX does not at the time of writing come with much of an installation guide, but there is a makefile which shows what is needed. It also includes a directory cooked, wherein you'll find some of the macro packages it needs, but not all, and not complete -- at least last we looked. Before building the jadetex.fmt format file, you'll probably want to edit the jadetex.ltx file, to change the configuration of Babel to suit your locality. The line to change looks something like \RequirePackage[german,french,english]{babel}[1997/01/23] and you should obviously list only the languages you actually need, and have configured Babel for. With JadeTeX working, you should be able to generate and format TeX output for the PostgreSQL manuals by giving the commands (as above, in the doc/src/sgml directory) jade -t tex -d /usr/local/share/docbook/print/docbook.dsl -D ../graphics postgres.sgml jadetex postgres.tex jadetex postgres.tex dvips postgres.dvi Of course, when you do this, TeX will stop during the second run, and tell you that its capacity has been exceeded. This is, as far as we can tell, because of the way JadeTeX generates cross referencing information. TeX can, of course, be compiled with larger data structure sizes. The details of this will vary according to your installation. sgml-tools v2.x supports jade and DocBook.