  This file documents the Kpathsea library for path searching.

  Copyright (C) 1993, 94 Karl Berry.

  Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

  Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided also that the
sections entitled "Freedom" and "GNU General Public License" are
included exactly as in the original, and provided that the entire
resulting derived work is distributed under the terms of a permission
notice identical to this one.

  Permission is granted to copy and distribute translations of this
manual into another language, under the above conditions for modified
versions, except that the sections entitled "Freedom" and "GNU General
Public License" may be included in a translation approved by the Free
Software Foundation instead of in the original English.

Kpathsea library
****************

  This manual documents how to install and use the Kpathsea library for
filename lookup.  It corresponds to version 2.1 (released in September
1994).

Introduction
************

  This manual corresponds to version 2.1 of the Kpathsea library,
released in September 1994.

  The library's fundamental purpose is to look up a file in a list of
directories specified by the user, similar to what shells do when
looking up program names to execute.

  The following software, all of which I maintain, uses this library:

   * Dviljk

   * Dvipsk (see Introduction (dvipsk))

   * GNU font utilities (see Introduction (fontu))

   * Web2c (see Introduction (web2c))

   * Xdvik

  The library is still under development. I do not promise to keep the
interface unchanged.  If you have comments or suggestions, please send
them to me (see Reporting bugs. for the address).

  Currently, I distribute the library under the GNU General Public
License (see Copying.).  In short, this means if you write a
program using the library, you must (offer to) distribute the source,
and allow anyone to modify the source and distribute their
modifications.

  If you have a problem with this, contact me. I would consider putting
the library under the GNU Library General Public License, which would
permit you to distribute the source only to the library, not to your
program using it.  But I will only do this if someone actually says they
will not use the library under the GPL conditions, and would use it
under the LGPL.

  If you know enough about TeX to be reading this manual, then you (or
perhaps your institution) should consider joining the TeX Users Group
(if you're already a member, great!).  TUG produces a periodical called
`TUGboat', sponsors an annual meeting (the proceedings of which are
published in `TUGboat'), and arranges courses on TeX for all levels of
users.  Given sufficient funding (which your joining will help) TUG
could sponsor more projects that will benefit the TeX community, such as
a successor to TeX pi .  Anyway, here is the address:

     TeX Users Group
     P.O. Box 869
     Santa Barbara, CA 93102 USA
     phone: (805) 899-4673
     email: `tug@tug.org'

Installation
************

  Here are the basic steps for configuration and installation:

  1. `make install'. This installs the library, header files, and
     documentation. Or `make install-data' to just install the
     architecture-independent files. Or `make install-exec' to just
     install the (binary) archive library file.

     Since I only distribute Kpathsea as part of another package, you
     will probably be doing the above in a top-level directory that
     contains a `Makefile', `kpathsea', and the other package.  But you
     can do the installation in `kpathsea' itself, if you only want to
     install the library, not the other package.

  2. `make distclean'.  This removes all files created by the build.

  See Filename database, for a description of an
externally-generated database that can help speed searches.

  See Debugging, for runtime debugging support that may help track
down problems.

Default paths
=============

  To summarize the chain of events that go into defining the default
paths:

  1. `configure' creates a `Makefile' from each `Makefile.in'.

  2. When Make runs in the `kpathsea' directory, it creates a file
     `texmf.sed' that substitutes the Make value of `$(var)' for a
     string `@var@'. The variables in question are the one that define
     the installation directories.

  3. `texmf.sed' (and a little extra magic--see the `Makefile') is
     applied to `texmf.cnf.in' to generate `texmf.cnf'. This is the
     file that will eventually be installed and used by the programs to
     look up programs.

  4. The definitions in `texmf.cnf' are changed into the form of C
     `#define''s, producing `paths.h'. These values will be the
     compile-time defaults; they are not used unless no `texmf.cnf' file
     can be found at runtime.

     (That's a partial lie: the compile-time defaults are what extra
     `:''s in `texmf.cnf' expand into; but the paths as distributed
     have no extra `:''s, and there's no particular reason for them to.)

  The purpose of this elaborate sequence is to avoid having the same
information in more than one place. If you change the installation
directories or top-level prefix before running `configure', those
changes will propagate through the whole mess. If you change the default
paths in `texmf.cnf.in', those changes are propagated to the
compile-time defaults.

  Alternatively, you can ignore the whole mess and edit `texmf.cnf'
after it is installed. Maybe even copying it into place beforehand so
you can complete the installation, if TeX or Metafont is having trouble
finding their input files.

  Unfortunately, editing `Makefile.in' *does not work* in one common
case--changing the `prefix' or `exec_prefix' variables. For these, you
must use the `-prefix' or `-exec-prefix' options to `configure'.  See
Running `configure' scripts (autoconf).
(That's another partial lie: editing does work, as long as a program
named `tex' is not in your `PATH'.)

  See TeX directory structure, for a
description of some ways to arrange the TeX library files, and some
features of the distributed paths that may not be obvious.  The file
`kpathsea/HIER' is a copy of that section.

  The Make definitions are all repeated in several `Makefile''s; but
changing the top-level `Makefile' should suffice, as it passes down all
the variable definitions, thus overriding the submakes. (The
definitions are repeated so you can potentially run Make in the
subdirectories.)

Common problems
===============

  Some common problems with compilation, linking, or execution are
described below.

Unable to find files
--------------------

  If a program complains it cannot find fonts (or other input files),
any of several things might be wrong:

   * You don't have the fonts (or whatever) installed. Nothing will
     automatically generate TFM files or TeX and Metafont sources for
     you (by default). See Obtaining Web2c (web2c).

     You can, however, configure TeX and Metafont to run a script to
     generate these input files, if you have (or write) such scripts.
     See `MakeTeX'... invocation (web2c)... invocation.

   * You have (perhaps unknowingly) told Kpathsea to use search paths
     that don't reflect where the files actually are. One common cause
     is having environment variables set, thus overriding what you
     carefully set in `texmf.cnf'. See TeX environment variables.

   * Your files reside in a directory that is only pointed to via a
     symbolic link, in a leaf directory.

     Unfortunately, Kpathsea's subdirectory searching has a (congenital)
     deficiency: If a directory D being searched for subdirectories
     contains plain files and symbolic links to other directories, but
     no true subdirectories, D will be considered a leaf directory,
     i.e., the symbolic links will not be followed.  See Subdirectory expansion,
     for an explanation of why this happens.

     You can work around this problem by creating an empty dummy
     subdirectory in D. Then D will no longer be a leaf, and the
     symlinks will be followed.

     The directory immediately followed by the `//' in the path
     specification, however, is always searched for subdirectories,
     even if it is a leaf.  This is since presumably you would not have
     asked for the directory to be searched for subdirectories if you
     didn't want it to be.

   * There is a bug in the library. See Reporting bugs.

  In any case, you may find the debugging options helpful in determining
precisely where the fonts (or whatever) are being looked for. See the
program's documentation for its debugging options, and also
Debugging..

Slow path searching
-------------------

  If your program takes an excessively long time to find fonts or other
input files, but does eventually succeed, here are some possible
culprits:

   bullet Most likely, you just have a lot of directories to search,
     and that takes a noticeable time. The solution is to create and
     maintain a separate `ls-R' file that lists all the files in your
     main TeX hierarchy. See Filename database.

   bullet Your recursively-searched directories (e.g.,
     `/usr/local/lib/tex/fonts//'), contain a mixture of files and
     directories. This prevents Kpathsea from using a useful
     optimization (see Subdirectory expansion.).

     It is best to have only directories (and perhaps a `README') in the
     upper levels of the directory structure, and it's very important
     to have *only* files, and no subdirectories, in the directories
     where the dozens of TFM, PK, or whatever files reside.

   bullet Finally, one simple-to-fix (but unlikely) cause: If you
     recursively search `$HOME' or `~', and you are running as `root',
     you will search every directory on the system.  This typically
     takes quite some time!

  In any case, you may find the debugging options helpful in determining
precisely when the disk or network is being pounded. See Debugging,
and also see the program's documentation.

`XtInherit'
-----------

  On DEC OSF/1 1.x systems, the loader has a bug that manifests itself
in the following error (all on one line, but for the sake of the paper
width it's broken here):

     xdvik/xdvi: /sbin/loader: Fatal Error: search_for_undefineds:
          symbol _XtInherit should not have any relocation entry

According to Michael Rickabaugh `<mjr@quarry.enet.dec.com>':

     This is a bug fixed in DEC OSF/1 2.0.

     If you know how, installing `/sbin/loader' from a 2.0 system onto a
     1.3 system will work.  Make sure that `/usr' is *not* mounted
     when you do this.  (If you forget about umounting `/usr', it is
     possible most of your filesystems will become corrupted.)

     Otherwise, I suggest getting a hold of a 2.0 CD and running
     `/usr/sbin/installupdate'.

  Alternatively, you may be able to use the freely available X11
libraries that come with the MIT distribution (on `ftp.x.org', for
example).

  Linking statically, perhaps only with some of the X libraries, may
also work.  (if you find the definitive workaround, please let me know.)

`wchar_t'
---------

  The upshot of all the following is that if you get error messages
regarding `wchar_t', try defining `FOIL_X_WCHAR_T'.

  `wchar_t' has caused infinite trouble. None of my code ever uses
`wchar_t'; all I want to do is include X header files and various
system header files, possibly compiling with GCC. This seems an
impossible task!

  The X11 header `<Xlib.h>' and GCC's `<stddef.h>' have conflicting
definitions for wchar_t.

  The particulars: `<X11/Xlib.h>' from MIT X11R5 defines `wchar_t' if
`X_WCHAR' is defined, which is defined if `X_NOT_STDC_ENV' is defined,
and we define *that* if `STDC_HEADERS' is not defined (`configure'
decides if STDC_HEADERS gets defined).  But when compiling with gcc on
SunOS 4.1.x, `STDC_HEADERS' is not defined (`string.h' doesn't declare
the `mem'* functions), so we do get X's `wchar_t'--and we also get
gcc's `wchar_t' from its `<stddef.h>'.  Conflict.

  On the other hand, SunOS 4.1.1 with some other X configurations
actually needs GCC to define `wchar_t', and fails otherwise.

  My current theory is to define `wchar_t' to a nonsense symbol before
the X include files are read; that way its definition (if any) will be
ignored by other system include files.  Going along with that, define
`X_WCHAR' to tell X not to use `<stddef.h>', that we've already
included, but instead to make its own definition.

  But this is not the end of the story. The X11 include files
distributed with DG/UX 5.4.2 for the Aviion have been modified to
include `<_int_wchar_t.h>' if `X_WCHAR', so our `#define' will not have
any typedef to change--but the uses of `wchar_t' in the X include files
will be changed to reference this undefined symbol. So there's nothing
to foil in this case. I don't know how to detect this automatically, so
it's up to you to define `NO_FOIL_X_WCHAR_T' yourself.

`ShellWidgetClass'
------------------

  This section is adapted from question 47 from the
`comp.sys.sun.admin' FAQ.

  If you are linking with Sun's OpenWindows libraries in SunOS 4.1.x,
you may get undefined symbols `_get_wmShellWidgetClass' and
`_get_applicationShellWidgetClass'. This problem does not arise with
the standard MIT libraries under SunOS.

  The cause is bugs in the `Xmu' shared library as shipped from Sun.
There are several fixes:

   * Get the Openwindows patches that apply to this problem.

   * Statically link the `Xmu' library into the executable.

   * Avoid using `Xmu' at all. For this last, if you are compiling
     Metafont, see Online Metafont graphics (Web2c).
     If you are compiling Xdvi, see the `-DNOTOOL' option in
     `xdvik/INSTALL'.

   * Ignore the errors. The binary runs fine regardless.

  Here is the information for getting the two patches:

     Patch ID: 100512-02
     Bug ID's: 1086793, 1086912, 1074766
     Description: 4.1.x OpenWindows 3.0 `libXt' jumbo patch

     Patch ID: 100573-03
     Bug ID: 1087332
     Description: 4.1.x OpenWindows 3.0 undefined symbols when using shared
     `libXmu'.

  The way to statically link with `libXmu' depends on whether you are
using a Sun compiler (e.g., `cc') or `gcc'. If the format, alter the
`x_libs' make variable to include

     -Bstatic -lXmu -Bdynamic

  If you are using `gcc', include `-static' in `LDFLAGS'; this will
link all libraries statically. If you want to link only `Xmu'
statically and everything else dynamically, you have to do it by hand:
run `gcc -v', grab the `ld' line, and add the `-B''s given above around
`-lXmu'.

  The reason is that gcc moves all linker options to the front of the
`ld' command line.  So you can't specify different options for
different libraries.  When I reported this to `bug-gcc', the reply was
that they would happily merge in the changes, but they didn't have the
time to do it themselves.

Pointer combination warnings
----------------------------

  When compiling with old C compilers, you may get some warnings about
"illegal pointer combinations".  These are spurious; just ignore them.
I decline to clutter up the source with casts to get rid of them.

  In general, if you have trouble with a system C compiler, I advise
trying the GNU C compiler. (And vice versa, unfortunately; but in that
case I also recommend reporting a bug to the GCC bug list.)

Reporting bugs
==============

  If you encounter problems, please report them to `tex-k@cs.umb.edu'.
Include the version number of the library, the system you are using, and
enough information to reproduce the bug in your report.  To get on this
mailing list yourself, email `tex-k-request@cs.umb.edu' with a message
whose body contains a line
     subscribe YOU@YOUR.PREFERRED.ADDRESS

  To avoid wasted effort and time (both mine and yours), I strongly
advise applying the principles given in the GNU C manual (see
Reporting Bugs (gcc)) to your bug reports.

  Please also report bugs in this documentation--not only factual
errors, but unclear explanations, typos, wrong fonts, ...

Debugging
*********

  Kpathsea provides a number of runtime debugging options, detailed
below by their names (and corresponding numeric values). You should use
these numbers only if you run the program under a debugger and set the
the variable `kpathsea_debug' yourself (just sum the numbers to set
more than one option).  Ordinarily, you can set the debugging options
with some runtime argument (e.g., `-debug') to the program; in that
case, you should use the numeric values described in the program's
documentation (which may be different from those below).

`KPSE_DEBUG_STAT'
     (1). Reports `stat'(2) calls. This is useful for verifying that
     your directory structure is not forcing Kpathsea to do many
     additional file tests (see Slow path searching. and
     Subdirectory expansion.). If you are using an up-to-date `ls-R'
     database (see Filename database.), this should produce no
     output unless a nonexistent file is searched for.

`KPSE_DEBUG_HASH'
     (2). Reports lookups in all hash tables, including `ls-R' (see
     Filename database.), font aliases (see Fontmap.), and config
     file values (see Config files.).  Useful when expected values
     are not being found, e.g.., file searches are looking at the disk
     instead of using `ls-R'.

`KPSE_DEBUG_FOPEN'
     (4). Reports file openings and closings. Especially useful when
     your system's file table is full, for seeing if some files have
     been opened but never closed. In case you want to set breakpoints:
     this works by redefining `fopen' (`fclose') to be
     `kpse_fopen_trace' (`kpse_fclose_trace').

`KPSE_DEBUG_PATHS'
     (8). Reports general path information for each file type Kpathsea
     is asked to search. This is useful when you are trying to track
     down how a particular path got defined--from `texmf.cnf',
     `config.ps', the compile-time default, an environment variable,
     etc.  This is the contents of a structure defined in `tex-file.h'.

`KPSE_DEBUG_EXPAND'
     (16). Reports the directory list corresponding to each path element
     Kpathsea searches in. This is only relevant when Kpathsea is
     searching the disk, since `ls-R' searches don't look through
     directory lists in this way (they go straight to the file using
     the hash table).

`KPSE_DEBUG_SEARCH'
     (32). Reports on each file search Kpathsea attempts: the name of
     the file searched for, the path searched in, whether or not the
     file must exist (when drivers search for `cmr10.vf', it need not
     exist), and whether or not we are collecting all occurrences of
     the file in the path (as with, e.g., `texmf.cnf' and
     `texfonts.map'), or just the first (as with most lookups).  This
     can help you correlate what Kpathsea is doing with what is in your
     input file.

  Debugging output from Kpathsea is always written to standard output,
and begins with `kdebug:'. (Except for hash table buckets, which just
start with the number.)

Logging
=======

  Kpathsea can record the time and filename found for each successful
search. This may be useful in finding good candidates for deletion when
your disk is full.

  To do this, define the environment or config file variable
`TEXMFLOG'. The value is the name of the file to append the information
to. The file is created if it doesn't exist.

  Each successful search turns into one line in the log file, with two
words separated by a space. The first word is the time of the search, as
the integer number of seconds since "the epoch", i.e., UTC midnight 1
January 1970 (more precisely, the result of the `time' system call).
The second word is the filename.

  For example, after `setenv TEXMFLOG /tmp/log', running Dvips on
`story.dvi' appends the following lines:

     774455887 /usr/local/lib/texmf/dvips/config.ps
     774455887 /usr/local/lib/texmf/dvips/psfonts.map
     774455888 /usr/local/lib/texmf/dvips/texc.pro
     774455888 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmbx10.600pk
     774455889 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmsl10.600pk
     774455889 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmr10.600pk
     774455889 /usr/local/lib/texmf/dvips/texc.pro

Only filenames that are absolute are recorded, to preserve some
semblance of privacy.

Path searching
**************

  This chapter describes the generic path searching mechanism Kpathsea
provides. For information about searching for particular file types
(e.g., TeX fonts), see the next chapter.

Searching overview
==================

  A "search path" is a colon-separated list of path elements, which are
directory names with some extra frills. A search path can come from (a
combination of) many sources; see below.  To look up a file `foo' along
a path `.:/dir', Kpathsea checks each element of the path in turn:
first `./foo', then `/dir/foo', (typically) returning the first one
that exists.

  The "colon" and "slash" mentioned here aren't necessarily `:' and `/'
on non-Unix systems. Kpathsea tries to adapt to other operating
systems' conventions.

  To check a path element E, Kpathsea first sees if a prebuilt database
(see below) applies to E, i.e., if the database is in a directory that
is a prefix of E. If so, the path specification is matched against the
contents of the database.

  If the database does not exist, or does not apply to this path
element, contains no matches, the filesystem is searched. Kpathsea
constructs the list of directories that correspond to this path
element, and then checks in them for the file being searched for. (To
help speed future lookups of files in the same directory, the directory
in which a file is found is floated to the top of the directory list.)

  Each path element is checked in turn: first the database, then the
disk. Once a match is found, the searching stops and the result is
returned. This avoids possibly-expensive processing of path
specifications that are never needed on a particular run.

  Although the simplest and most common path element is a directory
name, Kpathsea supports additional features in search paths: layers of
default values, environment variable names, config file values, users'
home directories, and recursive subdirectory searching. Thus, we say
that Kpathsea "expands" a path element, meaning getting rid of all the
magic specifications and getting down to the basic directory name or
names. This process is described in the sections below. It happens in
the same order as the sections.

  Exception to the above: If the filename being searched for is absolute
or explicitly relative, i.e., starts with `/' or `./' or `../',
Kpathsea simply checks if that file exists; it is not looked for along
any paths.

Path sources
============

  A search path can come from many sources.  In priority order (meaning
Kpathsea will use whichever it finds first):

  1. A user-set environment variable, e.g., `TEXINPUTS'.

  2. A program-specific configuration file, e.g., an `S /a:/b' line in
     Dvips' `config.ps'.

  3. A line in a Kpathsea configuration file `texmf.cnf', e.g.,
     `TEXINPUTS=/c:/d'. See section below.

  4. The compile-time default (specified in `kpathsea/paths.h').

In any case, once the path specification to use is determined, its
evaluation is independent of its source.  These sources may also be
combined via default expansion. See the next section.

  You can see each of these values for a given search path by using the
debugging options of Kpathsea or your program. See Debugging.

Config files
------------

  As mentioned above, Kpathsea reads "runtime configuration files"
named `texmf.cnf' for search path definitions. The path used to search
for them is constructed in the usual way, as described above (except
that configuration files cannot be used to define the path, naturally;
also, an `ls-R' database is not used to search for them, for technical
reasons).

  The environment variable used is `TEXMFCNF'.

  Kpathsea reads *all* `texmf.cnf' files in the search path, not just
the first one found; it uses the first definition of each variable
encountered. Thus, with the (default) search path of `.:$TEXMF', values
from `./texmf.cnf' override those from `$TEXMF/texmf.cnf'.

  Here is the format for `texmf.cnf' files:

     xx

Here is the fragment from the distributed file illustrating most of
these points:

     % TeX input files -- i.e., anything to be found by \input or \openin [...]
     latex209_inputs = .:$TEXMF/tex/latex209//:$TEXMF/tex//
     latex2e_inputs = .:$TEXMF/tex/latex2e//:$TEXMF/tex//
     TEXINPUTS = .:$TEXMF/tex//
     TEXINPUTS.latex209 = $latex209_inputs
     TEXINPUTS.latex2e = $latex2e_inputs
     TEXINPUTS.latex = $latex2e_inputs

  Although this format has obvious similarities to Bourne shell
scripts--change the comment character to `#', disallow spaces around
the `=', and get rid of the `.PROGRAM' convention, and it could be run
through the shell. But there seemed little advantage to doing this,
since all the information would have to passed back (with `echo''s,
presumably) to Kpathsea and parsed there anyway, since the `sh' process
couldn't affect its parent's environment.

Default expansion
=================

  If the highest-priority search path (in the list in the previous
section) contains an "extra colon" (i.e., leading, trailing, or
doubled), Kpathsea inserts the next-highest-priority search path that is
set at that point. If that search path has an extra colon, the same
happens with the next-highest. (An extra colon in the compile-time
default value has unpredictable results, and may cause the program to
crash, so installers beware.)

  For example, given

     setenv TEXINPUTS /home/karl:

and a `TEXINPUTS' value from `texmf.cnf' of

     .:$TEXMF//tex

then the final value used for searching will be:

     /home/karl:.:$TEXMF//tex

You can trace this by debugging "paths" (see Debugging.).

  Minor technical point: Since it would be useless to insert the default
value in more than one place, Kpathsea changes only one extra `:' and
leaves any others in place (where they will eventually be effectively
equivalent to `.', i.e., the current directory). It checks first for a
leading `:', then a trailing `:', then a doubled `:'.

Variable expansion
==================

  `$foo' or `${foo}' in a path element is replaced by (1) the value of
an environment variable `foo' (if it is set); (2) the value of `foo'
from `texmf.cnf' (if any such exists); (3) the empty string.

  If the character after the `$' is alphanumeric or `_', the variable
name consists of all consecutive such characters. If the character
after the `$' is a `{', the variable name consists of everything up to
the next `}' (braces are not balanced!).  Otherwise, Kpathsea gives a
warning and ignores the `$' and its following character.

  Remember to quote the `$''s and braces as necessary for your shell.

  *Shell* variable values cannot be seen by Kpathsea.

  For example, given

     setenv TEXMF /home/tex
     setenv TEXINPUTS .:$TEXMF:${TEXMF}new

the final `TEXINPUTS' path is the three directories:

     .:/home/tex:/home/texnew

You can trace this by debugging "paths" (see Debugging.).

Tilde expansion
===============

  A leading `~' or `~USER' in a path element is replaced by the current
or USER's home directory, respectively.

  If USER is invalid, or the home directory cannot be determined,
Kpathsea uses `.' instead.

  For example,

     setenv TEXINPUTS ~/mymacros:

will prepend a directory `mymacros' in your home directory to the
default path.

Subdirectory expansion
======================

  A `//' in a path element following a directory D is replaced by all
subdirectories of D: first those subdirectories directly under D, then
the subsubdirectories under those, and so on.  At each level, the order
in which the directories are searched is unspecified. (It's "directory
order", and definitely not alphabetical.)

  If you specify any filename components after the `//', only
subdirectories which contain those components are included.  For
example, `/a//b' would expand into directories `/a/1/b', `/a/2/b',
`/a/1/1/b', and so on, but not `/a/b/c' or `/a/1'.

  I should mention one related implementation trick, which I stole from
GNU find.  Matthew Farwell `<dylan@ibmpcug.co.uk>' suggested it, and
David MacKenzie `<djm@gnu.ai.mit.edu>' implemented it (as far as I
know).

  The trick is that in every real Unix implementation (as opposed to the
POSIX specification), a directory which contains no subdirectories will
have exactly two links (namely, one for `.' and one for `..').  That is
to say, the `st_nlink' field in the `stat' structure will be two.
Thus, we don't have to stat everything in the bottom-level (leaf)
directories--we can just check `st_nlink', notice it's two, and do no
more work.

  But if you have a directory that contains *one* subdirectory and five
hundred files, `st_nlink' will be 3, and Kpathsea has to stat every one
of those 501 entries.  Therein lies slowness.

  You can disable the trick by undefining `UNIX_ST_LINK' in
`kpathsea/config.h'. (It is undefined by default except under Unix.)

  Unfortunately, in some cases files in leaf directories are `stat''d:
if the path specification is, say, `$TEXMF/fonts//pk//', then files in
a subdirectory `.../pk', even if it is a leaf, are checked. The reason
cannot be explained without reference to the implementation, so read
`kpathsea/elt-dirs.c' (search for `may descend') if you are curious.
(And if you can find a way to *solve* the problem, please let me know.)

Filename database
=================

  Kpathsea goes to some lengths to minimize disk accesses for searches
(see Subdirectory expansion.).  Nevertheless, at installations with
enough directories, doing a linear search of each possible directory for
a given file can take an excessively long time ("excessive" depending
on the speed of the disk, whether it's NFS-mounted, how patient you are,
etc.). In practice, the union of font directories from the Dvips(k) and
Dviljk distributions is large enough for searching to be noticeably slow
on most machines.

  Therefore, Kpathsea can use an externally-built "database" that maps
files to directories, thus avoiding the need to exhaustively search the
disk.  By fiat, you must name the file `ls-R', and put it at the root
of the TeX installation hierarchy (`$TEXMF' by default).  Kpathsea does
variable expansion on the `$TEXMF', naturally, so you can use different
`ls-R''s for different trees, if you are testing new ones.  However,
one and only one `ls-R' is read; it is not searched for along any paths.

  You can build `ls-R' with the command `ls -R /YOUR/ROOT/DIR >ls-R',
if your `ls' produces the right output format (see the section below).
GNU `ls', for example, outputs in this format.  It is probably best to
do this via `cron', so changes in the installed files will be
automatically reflected (albeit with some delay) in the database.

  If your system uses symbolic links, the command `ls -LR' will be more
reliable than plain `ls -R'.  The former follows the symbolic links to
the real files, which is what Kpathsea needs.

  Because the database may be out-of-date for a particular run (e.g.,
if a font was just built with `MakeTeXPK'), if a file is not found in
the database, by default Kpathsea goes ahead and searches the disk. If a
particular path element begins with `!!', however, *only* the database
will be searched for that element, never the disk. If the database does
not exist, nothing will be searched. Because this can greatly surprise
users ("I see the font `foo.tfm' when I do an `ls'; why can't Dvips
find it?"), I do not recommend using this feature.

Database format
---------------

  The "database" read by Kpathsea is a line-oriented file of plain
text. The format is that generated by GNU (and perhaps other) `ls'
programs given the `-R' option, as follows.

   * Blank lines are ignored.

   * If a line begins with `/' and ends with a colon, it's the name of a
     directory.

   * All other lines name entries in the most recently seen directory.
     `/''s in such lines will produce possibly-strange results.

   * Files with no preceding directory line are ignored.

  For example, here's the first few lines of `ls-R' on my system:

     bibtex
     dvips
     fonts
     ini
     ls-R
     mf
     tex

     /usr/local/lib/texmf/bibtex:
     bib
     bst
     doc

     /usr/local/lib/texmf/bibtex/bib:
     asi.bib
     bibshare
     btxdoc.bib

On my system, `ls-R' is about 30K bytes.

TeX searching
*************

  Although the basic features in Kpathsea can be used for any type of
path searching, it came about (like all libraries) with a specific
application in mind: I wrote Kpathsea specifically for TeX system
programs.  I had been struggling with the programs I was using (Dvips,
Xdvi, and TeX itself) having slightly different notions of how to
specify paths; and debugging was painful, since no code was shared.

  Therefore, Kpathsea provides some TeX-specific features.  Indeed,
many of the supposedly generic path searching features were provided
because they seemed useful in conTeXt (particularly for font lookup).

TeX environment variables
=========================

  Kpathsea defines a sequence of environment variables to search for
each file type it supports.  This makes it easy for different programs
to check the same environment variables, in the same order.

  The following table lists the environment variables searched for each
file type in the order they are searched (and a brief description of the
file type).  That is, only if the first variable is unset is the second
variable checked, and so on.  If none are set, various other things are
checked; see Path sources..

`.base'
     (Metafont memory dump) `MFBASES'

`.bib'
     (BibTeX bibliography source) `BIBINPUTS'

`.bst'
     (BibTeX style file) `BSTINPUTS', `TEXINPUTS'

`.cnf'
     (Kpathsea runtime configuration files) `TEXMFCNF'

`.eps'
     (Encapsulated PostScript figures) `TEXPICTS', `TEXINPUTS'

`.fmt'
     (TeX memory dump) `TEXFORMATS'

`gf'
     (generic font bitmap) `PROGRAMFONTS', `GFFONTS', `GLYPHFONTS',
     `TEXFONTS'

`.mf'
     (Metafont source) `MFINPUTS'

`mf.pool'
     (Metafont program strings) `MFPOOL'

`.pict'
     (Other kinds of figures) Same as `.eps'.

`pk'
     (packed bitmap font) `PROGRAMFONTS', `PKFONTS', `TEXPKS',
     `GLYPHFONTS', `TEXFONTS'

`.tex'
     (TeX source) `TEXINPUTS'

`tex.pool'
     (TeX program strings) `TEXPOOL'

`.tfm'
     (TeX font metrics) `TFMFONTS', `TEXFONTS'

`.vf'
     (virtual font) `VFFONTS', `TEXFONTS'

These lists are defined in the source file `kpathsea/tex-file.h'.

  For the font variables, the intent is that:

  1. `TEXFONTS' is the default for everything.

  2. `GLYPHFONTS' is the default for bitmap (or, more precisely,
     non-metric) files.

  3. Each format has its own variable.

  4. Each program can and should have its own font override path as
     well; e.g., `DVIPSFONTS' for Dvipsk. Again, this is for bitmaps,
     not metrics.

Glyph lookup
============

  Kpathsea provides a routine (`kpse_find_glyph_format' in
`kpathsea/tex-glyph.c') which searches for a bitmap font in GF or PK
format (or either) given a font name (e.g., `cmr10') and a resolution
(e.g., 300).

  The search is based solely on filenames, not file contents--if a PK
file is named `cmr10.300gf', it will be found as a GF file.

  Here is an outline of the search strategy (details in the sections
below) for a file NAME at resolution DPI.  The search stops at the
first successful lookup.

  1. Look for an existing file NAME.DPI in the specified format(s).

  2. If NAME is an alias for a file F in the fontmap file
     `texfonts.map', look for F.DPI.

  3. Run an external script (typically named `MakeTeXPK') to generate
     the font.

  4. Look for FALLBACK.DPI, where FALLBACK is some last-resort font
     (typically `cmr10').

Basic glyph lookup
------------------

  When Kpathsea looks for a bitmap font NAME at resolution DPI in a
format FORMAT, it first checks each directory in the search path for a
file `NAME.DPIFORMAT'; for example, `cmr10.300pk'.  Kpathsea looks for
a PK file first, then a GF file.

  If that fails, Kpathsea looks for `dpiDPI/NAME.FORMAT'; for example,
`dpi300/cmr10.pk'. This is how fonts are typically stored on
filesystems (like DOS's) that permit only three-character extensions.

  If that fails, Kpathsea looks for a font with a close-enough DPI.
"Close enough" is defined (by the macro `KPSE_BITMAP_TOLERANCE' in
`kpathsea/tex-glyph.h') to be `DPI / 500 + 1', which is slightly more
than the 0.2% allowed by the DVI standard.

Fontmap
-------

  If a bitmap font is not found with the original name (see the previous
section), Kpathsea looks through any "fontmap" files for an "alias" for
the original font name.  These files are named `texfonts.map' and are
searched for along the usual glyph path.

  This feature is intended to help in two respects:

  1. An alias name is limited in length only by available memory, not
     by your filesystem.  Therefore, if you want to ask for
     `Adobe-Lucida-Bold-Sans=Typewriter' instead of `plcbst', you can.

  2. A few fonts have historically had multiple names: specifically,
     LaTeX's "circle font" has variously been known as `circle10',
     `lcircle10', and `lcirc10'.  Aliases can make all the names
     equivalent, so that it no longer matters what the name of the
     installed file is; TeX documents will find their favorite name.

  The format of fontmap files is straightforward: the first word on each
line is the true filename; the second word is the alias; subsequent
words are ignored.  A "word" is a sequence of non-whitespace
characters.  Blank lines are ignored; comments start with `%' and
continue to end-of-line.

  If an alias has an extension, it matches only those files with that
extension; otherwise, it matches anything with the same root,
regardless of extension.  For example, an alias `foo.tfm' matches only
when exactly `foo.tfm' is being searched for; but an alias `foo'
matches `foo.vf', `foo.300pk', etc.

  As an example, here are the fontmap entries that make the circle fonts
equivalent. These are in the distributed `texfonts.map' in the Web2C
distribution.

     circle10	lcircle10
     circle10	lcirc10
     lcircle10	circle10
     lcircle10	lcirc10
     lcirc10		circle10
     lcirc10		lcircle10

`MakeTeX'... scripts
--------------------

  If Kpathsea cannot find a bitmap font, by either its original name or
a fontmap alias, it can be configured to invoke an external program to
create it.  The same mechanism can be used for other nonexistent files.

  The script is passed the name of the file to create and possibly other
arguments, as explained below.  It must echo the full pathname of the
file it created (and nothing else) to standard output; it can write
diagnostics to standard error.

  By default, no `MakeTeX...' are run.

`MakeTeX'... script names
.........................

  The following table shows the default name of the script for each
possible file types.  (The source is the variable `kpse_make_specs' in
`kpathsea/tex-make.c'.)

`MakeTeXPK'
     Glyph fonts.

`MakeTeXTeX'
     TeX input files.

`MakeTeXMF'
     Metafont input files.

`MakeTeXTFM'
     TFM files.

These names are overridden by an environment variable specific to the
program--for example, `DVIPSMAKEPK' for Dvipsk.

  If a `MakeTeX...' script fails, the invocation is appended to a file
`missfont.log' in the current directory. If the current directory is
not writable and the environment variable `TEXMFOUTPUT' is set, its
value is used. Otherwise, nothing is written.

`MakeTeX'... script arguments
.............................

  The first argument to a `MakeTeX'... script is always the name of the
file to be created.

  For `MakeTeXPK', three or four additional arguments are also passed,
via corresponding environment variables:

  1. The dpi to make the font at (`KPATHSEA_DPI').

  2. The "base dpi" the program is operating at (`MAKETEX_BASE_DPI'),
     i.e., the assumed resolution of the output device.

  3. A "magstep" string suitable for the Metafont `mag' variable
     (`MAKETEX_MAG').

  4. Optionally, a Metafont mode name to assign to the Metafont `mode'
     variable (`MAKETEX_MODE').

  5. Optionally, a directory name. If the directory is absolute, it is
     used as-is. Otherwise, it is appended to the root destination
     directory set in the script (from environment variables `DESTDIR'
     or `MTP_DESTDIR' or a compile-time default). If this argument is
     not supplied, the mode name is appended to the root destination
     directory.

Kpathsea sets `KPATHSEA_DPI' appropriately for each attempt at building
a font.  It's up to the program using Kpathsea to set the others.
(see Calling sequence.)

  You can change the specification for the arguments passed to the
external script by setting the environment variable named as the script
name, but all capitals--`MAKETEXPK', for example.  If you've changed
the script name by setting (say) `DVIPSMAKEPK' to `foo', then the spec
is taken from the environment variable `FOO'.

  The spec can contain any variable references, to the above variables
or any others you might have set.  As an example, the default spec for
`MakeTeXPK' is:

     $KPATHSEA_DPI $MAKETEX_BASE_DPI $MAKETEX_MAG $MAKETEX_MODE

The convention of passing the name of the file to be created as the
first argument cannot be changed.

Fallback font
-------------

  If a bitmap font cannot be found or created at the requested size,
Kpathsea looks for the font at a set of "fallback resolutions".  You
specify these resolutions as a colon-separated list (like search paths).
Kpathsea looks first for a program-specific environment variable (e.g.,
`DVIPSSIZES' for Dvipsk), then the environment variable `TEXSIZES',
then a default specified at compilation time (the Make variable
`default_texsizes').  You can set this list to be empty if you prefer
to find fonts at their stated size or not at all.

  Finally, if the font cannot be found even at the fallback resolutions,
Kpathsea looks for a fallback font, typically `cmr10'.  Programs must
enable this feature by assigning to the global variable
`kpse_fallback_font' or calling `kpse_init_prog' (see Calling sequence.);
the default is no such fallback font.

TeX directory structure
***********************

  (This section obviously not written yet; sorry.)

  xx(from, and generate, HIER)

  If you want to include the device name (i.e., the Metafont mode) in a
path, perhaps to distinguish two different devices with the same
resolution, use `$MAKETEX_MODE', as in
     /usr/local/lib/texmf/fonts//pk/$MAKETEX_MODE

This is the default now.

  See `MakeTeX'... script arguments... script arguments.

Programming
***********

  This chapter is for programmers who wish to use Kpathsea. See
Introduction, for the conditions under which you may do so. (If you
do this, I'd appreciate a note, just to satisfy my curiousity.)