<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<!--Converted with LaTeX2HTML 98.2 beta6 (August 14th, 1998)
original version by:  Nikos Drakos, CBLU, University of Leeds
* revised and updated by:  Marcus Hennecke, Ross Moore, Herb Swan
* with significant contributions from:
  Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
<HTML>
<HEAD>
<TITLE>3.14 parser --
Access parse trees for Python code</TITLE>
<META NAME="description" CONTENT="3.14 parser --
Access parse trees for Python code">
<META NAME="keywords" CONTENT="lib">
<META NAME="resource-type" CONTENT="document">
<META NAME="distribution" CONTENT="global">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<LINK REL="STYLESHEET" HREF="lib.css">
<LINK REL="next" HREF="module-symbol.html">
<LINK REL="previous" HREF="module-imp.html">
<LINK REL="up" HREF="python.html">
<LINK REL="next" HREF="Creating_ASTs.html">
</HEAD>
<BODY bgcolor="#ffffff">
<div class=navigation>
<table align=center width="100%" cellpadding=0 cellspacing=2>
<tr>
<td><A HREF="Creating_ASTs.html"><img width=32 height=32 align=bottom border=0 alt="next"
 src="../icons/next.gif"></A></td>
<td><A HREF="python.html"><img width=32 height=32 align=bottom border=0 alt="up"
 src="../icons/up.gif"></A></td>
<td><A HREF="examples-imp.html"><img width=32 height=32 align=bottom border=0 alt="previous"
 src="../icons/previous.gif"></A></td>
<td align=center bgcolor="#99CCFF" width="100%">
 <b class=title>Python Library Reference</b></td>
<td><A HREF="contents.html"><img width=32 height=32 align=bottom border=0 alt="contents"
 src="../icons/contents.gif"></A></td>
<td><a
 href="modindex.html"><img width=32 height=32 align=bottom border=0 alt="modules"
 src="../icons/modules.gif"></a></td>
<td><A HREF="genindex.html"><img width=32 height=32 align=bottom border=0 alt="index"
 src="../icons/index.gif"></A></td>
</tr></table><b class=navlabel>Next:</b> <span class=sectref><A HREF="Creating_ASTs.html">3.14.1 Creating AST Objects</A></span>
<b class=navlabel>Up:</b> <span class=sectref><A HREF="python.html">3. Python Services</A></span>
<b class=navlabel>Previous:</b> <span class=sectref><A HREF="examples-imp.html">3.13.1 Examples</A></span>
<br><hr></div>
<!--End of Navigation Panel-->

<H1><A NAME="SECTION0051400000000000000000">
3.14 <tt class=module>parser</tt> --
         Access parse trees for Python code</A>
</H1>

<P>


<P>
<a name="l2h-479"></a>

<P>
The <tt class=module>parser</tt> module provides an interface to Python's internal
parser and byte-code compiler.  The primary purpose for this interface
is to allow Python code to edit the parse tree of a Python expression
and create executable code from this.  This is better than trying
to parse and modify an arbitrary Python code fragment as a string
because parsing is performed in a manner identical to the code
forming the application.  It is also faster.

<P>
There are a few things to note about this module which are important
to making use of the data structures created.  This is not a tutorial
on editing the parse trees for Python code, but some examples of using
the <tt class=module>parser</tt> module are presented.

<P>
Most importantly, a good understanding of the Python grammar processed
by the internal parser is required.  For full information on the
language syntax, refer to the <i>Python Language Reference</i>.  The
parser itself is created from a grammar specification defined in the file
<tt class=file>Grammar/Grammar</tt> in the standard Python distribution.  The parse
trees stored in the AST objects created by this module are the
actual output from the internal parser when created by the
<tt class=function>expr()</tt> or <tt class=function>suite()</tt> functions, described below.  The AST
objects created by <tt class=function>sequence2ast()</tt> faithfully simulate those
structures.  Be aware that the values of the sequences which are
considered ``correct'' will vary from one version of Python to another
as the formal grammar for the language is revised.  However,
transporting code from one Python version to another as source text
will always allow correct parse trees to be created in the target
version, with the only restriction being that migrating to an older
version of the interpreter will not support more recent language
constructs.  The parse trees are not typically compatible from one
version to another, whereas source code has always been
forward-compatible.

<P>
Each element of the sequences returned by <tt class=function>ast2list()</tt> or
<tt class=function>ast2tuple()</tt> has a simple form.  Sequences representing
non-terminal elements in the grammar always have a length greater than
one.  The first element is an integer which identifies a production in
the grammar.  These integers are given symbolic names in the C header
file <tt class=file>Include/graminit.h</tt> and the Python module
<tt class=module><a href="module-symbol.html">symbol</a></tt>.  Each additional element of the sequence represents
a component of the production as recognized in the input string: these
are always sequences which have the same form as the parent.  An
important aspect of this structure which should be noted is that
keywords used to identify the parent node type, such as the keyword
<tt class=keyword>if</tt> in an <tt class=constant>if_stmt</tt>, are included in the node tree without
any special treatment.  For example, the <tt class=keyword>if</tt> keyword is
represented by the tuple <tt>(1, 'if')</tt>, where <tt>1</tt> is the
numeric value associated with all <tt>NAME</tt> tokens, including
variable and function names defined by the user.  In an alternate form
returned when line number information is requested, the same token
might be represented as <tt>(1, 'if', 12)</tt>, where the <tt>12</tt>represents the line number at which the terminal symbol was found.

<P>
Terminal elements are represented in much the same way, but without
any child elements and the addition of the source text which was
identified.  The example of the <tt class=keyword>if</tt> keyword above is
representative.  The various types of terminal symbols are defined in
the C header file <tt class=file>Include/token.h</tt> and the Python module
<tt class=module><a href="module-token.html">token</a></tt>.

<P>
The AST objects are not required to support the functionality of this
module, but are provided for three purposes: to allow an application
to amortize the cost of processing complex parse trees, to provide a
parse tree representation which conserves memory space when compared
to the Python list or tuple representation, and to ease the creation
of additional modules in C which manipulate parse trees.  A simple
``wrapper'' class may be created in Python to hide the use of AST
objects.

<P>
The <tt class=module>parser</tt> module defines functions for a few distinct
purposes.  The most important purposes are to create AST objects and
to convert AST objects to other representations such as parse trees
and compiled code objects, but there are also functions which serve to
query the type of parse tree represented by an AST object.

<P>

<p><hr>
<!--Table of Child-Links-->


<UL>
<LI><A NAME="tex2html1110"
 HREF="Creating_ASTs.html">3.14.1 Creating AST Objects</A>
<LI><A NAME="tex2html1111"
 HREF="Converting_ASTs.html">3.14.2 Converting AST Objects</A>
<LI><A NAME="tex2html1112"
 HREF="Querying_ASTs.html">3.14.3 Queries on AST Objects</A>
<LI><A NAME="tex2html1113"
 HREF="AST_Errors.html">3.14.4 Exceptions and Error Handling</A>
<LI><A NAME="tex2html1114"
 HREF="AST_Objects.html">3.14.5 AST Objects</A>
<LI><A NAME="tex2html1115"
 HREF="AST_Examples.html">3.14.6 Examples</A>
<UL>
<LI><A NAME="tex2html1116"
 HREF="node51.html">3.14.6.1 Emulation of <tt class=function>compile()</tt></A>
<LI><A NAME="tex2html1117"
 HREF="node52.html">3.14.6.2 Information Discovery</A>
</UL></UL>
<!--End of Table of Child-Links-->
<p>
<div class=navigation><hr><table align=center width="100%" cellpadding=0 cellspacing=2>
<tr>
<td><A HREF="Creating_ASTs.html"><img width=32 height=32 align=bottom border=0 alt="next"
 src="../icons/next.gif"></A></td>
<td><A HREF="python.html"><img width=32 height=32 align=bottom border=0 alt="up"
 src="../icons/up.gif"></A></td>
<td><A HREF="examples-imp.html"><img width=32 height=32 align=bottom border=0 alt="previous"
 src="../icons/previous.gif"></A></td>
<td align=center bgcolor="#99CCFF" width="100%">
 <b class=title>Python Library Reference</b></td>
<td><A HREF="contents.html"><img width=32 height=32 align=bottom border=0 alt="contents"
 src="../icons/contents.gif"></A></td>
<td><a
 href="modindex.html"><img width=32 height=32 align=bottom border=0 alt="modules"
 src="../icons/modules.gif"></a></td>
<td><A HREF="genindex.html"><img width=32 height=32 align=bottom border=0 alt="index"
 src="../icons/index.gif"></A></td>
</tr></table><b class=navlabel>Next:</b> <span class=sectref><A HREF="Creating_ASTs.html">3.14.1 Creating AST Objects</A></span>
<b class=navlabel>Up:</b> <span class=sectref><A HREF="python.html">3. Python Services</A></span>
<b class=navlabel>Previous:</b> <span class=sectref><A HREF="examples-imp.html">3.13.1 Examples</A></span>
</div>
<!--End of Navigation Panel-->
<ADDRESS>
<hr>Send comments on this document to <a href="mailto:python-docs@python.org">python-docs@python.org</a>.
</ADDRESS>
</BODY>
</HTML>
