DocumentTemplate
Digital Creations, L.C.
910 Princess Anne Street Suite 300
Fredericksburg, VA 22401
Abstract
Many Web based applications must provide the capability for customers to be able to edit the look and feel of the application's Web pages. Futhermore, our experience has been that mixing Python and Hypertext Markup Language (HTML) together makes both less readable. We have developed a Python module named DocumentTemplate that allows documents to be created with embedded markup that causes data from Python objects to be inserted into documents when they are output from an application. The data insertion goes beyond simple insertion of data with C-language formats to include:
Two source formats are supported: extended Python format strings, and HTML with markup that uses server-side-include syntax. HTML document templates provide a web interface that supports through-the-web editing and management of document source.
Background
To make programmatically-created HTML pages easily editable, Digital Creations needed a way to separate them from the code. This would also allow users who are unfamiliar with Python to edit the page headers, footers and layout without compromising the reliability of the software.
Our first solution was called pyHTML which basically was a custom tag embedded into an HTML document that was read into a Python script, parsed and then sent to the user with all expressions evaluated. Initially pyHTML did the things we wanted it to, but after some use we found it had some security and standard compatibility problems. pyHTML allowed the evalution of arbitrary Python expressions and access to any variable in the present name space. These posed serious security problems that could allow the creation of malicious pyHTML that could execute functions from the os module and/or output values the programmer didn't intend to be displayed. Furthermore, pyHTML did not follow any HTML or Python guidelines and therefore, for lack of a better term, was ugly.
DocumentTemplate was designed to have the functionality we wanted but to solve the security and compatibility problems of pyHTML. DocumentTemplate will not allow arbitrary Python expressions to be evaluated or executed and only variables passed to the template by the programmer can be displayed. The syntax is also much easier to understand and blends into HTML better than the previous solution.
Benefits
The following are a few reasons to use DocumentTemplate:
Example
Here is a small example using DocumentTemplate:
form = DocumentTemplate.HTML(
'<HTML><HEAD><TITLE>Title</TITLE></HEAD>\n'
'<BODY>\n'
'<!--#var Param1--> <!--#var Param2-->\n'
'</BODY></HTML')
Param1 = "Spam"
Param2 = "is good for you."
return form(Param1=Param1,Param2=Param2)
The user will receive:
<HTML><HEAD><TITLE>Title</TITLE></HEAD>
<BODY>
Spam is good for you.
</BODY>
</HTML>
Features
DocumentTemplate provides for creation of textual documents, such as HTML pages, from template source by inserting data from Python objects and name-spaces. DocumentTemplate is especially useful when it is desirable to separate template text from Python program source. For example, HTML templates may be edited by people who know HTML but don't know Python, while associated Python code may be edited by people who know Python but not HTML.
When a document template is created, a collection of default values to be inserted may be specified with a mapping object and with keyword arguments.
DocumentTemplate may be called to create a document with values inserted. When called, an instance, a mapping object, and keyword arguments may be specified to provide values to be inserted. If an instance is provided, the document template will try to look up values in the instance using getattr, so inheritence of values is supported. If an inserted value is a function, method, or class, then an attempt will be made to call the object to obtain values. This allows instance methods to be included in documents.
DocumentTemplates masquerade as functions, so the Python Object Publisher (Bobo) will call templates that are stored as instances of published objects. Bobo will pass the object the template was found in and the HTTP request object.
Two source formats are supported:
Extended Python format strings (EPFS)
This format is based on the insertion by name format strings of Python with additional format characters, [ and ] to indicate block boundaries. In addition, parameters may be used within formats to control how insertion is done.
For example:
%(date fmt=DayOfWeek upper)s
causes the contents of variable date to be inserted using custom format DayOfWeek and with all lower case letters converted to upper case.
HTML
This format uses HTML server-side-include syntax with commands for inserting text. Parameters may be included to customize the operation of a command.
For example:
<!--#var total fmt=12.2f-->
is used to insert the variable total with the C format 12.2f.
Variable insertion parameters:
When inserting variables, parameters may be specified to control how the data will be formatted. In HTML source, the fmt parameter is used to specify a C-style or custom format to be used when inserting an object. In EPFS source, the fmt parameter is only used for custom formats, a C-style format is specified after the closing parenthesis.
Custom formats
A custom format is used when outputting user-defined objects. The value of a custom format is a method name to be invoked on the object being inserted. The method should return an object that, when converted to a string, yields the desired text. For example, the HTML source:
<!--#var date fmt=DayOfWeek-->
Inserts the result of calling the method DayOfWeek of the object bound to the variable date, with no arguments.
In addition to object methods, serveral additional custom formats are available:
html-quote
Convert characters that have special meaning in HTML to HTML character entities.
url-quote
Convert characters that have special meaning in URLS to HTML character entities using decimal values.
multi-line
Convert newlines and carriage-return and newline combinations to break tags.
whole-dollars
Show a numeric value with a dollar symbol.
dollar-with-commas
Show a numeric value with a dollar symbol and commas showing thousands, millions, and so on.
dollars-and-cents
Show a numeric value with a dollar symbol and two decimal places.
dollar-and-cents-with-commas
Show a numeric value with a dollar symbol and two decimal places and commas showing thousands, millions, and so on.
Note that when using the EPFS source format, both a C-style and a custom format may be provided. In this case, the C-Style format is applied to the result of calling the custom formatting method.
Null values
In some applications, and especially in database applications, data variables may alternate between "good" and "null" or "missing" values. A format that is used for good values may be inappropriate for null values. For this reason, the null parameter can be used to specify text to be used for null values. Null values are defined as values that:
For example, when showing a monitary value retrieved from a database that is either a number or a missing value, the following variable insertion might be used:
<!--#var cost fmt="$%.2d" null='n/a'-->
String manipulation
The parameters lower and upper may be provided to cause the case of the inserted text to be changed.
The parameter capitalize may be provided to cause the first character of the inserted value to be converted to upper case.
The parameter spacify may be provided to cause underscores in the inserted value to be converted to spaces.
DocumentTemplate supports conditional and sequence insertion
DocumentTemplates extend Python string substitition rules with a mechanism that allows conditional insertion of template text and that allows sequences to be inserted with element-wise insertion of template text.
Conditional insertion
Conditional insertion is performed using if and else commands.
To include text when an object is true using the EPFS format, use:
%(if name)[
text
%(if name)]
To include text when an object is true using the HTML format, use:
<!--#if name-->
text
<!--#/if name-->
where name is the name bound to the object.
To include text when an object is false using the EPFS format, use:
%(else name)[
text
%(else name)]
To include text when an object is false using the HTML format, use:
<!--#else name-->
text
<!--#/else name-->
To include text when an object is true and to include different text when the object is false using the EPFS format, use:
%(if name)[
true text
%(if name)]
%(else name)[
false text
%(else name)]
To include text when an object is true and to include different text when the object is false using the HTML format, use:
<!--#if name-->
true text
<!--#else name-->
false text
<!--#/if name-->
Sequence insertion
A sequence may be inserted using an in command. The in command specifies the name of a sequence object and text to be inserted for each element in the sequence.
The EPFS syntax for the in command is:
%(in name)[
text
%(in name)]
The HTML syntax for the in command is:
<!--#in name-->
text
<!--#/in name-->
See the example below that shows how if, else, and in commands may be combined to display a possibly empty list of objects.
The text included within an in command will be refered to as an in block.
Within an in block, variables are substituted from the elements of the iteration. The elements may be either instance or mapping objects. In addition, the variables below are defined for each element:
sequence-item
The element.
sequence-index
The index, starting from 0, of the element within the sequence.
sequence-number
The index, starting from 1, of the element within the sequence.
sequence-letter
The index, starting from a, of the element within the sequence.
sequence-Letter
The index, starting from A, of the element within the sequence.
sequence-roman
The index, starting from i, of the element within the sequence.
sequence-Roman
The index, starting from I, of the element within the sequence.
sequence-start
A variable that is true if the element being displayed is the first of the displayed elements, and false otherwise.
sequence-end
A variable that is true if the element being displayed is the last of the displayed elements, and false otherwise.
Normally, in blocks are used to iterate over sequences of instances. If the optional parameter mapping is specified after the sequence name, then the elements of the sequence will be treated as mapping objects.
An in command may be used to iterate over a sequence of dictionary items. If the elements of the iteration are two-element tuples, then then the template code given in the in block will be applied to the second element of each tuple and may use a variable, sequence-key to access the first element in each tuple.
Batch sequence insertion
When displaying a large number of objects, it is sometimes desireable to display just a sub-sequence of the data. An in command may have optional parameters, as in:
<!--#in values start=start_var size=7-->
The parameter values may be either integer literals or variable names.
Up to five parameters may be set:
start
The number of the first element to be shown, where elements are numbered from 1.
end
The number of the last element to be shown, where elements are numbered from 1.
size
The desired number of elements to be shown at once.
orphan
The desired minimum number of objects to be displayed. The default value for this parameter is 3.
overlap
The desired overlap between batches. The default is no overlap.
Typically, only start and size will be specified.
When batch insertion is used, several additional variables are defined for use within the sequence insertion text:
sequence-step-size
The batch size used.
previous-sequence
This variable will be true when the first element is displayed and when the first element displayed is not the first element in the sequence.
previous-sequence-start-index
The index, starting from 0, of the start of the batch previous to the current batch.
previous-sequence-end-index
The index, starting from 0, of the end of the batch previous to the current batch.
previous-sequence-size
The size of the batch previous to the current batch.
previous-batches
A sequence of mapping objects containing information about all of the batches prior to the batch being displayed.
Each of these mapping objects include the following variables:
batch-start-index
The index, starting from 0, of the beginning of the batch.
batch-end-index
The index, starting from 0, of the end of the batch.
batch-end-index
The size of the batch.
next-sequence
This variable will be true when the last element is displayed and when the last element displayed is not the last element in the sequence.
next-sequence-start-index
The index, starting from 0, of the start of the batch after the current batch.
next-sequence-end-index
The index, starting from 0, of the end of the batch after the current batch.
next-sequence-size
The size of the batch after the current batch.
next-batches
A sequence of mapping objects containing information about all of the batches after the batch being displayed.
Each of these mapping objects include the following variables:
batch-start-index
The index, starting from 0, of the beginning of the batch.
batch-end-index
The index, starting from 0, of the end of the batch.
batch-end-index
The size of the batch.
For each of the variables listed above with names ending in "-index", there are variables with names ending in "-number", "-roman", "-Roman", "-letter", and "-Letter" that are indexed from 1, "i", "I", "a", and "A", respectively. In addition, for every one of these variables there are variables with names ending in "-var-xxx", where "xxx" is an element attribute name or key.
Summary statistics
When performing sequence insertion, special variables may be used to obtain summary statistics. To obtain a summary statistic for a variable, use the variable name: statistic-name, where statistic is a statistic name and name is the name of a data variable.
Currently supported statistic names are:
total
The total of numeric values.
count
The total number of non-missing values.
min
The minimum of non-missing values.
max
The maximum of non-missing values.
median
The median of non-missing values.
mean
The mean of numeric values values.
variance
The variance of numeric values computed with a degrees of freedom qeual to the count - 1.
variance-n
The variance of numeric values computed with a degrees of freedom qeual to the count.
standard-deviation
The standard deviation of numeric values computed with a degrees of freedom qeual to the count - 1.
standard-deviation-n
The standard deviation of numeric values computed with a degrees of freedom qeual to the count.
Missing values are either None or the attribute Value of the module Missing, if present.
Implementation Mechanisms
A DocumentTemplate may be created using any of these four methods:
DocumentTemplate.String
Creates a DocumentTemplate from a string using an extended form of Python string formatting.
DocumentTemplate.File
Creates a DocumentTemplate bound to a named file using an extended form of Python string formatting. If the object is pickled, the file name, rather than the file contents is pickled. When the object is unpickled, then the file will be re-read to obtain the string. Note that the file will not be read until the document template is used the first time.
DocumentTemplate.HTML
Creates a DocumentTemplate from a string using HTML server-side-include rather than Python-format-string syntax.
DocumentTemplate.HTMLFile
Creates an HTML DocumentTemplate from a named file.
Issues
The following lists some issues concerning future modifications to DocumentTemplate: