JINJA2 is a powerful yet easy to use templating engine that enables easily generating documents on the fly blending one or more objects with a template.
Since the syntax is heavily inspired by Django and Python, JINJA2 is not only commonly used inside Python scripts: it is very often exploited by Ansible, for example when generating configuration files on the fly.
In "JINJA2 With Python Tutorial JINJA Ansible Howto" post we will learn all the must know about JINJA2, and experiment with it by writing a real-life JINJA2 template file, rendering it by merging it with a JSON document using a custom Python script and an Ansible playbook.
JINJA2 Templates in A Nuthsell
JINJA2 provides a very extensive documentation - thoroughly documenting JINJA2 here would just mean duplicating it, so I’m just mentioning its most likely to be used features, describing them the easiest and more straightforward way I can. The official and complete JINJA2 template documentation is available here.
Extensions
First and foremost it is important to know that JINJA2 supports extensions that can add extra features: if you are not aware of this you will be easily confused when trying to render templates containing statements requiring them.
Extensions can be added either:
- at JINJA2 environment creation time
- later on, by invoking the "add_extension()" method passing as input parameter the string with the extension name to add
Later on in this post we will see adding an extension at JINJA2 environment creation time.
Examples of extensions are:
- jinja2.ext.debug
- jinja2.ext.loopcontrols
- jinja2.ext.i18n
More information on JINJA2 extensions are available in the official documentation here.
Variables
Variables are simply referenced by their object's name; when dealing with structured objects with attributes, you can access each single attribute by either using the dot (".") or the square brackets as by the below example:
foo.bar
foo['bar']The variables' value or the variables' attributes value can be easily printed by enclosing them between braces "{{ ... }}".
For example:
{{ foo['bar'] }}Although variables are most of the time defined outside the template, if necessary it is possible to define variables using the "set" keyword. For example:
{% set author="Marco Carcano" %}Statements
The JINJA2 template engine supports either conditional as well as loop statements.
Conditionals
At its most basic form, a conditionals statement in JINJA2 looks like the below one:
{% if True %}
this line will be printed
{% else %}
this line will never be printed
{% endif %}that is a very classic if-else structure very similar to the pattern of many programming languages.
You can negate a boolean value by using the "not" operator - for example:
{% if not False %}
this line will be printed
{% endif %}It is of course it is possible to have a more fine-grained control on the conditionals by adding "else if "cases, for example:
{% if outcome == 'success' %}
success
{% elif outcome == 'skipped' %}
skipped
{% else %}
failed
{% endif %}"Conditionals can be comparison operators ( the same used by Python= != < > <= >= ) as well as built-in tests.
The built-in tests that are more likely to be used are:
defined
Returns True if the variable is defined, or False if it is not. For example:
{% if foo_var is defined %}divisibleby
Checks if a variable is divisible by a number. For example:
{% if foo_number is divisibleby(3) %}even
Returns True if the variable is even. For example:
{% if foo_var is even %}odd
Returns True if the variable is odd. For example:
{% if foo_var is odd %}in
Checks if a value is part of a list. For example:
{% if item in list %}Loops
JINJA2 loops enable it to iterate between list's items.
At its most basic form, the syntax is as follows:
{% for item in items %}
{{ item }}
{% endfor %}it enables you to iterate between a list's elements.
It's also possible to iterate between a dictionary's items, accessing both each single key and related value by invoking the "items" method as follows:
{% for key, value in my_dict.items() %}
the value of {{ key }} key is {{ value }}
{% endfor %}Other methods you may be interested in are the "keys()" for getting the keys and "values()" for getting the values. Anyway, mind that the actual method to use depends on the Python version you are using: if you are dealing with Python2, you must prepend the word "iter" to the method's name. For example, "items()" becomes "iteritems()".
when necessary, by combining the loop with an "else" conditional, it's possible to define a block to be processed when the iterable object is empty - for example an empty list.
For example:
{% for item in items %}
{{ item }}
{% else %}
the list is empty
{% endfor %}As in many languages, you can force the loop to continue from the next iteration:
{% for item in items %}
{% if loop.index is even %}{% continue %}{% endif %}
...
{% endfor %}or break out of the loop:
{% for item in items %}
{% if loop.index >= 10 %}{% break %}{% endif %}
{% endfor %}
Be wary that both "continue" and "break" statements are extensions: in order to use them you must import the "jinja2.ext.loopcontrols" extension in the JINJA2 environment.
special variables
Within a loop JINJA2 automatically defines a few special variables that can be used to perform conditionals checks as necessary.
The ones that are more likely to be used are:
loop.index
the current iterator in a loop (using 1 as the first element's index)
loop.index0
the current iterator in a loop (using 0 as the first element's index)
loop.cycle
helper function to cycle between a list of sequences - please see the example next in this post
loop.first
True on the first iteration
loop.last
True on the last iteration
loop.length
the number of items in the sequence
loop.previtem
the item from the previous iteration of the loop - obviously this special variable is not defined during the first iteration
loop.next
the item from the next iteration of the loop - obviously this special variable is not defined during the last iteration
Suspending The Engine
Sometimes it may be necessary to suspend the engine for a block of text, to avoid processing text fragments that can be confused as statements.
This can be easily accomplished by defining a "raw" block as follows:
{% raw %}
{{ a }} first topic
{{ b }} second topic
{{ c }} third topic
{% endraw %}if you don't need to suspend for a whole block of text, such as for only a few characters you can escape them by enclosing between single quotes.
For example:
these '{{' will be printedExpressions
When describing statements we already saw how to write expressions: the only missing bit is that JINJA2 enables to use the pipe (|) character to pass values from a variable through one or more filters that modifies it.
Filters
The typical use of filters is when printing a value, for example
<P>{{ myvar | lower | escape }}</P>In this case the value of "myvar" piped to the "lower" filter that turns it to lowercase, and the piped to the "escape" filter, that turns it into HTML safe format.
Mind anyway that filters are not used only for printing: another very common use is in conditionals, for example to make sure a value is lowercase before comparison.
The most likely to be used built-in filters are:
default
sets a default value if the object is undefined
lower
converts the value to lowercase
upper
converts the value to uppercase
capitalize
turns the first character of the value uppercase, all the rest lowercase.
title
turns the first character of each word uppercase, all the rest lowercase.
escape
turns the value into HTML safe format - mind that this filter can be shortened by just specifying "e"
first
returns the first element of the list
last
returns the last element of the list
length
returns the length of the list
unique
returns a list of non-duplicated items from the given list
join(delimiter)
returns a string by joining each item of the list using the delimiter as item separator
White spacing
JINJA2 default configuration has both "trim_blocks" and "lstrip_blocks" options disabled - the outcome is that:
- a single trailing newline is stripped if present
- other whitespace (spaces, tabs, newlines etc.) is returned unchanged
This is a good default, but sometimes it is necessary to alter it, for example stripping spaces from the template.
You can easily achieve that by adding a minus sign (-) to the start or end of a block: this will cause whitespaces before or after that block to be removed.
The example template later on in the post shows this feature in action.
Hands On Lab
My loyal readers know that after some theory my posts always have the hands-on lab: in this post we are creating a very simple script that generates an HTML file embedding its own CSS merging a JINJA2 template with a JSON report .
The script is intentionally very basic, focusing only on using JINJA2: this way we will use it as a basis, improving it in a few other posts I'm going to write, this time talking about "argparse" and "smtplib".
The Project Directory
Let's start from creating a project directory tree:
mkdir -m 755 ~/myproject
cd ~/myprojectinside the project's root directory we create a directory dedicated to configuration files:
mkdir -m 755 confand a directory for the spool data files:
mkdir -m 755 spoolJSON Document
In this example , we are dealing with a JSON report generated by an application's health-check, saving the JSON document into a file in the "spool" directory.
Create the "spool/report.json" JSON formatted file with the following contents:
{
"testsuite": {
"name": "Readiness Probe",
"target": "Fancy App - P1",
"started": "2024-12-01T21:02:59Z",
"ended": "2024-12-01T21:03:32Z",
"host": "app-ca-up1a003.prod.carcano.corp",
"outcome": "failed",
"testcases": [
{
"name": "check-ldap-endpoint",
"message": "Connection timeout",
"started": "2024-12-01T21:02:59Z",
"elapsed": "30",
"outcome": "failed"
},
{
"name": "check-ldap-authentication",
"message": "Not run because of parents failures",
"started": "",
"elapsed": "0",
"outcome": "skipped"
},
{
"name": "check-mysql-endpoint",
"message": "Connection was successful",
"started": "2024-12-01T21:03:29Z",
"elapsed": "1",
"outcome": "success"
},
{
"name": "check-mysql-authentication",
"message": "Authentication failed",
"started": "2024-12-01T21:03:30Z",
"elapsed": "1",
"outcome": "failed"
},
{
"name": "check-redis-endpoint",
"message": "Connection was successful",
"started": "2024-12-01T21:03:31Z",
"elapsed": "1",
"outcome": "success"
}
]
}
}the report contains:
- the "testsuite" dictionary, providing global information and a list of "testcase"
- each single testcase provides detailed information about its outcome
JINJA2 Template
As we said JINJA2 merges structured data with a template, so it's time to create the template - create the "conf/message.jn2" file with the following contents:
<!DOCTYPE html>
<html lang="en">
<head>
<style>
.failed{
color: #c64c00;
}
.success{
color: #00aa36;
}
.skipped{
color: #ff9900;
}
.divTable{
display: table;
width: 100%;
}
.divTableBody {
display: table-row-group;
}
.divTableRow {
display: table-row;
}
.divTableHead {
border: 1px solid #b0b0b0;
background-color: #b0b0b0;
color:#ffffff;
font-weight: bold;
display: table-cell;
text-align: center;
vertical-align: middle;
padding: 3px 10px;
}
.divTableCellOdd {
border: 1px solid #999999;
background-color: #e7e7e7;
display: table-cell;
text-align: center;
vertical-align: middle;
padding: 3px 10px;
}
.divTableCellEven {
border: 1px solid #999999;
background-color: #f4f4f4;
display: table-cell;
text-align: center;
vertical-align: middle;
padding: 3px 10px;
}
</style>
</head>
<body>
<h1>{{ testsuite.name | e }}</h1>
<p>The run of the <b>{{ testsuite.target | e }}'s </b> {{ testsuite.name | e }}
on <b>{{ testsuite.host | e }}</b> started at "<em>{{ testsuite.started | e}}</em>"
and ended at "<em>{{ testsuite.ended | e }}</em>" is
<b>{{ testsuite.outcome | e }}</b>.
</p>
<div class="divTable">
<div class="divTableBody"></div>
<div class="divTableRow">
<div class="divTableHead">Testcase</div>
<div class="divTableHead">Message</div>
<div class="divTableHead">Started At</div>
<div class="divTableHead">Elapsed</div>
<div class="divTableHead">Outcome</div>
</div>
{% for testcase in testsuite.testcases -%}
<div class="divTableRow">
<div class="divTableCell{{ loop.cycle('Odd', 'Even') }}">
<p class="">{{ testcase.name | e}}</p>
</div>
<div class="divTableCell{{ loop.cycle('Odd', 'Even') }}">
<p class="">{{ testcase.message | e }}</p>
</div>
<div class="divTableCell{{ loop.cycle('Odd', 'Even') }}">
<p class="">
{% if testcase.started == '' -%}
{%- else -%}
{{ testcase.started | e }}
{%- endif %}
</p>
</div>
<div class="divTableCell{{ loop.cycle('Odd', 'Even') }}">
<p class="">{{ testcase.elapsed }}</p>
</div>
<div class="divTableCell{{ loop.cycle('Odd', 'Even') }}">
<p class="{%- if testcase.outcome == 'success' -%}success
{%- elif testcase.outcome == 'skipped' -%}skipped
{%- else -%}failed{%- endif -%}">
{{ testcase.outcome | e }}
</p>
</div>
</div>
{% endfor %}
</div>
</div>
<p>Yours truly, the automated monitoring suite.</p>
</body>
</html>as you see, this is the template of a HTML document embedding its own CSS (lines 4 - 50).
The CSS defines:
- the color classes "failed", "success" and "skipped" (lines 5 - 13); these are used later on in the template to print the outcome using different colors, so to visually help into getting an unexpected outcome at a glance.
- the classes to render each "div" element as a table element (Table, Body, Row, Head and Cell - lines 14 - 49); more specifically, it defines a class dedicated to Cells belonging to odd rows (lines 34 - 41) and a class dedicated to Cells belonging to even rows (lines 42 - 49).
The JINJA2 statements are in the document's body (lines 52 - 100):
- placeholders enclosed between "{{" and "}}", such as "{{ testsuite.name }}" (line 53), are replaced with the related value from the report file.
- placeholders enclosed between "{%" and "%}" are used to mark the beginning or the end of control structures.
For example:
"{% for testcase in testsuite.testcases %}" (line 68) iterates over each single testcase entry in the report file, storing it into the "testcase" variable. The snippet subjected to the loop ends when the "{% endfor %}" statement is found (line 96).
the other use case of placeholders enclosed between "{%" and "%}" are conditional blocks, such as:
{%- if testcase.outcome == 'success' -%}success
{%- elif testcase.outcome == 'skipped' -%}skipped
{%- else -%}failed
{%- endif -%}this conditional block is used to print "success", "skipped", or "failed" depending on the contents of "testcase.outcome"; in this specific context, the printed word is used to select the color class to be used. Please note also the use of "-" to continue on the same lines.
Rendering With Python
We can now create the script that performs the rendering - create the "render.py" file with the following contents:
#!/usr/bin/python3
import os
import json
import logging
from jinja2 import Environment, FileSystemLoader
# https://docs.python.org/3/library/logging.html#levels
logging_loglevels = {"NOTSET": 0, "DEBUG": 10, "INFO": 20, "WARNING": 30, "ERROR": 40, "CRITICAL": 50}
report_file = "report.json"
template_file = "message.jn2"
def setup_logging():
"""
Setups and return a logger object with a default formatter
:return: the logger object
"""
log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())
log_handler = logging.getLogger()
handler = logging.StreamHandler()
formatter = logging.Formatter("%(asctime)s %(name)-12s %(levelname)-8s %(message)s")
handler.setFormatter(formatter)
log_handler.addHandler(handler)
log_handler.setLevel(logging.INFO)
return log_handler
def render_message():
"""
Renders the report_file by merging it using the template_file
:return: the rendered report
"""
with open(os.path.join(spool_dir, report_file), "r") as report_file_handler:
report = json.load(report_file_handler)
logger.debug("Loaded report {}".format(os.path.join(spool_dir, report_file)))
jinja_env = Environment(loader=FileSystemLoader(config_dir),
extensions=["jinja2.ext.loopcontrols"]
)
logger.debug("Merging JINJA2 template {} with the loaded report".format(
os.path.join(config_dir, template_file)))
msg = jinja_env.get_template(template_file).render(report)
subj = "{} {}".format(report["testsuite"]["name"], report["testsuite"]["target"])
return subj, msg
if __name__ == '__main__':
config_dir = "conf"
spool_dir = "spool"
logger = setup_logging()
logger.setLevel(logging_loglevels[os.getenv("LOGLEVEL", "INFO")])
logger.debug("Script started")
subject, message = render_message()
with open("{}/{}".format(spool_dir, "message.htm"), "w") as message_file_handler:
message_file_handler.writelines(message)
logger.debug("Script ended")
the actual rendering is made by the "render_message()" function, that:
- loads the report file
- initializes the JINJA2 environment, loading extensions and the template file
- performs the rendering, storing the outcome in the msg variable
Please note also the use of the Python standard logging facility, initialized by the "setup_logging" function: this is a best practice for every Python script.
After creating the above script, set is as executable:
chmod 755 render.pyWe are now ready to test everything - just run it as follows:
./render.pyit must generate the "spool/message.htm" HTML file.
You can have a look at it with a browser.
Since we are using the Python standard logging facility, let's check what happens when exporting the "LOGLEVEL" variable with "DEBUG" as value:
export LOGLEVEL=DEBUGre-run the script:
./render.pythis time the output, as expected, is more verbose:
2024-12-04 21:51:53,752 root DEBUG Script started
2024-12-04 21:51:53,752 root DEBUG Loaded report spool/report.json
2024-12-04 21:51:53,756 root DEBUG Merging JINJA2 template conf/message.jn2 with the loaded report
2024-12-04 21:51:53,756 root DEBUG Script endedRendering With Ansible
If you are using Ansible you can very easily exploit the power of the JINJA2 template engine without even needing to write a single line of code.
The same functionalities provided by the above "render.py" script can indeed be easily achieved by running a 1 task only Ansible playbook.
To see this in action, you of course need to install Ansible on your system (if it isn't installed yet).
Since Ansible is actually a pip package, to install it is necessary to run a pip statement: if pip is not installed yet, install it using your linux distribution package manager.
For example, on a Red Hat family system, type:
sudo yum install -y python3-piponce you got pip installed, install Ansible by running the following statement:
python3 -m pip install --user ansibleNow, create the "render.yml"playbook with the following contents:
- hosts: localhost
gather_facts: false
vars:
testsuite: "{{ lookup('file', 'spool/report.json') | from_json | json_query('testsuite') }}"
tasks:
- name: display testsuite value
ansible.builtin.debug:
var: testsuite
verbosity: 2
- name: generate the message.htm file
ansible.builtin.template:
src: conf/message.jn2
dest: spool/message.htmthe above playbook consists of a single play (lines 1-13) that:
- runs on the localhost (line 1)
- disable Ansible facts gathering(line 2)
- loads the contents of the JSON document from the "spool/report.json" file storing them into the "testsuite" variable (line 4)
- run a task (lines 10-13) to generate the "spool/message.htm" file from the "conf/message.jn2" template using the "ansible.builtin.template" Ansible module (this module leverages on the JINJA2 template engine)
The first task (lines 6-9) is not really necessary: I just added it for didactical purposes to show how to display the contents of a variable: on a normal "ansible-playbook" run it is skipped - it is enabled only when specify the "-vv" option of the "ansible-playbook" command.
If you want to know more on Ansible, don't miss my other posts on it.
Please mind that the "json_query" filter requires having the "jmespath" Python package installed on the management host. If you don't have it already installed, install it. For example, on a Red Hat family system, type:
sudo yum install -y python3-jmespathWe are finally ready to run the playbook - just type
ansible-playbook render.yml Once the run completes, have a look at the "spool/message.htm" file freshly generated.
Please mind that if the template file there are statements requiring JINJA2 extensions - such as "{% continue %}" or "{% break %}", you must explicitly enable them while launching the "ansible-playbook" command by setting the "ANSIBLE_JINJA2_EXTENSIONS" environment variable.
For example:
ANSIBLE_JINJA2_EXTENSIONS=jinja2.ext.loopcontrols ansible-playbook render.yml You can of course avoid that by adding to the "ansible.cfg" default stanza the "jinja2_extensions" directive with the list of extensions to enable - for example:
[default]
...
jinja2_extensions=jinja2.ext.loopcontrols
...Footnotes
As you see JINJA2 is a powerful yet easy to use tool not only suitable to be used in Python scripts, but even with Ansible.
As I said the example Python script used in this post is intentionally basic to keep the focus on JINJA2, but we will see how to improve it adding command line parsing arguments using "argparse"; it would also be nice to improve the script adding notification capabilities, such as send the HTML document as a mail message: those will be the the topics of the next posts.
If you appreciate this strive please and if you like this post and any other ones, just share this and the others on Linkedin - sharing and comments are an inexpensive way to push me into going on writing - this blog makes sense only if it gets visited.
