The Python Argparse Tutorial - Argparse HowTo post provides a practical example of how to parse command line parameters and options using the argparse module.
Knowing how to add command line paramenters and options to Python scripts is a must-have skill, since it enables them to better adapt to customers' needs, allowing them to enable or disable specific script features, or providing information on the operational environment structure, such as specific directory paths different from the default ones.
As a professional, you are supposed to always add command line arguments support to your scripts, so to make them always fit to the user-specific scenarios.
What Is Argparse
Argparse is a built-in Python module that streamlines parsing command-line arguments and options in scripts. It allows developers to define expected inputs, such as flags (e.g., --verbose), optional parameters, and positional arguments, while providing automatic type checking, default values, and help message generation.
Beyond that, it offers robust type validation and user-friendly help output. This comprehensive feature set makes it indispensable for any professional-grade Python script.
Without argparse or equivalent support, Python scripts often lack command-line flexibility: they might rely on hardcoded values or manual sys.argv parsing, which is tedious and prone to errors. Worse, they could depend solely on positional arguments, requiring inputs in a strict order without descriptive names, leading to inflexibility for users who forget sequences or need to skip options. This results in brittle, hard-to-maintain code.
Incorporating argparse is vital for professional scripts, as it boosts usability through built-in help messages (via -h or --help), validates inputs to avoid runtime errors, supports subcommands for complex tools, and ensures scalability as projects expand. Ultimately, it makes your code feel intuitive and robust, akin to well-designed CLI applications.
Improving The Command Line
In the JINJA2 With Python Tutorial JINJA Ansible Howto post we developed the "render.py" script to illustrate the use of JINJA in Python. As we anticipated in that post, despite the script is already properly working, it still lacks the versatility typical of a professional product, since it does not still support command line arguments to fit the target installation environment.
More specifically, in a Linux filesystem compliant installation:
- The configuration directory would be beneath the "/etc" directory tree
- The spool directory would be beneath the "/var/spool" directory tree
- The script itself would be beneath the "/usr/local/bin" or in a dedicated directory tree beneath "/opt"
Since the script expects both the configuration and the spool directories to be subdirectories of the current working directory, the script is definitely still in the "works for me" stage.
The first step for using argparse is importing its package as follows:
import argparsethen, we move straight to the main block.
First as first, we modify the default value for the "config" and "spool" directories so to be subdirectories of the directory where the script is installed - just replace the lines:
config_dir = "conf"
spool_dir = "spool"
with:
config_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "conf")
spool_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "spool")we can now instantiate the parser variable as ArgumentParser, providing as description the help message that will printed when the script will be invoked with mismatching parameters or when the help message is intentionally invoked.
parser = argparse.ArgumentParser(
description="Generate an HTML document from a JSON report")
then, let's add the statements for parsing the command line:
parser.add_argument("-c", "--config-dir", dest="config_dir", type=str, default=None,
help="path to the directory containing the configuration files - if omitted, it assumes it's the same"
"containing the script itself")
parser.add_argument("-p", "--spool-dir", dest="spool_dir", type=str, default=None,
help="path to the directory containing the report files - if omitted, it assumes it's the same containing"
" the script itself")
parser.add_argument("-f", "--report-file", dest="report_file", type=str, default=None,
help="the file name of the JSON report - if omitted, it assumes '{}'".format(report_file))
here you can add as many command line arguments as needed by your script.
Please note how:
- we provide both the single character and the long parameter (for example -c and --config-dir)
- we specify the attribute where the parameter's value will be stored using the dest parameter
- we specity the expected format using the type parameter
- we provide a default using default parameter
- we provide the help message for each single argument using the help parameter
after specifying all the necessary parameters, we invoke the parse_args method to add their values as attributes ot the args object:
args = parser.parse_args()To eliminate (or at least reduce to the bare minimum) unpredictable outcome caused by imporper values entered by the user, professional scripts must also validate the input passed by the command line arguments, failing with a easy to understand message explaining the user's error.
Since aby best practices require error messages to be printed to the standard error, we must import the "sys" package, adding it to the beginning of the script:
import sysIn this script's context, input validation consists in checking that the specified directories exist and contain the expected files.
We can implement this in two distinc functions - the validate_spool function:
def validate_spool(directory):
"""
After overriding spool_dir with the related argparse argument (if specified on the command line),
it validates spool_dir's contents, ensuring it exists and that it contains the report_file file.
:param directory: the default spool directory
:return: the actual spool directory (overridden as necessary)
"""
global report_file
if args.spool_dir is not None:
directory = args.spool_dir
try:
if not os.path.isdir(directory):
raise FileNotFoundError
except FileNotFoundError:
print("Directory {} not found".format(directory), file=sys.stderr)
exit(1)
if args.report_file is not None:
report_file = args.report_file
try:
if not os.path.isfile(os.path.join(directory, report_file)):
raise FileNotFoundError
except FileNotFoundError:
print("File {} not found".format(os.path.join(directory, report_file)), file=sys.stderr)
exit(1)
return directory, report_fileand the validate_config function:
def validate_config(directory):
"""
After overriding config_dir with the related argparse argument (if specified on the command line),
it validates config_dir's contents, ensuring it exists and that it contains the template file.
:param directory: the default configuration directory
:return: the actual configuration directory (overridden as necessary)
"""
if args.config_dir is not None:
directory = args.config_dir
try:
if not os.path.isdir(directory):
raise FileNotFoundError
except FileNotFoundError:
print("Directory {} not found".format(directory), file=sys.stderr)
exit(1)
try:
if not os.path.isfile(os.path.join(directory, template_file)):
raise FileNotFoundError
except FileNotFoundError:
print("File {} not found".format(os.path.join(directory, template_file)), file=sys.stderr)
exit(1)
return directorywe must now focuson on the main part of the script.
Back in the main block, we must add the call to the above two functions:
spool_dir, report_file = validate_spool(spool_dir)
config_dir = validate_config(config_dir)as you probably inferred, their purpose is twofold:
- they validate the directory and contents
- they override the default values with the ones passed through the command line arguments
We have finally reached the time for having a go with the freshly added argparse feature:
./notify.py -c conf -p spool -f report.json
please remind that if you need to troubleshoot you can export the "LOGLEVEL" variable increasing the verbosity. For example:
export LOGLEVEL=DEBUGand then re-run the sript.
The Whole Script
For your convenience, here is a copy of the whole script integrated with the argparse additions.
Mind that an offering using the below cup is always welcome, since it helps me running this blog:
#!/usr/bin/python3
import os
import sys
import json
import logging
import argparse
from jinja2 import Environment, FileSystemLoader
# https://docs.python.org/3/library/logging.html#levels
logging_loglevels = {"NOTSET": 0, "DEBUG": 10, "INFO": 20, "WARNING": 30, "ERROR": 40, "CRITICAL": 50}
report_file = "report.json"
template_file = "message.jn2"
def setup_logging():
"""
Setups and return a logger object with a default formatter
:return: the logger object
"""
log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())
log_handler = logging.getLogger()
handler = logging.StreamHandler()
formatter = logging.Formatter("%(asctime)s %(name)-12s %(levelname)-8s %(message)s")
handler.setFormatter(formatter)
log_handler.addHandler(handler)
log_handler.setLevel(logging.INFO)
return log_handler
def render_message():
"""
Renders the report_file by merging it using the template_file
:return: the rendered report
"""
with open(os.path.join(spool_dir, report_file), "r") as report_file_handler:
report = json.load(report_file_handler)
logger.debug("Loaded report {}".format(os.path.join(spool_dir, report_file)))
jinja_env = Environment(loader=FileSystemLoader(config_dir),
extensions=["jinja2.ext.loopcontrols"]
)
logger.debug("Merging JINJA2 template {} with the loaded report".format(
os.path.join(config_dir, template_file)))
msg = jinja_env.get_template(template_file).render(report)
subj = "{} {}".format(report["testsuite"]["name"], report["testsuite"]["target"])
return subj, msg
def validate_spool(directory):
"""
After overriding spool_dir with the related argparse argument (if specified on the command line),
it validates spool_dir's contents, ensuring it exists and that it contains the report_file file.
:param directory: the default spool directory
:return: the actual spool directory (overridden as necessary)
"""
global report_file
if args.spool_dir is not None:
directory = args.spool_dir
try:
if not os.path.isdir(directory):
raise FileNotFoundError
except FileNotFoundError:
print("Directory {} not found".format(directory), file=sys.stderr)
exit(1)
if args.report_file is not None:
report_file = args.report_file
try:
if not os.path.isfile(os.path.join(directory, report_file)):
raise FileNotFoundError
except FileNotFoundError:
print("File {} not found".format(os.path.join(directory, report_file)), file=sys.stderr)
exit(1)
return directory, report_file
def validate_config(directory):
"""
After overriding config_dir with the related argparse argument (if specified on the command line),
it validates config_dir's contents, ensuring it exists and that it contains the template file.
:param directory: the default configuration directory
:return: the actual configuration directory (overridden as necessary)
"""
if args.config_dir is not None:
directory = args.config_dir
try:
if not os.path.isdir(directory):
raise FileNotFoundError
except FileNotFoundError:
print("Directory {} not found".format(directory), file=sys.stderr)
exit(1)
try:
if not os.path.isfile(os.path.join(directory, template_file)):
raise FileNotFoundError
except FileNotFoundError:
print("File {} not found".format(os.path.join(directory, template_file)), file=sys.stderr)
exit(1)
return directory
if __name__ == '__main__':
config_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "conf")
spool_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "spool")
logger = setup_logging()
logger.setLevel(logging_loglevels[os.getenv("LOGLEVEL", "INFO")])
parser = argparse.ArgumentParser(
description="Send an email notification")
parser.add_argument("-c", "--config-dir", dest="config_dir", type=str, default=None,
help="path to the directory containing the configuration files - if omitted, it assumes it's the same"
"containing the script itself")
parser.add_argument("-p", "--spool-dir", dest="spool_dir", type=str, default=None,
help="path to the directory containing the report files - if omitted, it assumes it's the same containing"
" the script itself")
parser.add_argument("-f", "--report-file", dest="report_file", type=str, default=None,
help="the file name of the JSON report - if omitted, it assumes '{}'".format(report_file))
args = parser.parse_args()
spool_dir, report_file = validate_spool(spool_dir)
config_dir = validate_config(config_dir)
logger.debug("Script started")
subject, message = render_message()
with open("{}/{}".format(spool_dir, "message.htm"), "w") as message_file_handler:
message_file_handler.writelines(message)
logger.debug("Script ended")
Footnotes
We have reached the end of this short post. As you see, argparse provides a convenient and easy way to parse command line arguments in Python scripts. It not only eases parsing the arguments, but also provides automatic generation of help messages and automatic format validation.
It the next post we will improve the scritp once more, adding email notification in HTML format using the Python SMTPlib library. Stay tuned.
If you appreciate this post any other in this blog, just share this and the others on Linkedin - sharing and comments are an inexpensive way to push me into going on writing - this blog makes sense only if it gets visited.
Also concrete contributions to the maintenance of this blog are also very welcome, ... just put your tip in the below cup.
