I sometimes heard claims such as “shell scripting is no more necessary since now we have Python“, or “I don’t like Python, I can do everything with shell scripts“.

Both of the above claims are too simplistic: skilled professionals that know both of them for true know also that they both have pros and cons. So the right approach probably is to use the one that best fit the use case in terms of delivery time and maintainability.

For example:

if the use case is housekeeping of files and directories, or automating tasks that run a lot of shell commands, a shell script certainly wins over a Python script in terms of delivery time
when it comes to data search and manipulation, in simplest use cases may be still convenient to use shell scripting, but when complex formats such as JSON, XML or YAML are involved, it is certainly more effective to develop using Python
the same applies when developing scripts that connect to REST API: it is straightforward that Python is the best choice

So, to keep it short, professionals should know both shell scripting as well as Python, so to pick the right one for the particular use case they are working onto.

Git With Python HowTo GitPython Tutorial And PyGit2 Tutorial

Posted on September 11, 2024November 14, 2024 by Marco Antonio Carcano

Interacting with Git using Python is a very common use case in the DevOps field: very often it is necessary to checkout application’s or scripts along with their configuration or even just checkout versioned configurations. Although more rare, it is sometimes necessary to update the checked out contents and push the committed version back to the “origin” remote repository. In the "Git With Python HowTo GitPython Tutorial And PyGit2 Tutorial" post we play with the two most commonly used Python libraries used to interconnect to Git: gitpython and pygit2.

Python Serialization as JSON or YAML exploiting YAML TAGS

Posted on November 30, 2022November 14, 2024 by Marco Antonio Carcano

Python enables you to easily serialize objects as either JSON or YAML: very often it is very convenient to leverage on these features exploiting them to enhance your own object. YAML serialization comes almost for free if you derive your classes from the YAML object, whereas automatic instantiation of objects from a YAML document requires a little bit of work and leverages on YAML tags. As for JSON serialization, it is bloody useful and enables you to quickly and easily serialize the contents of your objects into JSON documents that can be exploited for example to interact with or to develop a REST API.

This post, Python Serialization as JSON or YAML exploiting YAML TAGS, shows you how to develop a Python package that provides such kinds of objects. As usual we take particularly care of the code style, using a very clean and portable design and adhering best practices.

The operating environment used in this post is Red Hat Enterprise Linux 9 or Rocky Linux 9 - using a different environment may lead to having to adapt or even change things.

The Ultimate AWK Tutorial For Professionals

Posted on June 14, 2022November 14, 2024 by Marco Antonio Carcano

AWK is a powerful pattern scanning and processing language developed by Alfred Aho, Peter Weinberger and Brian Kernighan at Bell Labs - the name of this tool is indeed derived by concatenating the letter of their surnames to one another. It is one of that tools that every Linux professionals (not only the more seasoned ones) must be skilled on, since it is broadly used in a lot of shell scripts that very often are inherited from predecessors and that must be maintained: the sad truth is that very often is not worth the effort to rewrite them using other more modern languages, so knowing how to deal with it can really ease your life. And anyway, ... sometimes it requires much less time to code an AWK one liner than a Python script, so knowing how and when to use AWK is certainly a valuable skill still nowadays.
The aim of "The Ultimate AWK Tutorial For Professionals" is not to provide a complete explain about how to code with AWK - there are more modern and handy languages such as Python nowadays: I just want to provide a very quick yet comprehensive walkthrough on it focusing on how to write AWK one-liners that are often embedded in shell scripts or that you can use to sort out common system administration tasks. That's why I'm also showing some real-life use cases with AWK one-liners that can very quickly and easily sort things out.

Sed – the Stream EDitor

Posted on December 30, 2021November 14, 2024 by Marco Antonio Carcano

Sed is a command line tool that can really do amazing stream manipulations: despite it certainly being a "seasoned" tool, it is very likely that there are a lot of sed one-liners inside your Company's scripts.

Having at least an understanding of it is a must if you want to be able to maintain this legacy stuff that very often is not worth the effort to rework.

And anyway, when having to deal with quick and dirty solutions that rely on shell scripts, or when writing documentation with shell commands that can be easily replaced by a copy and paste by the reader... it's still an excellent tool honestly I cannot work without.

The aim of this post is to provide an easy tutorial to quickly learn how to use sed in every situation that can be easily sorted out with a sed one-liner.

In memory of Lee E. McMahon, contributor to early versions of the Unix operating system, ... and of course in particular of the sed stream editor.

Packaging a Python Wheel as RPM

Posted on October 31, 2021November 14, 2024 by Marco Antonio Carcano

This is the last post of the trilogy dedicated to how to set up a well structured Python project, developed with professional style, suitable to be used within the context of a Continuous Integration toolchain. This time we focus on how to package all we have done so far as RPM packages, showing how to break down everything into subpackages that also perform post installation tasks.
If you haven't read the previous two posts, you must do it right now since they are requisite to understand this post. In addition to that this post relies onto objects that are being created in the previous posts.

Read them in the following order:

The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.

Python Setup Tools

Posted on October 10, 2021November 14, 2024 by Marco Antonio Carcano

This post is focused on packaging distribution modules using "setuptools" and publish them onto PyPI. To better understand these concepts we will clarify the concept of module and, since most of the people use the term "package" in place of either "import package" and "distribution package", we will also clarify the term "package" too to avoid confusion. In addition to that, we will highlight the differences, pros and cons of source, binary and wheel distribution packages. All of this taking care of "styling" things so that they can easily be used within a Continuous Integration environment.

The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.