I sometimes heard claims such as “shell scripting is no more necessary since now we have Python“, or “I don’t like Python, I can do everything with shell scripts“.

Both of the above claims are too simplistic: skilled professionals that know both of them for true know also that they both have pros and cons. So the right approach probably is to use the one that best fit the use case in terms of delivery time and maintainability.

For example:

  • if the use case is housekeeping of files and directories, or automating tasks that run a lot of shell commands, a shell script certainly wins over a Python script in terms of delivery time
  • when it comes to data search and manipulation, in simplest use cases may be still convenient to use shell scripting, but when complex formats such as JSON, XML or YAML are involved, it is certainly more effective to develop using Python
  • the same applies when developing scripts that connect to REST API: it is straightforward that Python is the best choice

So, to keep it short, professionals should know both shell scripting as well as Python, so to pick the right one for the particular use case they are working onto.

AWK is a powerful pattern scanning and processing language developed by Alfred Aho, Peter Weinberger and Brian Kernighan at Bell Labs - the name of this tool is indeed derived by concatenating the letter of their surnames to one another. It is one of that tools that every Linux professionals (not only the more seasoned ones) must be skilled on, since it is broadly used in a lot of shell scripts that very often are inherited from predecessors and that must be maintained: the sad truth is that very often is not worth the effort to rewrite them using other more modern languages, so knowing how to deal with it can really ease your life. And anyway, ... sometimes it requires much less time to code an AWK one liner than a Python script, so knowing how and when to use AWK is certainly a valuable skill still nowadays.
The aim of "The Ultimate AWK Tutorial For Professionals" is not to provide a complete explain about how to code with AWK - there are more modern and handy languages such as Python nowadays: I just want to provide a very quick yet comprehensive walkthrough on it focusing on how to write AWK one-liners that are often embedded in shell scripts or that you can use to sort out common system administration tasks. That's why I'm also showing some real-life use cases with AWK one-liners that can very quickly and easily sort things out.

Read more >

Sed is a command line tool that can really do amazing stream manipulations: despite it certainly being a "seasoned" tool, it is very likely that there are a lot of sed one-liners inside your Company's scripts.

Having at least an understanding of it is a must if you want to be able to maintain this legacy stuff that very often is not worth the effort to rework.

And anyway, when having to deal with quick and dirty solutions that rely on shell scripts, or when writing documentation with shell commands that can be easily replaced by a copy and paste by the reader... it's still an excellent tool honestly I cannot work without.

The aim of this post is to provide an easy tutorial to quickly learn how to use sed in every situation that can be easily sorted out with a sed one-liner.

In memory of Lee E. McMahon, contributor to early versions of the Unix operating system, ... and of course in particular of the sed stream editor.

Read more >

This is the last post of the trilogy dedicated to how to set up a well structured Python project, developed with professional style, suitable to be used within the context of a Continuous Integration toolchain. This time we focus on how to package all we have done so far as RPM packages, showing how to break down everything into subpackages that also perform post installation tasks.
If you haven't read the previous two posts, you must do it right now since they are requisite to understand this post. In addition to that this post relies onto objects that are being created in the previous posts.

Read them in the following order:

  1. Python Full Featured Project
  2. Python Setup Tools
The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.

Read more >

This post is focused on packaging distribution modules using "setuptools" and publish them onto PyPI. To better understand these concepts we will clarify the concept of module and, since most of the people use the term "package" in place of either "import package" and "distribution package", we will also clarify the  term "package" too to avoid confusion. In addition to that, we will highlight the differences, pros and cons of source, binary and wheel distribution packages. All of this taking care of "styling" things so that they can easily be used within a Continuous Integration environment.

The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.

Read more >

When it comes to talk about scripting, we cannot avoid talking about the probably most famous of the shells: the Bourne Again SHell. Thoroughly explaining it would require a whole book, so as usual in this post we explore only the features that it’s theory likely the reader should learn. The post is not intended to be easily understood by new-bies: it is structured as a cheat sheet, so the reader can use it as a quick reference when needed, but this approach has the drawback that there’s not much room to elaborate things enough.

Read more >

This is the first of a set of three posts aimed at showing how to create a full featured Python3 project: since the topic is quite massive, I decided to split it into three different posts. In this post we do not only quickly see how to develop a full featured Python application, since I wanted to do something that shows a lot of things, such as:

  • creating Python objects
  • put the custom Python objects inside a Python package within the scope of our own namespace
  • develop accordingly to encapsulation rules, by implementing getters and setters methods that look like regular attributes by exploiting decorators
  • use the standard Python logging facility, configuring everything with an external settings file
  • altering the __eq__ comparison so to consider two objects as equal when one of their attribute has the same value
  • implementing comparison methods and the __iter__ method, so to be able to use Python standard functions such as sorted() to sort it also in reverse order
  • exploit total_ordering to make an object fully sortable

The next parts of this post will be on the following topics

The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.

Read more >