Regular Expressions Quick Guide

A pillar every Linux professional must have are certainly Regular Expressions: they are broadly used every time is required to lookup or substitute a pattern that matches some criteria. Tools such as grep and sed miss almost all of their power if who is using them does not have a good understanding of Regular Expressions. This is really a huge topic: there's more than one book that is fully focused on regular expressions - this post is only a quick guide: the aim of this post is to let the reader get the gist of what Regular Expressions are by explaining everything that is needed to know to face the common use cases that may arise during daily work..

The purpose of Regular Expressions

Regular expressions provide a way to express a pattern to match by using a standard syntax made of a sequence of characters. This post illustrate their usage by examples using grep.

When dealing with Perl-Compatible Regular Expression (PCRe), you can get some handy documentation
by installing pcre-devel RPM package: this let you use PCRE syntax man page as follows:

man pcresyntax

Meta-Characters

Metacharacters are reserved symbols used to assist in matching: the following list summarizes the most commonly used meta-characters:

it is the wildcard character: this means that can match any single character

the Caret character means the beginning of the line

grep "^Nov 10" messages.1

the Dollar character means the end of the line

grep "terminating$" messages

the following pattern searches for a line of only three characters in "foo.txt" file:

grep "^...$" foo.txt

The following example can be used to print the contents of Python files stripping the comments:

grep -v "^#\|^'\|^\/\/" mycode.py

commented lines in Python begin by character #, ' or // : the supplied regular expression matches only when a commented out line is found, then the "-v" option of grep reverts the match.

Sometimes it is needed to use a pattern that includes dots that should not be evaluated as a wildcard: simply escape the dot character using a backslash as in the following example:

grep "192\.168\.50\.31"  /var/log/secure

Class of characters

It is possible to express a group (class) of characters by enclosing the characters between square brackets [] using a specific syntax. For example:

[hc]

is a class of characters that contains only h and c characters

[0-9]

is a class of characters that contains all the digits (it goes from 0 to 9)

[a-zA-Z]

is a class of characters that contains all the alphas

Extended regular expressions introduce some additional built-in character classes - the most used are:

[:digit:]

all the digits from 0 to 9

[:alnum:]

any alphanumeric character - from 0 to 9 OR from A to Z or from a to z

[:alpha:]

any alpha character from A to Z or from a to z

[:lower:]

any lowercase alpha character from a to z

[:upper:]

any uppercase alpha character from A to Z

[:blank:]

space and TAB characters only

[:punct:]

punctuation characters - [][!"#$%&'()*+,./:;<=>?@\^_`{|}~-]

grep does not evaluate extended regular expressions by default: their support must be explicitly enabled
by supplying the -e command switch when needed. For example:

grep -e "ntpd\[[[:digit:]]\+\]" /var/log/messages

The above command checks for all of the matches of words that begin with "ntpd" followed by a number of digits

Negating a class of characters

When the ^ (Caret) meta-character precedes a character class it negates the class of characters to match: for example

grep -i  "^[^aeiou]" /usr/share/dict/linux.words

matches only when it find words without vowels.

Quantification

The following symbols are used to quantify the number of times the meta-character or class of characters should be repeated to have a match :

The previous character or class should match at most one time - this means that it can also match
zero times

grep "hi \?hello" input

the previous character or class match zero or more times.

the previous character or class match at least one time

grep "hi \+hello" input

Are you enjoying these high quality free contents on a blog without annoying banners? I like doing this for free, but I also have costs so, if you like these contents and you want to help keeping this website free as it is now, please put your tip in the cup below:

Even a small contribution is always welcome!

Conversely from GNU and Posix RegEx, that implements only the greedy behavior, Perl RegEx implements three different flavors of quantification: let's test them with the string "http://grimoire.carcano.ch/webfolder/foofile.html" - issue:

URL="http://grimoire.carcano.ch/webfolder/foofile.html"

greedy

the default – it enlarges the match as much as possible - for example:

echo $URL|perl -pe 's|(http://.*/).*|\1|'

the output is

http://grimoire.carcano.ch/webfolder/

reluctant

sometimes told also lazy – it considers the match as small as possible – for example:

echo $URL|perl -pe 's|(http://.*?/).*|\1|'

the output is

http://grimoire.carcano.ch/

possessive

perform a greedy match, but it returns the whole string – for example:

echo $URL|perl -pe 's|(http://.*+/).*|\1|'

the output is:

http://grimoire.carcano.ch/webfolder/foofile.html

that is the whole string indeed.

Neither basic nor extended Posix/GNU regex recognize the non-greedy quantifier
– we used Perl in the examples indeed. In addition to that take in account that the reluctant
quantifier "costs" much more the greedy one, … so use it only when you really need it.

The following table summarizes quantifiers and the flavour-specific syntax:

GNU/Posix

Perl

Quantifier

Greedy

Lazy

Possessive

Zero or more

One or more

Zero or none

it is also possible to state the number of times the match should happen:

{n}

the previous character or class match exactly n times

grep  "^[0-9]\{5\}$" number

{n,}

the previous character or class match n or more times

grep "[0-9]\{5,\}" number

{,m}

the previous character or class match at most m times

{n,m}

the previous character or class match at least n times, but not more than m times

grep  "^[0-9]\{1,5\}$" number

Is text filtering just about matching raw string patterns? Successfully mastering RegEx backreferences, handling POSIX alphanumeric classes, and debugging complex IPv4 validation structures via custom terminal statements is an excellent operational milestone.

However, writing independent data filtering strings utility-by-utility introduces invisible maintenance debt as application code bases grow. True enterprise data consistency requires you to look past raw command execution: you must know how to govern your automation code through modern project layouts, strict pre-commit compliance scanning, and cloud-agnostic architectures.

If you want to check whether your engineering foundations are ready to scale toward advanced continuous integration, jump directly to the Apress Blueprint Box below to discover how to boost and evolve your career using a self-paced learning path.

Examples

The following example uses a class of only digits:

grep "[0-9]\+ times" /var/log/messages

it matches every time at least a digit is found - note how we had to escape the plus quantification meta-character

This example validates an IPv4 address:

egrep  '\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' input

this last example uses the OR operator to matches some variations of “Object Oriented”:

grep "OO\|\([oO]bject\( \|\-\)[oO]riented\)"

ls |xargs rm

Beyond Stream Matching: Building the DevOps and DevSecOps Foundations

Successfully validating multiline record patterns, isolating text configurations, and leveraging advanced greedy or lazy expressions to automatically parse runtime profiles are top-tier systems engineering skills. Yet, formatting standalone filter criteria and local terminal matchers represents just one single technical brick in the massive wall of modern DevOps and DevSecOps engineering on Linux. In complex enterprise networks, you cannot operate within isolated, unvalidated script parameters: you must master the upstream methodology required to develop, test, and validate your code through strict automated compliance gates before any data asset enters your production delivery infrastructure.

When you evaluate your daily systems engineering and scripting tasks, are you completely confident in your ability to build rock-solid software supply chains from the first line of code? Or are you simply invoking local matching utilities and parsing text strings without mastering the underlying project governance and quality frameworks underneath?

My book, "DevSecOps and DevOps for Linux: The Foundations", published by Apress, was specifically designed to bridge this critical gap between basic text manipulation and automated cloud-native infrastructure delivery. Through a comprehensive, lab-driven blueprint, the volume guides you step-by-step through the mechanics of the Data & Core Automation Brick—mastering advanced structured data parsing (XML, JSON, YAML) using Python, jq, and yq—alongside robust advanced shell scripting architectures governed by strict automated pre-commit compliance linters and testing. Built entirely on cloud-agnostic, open-source principles, this practical learning path ensures you systematically identify and fill your technical gaps, allowing you to turn basic regex strings into a secure, fully automated pipeline hosted on Kubernetes.

Key insights covered in this volume:

The Holistic Skills Set Brick: Bridge technical engineering with team management frameworks. Master Scrum, Kanban, and Lean methodologies to design system architectures aligned with real corporate workflows.
The Shell Scripting & Unix Tools Brick: Build rigorous operational foundations. Master advanced Bash shell scripting architecture while learning how to combine core Unix tools into robust, repeatable, and enterprise-ready host automations.
The Version Control Engineering Brick: Move past basic commits. Dive deep into Git version control, mastering feature-branch workflows, repository lifecycle management, and complex conflict resolution.
The Data & Core Automation Brick: Build bulletproof data processing setups. Learn advanced RegEx, how to operate using evergreen tools such as Grep, Sed, and AWK, and how to master structured data parsing (XML, JSON, YAML) using Python and tools like xmlstarlet, jq, and yq.
The Modern Python & Automation Brick: Develop a modern Python project using pyproject.toml with pytest-based unit tests, governing the project with GNU Make for testing, building, and digitally signing RPM packages. The project is presented in an evolving fashion, showing how features are added step by step, highlighting how a properly structured Python project can be improved and evolved with minimal or no rework at all.
The Linux OS Hardening & PKI Brick: Learn the real mechanics of security. Implement X.509/PKI architectures, TLS configurations, and GPG encryption and signing, while mastering low-level kernel defenses like SELinux and Linux Capabilities.
The Compliance Check and Shift-Left Security Brick: Learn how to leverage the pre-commit framework to automate compliance checks with Pylint and Flake8, and perform security scans with Bandit and Safety, extending the security audit to the full software supply chain.
The Application Integration Brick: Master the foundational protocols used to securely interconnect enterprise microservices, including HTTP, REST, OpenAPI, SOAP, and LDAP/LDAPS.
The Infrastructure Delivery Brick: Put theory into practice with vertical, real-world labs. Move from basic scripts to engineering Ansible architectures, rootless Podman setups, image creation via Buildah, and complete Pulp3 deployments using Docker Compose.
The Enterprise GitOps Pipeline Brick: Tie everything together by automating your software supply chain. Build complete continuous deployment workflows using Gitea CI pipelines hosted natively on Kubernetes (RKE2).

Footnotes

Here it ends the quick-guide: although I use regular expressions, I cannot always remember everything, so I wrote it for my own needs, but as it grows I thought that it has become quite mature and that somebody else may benefit if I publish it. So here it is: I hope you enjoyed it.

I hate blogs with pop-ups, ads and all the (even worse) other stuff that distracts from the topics you're reading and violates your privacy. I want to offer my readers the best experience possible for free, ... but please be wary that for me it's not really free: on top of the raw costs of running the blog, I usually spend on average 50-60 hours writing each post. I offer all this for free because I think it's nice to help people, but if you think something in this blog has helped you professionally and you want to give concrete support, your contribution is very much appreciated: you can just use the above button.