Wayland Tutorial – A Wayland HowTo

The first Wayland release is dated 2012, and it was of course in a very early stage. Now 16 years have passed since the initial design, they say Wayland is mature enough that it is safe to remove X11 sessions. In the Wayland Tutorial - A Wayland HowTo post we will go through everything it is worth the effort to know about Wayland, trying to answer the question: do Wayland replace the whole X Window system, or do it replace just the X11 protocol?

Why Wayland Was Started

As we saw in the X Window Tutorial - X Display Server HowTo And Cheatsheet post, the X Window System is really seasoned (1984, so it's 40 years old): the software has been maintained and reworked a lot of times in all these decades, and of course the X11 protocol needed several revisions. In 2008 they decided going on with the improvement of the X11 protocol was not worth the effort and so it was a dead end: they decided drastically cut budget from the X11 protocol's maintenance and spin up a new project called Wayland, being aware that it meant a complete rewrite of the software's full stack.

Please, read this carefully:

replace the X11 protocol

that is not:

replace the whole X Window System

Indeed they are replacing the X11 protocol with the Wayland protocol.

IMHO it is not correct (as instead I very often hear people saying) that Wayland replaces the X Window System - you can replace anything when you provide all its core features or provide a valid replacement of them - they clearly stated they are not to implement XDMCP neither to provide a replacement - this means no longer remote logins to display managers, so graphical terminal gone forever. That is a core feature of the X Window System. Avoiding implementing it by just saying "nobody or few are currently using it" is a very questionable and too dictatorial answer for the free software world, and very distant from Linux's way.

The project was started in 2008 by Kristian Høgsberg (a Red Hat's developer), but we had to wait until 2012 for the project to be mature enough for its first release.

Since we are speaking of a completely new protocol (actually a suite of protocols), it is clear we are talking about something that breaks everything with the past: for that reason they implemented a broker component called Xwayland meant to provide an X11 Display Server to be used by the graphical applications during the expected very long transition time.

In addition to that the Display Managers are providing an option for configuring the system to either run X11 or Wayland graphical sessions.

This year (2024) Fedora (and actually some other distributions) announced they are switching to Wayland-only graphical sessions: this means that, despite the Xwayland broker being still available, it will no longer be possible to configure the system to run legacy X11 graphical sessions.

A decision I personally don't like at all: I'm not against Wayland, but they don't care about remote graphical Terminals, and so this change means they are shutting down the Remote Display server feature, which is one of the X Window system key features.

If you have not read it yet, I advise reading the X Window - Display Manager (Remote XDMCP) And Thin Clients post, since it provides a nice lab showing how to set up Display Servers querying a remote Display Manager, protecting the traffic through a VPN from the Terminal client to the remote server.

The Wayland protocols

Wayland is a set of protocols governing how a compositor draws graphics on the screen and how the applications interact with the compositor's drawing facilities.

The protocol implementation is provided by the "libwayland-client" and "libwayland-server" libraries, which provide a versioned API with a very limited features set that can be exploited by compositors.

Some features you would expect from it, and instead are missing, are:

• screen locking
• screen sharing
• cross-app window activation
• non-integer scaling
• ....

Compositors must of course implement the missing features on their own, that in turn means they are doing it their own way: this leads to fragmentation of the code base across the different compositors.

This fragmentation leaded to a lot of criticism since, besides being cumbersome to develop and maintain, it can lead to unpredictable side effects such as having clients perfectly working on a compositor having compatibility problems with other ones, besides having to wait for a long time for a feature being developed/fixed on every compositors and such.

Wayland's very small feature set (especially when compared to X11) led to several criticisms. Anyway during the years some extensions have been published, so currently the formerly missing features such as screen sharing, fractional scaling or global shortcuts are now available. They even developed a few frameworks to ease the development of compositors, so to avoid the fragmentation of the code base across the different compositors: the only mature one is "wlroots" - "wlc", "smithay" and "libweston" are mostly deprecated. The only pity is that blasoned compositors such as Mutter and KWin were developed before these libraries were mature, and so they are still (and probably will be for a long time) using their own libraries.

Wayland Compositors

As we saw,a Wayland compositor is a software implementation that covers the same use case of a X11 Display Server with a Window Manager - as we saw in the "X Window Tutorial - X Display Server Howto And Cheatsheet", a Display Server is a machine running an instance of a Display Server software: for example, when dealing with X Window, the Display Server is the X.org server running the X11 protocol on the user's workstation.

It is said "Compositor" rather than "Display Server" because it is actually the Wayland compositor that embeds the Display Server, hence why there are several Wayland compositors around, and why each Desktop Environment has its own.

The obvious consequence is that, in order to implement Wayland, blasoned Window Manager such as "Mutter" (the GNOME Window Manager) or "Kwin" (the KDE Window Manager) had to embed and implement the Wayland Display Server too - currently they implement a dual personality: the legacy X Windows Manager (so basically an X Window client registering itself for Substructure Redirection,) or an actual Wayland Display Server.

If you have not read it yet, I strongly advise reading the X Window Tutorial - X Display Server HowTo And Cheatsheet post, since it provides a quick but very detailed overview of the X Windows System, explaining  concepts such as the Window Manager in detail.

The Lab Layout

After this very short summary of Wayland we can start playing with a Wayland compositor: the examples in this post leverage on a lab with the following machines:

sudo dnf install -y nautilus gedit

In this post I'm using OracleLinux 9.3, but things work the same way in every Red-Hat family Linux distribution. Mind anyway that with a minimal rework you can replicate this setup also on other Linux distributions.

Weston- The Wayland's Reference Compositor

Weston is a minimalistic Wayland compositor developed by the Wayland project as the reference implementation: it can be used for compatibility testing or just to get confidence with Wayland.

Mind that Weston is just a reference implementation of a Wayland compositor, its minimal design and fast performances makes it suitable for many embedded and mobile use cases.

Because of its minimal design and strict compliance to the Wayland protocol, we start our walkthrough from it.

Let's install Weston on the "wks-ca-ut1a001"workstation as follows:

sudo dnf install -y weston

then launch Weston:

weston

The obvious requisite for running Weston is being interactively logged to a console.

Despite it may look "empty", compared to the behaviour of an X Display Server, it is instead displaying much more things: there is a bar on the top with date and time on the right and a terminal icon on the left.

Click on the terminal icon: this launches the "weston-terminal" Wayland application.

Even here there are differences compared to the plain x-term launched on a minimalistic X Display Server (without the Window Manager): the "weston-terminal" has a title bar and buttons to iconize, maximise or close it indeed.

The key difference here is that Weston is a Compositor embedding the code of a Display Server too - since it is a compositor, it implements the Window Manager: as we already saw in the "X Window Tutorial - X Display Server Howto And Cheatsheet" post, title bars, buttons and such are actually decorators added by embedding and reparenting the window within a frame window, or just compositing it.

If you come from the X11 world, you may be wondering how Wayland applications locate the Display to use, ... they do this the same way as the X11 applications, by checking an environment variable. In this case it is the "WAYLAND_DISPLAY" environment variable:

echo ${WAYLAND_DISPLAY}

the outcome is:

wayland-0

In order to have a quick walkthrough with Wayland, the Weston project also provides a few demo applications you can install to get acquainted with Weston and Wayland: these are packaged into the "weston-demo" RPM.

We can install them right from the "weston-terminal" just opened as follows:

sudo dnf install -y weston-demo

let's list the available applications:

rpm -ql weston-demo |egrep "(/usr/bin/)"

on my system it is as follows:

/usr/bin/weston-calibrator
/usr/bin/weston-clickdot
/usr/bin/weston-cliptest
/usr/bin/weston-confine
/usr/bin/weston-content_protection
/usr/bin/weston-dnd
/usr/bin/weston-editor
/usr/bin/weston-eventdemo
/usr/bin/weston-flower
/usr/bin/weston-fullscreen
/usr/bin/weston-image
/usr/bin/weston-multi-resource
/usr/bin/weston-presentation-shm
/usr/bin/weston-resizor
/usr/bin/weston-scaler
/usr/bin/weston-simple-damage
/usr/bin/weston-simple-dmabuf-egl
/usr/bin/weston-simple-dmabuf-v4l
/usr/bin/weston-simple-egl
/usr/bin/weston-simple-shm
/usr/bin/weston-simple-touch
/usr/bin/weston-smoke
/usr/bin/weston-stacking
/usr/bin/weston-subsurfaces
/usr/bin/weston-touch-calibrator
/usr/bin/weston-transformed

Just to have a go, we can try launching the "weston-image" application.

First let's install some backgrounds, so that we have something to display:

sudo dnf install -y oracle-backgrounds

Now we can launch "weston-image", having it displaying the "/usr/share/backgrounds/day.jpg" background:

weston-image /usr/share/backgrounds/day.jpg

Obviously Weston is not limited to run only applications from its own demo suite: several old fashioned X11 applications have gradually been ported to the Wayland protocol.

Let's try for example running the GNOME Editor - of course first we need to install it:

sudo dnf install -y gedit

let's now launch it:

gedit

as expected, the GNOME Editor gets immediately displayed on our screen.

Let's try the same with some applications from the KDE Desktop Environment family, such as "kate" and "konsole":

sudo dnf install -y kate konsole

let's fist launch "kate":

kate

the program immediately exits back to the shell and nothing gets displayed.

This may look odd, since they claimed KDE applications do already support Wayland.

The trouble is that we are launching it without explicitly saying we are running a Wayland session: since KDE's family applications leverage on the QT toolkit, Wayland support has been added to the toolkit itself.

While designing the QT's Wayland support, they decided that the toolkit must guess whether to enable the "X11" or "Wayland" client by checking the value of the "QT_QPA_PLATFORM" environment variable.

This means that if we want to start the KDE (and so QT) applications using a Wayland compositor as the rendering service, we must first set the "QT_QPA_PLATFORM" environment variable as follows:

export QT_QPA_PLATFORM=wayland

let's now launch "kate" again:

kate

this time it gets properly rendered on the screen.

Since we exported the "QT_QPA_PLATFORM" environment variable, we can now launch any QT application and be sure they run their Wayland code.

For example, we can launch the "konsole" by just typing:

konsole

You are certainly wondering how come the GNOME editor, that same ways as kate is an X11 application, just worked with Wayland: the answer is that GTK - the GNOME Toolkit framework - implemented Wayland support so that applications attempt both an X11 connection and a Wayland one in sequence. If necessary, this behavior can be altered by setting the "GDK_BACKEND" environment. More information on this topic will be provided later on.

As we saw, the requirement for an application for using a Wayland Compositor is using the Wayland protocol. This of course means that every current X11 application must be ported to Wayland, or support both X11 and Wayland and provide a mechanism to specify which protocol to use.

This transition started years ago, so a lot of applications are currently dual protocols (X11 and Wayland), but some have not been ported yet, and some even will never be ported at all.

Let's see now how to deal with an X11 application that has not been ported to Wayland yet: "xclock".

Of course first we need to install it:

sudo dnf install -y xclock

let's now launch it:

xclock

as we should expect, the outcome is:

Error: Can't Open Display:

As we said the Wayland guys designed the XWayland broker for this specific use case.

Kill the current Weston instance - it uses the same key sequence as X11 (CTRL+ALT+BACKSPACE), getting back to the text based console.

First we need to install the Xwayland component:

sudo dnf install -y xorg-x11-server-Xwayland

then we must set up Weston to trigger an XWayland instance on the fly when a legacy X11 application tries to connect to the X11 Display Server socket.

Create the "~/.config/weston.ini" file with the following contents:

[core]
xwayland=true

[xwayland]
path=/usr/bin/Xwayland

launch Weston again:

weston

then open the "weston-terminal" and inside it launch the "xclock" application again:

This time it gets displayed.

Xwayland is certainly a topic that is worth the effort to dig into: we'll do that later on, in a real use case scenario within a GNOME Wayland session.

Using A Remote Compositor

What made the X Window System unique is implementing a network based Display Server: Wayland can replicate this feature through the "waypipe" command line tool.

The "waypipe" utility must be installed both on the remote server as well as on the workstation.

On both "app-ca-ut1a001" and "wks-ca-ut1a001", run:

sudo dnf install -y waypipe

As its name suggests, "waypipe" pipes the traffic through an SSH connection, so the syntax is as follows:

Let's try launching the "Nautilus" application (the GNOME file manager) we previously installed on the "app-ca-ut1a001", displaying it on the "wks-ca-ut1a001" workstation.

From the "weston-terminal" already opened, just type:

waypipe ssh app-ca-ut1a001 nautilus

after entering the password, the GNOME files gets immediately displayed:

If you are used to working with it, you certainly see that it looks different: that's why we are using Weston, which embeds its own Window Manager.

The current "waypipe" implementation does not enable compression by default.

Let's have another go, this time with the GNOME Editor, enabling the "lz4" compression:

waypipe -c lz4=9 ssh app-ca-ut1a001 gedit

the outcome is as follows:

As expected, also the GNOME editor is displayed with a different layout.

Gnome Display Manager

As we said, to meet the transition to Wayland, Desktop Environments has started developing their own Wayland compositor - since compositors in the X11 world are the Window Managers, they actually added Wayland support to their Window Managers.

The major difference is that, when running as Wayland compositor, the Window Managers no longer need to connect to a Display Server since they actually embed the necessary code to run it on themselves.

For example GNOME improved Mutter adding a dual personality:

• X11 Window Manager and Compositing Manager: in this flavor Mutter provides functionalities related to window management, window compositing, focus tracking, workspace management, keybindings and monitor configuration and some other few things. This is the original and now legacy behavior.
• Wayland Display Server / XWayland application: in this flavor Mutter runs on top of KMS and libinput and implements the compositor side as well as various extensions of the Wayland core protocol. It also provides facilities related to running X11 applications using Xwayland, so they enable a compatibility layer to graphical applications that have not been migrated to the Wayland protocol.

This change obviously required also modifying the GNOME DIsplay Manager to enable it to launch both flavors when the graphical target is reached at the system boot.

The graphical session to start  (Wayland or X11) is set by the "WaylandEnable" configuration parameter in the "/etc/gdm/custom.conf" file.

For example:

WaylandEnable=True

enables the Wayland session.

Conversely, setting the "WaylandEnable" flag to "False" enables the legacy X11 session.

Let's have a go with it running a Wayland session.

Configure GDM At Boot in Graphical Mode

Since the GNOME Desktop Environment is the default Desktop Environment in Red Hat like distributions, the easiest way to install it is just installing the "Server with GUI" dnf group of RPM packages.

On the "wks-ca-ut1a001" workstation, type:

sudo dnf groupinstall -y "server with GUI" 

since we will use it in our examples, install also the GNOME terminal:

sudo dnf install -y gnome-terminal

then configure the system to boot in graphical mode:

sudo systemctl set-default graphical

enable the "gdm" (the actual Gnome Display Manager) Systemd unit to be started at system boot:

sudo systemctl enable gdm.service

and of course restart the system:

sudo shutdown -r now

Check The GDM Session Type

When running a GNOME session, the easiest and official way to check if GDM is running a Wayland or a X11 session is checking the value of the "XDG_SESSION_TYPE" environment variable.

On the "wks-ca-ut1a001" workstation, just launch the GNOME terminal and type:

echo $XDG_SESSION_TYPE

if the output is :

wayland

then we are running a Wayland session.

X Applications On Wayland

Let's see how the GNOME session deals with a legacy X11 application: on the "wks-ca-ut1a001" workstation,  as we already did with Weston, let's launch the "xclock" application:

xclock

It gets displayed - this means that the Xwayland broker is enabled: let's have a closer look at this enchantment.

The Xwayland Broker

This "magic" is accomplished by the Xwayland application: this is a special application running a X11 display server acting as a broker that, after applying a proper translation, pipes everything to the actual Wayland compositor.

Let's have a closer look at this interesting compatibility feature.

On the "wks-ca-ut1a001" workstation, type:

pstree | egrep '(way|systemd-\+-NetworkManager|systemd-\+-\(sd-pam\))'

the above egrep expression takes care of cutting a snippet with only the part interesting in our investigation:

systemd-+-NetworkManager---2*[{NetworkManager}]
        |-gdm-+-gdm-session-wor-+-gdm-wayland-ses-+-gnome-session-b---3*[{gnome-session-b}]
        |     |                 |                 `-2*[{gdm-wayland-ses}]
        |-systemd-+-(sd-pam)
        |         |-gnome-shell-+-Xwayland---{Xwayland}

what we infer from the output is:

• Systemd started the "gdm" process (the GNOME Display Manager), that started a processes subtree leading to the Wayland session ("gdm-wayland-ses")
• Systemd also started the "gnome-shell" process that started the Xwayland brooker

Mind that, since it is started by the "gnome-shell" process, the "Xwayland" process runs as the user actually connected to the session - In this example it is the "vagrant" user.

Let's have a closer look to the Xwayland process:

ps aux | grep [X]wayland

on my system the output is:

vagrant     9082  0.0  3.7 242696 65632 ?        Sl   22:06   0:00 /usr/bin/Xwayland :0 -rootless -noreset -accessx -core -auth /run/user/1000/.mutter-Xwaylandauth.IXNIM2 -listenfd 4 -listenfd 5 -displayfd 6 -initfd 7

the information we can infer from here are:

• from the ":0" command line parameter, we know that XWayland is emulating an X Display Server with Display Number 0
• form the "-auth  /run/user/1000/.mutter-Xwaylandauth.IXNIM2" we know that the XWayland process is enforcing XAuthority authentication using the "/run/user/1000/.mutter-Xwaylandauth.IXNIM2" file as the user's database

Since the Xwayland process runs as the logged in user, it means that every logged user launches its own Xwayland instance, each with its own Display Number.

Let's have a close look to the contents of the XAuthority user database:

xauth -f /run/user/1000/.mutter-Xwaylandauth.IXNIM2 list

on my system the output is:

wks-ca-ut1a001.mgmt.carcano.local/unix:  MIT-MAGIC-COOKIE-1  20773679a53c26e477ee8c7083d2d680
#ffff#776b732d63612d757431613030312e6e6574646576732e63617263616e6f2e6c6f63616c#:  MIT-MAGIC-COOKIE-1  20773679a53c26e477ee8c7083d2d680

Since XAuthority is meant to provide user authentication to the X Display Servers, and it can of course be used to enforce access control on networking connections, we can guess the Xwayland server is also providing a networking endpoint.

If that is true, it means that we can run remote legacy X11 applications using SSH's X forwarding.

Before going on with the lab and check this, uninstall the "xclock" application as follows:

sudo dnf remove -y xclock

X Forwarding With XWayland

In the current Wayland session, we are about to try launching a remote instance of the "xclock" legacy X11 application installed on the "app-ca-ut1a001" server.

On the GNOME console already opened on the "wks-ca-ut1a001" workstation.just type:

ssh -X app-ca-ut1a001 xclock

the "xclock" application is actually displayed: this confirms Xwayland does not break the SSH's X forwarding feature.

Wayland Environment Variables

Last but not least, let's have a recap of the most interesting environment variables somehow bound to Wayland.

Footnotes

Here it ends our post on the Wayland protocol: I think it is mature enough to be used, for sure it still has shortcomings, but they are working hard on fixing it.

What I disagree with is they completely “forgot” about having clients be able to register themselves to remote Display Managers and so not supporting XDMCP nor developing a replacement.

By doing so they are forcibly shutting down one of the core and most useful X Window system’s features.

I hope they re-think wisely on this and fill this huge gap..