JSON Web Token – JWT

JWT provides a convenient and standard way to securely transport claims from an issuer to its audience over HTTP. It can be used for authentication and authorization purposes, as well as non repudiable information exchange. It's easiness of setup along with the pros of being transportable over a simple and plain-text protocol such as HTTP and of being stateless (session data can simply be put inside the claims) made it be broadly adopted by developers, whenever it does not worth to implement more cumbersome frameworks such as OpenID or OAuth. In this post we explore JWT and its related standards JWS and JWE.

JSON WEB Security standards and JWT

JSON Web Security standards are a set of web standards that have been designed to enumerate the available security protocols and to define a standard way of using them for digitally signing and encrypting contents using an agreed serialization format.

The available base standards are:

• JWK (JSON Web Key - IETF RFC 7517) - a data structure used to store a cryptographic key along with its attributes, such as key usage
• JWA (JSON Web Algorithms - IETF RFC 7518) - a set of algorithms and their identifiers that can be used to encrypt or sign messages
• JWS (JSON Web Signature – IETF RFC 7515) – a standard that describes the processes and formats necessary to create and validate a signed payload
• JWE (JSON Web Encryption – IETF RFC 7516) – a standard that describes the processes and formats necessary to encrypt and decrypt an encrypted payload

All of the above standards have been eventually put together into the JWT (JSON Web Token - IETF RFC 7519), that defines the standard format to serialize a container to safely transport claims from an issuer to the audience over HTTP.

These claims can be either:

• validated by the audience by verifying the issuer's signature with its public key (JWS)
• decrypted using the audience's private key (JWE)

A JSON Web Token is made of the following parts:

JWT in action

We are now skilled enough to take a deeper look at all of these standards: JWK, JWA, JWS and JWE.

JWK

A key and all of the attributes required to describe it are stored using a JSON as in the following snippet:

{
    "kty":"EC",
    "crv":"P-256",
    "x":"MKBCTNIcKUSDii11ySs3526iDZ8AiTo7Tu6KPAqv7D4",
    "y":"4Etl6SRW2YiLUrN5vfvVHuhp7x8PxltmWWlbbM4IFyM",
    "use":"enc",
    "kid":"1"
}

kty is a mandatory parameter that specifies the key type: at the time of writing this post the specifications describes the following three key types:

• EC - for Elliptic Curve
• RSA- for RSA
• oct octet sequence, denoting the shared symmetric key

The other parameters depend on the key type. The previous snippet describes the elliptic curve key that has been supplied:

• crv identifies the curve
• x and y coordinate of point
• use optional parameter that denotes the intended usage of the key
• kid the ID of the key

JWA

The following box contains the key types that are usable for JWS and JWE:

"enc": 
    A128CBC-HS256, A192CBC-HS384, A256CBC-HS512 (AES in CBC with HMAC), 
    A128GCM, A192GCM, A256GCM

"alg" for JWS: 
    HS256, HS384, HS512 (HMAC with SHA), 
    RS256, RS384, RS512 (RSASSA-PKCS-v1_5 with SHA), 
    ES256, ES384, ES512 (ECDSA with SHA), 
    PS256, PS384, PS512 (RSASSA-PSS with SHA for digest and MGF1)

"alg" for JWE: 
    RSA1_5, RSA-OAEP, RSA-OAEP-256, 
    A128KW, A192KW, A256KW (AES Keywrap), 
    dir (direct encryption), 
    ECDH-ES (EC Diffie Hellman Ephemeral+Static key agreement), 
    ECDH-ES+A128KW, ECDH-ES+A192KW, ECDH-ES+A256KW (with AES Keywrap), 
    A128GCMKW, A192GCMKW, A256GCMKW (AES in GCM Keywrap), 
    PBES2-HS256+A128KW, PBES2-HS384+A192KW, PBES2-HS512+A256KW 
    (PBES2 with HMAC SHA and AES keywrap)

alg is the key algorithm used to:

• sign the claim (JWS)
• encrypt the claim (JWE with direct encryption)

When a JWE is generated for multiple recipients, a random symmetric encryption key (said Content Encryption Key - CEK) is generated using what is specified by enc parameter, and this key is encrypted using the algorithm specified by alg field on a per recipient basis.

JWS

It describes the process of creation and validation of data structures representing signed payload. Please note that the payload can be of any format, although JSON is used the very most of the time, also because it is part of the JWT standard.

The signing algorithm is described using a base64 encoded JSON with the following format:

{ "alg": "ES384" }

The value of the alg field is of course one of the list of JWA algorithms we saw above. If the claims does not need to be signed, alg should be set to "none".

When generating a JWS JSON Web Token, eventually everything is serialized into a JSON with a standard structure.

The general serialization of JWS/JWT enables the use of  per-recipient signatures. It looks like as follows:

{
  "payload": "SX...B0by4",
  "signatures": [
    {
      "protected": "iOiJSUzI...",
      "header": { 
        "kid": "macarcano@carcano.ch" 
      },
      "signature": "MItVlOpa7..."
    },
    {
      "protected": "As0x924DU...",
      "header": { 
        "kid": "otheruser@foo.tld" 
      },
      "signature": "VGhDlOpa7..."
    },
    { 
      "protected": "UzI1NiIsImtpZCI...",
      "signature": "BBBkLspW1h84VsJZF..."
    } 
  ]
}

Let's see JWS/JWT in action using Python: this of course requires the installation of a Python module that implements JWT. In the following examples we use jwcrypto module, that implements both JWS and JWE.

Install it using pip3 - yes, I'm using Python 3

sudo pip3 install jwcrypto

we need a key pair to be used for signing and verification purposes: since this is just a lab to learn JWT, we are not very concerned about security, so we do not  need to use encrypted keys. Let's generate an unsecured EC private key using secp384r1 curve and store it into jwt.key file:

openssl ecparam -name secp384r1 -genkey > jwt.key

The generated EC key is not encrypted: we are doing so since this is only a lab: in real life the key should be encrypted so to have it password protected.

now let's extract the public key from the private key and store it into jwt .pub file:

openssl ec -in jwt.key -pubout -out jwt.pub

Before going on you may want to review asymmetric encryption: don't miss Cryptography quick guide - understand symmetric and asymmetric cryptography and how to use it with OpenSSL.

now we need some claims to sign: create claims.json file with some public claims:

{
  "family_name": "Carcano",
  "given_name": "Marco Antonio",
  "name": "Marco Antonio Carcano",
  "zoneinfo": "Europe/Zurich",
  "locale": "it_CH"
}

please note that the above claims are some of the ones that are used by a OpenID Connect profile.

OpenID Connect and OAuth2 are probably the most used frameworks when it comes to delegated authentication and authorization. You may be interested in the "OpenID Connect With Kratos And Hydra - Gitea OAuth" post: This is a hands-on lab showing not only how to set them up, but also provides a real life use case with Gitea using OpenID Connect as the authentication source. As usual the post provides a short but comprehensive overview of these two very complex frameworks so as to make sense of the configuration applied to the Lab.

Create jws-generate.py, a Python script that generates a signed JWS/JWT with the claims contained into claims.json file:

#!/usr/bin/python3
import sys
from jwcrypto import jwt,jws
from jwcrypto.common import json_encode

key = jwt.JWK()
            
try:
   with open("jwt.key", "rb") as key_file:
      key_data = key_file.read()
   with open("claims.json") as cf:
      payload = cf.read().rstrip('\n')
except Exception as e:
      print("Error loading key file: %s" % str(e), file=sys.stderr)
      exit(1)
key.import_from_pem(key_data)

jwstoken = jws.JWS(payload.encode('utf-8'))
jwstoken.add_signature(key, None, json_encode({"alg": "ES384"}), json_encode({"kid": key.thumbprint()}))
sig = jwstoken.serialize()
print(sig)

and create jws-verify.py, a Python script that verifies the signature of a JWS/JWT stored into jws.json file:

#!/usr/bin/python3
import sys
import json
from jwcrypto import jwt,jws

key = jwt.JWK()
            
try:
   with open("jwt.pub", "rb") as key_file:
      key_data = key_file.read()
   with open("jws.json") as cf:
      jws_data = cf.read()
except Exception as e:
      print("Error loading key file: %s" % str(e), file=sys.stderr)
      exit(1)
key.import_from_pem(key_data)
jwstoken = jws.JWS()
jwstoken.deserialize(jws_data)
try:
   jwstoken.verify(key)
except:
   print("FAILED - Invalid signature", file=sys.stderr)
   exit(1)
payload = jwstoken.payload

print(payload.decode("utf-8") )

we are ready to play with JWS/JWT - in the following statement we use tee and jq to write the output of the script to jws.json file while to pretty-printing it to standard too :

./jws-generate.py | tee jws.json | jq

the output should look like as follows:

{
  "header": {
    "kid": "_91u8kDI4UEHfOaFPp2dd2YIlldikPVK42ldwOATAJo"
  },
  "payload": "ewogICJmYW1pbHlfbmFtZSI6ICJDYXJjYW5vIiwKICAiZ2l2ZW5fbmFtZSI6ICJNYXJjbyBBbnRvbmlvIiwKICAibmFtZSI6ICJNYXJjbyBBbnRvbmlvIENhcmNhbm8iLAogICJ6b25laW5mbyI6ICJFdXJvcGUvWnVyaWNoIiwKICAibG9jYWxlIjogIml0X0NIIgp9",
  "protected": "eyJhbGciOiJFUzM4NCJ9",
  "signature": "oQPttSHLSm4w8q90XUXYXguUEvwni5nakUPSz1HK6p1qGcWa7wnBfS89ISyV3nnOtO6jMbAmiK1ZNeDurrK5hb-BU6Lc-SC98B06QjMz7KxERruZ_YK4jI3FfZqFUYvV"
}

The above serialization is told flattened serialization, conversely from the general serialization we saw before.

Now launch jws-verify.py script to verify if the JWS/JWT we wrote into jws.json file is actually valid:

./jws-verify.py

if the signature verification succeeds, then the output should be the original claim that have been signed:

{
  "family_name": "Carcano",
  "given_name": "Marco Antonio",
  "name": "Marco Antonio Carcano",
  "zoneinfo": "Europe/Zurich",
  "locale": "it_CH"
}

try also to rerun the verify script after modifying jws.json altering some of the characters of the signature: this time the outcome should be as follows:

FAILED - Invalid signature

JWE

JWE uses the same logic as JWS, but focusing on encryption: unless Direct Encryption is used, it generates a symmetric encryption (CEK) key and encrypts things with it. This key itself is then encrypted into a per-recipient header using the public key of the recipient.

The general serialization of JWE/JWT enables the use of  per-recipient headers. It looks like as follows:

Are you enjoying these high quality free contents on a blog without annoying banners? I like doing this for free, but I also have costs so, if you like these contents and you want to help keeping this website free as it is now, please put your tip in the cup below:

Even a small contribution is always welcome!

{
  "protected": "OiJBMTI4Q0JDL...",
  "recipients": [
    {
      "header": {
        "alg": "RSA1_5",
        "kid": "2019-01-21",
        "epk": {}
      },
      "encrypted_key": "..."
    },
jwe    {
      "header": {
        "alg": "A128KW",
        "kid": "2"
      },
      "encrypted_key": "..."
    }
  ],
  "iv": "Y20EnzUtZFl2RpB1g...",
  "ciphertext": "u3a_k1C55kCQ_3xlkcVKC5yr__Is48VOoK0k63_QRM...",
  "tag": "Mz-VPPyU4Rl..."
}

This trick does not only let have multiple recipients being able to decrypt the same encrypted payload, but also to encrypt the CEK using different algorithms, as specified by the alg field of each header - in turns we can encrypt using RSA for a recipient, EC to another one and so on.

Conversely, when Direct Encryption is used, the algorithm specified by the alg field contains the value "dir", so to inform users to directly use the specified algorithm to encrypt the payload.

Let's see JWE in action now: create jwe-generate.py, a Python script that generates a JWE/JWT from the contents of claims.json file:

#!/usr/bin/python3
import sys
from jwcrypto import jwt,jwe

key = jwt.JWK()
            
try:
   with open("jwt.pub", "rb") as key_file:
      key_data = key_file.read()
   with open("claims.json") as cf:
      payload = cf.read().rstrip('\n')
except Exception as e:
      print("Error loading key file: %s" % str(e), file=sys.stderr)
      exit(1)
key.import_from_pem(key_data)

protected_header = {
   "typ": "JWE",
   "alg": "ECDH-ES+A256KW",
   "enc": "A256CBC-HS512",
   "kid": key.thumbprint(),
}

jwetoken = jwe.JWE(payload.encode('utf-8'),recipient=key,protected=protected_header)
enc = jwetoken.serialize()
print(enc)

and create jwe-decrypt.py, a Python script that decrypts the JWE/JWT stored into jwe.json file

#!/usr/bin/python3
import sys
import json
from jwcrypto import jwt,jwe

key = jwt.JWK()
            
try:
   with open("jwt.key", "rb") as key_file:
      key_data = key_file.read()
   with open("jwe.json") as cf:
      jwe_data = cf.read().rstrip('\n')
except Exception as e:
      print("Error loading key file: %s" % str(e), file=sys.stderr)
      exit(1)
key.import_from_pem(key_data)

jwetoken = jwe.JWE()
try:
   jwetoken.deserialize(jwe_data, key=key)
except:
   print("FAILED - Decryption failed", file=sys.stderr)
   exit(1)

payload = jwetoken.payload

print(payload.decode("utf-8") )

We are ready to play with JWE/JWT - in the following statement we use tee and jq to write the output of the script to jwe.json file while to pretty-printing it to standard too :

./jwe-generate.py | tee jwe.json | jq

the output should look like as follows:

{
  "ciphertext": "y5Dnz7bt17nHKPDaUR3sT3KFuIepzHPOCcv09xnK3BzKu1gicsjbTBG16Hk9WilfX8nVt78Hw2NKhANGQ59W09fvtIYlVX2gSQHsJR6dfgXqxrPa7hUbV1qCHjgV1i6Slfcn7oXoh69fHvVUJfxD7TfcVFebs1iskP_6U2bzbak51TsUn-R0v-oF8VUajK3vjteVcgov7aqhSitZW6Hnbw",
  "encrypted_key": "H5dSdEDWjBOXRfrpDLcz_Mn4m7EAVKhzCRyfWCLHXOe4OhvIkV3KCMldkQL1wHJidQ8E7qOW1ImFaK36BJflyaVhCQd0o5RT",
  "header": {
    "epk": {
      "crv": "P-384",
      "kty": "EC",
      "x": "6pySccNEbo8tPigI2CtPP9Y-NIKvJY7ytXBp1KVK8tile0YfGnTQ47BHsJkkuxMS",
      "y": "JqNKG3LBaEwZt_wX79-nm1ZQBpcUgKLEB9P9fcpHPiHOXjSGrvWqcHzB3Ec89Qdb"
    }
  },
  "iv": "cT83StoZ09exRsLBh17LYg",
  "protected": "eyJhbGciOiJFQ0RILUVTK0EyNTZLVyIsImVuYyI6IkEyNTZDQkMtSFM1MTIiLCJraWQiOiJfOTF1OGtESTRVRUhmT2FGUHAyZGQyWUlsbGRpa1BWSzQybGR3T0FUQUpvIiwidHlwIjoiSldFIn0",
  "tag": "BVZ3Z6hVhRDKEZ0HBIafqkF5ZfYgMr3fZ1oqRfd1OvA"
}

The above serialization is told flattened serialization, conversely from the general serialization we saw before.

Now launch jws-decrypt.py script to decrypt the JWE/JWT we have just wrote into jwe.json file:

./jwe-decrypt.py

if the decryption succeeds, then the output should be contain all of the original claims:

{
  "family_name": "Carcano",
  "given_name": "Marco Antonio",
  "name": "Marco Antonio Carcano",
  "zoneinfo": "Europe/Zurich",
  "locale": "it_CH"
}

try also to rerun the verify script after modifying jwe.json altering some of the characters of the encrypted payload: this time the outcome should be as follows:

FAILED - Decryption failed

COMPACT Serialization

JWS/JWT COMPACT Serialization

When working on the web, the JWS/JWT token should be encoded into the authentication header that is sent from the client to the server. For this use case it is used the Compact Serialization, that besides being the most commonly used format, it has specifically been designed for the web.
It's format is as depicted by the following box:

BASE64URL(UTF8(JWS Protected Header)) || '.' ||
BASE64URL(JWS Payload) || '.' ||
BASE64URL(JWS Signature)

for your convenience I put into the following box the JWS/JWT we previously generated

{
  "header": {
    "kid": "_91u8kDI4UEHfOaFPp2dd2YIlldikPVK42ldwOATAJo"
  },
  "payload": "ewogICJmYW1pbHlfbmFtZSI6ICJDYXJjYW5vIiwKICAiZ2l2ZW5fbmFtZSI6ICJNYXJjbyBBbnRvbmlvIiwKICAibmFtZSI6ICJNYXJjbyBBbnRvbmlvIENhcmNhbm8iLAogICJ6b25laW5mbyI6ICJFdXJvcGUvWnVyaWNoIiwKICAibG9jYWxlIjogIml0X0NIIgp9",
  "protected": "eyJhbGciOiJFUzM4NCJ9",
  "signature": "oQPttSHLSm4w8q90XUXYXguUEvwni5nakUPSz1HK6p1qGcWa7wnBfS89ISyV3nnOtO6jMbAmiK1ZNeDurrK5hb-BU6Lc-SC98B06QjMz7KxERruZ_YK4jI3FfZqFUYvV"
}

since the recommended way to send the JWT token is using the Authorization header with the Bearer scheme, the header should look like as follows:

Authorization: Bearer eyJhbGciOiJFUzM4NCJ9.ewogICJmYW1pbHlfbmFtZSI6ICJDYXJjYW5vIiwKICAiZ2l2ZW5fbmFtZSI6ICJNYXJjbyBBbnRvbmlvIiwKICAibmFtZSI6ICJNYXJjbyBBbnRvbmlvIENhcmNhbm8iLAogICJ6b25laW5mbyI6ICJFdXJvcGUvWnVyaWNoIiwKICAibG9jYWxlIjogIml0X0NIIgp9.oQPttSHLSm4w8q90XUXYXguUEvwni5nakUPSz1HK6p1qGcWa7wnBfS89ISyV3nnOtO6jMbAmiK1ZNeDurrK5hb-BU6Lc-SC98B06QjMz7KxERruZ_YK4jI3FfZqFUYvV

Validating the JWS/JWT using the official online tool

The JWS/JWT header can be easily validates by using the official online tool at https://jwt.io/: the steps are as follows:

• put the serialized JWS/JWT, without the "Authorization: Bearer " part, in the Encoded box.  It should automatically detect the Algorithm and decode the payload.
• put your public key (the contents of jwt.pub file) into the Public Key box within the Verify Signature box

If everything is OK, you should get the "Signature Verified" message and the outcome should be as follows:

JWE/JWT COMPACT Serialization

When working on the web, the JWE/JWT token should be encoded into the authentication header that is sent from the client to the server. Same way as for JWS/JWT, in this use case it is used the Compact Serialization. It's format is as depicted by the following box:

BASE64URL(UTF8(JWE Protected Header)) || '.' || BASE64URL(JWE Encrypted Key) || '.' || BASE64URL(JWE Initialization Vector) || '.' || BASE64URL(JWE Ciphertext) || '.' || BASE64URL(JWE Authentication Tag)

for your convenience I put into the following box the JWE/JWT we previously generated

{
  "ciphertext": "y5Dnz7bt17nHKPDaUR3sT3KFuIepzHPOCcv09xnK3BzKu1gicsjbTBG16Hk9WilfX8nVt78Hw2NKhANGQ59W09fvtIYlVX2gSQHsJR6dfgXqxrPa7hUbV1qCHjgV1i6Slfcn7oXoh69fHvVUJfxD7TfcVFebs1iskP_6U2bzbak51TsUn-R0v-oF8VUajK3vjteVcgov7aqhSitZW6Hnbw",
  "encrypted_key": "H5dSdEDWjBOXRfrpDLcz_Mn4m7EAVKhzCRyfWCLHXOe4OhvIkV3KCMldkQL1wHJidQ8E7qOW1ImFaK36BJflyaVhCQd0o5RT",
  "header": {
    "epk": {
      "crv": "P-384",
      "kty": "EC",
      "x": "6pySccNEbo8tPigI2CtPP9Y-NIKvJY7ytXBp1KVK8tile0YfGnTQ47BHsJkkuxMS",
      "y": "JqNKG3LBaEwZt_wX79-nm1ZQBpcUgKLEB9P9fcpHPiHOXjSGrvWqcHzB3Ec89Qdb"
    }
  },
  "iv": "cT83StoZ09exRsLBh17LYg",
  "protected": "eyJhbGciOiJFQ0RILUVTK0EyNTZLVyIsImVuYyI6IkEyNTZDQkMtSFM1MTIiLCJraWQiOiJfOTF1OGtESTRVRUhmT2FGUHAyZGQyWUlsbGRpa1BWSzQybGR3T0FUQUpvIiwidHlwIjoiSldFIn0",
  "tag": "BVZ3Z6hVhRDKEZ0HBIafqkF5ZfYgMr3fZ1oqRfd1OvA"
}

since the recommended way to send the JWT token is using the Authorization header with the Bearer scheme, the header should look like as follows:

Authorization: Bearer eyJhbGciOiJFQ0RILUVTK0EyNTZLVyIsImVuYyI6IkEyNTZDQkMtSFM1MTIiLCJraWQiOiJfOTF1OGtESTRVRUhmT2FGUHAyZGQyWUlsbGRpa1BWSzQybGR3T0FUQUpvIiwidHlwIjoiSldFIn0.H5dSdEDWjBOXRfrpDLcz_Mn4m7EAVKhzCRyfWCLHXOe4OhvIkV3KCMldkQL1wHJidQ8E7qOW1ImFaK36BJflyaVhCQd0o5RT.cT83StoZ09exRsLBh17LYg.y5Dnz7bt17nHKPDaUR3sT3KFuIepzHPOCcv09xnK3BzKu1gicsjbTBG16Hk9WilfX8nVt78Hw2NKhANGQ59W09fvtIYlVX2gSQHsJR6dfgXqxrPa7hUbV1qCHjgV1i6Slfcn7oXoh69fHvVUJfxD7TfcVFebs1iskP_6U2bzbak51TsUn-R0v-oF8VUajK3vjteVcgov7aqhSitZW6Hnbw.BVZ3Z6hVhRDKEZ0HBIafqkF5ZfYgMr3fZ1oqRfd1OvA

Footnotes

Here it ends our deep dive into the JWT standards: being skilled onto JWT is mandatory when dealing with system integrations, since it is a protocol broadly adopted and there are several other broadly adopted frameworks that rely on it, such as OpenID Connect or OAUth.

JWT can be issued to clients by Identity Managers (IdM) after authenticating them, then the client can use the received JWT to authenticate to all of the services that trusts that IdM.

Using JWT enabled services is very convenient, since the service itself can be implemented stateless and does not need to directly speak to the IdM to authenticate the clients.

However using JWT have some pitfalls that should be carefully considered when designing JWT based solutions, such as how to deal when it comes to revoke JWTs, implementing an expiration policy and so on.

 But this may be the topic of another post, that probably should get classified also in the Designing category of this blog.

I hate blogs with pop-ups, ads and all the (even worse) other stuff that distracts from the topics you're reading and violates your privacy. I want to offer my readers the best experience possible for free, ... but please be wary that for me it's not really free: on top of the raw costs of running the blog, I usually spend on average 50-60 hours writing each post. I offer all this for free because I think it's nice to help people, but if you think something in this blog has helped you professionally and you want to give concrete support, your contribution is very much appreciated: you can just use the above button.