How to store signed and encrypted data on IPFS

Storing authenticated and encrypted data on IPFS is a core building block for many Web3 applications, but to date there has been no standardized way to encode this type of data.

Without standards, many developers are forced to create custom formats for their signed and encrypted data. This has hindered the openness and interoperability of information stored in IPFS by storing data in a specific implementation of IPFS. Another way to verify data is to put the data in IPFS and put the CID of the data in a smart contract on a blockchain such as Ethereum. Essentially, this is an expensive way to add a signature on top of the data and persist the signature record on the blockchain.

With the introduction of EIP-2844, a standard that allows wallets to support some new ways to sign and decrypt data based on DIDs and the dag-joseIPLD codec, we can now simply put authenticated and encrypted data directly into IPFS.

DID is a W3C standard for decentralized identifiers.

For more details, please refer to our previous article: Astral builds a new world. This article only briefly introduces that DID specifies a general method from a string identifier (such as did:3:bafy...) to a DID document containing a public key for signature verification and key exchange. In most DID methods, the document can be updated when the key is rotated for security reasons.

JOSE is an IETF (Internet Engineering Task Force) standard that stands for JSON Object Signature and Encryption, which pretty much explains what it means . The standard has two main primitives: JWS (JSON Web Signature) and JWE (JSON Web Encryption). Both formats allow for multiple participants: in JWS, there can be one or more signatures on the payload, while in JWE, there can be one or more recipients of the encrypted plaintext.

As we built Ceramic using dag-jose and EIP-2844 as basic building blocks, we created some low-level tooling that allows us to use these technologies more easily.

js-3id-did-provider is an implementation of EIP-2844 using 3ID as a DID method. It can be used as a DID provider on its own or more conveniently in the 3ID Connect library. 3ID Connect allows users to access the DID Provider using their Ethereum wallet (support for more blockchains is coming soon).

key-did-provider-ed25519 is an implementation of EIP-2844 using the Key DID method. It is the simplest DID provider that supports both signing and encryption.

js-did is a library that allows developers to represent users in the form of DIDs. This is the main interface we will be looking at in this tutorial. It enables us to sign data with the currently authenticated user, encrypt data to any user (DID), and decrypt data with the currently authenticated user.

By using the dag-jose IPLD codec we can create linked and signed data structures. This is done by creating a JSON Web Signature (JWS) that contains links to other data. One of the main issues that the dag-jose codec solves is that the payload of a JWS is traditionally encoded as: base64url This means that if there are IPLD links in it, you will not be able to traverse those links.

Instead, what we do with DagJWS is to coerce the payload into the bytes of a CID. The codec then converts the payload into a CID instance and sets it as a link property of DagJWS. This allows us to easily traverse the resulting DAG.

Since dag-jose is a new IPLD codec, it is not yet included in js-ipfs by default. It also implements the new IPLD codec API, which js-ipfs does not support yet. Therefore, when creating an IPFS instance, you need to do the following:

Make sure you install the correct multi-format version:

In the example setup below, we use key-did-provider-ed25519. If you choose to use the network from above, 3ID Connect and js-3id-did-provider will be used behind the scenes.

Now we can start signing and adding data to IPFS! First, let's create a simple function that takes a payload, signs it using the did.createDagJWS method, and then adds the resulting data to IPFS.

As we can see in the code below, we get two objects from this method: jwsDagJWS itself and the raw bytes of the linkedBlock encoded payload. What happens in the background is that the payload is encoded using dag-cbor, after which the CID of the encoded payload is used as the payload created by jws. We can access the payload CID on the DagJWS instance through jws.link.

Using this function, let's create our first signed data object:

Note that since the payload will be signed by your DID, the values ​​for CID and JWS will be different for you.

Verifying a JWS is very simple. Simply retrieve the JWS object and pass it to the verifyJWS method. If the signature is invalid, this function will throw an error. If the signature is valid, it will return the DID (with the key fragment) used to sign the JWS.

Signing data in IPFS is a difficult problem, but perhaps more interesting is encrypting data. By using dag-jose and EIP-2844, we can encrypt data into one or more DIDs and store it directly in IPFS. Below, we demonstrate how to do this using the convenient tools provided by the js-did library.

There is a simple method to create a DagJWE object that is encrypted to one or more DIDs createDagJWE. This method accepts an IPLD object (which may also include a JSON object of CID links) and an array of DIDs.

It will parse the DID to retrieve the public encryption keys found in its DID document and create a JWE encrypted to those keys. First, let's create a helper function that creates a JWE and puts it into IPFS.

Once we have this functionality we can create some encrypted objects. In the following example we first create a simple encrypted object and then create an additional encrypted object that is linked to the previous object.

Note that in the example above, we used [did.id]( ) to encrypt the data to the currently authenticated DID. We can of course also encrypt the data to a DID of a user that is not locally authenticated (e.g. another user)!

After retrieving the data from IPFS, we will only get the encrypted JWE. This means we need to decrypt the data after getting it. Since we have already created objects that are linked to each other, let's create a function to retrieve these objects and recursively decrypt them.

The function above is an example that simply logs the decrypted objects. We can use it to view the contents of these objects.