What improvements have been made in IPFS 0.5.0, the largest upgraded version of IPFS?

go-ipfs 0.5.0 is an important step on our path to a more efficient, secure, and resilient network to protect and advance human knowledge.

We are excited to share that IPFS 0.5.0, the biggest upgrade to the IPFS protocol and public network, has been released. The result of extensive community interest and participation, IPFS 0.5.0 makes major performance and reliability updates that allow you to distribute content peer-to-peer more efficiently and cost-effectively.

What IPFS 0.5.0 means for the web

Today's web is fragile, inefficient, and expensive. It is built on a client-server model that finds and serves content based on its location on a specific host. This system has failed to adapt to the highly dynamic ways we communicate, create, work, consume, and connect.

The distributed web is changing all of this. IPFS re-architects the web to work peer-to-peer, processing data by what it is, not where it is. In doing so, IPFS powers a resilient and scalable web. Our latest release, IPFS 0.5.0, delivers improved performance and reliability to meet the growing demands of the web today and in the future.

Community Feedback

The IPFS public network has experienced tremendous growth (30x in size in 2019), bringing new opportunities to IPFS users and the broader interplanetary ecosystem. Major deployments and applications that require fast and reliable connectivity have begun to build on top of IPFS and support IPFS:

• Opera & Brave: Shortly after Brave began directly embedding the IPFS Companion extension (with a built-in js-ipfs node), Opera became the first major web browser to offer default IPFS support on Android. This means that millions of people around the world can now access the decentralized web directly built into their browser.

• Microsoft ION: Microsoft has used IPFS as a content-addressed storage layer to power its decentralized identity system.

• Netflix: We started working with Netflix to integrate peer-to-peer services into Netflix developer tools to speed up the distribution of container images in their build pipelines.

• Cloudflare: Cloudflare operates a distributed web gateway that can serve content hosted on IPFS easily, quickly, securely, and efficiently.

We worked with these organizations and the broader community of active IPFS users to gather feedback and set priorities for this release and the rest of 2020. With their help, we identified two major pain points: performance and reliability.

Performance and security upgrades in IPFS 0.5.0

Our performance work revolves around content routing, the process of finding and fetching the data you need to load a website, watch a video, or use an app. Now that our distributed network has scaled to 100,000s of nodes, we need to be more rigorous about which nodes should participate in routing requests and where new content is announced to improve overall reliability.

Here are some of the improvements made in IPFS 0.5.0:

• Faster content discovery: IPFS 0.5.0 makes finding and sharing data in the network much faster, thanks to major improvements to DHT logic. More efficient routing and serving (2-3 times faster), reduced bandwidth usage and background traffic. We have also updated the experimental QUIC transport to reduce connection latency and round trips, with the goal of making it the default configuration for the next version.

• Faster file transfers: We sped up Bitswap, our file transfer mechanism, to enable faster sharing of data between many nodes. Read more about how we worked with Netflix and how we benchmark and stress test!

• Faster file additions: Adding files to the IPFS network is now 2x faster using the Badger datastore on Linux and MacOS. Huzzah for asynchronous writes! We’ve also configured Badger to use less memory.

• Faster, more reliable mutable links: The InterPlanetary Name System (IPNS), the system we use to create mutable links for content addresses, now provides faster naming lookups and has a new experimental pubsub transport to speed up record distribution. In a 1K node network simulation, serving IPNS records is now 30-40 times faster!

To ensure the reliability of all of these massive improvements, we created Testground, a new testing tool for large-scale peer-to-peer networks. This allows us to validate and test every improvement with 1K+ node simulations in many network configurations, increasing development speed and rigor.

And much more

After all, this is a big release. In IPFS 0.5, we’ve also made great progress in developer tooling and security, including:

• Wider support for security and encryption protocols: IPFS now uses Transport Layer Security (TLS) by default, an open standard for transport security that helps maintain secure connections between clients and servers by default.

• Developer experience improvements: IPFS 0.5.0 supports many existing tools and workflows, making decentralized capabilities accessible to more developers. Updates include:

• Subdomains in HTTP Gateway: Developers can now place dapps (distributed applications) in isolated origins to protect the security of their applications and web content. This applies to hashes, IPNS, DNSLink, ENS, and others.

• /ipfs/peer_id → /p2p/peer_id: We now use /p2p/ to disambiguate ipfs files from peer addresses.

• .eth Link Support: Blockchain-based .eth links are supported by default, further opening up IPFS to decentralized application developers (support for more distributed TLDs will be coming soon).

Please see our changelog for a comprehensive list of IPFS 0.5.0 updates, as well as our in-depth look at all the important features in this release.

The significance of IPFS 0.5.0

Through our new testing and release process, IPFS 0.5.0 is already being used in various production environments. This will make a difference for IPFS users:

• Matt Ober of Pinata: “IPFS 0.5 makes a major improvement to Bitswap that deduplicates data between nodes. This means that data providers like Pinata, which transfer large amounts of data around the world every day, can only send each block once and then arrive at its destination. This significantly reduces our bandwidth costs. What’s more, it’s faster, too.”

• Michael Godsey, Infura: “Our collaboration with Protocol Labs and the latest go-ipfs v0.5 release has been critical to our upcoming advanced IPFS product later this year. The upgraded subdomain gateway support has greatly improved the security of our product while helping us support some great new features. Improvements to the DHT and stability of peer connections have allowed us to deliver an even more reliable product and continue to get IPFS into the hands of decentralized application developers.”

• Dr. Ian Preston, Peergos: “We’ve seen that an inactive IPFS instance now uses less bandwidth maintaining the DHT and other background traffic. Under IPFS 0.5, this load has dropped by a factor of four: from about 800 kb per second, now 200 kb per second. It doesn’t sound like much, but it adds up quickly, so the difference is significant.”

• Brendan O'Brien, Qri: "Our engineering time is one of our most valuable resources. Now that we're seeing a mature version of IPFS, it means we can spend more time on our own code and less on dependencies. Upgrading to IPFS 0.5.0 took less than a day, allowing us to spend more engineering time building and fixing our own stuff."

If you’re running IPFS, we strongly recommend you upgrade as soon as possible to take advantage of these improvements! Check out our release notes, and upgrade today!

Thank you

IPFS 0.5.0 wouldn’t be our biggest update yet without the help of our contributors. Special thanks to our early testers for releasing this version, especially:

@MichaelMure (Infura)

@carsonfarmer

@b5 (Qri)

@obo20

@ianopolous (Peergos)

@postables (RTrade)

@meeDamian

@pataquets

Also, a huge thank you to everyone who contributed patches and improvements to this release, you all have been awesome! We couldn't have done it without your help and feedback.

How to participate

• We want to know what you think and answer any questions you may have. Please join us for a virtual meeting on Friday, May 1st to dive deeper into areas for improvement in this release, as well as to get live feedback.

• In the meantime, start installing go-ipfs or upgrading to go-ipfs 0.5 to check the progress yourself!

• Of course, there are many other ways to get involved with IPFS, depending on your skills, interests, and availability. Check out our page on GitHub for guidance and next steps.

• This is an exciting time for IPFS and the web as a whole. We welcome you to join us!