Understanding GPU Internet io.net

Note: On March 6, 2024, Solana Ecosystem DePIN Protocol io.net announced the completion of a $30 million Series A financing. io.net said it will use the funds raised to build the world's largest decentralized GPU network and solve the AI ​​computing shortage problem. Multicoin Capital participated in the investment. Multicoin Capital partner Shayon Sengupta wrote an article about why he invested in io.net. Translated by Jinse Finance xiaozou.

We are excited to announce our investment in io.net, the leading distributed computing marketplace for AI workloads. We led its seed round and participated in its Series A round. io.net has successfully raised $30 million from Multicoin, Hack VC, 6th Man Ventures, Modular Capital and a well-connected large angel investor to build an on-demand computing marketplace.

I first met io.net founder Ahmad Shadid at the Austin Solana Hacker House in April 2023, and I was immediately drawn to his focus on democratizing access to compute resources for ML workloads.

Since then, the io.net team has been executing on this idea at the speed of light. Today, the network aggregates tens of thousands of distributed GPUs and has delivered over 57,000 compute hours to AI enterprises. We are excited to be working with them as they drive the next decade of AI renaissance.

1. Global computing shortage

AI computing demand is growing at an alarming rate, and the demand is insatiable. With AI workload data center revenue exceeding $100 billion in 2023, even under the most conservative scenario, AI demand outstrips chip supply.

In a time of high interest rates and scarce capital, new data centers that can accommodate this type of hardware require large upfront investments. The crux of the matter is that advanced chips such as NVidia A100 and H100 are production-limited. While GPU performance continues to improve and costs steadily decrease, the physical manufacturing process is not accelerating fast enough, and shortages of raw materials, components, and production capacity have limited the pace of development.

Despite the promise of AI, its physical footprint is expanding, and the need for space, power, and cutting-edge equipment is straining budgets around the world. io.net paves the way for a world where the pace of development is not constrained by current supply chains.

io.net is a classic example of the DePIN theory: using token incentives to structurally reduce the cost of acquiring and retaining supply-side resources, and ultimately reduce costs for end consumers. The network brings together a large number of heterogeneous GPUs into a shared pool for use by AI developers and companies. Today, the network includes thousands of GPUs from data centers, mining farms, and consumer devices.

While this aggregation of resources is valuable, AI workloads do not automatically scale from centralized enterprise-grade hardware to distributed networks. In the history of crypto, there have been several attempts to build distributed computing networks, most of which have not generated meaningful demand-side volume.

Coordinating and scheduling workloads across heterogeneous hardware (with different memory, bandwidth, and storage configurations) is not an easy task. We believe the io.net team has the most practical solution on the market today to make this hardware aggregation beneficial and cost-effective for end customers.

2. Paving the way for clusters

Throughout the history of computing, software frameworks and design patterns have molded themselves around the hardware configurations available on the market. Most frameworks and libraries for AI development rely heavily on centralized hardware resources, but over the past decade, significant progress has been made in distributing workloads across geographically distributed hardware.

io.net leverages potential hardware around the world, deploys a custom network and coordination layer on top of it, brings them online, and creates a super-scalable GPU internet. The network leverages Ray, Ludwig, Kubernetes, and various other open source distributed computing frameworks to allow machine learning engineering and operations teams to scale their workloads with only minor adjustments on the GPU network.

ML teams can parallelize workloads on io.net GPUs by launching clusters on demand and leveraging these libraries to handle coordination, scheduling, fault tolerance, and scaling. For example, if a group of motion graphics designers contribute their home GPUs to the network, io.net can build a cluster that gives image diffusion model developers anywhere in the world access to collective computing resources.

BC8.ai, a fine-tuned variant of Stable Diffusion trained entirely on io.net hardware, is a good example of this. The io.net browser shows live inference and payments to network contributors.

Each inference is recorded on-chain to provide traceability. This special image generation was completed by 6 consumer-grade gaming GPUs RTX 4090.

Today, there are tens of thousands of devices on the network, spanning mining farms, underutilized data centers, and Render Network consumer nodes. In addition to creating new GPU supply, io.net is also able to often provide cheaper resources to compete on cost with traditional cloud providers.

They achieve cost savings by outsourcing GPU coordination and overhead to the protocol. On the other hand, cloud providers mark up infrastructure costs because they also have staff costs, hardware maintenance fees, and data center overhead. The opportunity cost of consumer clusters and mining farms is significantly lower than the cost acceptable to hyperscalers, so there is a structural arbitrage, and resources on io.net are priced lower than the ever-increasing cloud rates in real time.

3. Building the GPU Internet

Io.net has a unique advantage of being asset-light and reducing the marginal cost of serving any given customer to almost zero while building direct relationships with both the supply and demand sides of the market. They are well-positioned to serve the thousands of new businesses that need access to GPUs to build competitive products that everyone will interact with in the future.

We are excited to work with Ahmad and his team as they build and accelerate the development of artificial intelligence around the world. If you are building compute-intensive applications, you can sign up to access io.net's resources. If you have idle GPUs, you can also contribute them to the network to earn points.