Why are Solana transactions always failing due to recent severe congestion?

1. Let’s start with the basics. From a user’s perspective, when they make a transaction, essentially three things may happen:

a. Tx is executed successfully without any errors.

b. Executed but failed — Gas was paid, but the execution returned an error. This happens when conditions may not be met, such as the mint they were trying to buy was sold out, or the slippage was exceeded due to a very fast price movement, etc.

c. Dropped transactions: The tx does not appear anywhere, in this case the tx has not yet reached the block leader. This is what most users see currently. This is a network layer issue, not a consensus/execution issue.

2. Now you might ask what is the network layer, but later we will discuss these dropped transactions and why they are the main cause of the recent congestion on the Solana network.

First let's understand how failed trades occur, as this is important to understanding why failed trades are not a major problem.

3. You will be surprised to find that only about 8% of these failed trades are made by actual users, the rest are failed arbitrage trades made by on-chain bots. Arbitrageurs send spam on-chain because the cost of spamming is negligible compared to the ultimate reward of successful arbitrage.

4. For example: They could spam the chain multiple times a day, which would cost them a few hundred dollars (because Solana has lower fees), and they would make over $100,000 in profit just by completing one transaction.

5. Therefore, it is worth noting that these failed transactions do not mean that the blockchain has failed. The blockchain is operating as expected. These are just a few robot transactions that ultimately failed because the transaction conditions were not met. This is not the reason why the user experience on Solana is poor now.

6. In fact, Solana’s transaction failure rate has remained around 50% since November last year. If you look back at the failure and success chart I listed above, you will find that it was similar before.

7. Now let’s discuss the main cause of Solana congestion in the past few days - “transactions that never made it”. As mentioned earlier in this article, these transactions failed to reach the block leader and were discarded due to problems at the network layer.

8. The network layer is the communication layer of the Internet, which is used to send packets from one connection to another. For example - TCP, UDP, QUIC (designed by google), etc.

9. Solana recently upgraded to QUIC as its network layer , which helps to establish connections between users and block leaders. Since Solana has continuous block production and no memory pool, losing connection will mean that tx will not land on the block.

10. QUIC enables block leaders to disconnect or rate-limit certain user connections based on specific criteria. As a result, block leaders can now disconnect certain connections during periods of high demand.

11. This new architecture prevents Solana from downtime during periods of increased network activity. It may be too congested to be used, but at least it won’t go down this time, right?

12. So you might ask, if everything about QUIC is so well designed, why is Solana so bad? So the problem is that although block leaders can now limit certain connections, the logic for limiting connections is poorly implemented and has errors.

13. To understand better, let’s imagine a situation where each block leader has X number of connections it can communicate with. Now during peak activity, the block leader starts receiving requests for 10-100 times the number of connections…

14. This is when the block leader can choose to drop certain connections. However, currently these connections are not dropped according to set criteria (such as dropping all connections with a fee lower than x), but are dropped randomly...

15. Essentially, to win you have to spam more than everyone else, and because there are multiple bots that spam the network with connection requests, it becomes increasingly difficult for regular users to establish connections and complete transactions.

16. That’s the gist of the main issues. Teams like firedancer from jump, Anz, and Solana are working hard to deliver fixes to the network layer. As I write, these patches are rolling out, and it is said that some major patches will be released in the next few weeks.

17. So does this solve the problem? Will Solana surge again? Well, not exactly. There is still a long way to go for 3 reasons:

a. There is no guarantee that the current fix will prevent Solana

How effective it will be in terms of the next congestion problem. We will only know when we see its actual effect.

b. Firedancer may indeed solve these problems, but it won't be available until the end of this year.

C. Network Spam Issues: There are multiple issues with transaction economics on Solana, and there is no way to prevent bad actors from sending spam to the chain.

18. Finally, I want to end with this: I believe Solana is fighting to defend the right trade-offs. Just as Ethereum has gotten rid of its fair share of problems, so should Solana.