Dr. Cao Feng: Is blockchain the end or rebirth of database? — Blockchain, database, virtual reality and movies

On April 19, I attended the ChinaLedger China Distributed Ledger Basic Protocol Alliance press conference as a co-sponsor. In the evening, I suddenly received a call from my doctoral supervisor, Professor Zhou. It turned out that he was organizing an academic forum, inviting experts and scholars from the database academic community at home and abroad, hoping to attract the attention of the academic community to blockchain technology and discuss such a new technology from the perspective of database research. He hoped that I could give a speech at this conference. Professor Zhou is a leading figure in database research at home and abroad, and has always maintained a high degree of foresight for new technologies and new directions in the database field. For example, he took the lead in establishing the first cloud computing and big data research center in China in 2013, as well as many internationally leading achievements.

The invitation from my supervisor coincides with my current research direction. I have always wanted to organize my thoughts on blockchain and database into an article. Due to work reasons, this article was not completed until today! It can also be regarded as a warm-up for the academic seminar on blockchain and database organized by my supervisor, and at the same time pay tribute to my supervisor's unique academic insights.

IBM is one of the earliest international companies involved in blockchain research and development, such as the well-known IBM and Samsung blockchain cooperation project and Linux/IBM joint project last year. As the head of IBM blockchain technology in China and co-chair of the patent review committee, my team and I also submitted and obtained the US patent for blockchain in October last year. This is the first international blockchain patent obtained by Chinese. Then in December, we organized the IBM Global Fintech Summit and internal closed-door meeting, inviting academicians, DE, VP and CTO from IBM's 13 major research institutes in the blockchain-related field to gather in Shanghai for a two-day seminar. The result of the discussion was that the company quickly promoted blockchain to the company's global strategy this year. And at this summit, one of the core issues that everyone debated was, what is the relationship between blockchain and database? What is the essential difference between blockchain and distributed database? In February this year, I attended WSDM 2016 and Microsoft Global Technology Leadership Summit in the San Francisco Bay Area, and some teams have begun to study data analysis based on blockchain. Recently, Oracle has also begun to pay attention to the topic of blockchain and database, which has further heated up.

More than 10 years ago, my first job was to develop the ASE database at Sybase, where I was responsible for the SQL query optimization module. You may not be familiar with ASE, but it is actually the predecessor of the famous Microsoft SQLServer. At that time, the domestic database market was basically monopolized by foreign capital, and the few database products in China were not technically mature enough. Sybase was one of the few companies in China that could really give Chinese employees the opportunity to access the underlying code of first-class foreign databases. This underlying coding experience laid a solid foundation for me to be responsible for many large-scale projects such as Internet finance and man-machine wars at IBM later, and also gave me the opportunity to understand the relationship between databases and blockchains from a more basic level.

Database is actually a very old research field in the IT industry. From the initial file system to the later ER entity-relationship model. The introduction of the entity-relationship model gave birth to a series of great database companies and software, such as IBM's DB2, Sybase, Oracle, Microsoft's SQLServer, MySQL, etc. And, this led to the three major achievements of traditional databases, relational models, transaction processing, and query optimization. Later, with the prevalence of the Internet, NOSQL databases represented by MangoDB emerged. Database technology itself is constantly evolving and has always been a hot direction, including semi-structured represented by XML, unstructured data processing based on text, voice and images, etc. For example, the big data that everyone has been talking about in recent years is actually a small branch of database research.

Speaking of the relationship between database and blockchain, I personally think that it is somewhat similar to the relationship between virtual reality and movies. Movies are an even older industry. From the earliest silent movies, to black and white movies, to color movies, and now to the 3D movies and IMAX movies that everyone often watches. Today's film industry has developed beyond everyone's imagination. Especially with the emergence of television, everyone thought it was the end of the film industry. But in fact, the development of television and movies have taken completely different dimensions. The film industry continues to actively absorb new technologies to bring a new viewing experience to the audience. And the next new technology that may be included in the film may be virtual reality. Virtual reality technology is both a challenge and an opportunity. It can unleash all the imagination of artists and allow them to create a world that could not have existed before. Just like the movie "Inception", artists can imagine a world of anti-gravity and bring this world to the audience. In other words, movies can rely on technology to achieve a new visual space and enjoyment.

The relationship between blockchain and database also seems to be such a relationship. From the evolution of database technology, we can find that it has such a continuous vitality to meet new business needs and create various unique data processing technologies. It can be said that each generation of database technology innovation is caused by practical needs. For example, why did the file system evolve into the ER entity-relationship model from the beginning? In fact, it was due to the development of financial banks. Everyone has a very practical demand for fast accounting and support for high-concurrency data writing and access, which led to the generation and rapid development of the actual relationship model. Later, NOSQL appeared, which was actually due to the rapid development of the Internet that put forward new requirements for databases. The time for Internet project development is very tight, and the project needs to be iterated quickly. There is no way to tolerate the original traditional development method based on SQL database. Because in the entity-relationship model, the design of the database table needs to be defined very accurately according to the business logic. The rapid iteration characteristics of Internet projects are destined to make its business logic constantly and quickly adjusted. If everything is redesigned from scratch, and the access layer in the middle is written, it will cause interruptions, which everyone cannot stand, so it led to the emergence of NOSQL. Everyone can write a particularly good database application without knowing SQL statements. Of course, NOSQL supports more and richer connotations. For example, it focuses on fast data writing operations, and does not require too many modification and deletion operations. In essence, we can think of the entire Internet as a large database.

The problem is that things are constantly evolving. When we solved the problem of processing massive amounts of Internet data through technologies such as NOSQL databases and cloud storage, the next problem followed. That is, how to solve the problem of data authenticity and validity in a scalable way? In fact, this real and effective demand is also very easy to understand. From a certain perspective, it is like our human demand for food. At the beginning, there were many people and little food, and food was scarce. The first thing to solve was to let everyone have the opportunity to eat enough, that is, the problem of food scale. Then one day, we found that everyone could eat enough. Because food production, especially meat production, is already an industrialized way of operation. At this time, everyone turned around and pursued organic food, pursuing these foods that return to nature. At this time, a new organic food industry was born. The development of databases is the same. When we use ER entity relationship models and NOSQL, we can solve the scalability problem of data storage and data access very well. The next problem that everyone needs to solve and care about must be the problem of authenticity and validity. Just like our current demand for natural and organic food.

Therefore, the demand for data organization that is authentic, valid, unforgeable, and tamper-proof, represented by blockchain, is a new starting point and new requirement for existing databases. I feel more and more clearly that the trend of database and blockchain integration is almost unstoppable, just like the development of movies has been unstoppable towards virtual reality technology.

From the perspective of database, blockchain is actually a new way of organizing data. In line with the development of history, the financial industry has once again become the initial advocate and user of this new technology. The demand for data authenticity, non-forgeability and tampering is likely to trigger and create several blockchain database companies. Due to the cornerstone status of database in the IT industry, it can be inferred that the impact of blockchain on the IT industry may be very far-reaching.

The future is here, and the financial applications of blockchain are just the beginning.

Postscript: Since many non-computer science friends pointed out that my previous article "Blockchain Reconstructs Big Data and Changes Everything" was too academic and difficult to understand, this article does not explain the relationship between blockchain and database from a purely technical perspective. Instead, it uses more analogies, hoping to do some work in promoting and popularizing blockchain. If there is anything inappropriate, please criticize and correct me.

Author: Dr. Cao Feng, Vice President of CIO and Chief Scientist of Henlius