Brian, an important Rust developer, has a first experience of building and running CKB

In the original text, we can find that words such as confuse and stuck often appear in front of us, and we can feel Brian's seriousness and thoughtfulness. At the end of the article, we have also sorted out some of the pitfalls or suggestions he encountered, welcome to refer to. If you are a friend who has never been exposed to CKB before and want to develop and play on Nervos CKB, then you are very welcome to watch Brian's practical ideas. In this article, you can see the pitfalls he has encountered, maybe this can help you save some time.

Original link:

https://talk.nervos.org/t/experience-report-first-time-building-and-running-ckb/4518

Translation & Proofreading: Brother Mus & Sister Kelly

I want to write a software that can run on CKB, but first I have to learn how to build, run CKB and use its tools. Although I don’t have any experience in blockchain development, I still have some understanding of CKB based on my past studies. I have also created a project on CKB VM:

http://www.ferrisfencing.org/

I started by downloading the main repositories on Nervos’ GitHub, which led me to the Nervos website. I needed to find the documentation for building and running a private testnet on CKB, because once I run a test node, I will need documentation on how to write, deploy, and test scripts.

Building CKB

I clicked into the Nervos development documentation (https://docs.nervos.org/introduction/welcome.html) and was surprised to find that there was nothing about building CKB. I think this may be because the document is aimed at developers building on CKB and assumes that they don’t need to build CKB themselves. But personally, I want to build and understand my entire development stack, especially for a platform like CKB that is still in the development stage.

I looked through the README on GitHub and didn't see any documentation on how to build CKB.

Although there are two links called "Get CKB" and "Quick Start", the "Get CKB" document also contains a "Build from source" section, which says to use make prod to build CKB. But since CKB is a Rust project, I didn't want to do that, and instead wanted to try using cargo build, so I did that, and it worked.

I'm always curious what other items are added to their Makefiles to enhance cargo, in this case it just adds --cfg disable_faketime. I have no idea what this is, but I guess faketime is a field that can be used during development. I note that this is a "negative" cargo feature and is generally discouraged.

Explore the developer documentation

I seemed to be quickly drifting off my “happy path” of exploring the documentation, as I had neglected the docs website and dug into the markdown of the GitHub repo, so I backtracked and started over to the docs website.

To make sure I wasn’t missing something I was looking for in the main documentation, I opened the Getting Started/Introduction page. This is about running a node, using a wallet, and transacting, not developer documentation.

I swear I got here by clicking on the “Developer Documentation” link. I double-checked—I clicked “Developer” → “Documentation” on nervos.org, which landed me at docs.nervos.org, which describes itself as:

I'm confused as to who the intended audience for this document would be.

There is another document on this site called “Developer Guide”, which also does not give any hints on how to build CKB. I also noticed that it emphasizes using a testnet called “Aggron”, which makes me feel that the method of running your own testnet may not be popular.

Read the testnet documentation

Based on the documentation, my new plan is to learn using the testnet. Although I plan to use my own CKB creation regardless of whether the documentation requires me to download a version.

The first paragraph of the Testnet Aggron page really confused me:

I think the "we" here should refer to the Nervos project, not the "royal we" often used in tutorial documents to refer to "you". And this seems to have been written a long time ago, when the testnet was just deployed. (Translator's note: This page was written on December 9, 2019).

This page asked us to download a new version of CKB. Then I checked the CKB release page and found that the latest version was v0.29.0.

I checked the v0.29.0 tag and rebuilt CKB in debug mode.

At this point I discovered that ckb-cli was not part of the ckb repo. I was able to find the ckb-cli repo easily, but encountered a stumbling block.

Create ckb-cli

There is no v0.29.0 tag in ckb-cli. I don't know which version of ckb-cli matches ckb. This discrepancy makes me doubt whether the tarballs released from the ckb repo include ckb-cli.

I downloaded the 0.29 tarball and found that it did contain ckb-cli . I wanted to confirm which version of ckb-cli it was, and it seemed that I had no choice but to run the latest version. So I did that, and saw the response:

So ckb 0.29.0 is adapted from ckb-cli 0.27.1, and I guess these two versions are loosely coupled and I should always use the latest version of one of them.

I checked the latest version of ckb-cli and started building. This seemed to involve building another copy of ckb. A bit annoying, but that's it.

In a new window, I set up PATH to give me access to ckb and ckb-cli.

I noticed on the testnet page that the example versions of ckb and ckb-cli are from November 2019 and have the same version number:

Using the Testnet

Now I want to connect to the testnet and verify that I have the tools working.

Here are the original instructions:

• Create aggron.toml in the directory containing the CKB binary

• Initialize the CKB node with the command ckb init --import-spec ./aggron.toml --chain testnet .

The link to aggron.toml points to a gist from December of last year. I hope it's still valid.

I created a new directory ckb-testing to hold our testnet data and included the aggron.toml example in it.

Run ckb init:

 ~/ckb-testing$ ckb init --import-spec ./aggron.tml --chain testnet

The output is:

This looked like it was successful, copying aggron.toml into specs/testnet.toml and creating ckb.toml and ckb-miner.toml . I fiddled around with the two new toml files.

ckb.toml contains the following lines:

 [chain]# Choose the kind of chains to run, possible values:# - { file = "specs/dev.toml" }# - { bundled = "specs/testnet.toml" }# - { bundled = "specs/mainnet.toml" }spec = { file = "specs/testnet.toml" }

It shows that there is a built-in testnet definition, which makes me wonder if it is consistent with our own testnet definition. Let's see if we can find the built-in definition.

I searched for the configuration in the ckb repo using fd:

 ~/ckb$ fd "testnet.toml"resource/specs/testnet.toml

It’s true!

Is this built-in definition the same as the aggron.toml recommended by the developer documentation?

 ~/ckb$ diff ../ckb-testing/aggron.toml resource/specs/testnet.toml10c10< # run `cargo run cli hashes -b` to get the genesis hash---> # run `cargo run list-hashes -b` to get the genesis hash

Yes, except for one outdated comment.

Keep following the testnet documentation guys...

There is another description in the gist containing aggron.toml:

But it doesn’t say how to verify it. The genesis hash is not output by ckb init . The next step in the testnet documentation is ckb run , which I executed. Windows Firewall asked for my permission, which I agreed to (I was running on WSL at the moment). I saw a lot of log output, and ckb had started running:

 2020-03-20 15:10:22.445 -06:00 main INFO sentry **Notice**: The ckb process will send stack trace to sentry on Rust panics. This is enabled by default before mainnet, which can be opted out by setting the option `dsn` to empty in the config file. The DSN is now https://[email protected]/14227952020-03-20 15:10:23.354 -06:00 main INFO main Miner is disabled, edit ckb.toml to enable it2020-03-20 15:10:23.374 -06:00 main INFO ckb-db Initialize a new database2020-03-20 15:10:23.624 -06:00 main INFO ckb-db Init database version 201911271355212020-03-20 15:10:23.636 -06:00 main INFO ckb-chain Start: loading live cells ...2020-03-20 15:10:23.636 -06:00 main INFO ckb-chain Done: total 2 transactions.2020-03-20 15:10:23.653 -06:00 main INFO main ckb version: 0.29.0 (a6733e6 2020-02-26)2020-03-20 15:10:23.654 -06:00 main INFO main chain genesis hash: 0x184ac4658ed0c04a126551257990db132366cac22ab6270bbbc1f8c3220f302d2020-03-20 15:10:23.654 -06:00 main INFO ckb-network Generate random key2020-03-20 15:10:23.655 -06:00 main INFO ckb-network write random secret key to "/home/brian/ckb-testing/data/network/secret_key"2020-03-20 15:10:23.668 -06:00 main INFO ckb-network Listen on address: /ip4/0.0.0.0/tcp/8115/p2p/Qme9oaLbtaqF6JyZnZhrNPoW3dSPkzJSW1PGag3fKdEuaj2020-03-20 15:10:23.718 -06:00 NetworkRuntime-1 INFO ckb-network p2p service event: ListenStarted { address: "/ip4/0.0.0.0/tcp/8115" }2020-03-20 15:10:24.415 -06:00 NetworkRuntime-1 INFO ckb-relay RelayProtocol(1).connected peer=SessionId(1)2020-03-20 15:10:24.415 -06:00 NetworkRuntime-0 INFO ckb-sync  SyncProtocol.connected peer=SessionId(1)2020-03-20 15:10:24.474 -06:00 NetworkRuntime-2 INFO ckb-sync Ignoring getheaders from peer=SessionId(1) because node is in initial block download2020-03-20 15:10:25.319 -06:00 NetworkRuntime-5 INFO ckb-relay RelayProtocol(1).connected peer=SessionId(2)2020-03-20 15:10:25.433 -06:00 NetworkRuntime-4 INFO ckb-relay RelayProtocol(1).connected peer=SessionId(3)2020-03-20 15:10:25.712 -06:00 NetworkRuntime-3 INFO ckb-relay RelayProtocol(1).connected peer=SessionId(4)2020-03-20 15:10:26.346 -06:00 NetworkRuntime-1 INFO ckb-sync SyncProtocol.connected peer=SessionId(2)2020-03-20 15:10:26.346 -06:00 NetworkRuntime-1 INFO ckb-sync Ignoring getheaders from peer=SessionId(2) because node is in initial block download2020-03-20 15:10:26.347 -06:00 NetworkRuntime-1 INFO ckb-sync SyncProtocol.connected peer=SessionId(3)2020-03-20 15:10:26.347 -06:00 NetworkRuntime-1 INFO ckb-sync Ignoring getheaders from peer=SessionId(3) because node is in initial block download2020-03-20 15:10:26.347 -06:00 NetworkRuntime-1 INFO ckb-sync SyncProtocol.connected peer=SessionId(4)2020-03-20 15:10:26.348 -06:00 NetworkRuntime-1 INFO ckb-sync Ignoring getheaders from peer=SessionId(4) because node is in initial block download2020-03-20 15:10:28.039 -06:00 ChainService INFO ckb-chain block: 1, hash: 0xcfbc1525e97bdb6e3202ff19270bddc1fcd6d8f1aee14c719064a8b989909f5d, epoch: 0(1/1000), total_diff: 0x1800060, txs: 1

This log indicates that the testnet has the correct genesis hash and we are syncing blocks.

I am currently at block 2704. I would like to know how many blocks there are on the testnet so I know how long I have to wait.

Is this a testnet-only block explorer? I googled “nervos block explorer” and found this:

There is also a test network browser:

There are 567493 blocks on the testnet, but I'm not particularly excited to wait for it to download.

My data directory is about 7MB in size, so I estimate that I will need (600_000/3000 * 7) = ~1.4GB of space for testnet development, which is still ok. But I really don’t want to wait for sync, so let’s move on to the next part.

The next document is the "Dev Chain", which hopefully will tell us how to use our own chain.

I temporarily stopped ckb which was syncing the testnet, moved all its data to ckb-testing/testnet , and then continued running ckb to sync the testnet.

Then I created ckb-testing/devnet. According to the dev chain documentation, I executed:

 ckb init --chain dev

It outputs:

 ~/ckb-testing/devnet$ ckb init --chain devWARN: mining feature is disabled because of lacking the block assembler config optionsInitialized CKB directory in /home/brian/ckb-testing/devnetcreate specs/dev.tomlcreate ckb.tomlcreate ckb-miner.toml

The dev chain documentation wants me to customize specs/dev.toml and set genesis_epoch_length to 1000. I looked at my dev.toml and genesis_epoch_length was already 1000.

It wants me to customize ckb-miner.toml to set the idle time interval (often referred to as value ) to 500. I glanced at ckb-miner.toml and value was already set to 500. This document seems to be out of date.

Translator's note: The document shows that the setting value is 5000 (the screenshot is from 2020/03/10)

As mentioned in ckb init, an initialized chain is not yet configured for mining. Step 4 of the dev chain document is titled "Configure block-assembler for mining in ckb.toml" and it says:

I really wish it explained what lock_arg is, or linked to more information.

Then I ran ckb-cli account new.

It asked me to set a password, I set it to "foo", but the CLI didn't give me any response for a long time. I really wondered if it was broken.

Aimee wonders why we need an account. I guess it is connected to the public key of our miner, and the account will provide permission for the miner to store the mined CKB.

This part of the documentation is flawed.

Finally, the account creation command is completed and outputs:

 ~/ckb-testing/devnet$ time ckb-cli account newYour new account is locked with a password. Please give a password. Do not forget this password.Password: Repeat password: address: mainnet: ckb1qyq9u782efpvk8m3q88gplxlzadpct5d7ypqythss9 testnet: ckt1qyq9u782efpvk8m3q88gplxlzadpct5d7ypqewf0uelock_arg: 0x5e78eaca42cb1f7101ce80fcdf175a1c2e8df102lock_hash: 0xe68befeae28e69dc658565be71bc7146fd33a9a4bea3f7edba8de8c8c46865ff

Uh, looking at the output, it still needs some work. You can only see the labels (address, mainnet are also dark blue which is not visible in my terminal).

It took me 168 seconds to run the debug binary and sync the testnet in the background.

For the experiment, I stopped syncing to the testnet, ran ckb-cli account new again in the debug version; then rebuilt ckb and ckb-cli in release mode.

• debug: 99 seconds
• release: 15 seconds

 2020-03-24 21:50:22.644 -06:00 NetworkRuntime-3 INFO ckb-network Ban peer PeerId(QmT6DFfm18wtbJz3y4aPNn3ac86N4d4p4xtfQRRPf73frC) for 300 seconds, reason: The nodes are not on the same network

• lock_arg must be prefixed with 0x and followed by the text in args.
• Fill in 0x in message

 # Set the lock script to protect mined CKB.## CKB uses CS architecture for miner. Miner process (ckb miner) gets block# template from the Node process (ckb run) via RPC. Thus the lock script is# configured in ckb.toml instead of ckb-miner.toml, and the config takes effect# after restarting Node process.## The `code_hash` identifies different cryptography algorithm. Read the manual# of the lock script provider about how to generate this config.## CKB provides an secp256k1 implementation, it requires a hash on the# compressed public key. The hash algorithm is blake2b, with personal# "ckb-default-hash". The first 160 bits (20 bytes) are used as the only arg.## You can use any tool you trust to generate a Bitcoin private key and public# key pair, which can be used in CKB as well. CKB CLI provides the function for# you to convert the public key into block assembler configuration parameters.## Here is an example using ckb-cli to generate an account, this command will# print the block assembler args(lock_arg) to screen:## ckb-cli account new## If you already have a raw secp256k1 private key, you can get the lock_arg by:## ckb-cli util key-info --privkey-path## The command `ckb init` also accepts options to generate the block assembler# directly. See `ckb init --help` for details.## ckb init## secp256k1_blake160_sighash_all example:# [block_assembler]# code_hash = "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8"# args = "ckb cli blake160"# hash_type = "type"# message = "A 0x-prefixed hex string"

Well, there are still a lot of things that are not explained clearly here:

• What is the "CS architecture of miners"? I don't understand.
• "Please refer to the lock script provider's manual." I don't know what a lock script provider is, nor do I know where the manual is.
• "The hashing algorithm is blake2b with a personal "ckb-default-hash"" What is a personal "ckb-default-hash"?

 # [block_assembler]# code_hash = "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8" # args = "ckb cli blake160" # hash_type = "type" # message = "A 0x-prefixed hex string"

• Forget code_hash
• Replace all args with the value of lock_arg
• Don't touch hash_type again
• Set message to "0x"

• Explore the capabilities of ckb-cli;
• Try to use ckb-cli to make a transaction on devnet.

 CKB> account newYour new account is locked with a password. Please give a password. Do not forget this password.Password: Repeat password: address: mainnet: ckb1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqw9e2hp testnet: ckt1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqnq84malock_arg: 0xa0ea4869c3307e6825709adfd8f68313c47c4160lock_hash: 0xb686ca47423942cd64f4d091829450ad3b968b485d922b773a4672616a084c0c

 CKB> wallet get-capacity --lock-arg 0xa0ea4869c3307e6825709adfd8f68313c47c4160total: 0.0 (CKB)

 CKB> wallet get-live-cells --address=ckt1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqnq84macurrent_capacity: 5000.0 (CKB)current_count: 1live_cells: - capacity: 5000.0 (CKB) data_bytes: 0 index: output_index: 0 tx_index: 1 lock_hash: 0xb686ca47423942cd64f4d091829450ad3b968b485d922b773a4672616a084c0c mature: true number: 55611 tx_hash: 0x1a8a1650dcf8bc21a210a970073b6d1a47eff13afdd649e5cb3ecc3d56c06083 tx_index: 0 type_hashes: ~total_capacity: 5000.0 (CKB)total_count: 1

 CKB> wallet transfer --capacity 1000 --from-account 0x2af2ea2efbffce44f51cd4d7731507fdbbd63629 --to-address ckt1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqnq84ma --tx-fee 1PassworD:0xc0a6973ac921cea95d27fa9e8a30ae6fca18de2404f768e3a077dd1e5f345ec0

 CKB> wallet get-capacity --address=ckt1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqnq84matotal: 6000.0 (CKB)CKB> wallet get-live-cells --address=ckt1qyq2p6jgd8pnqlngy4cf4h7c76p383rug9sqnq84macurrent_capacity: 6000.0 (CKB)current_count: 2live_cells: - capacity: 5000.0 (CKB) data_bytes: 0 index: output_index: 0 tx_index: 1 lock_hash: 0xb686ca47423942cd64f4d091829450ad3b968b485d922b773a4672616a084c0c mature: true number: 55611 tx_hash: 0x1a8a1650dcf8bc21a210a970073b6d1a47eff13afdd649e5cb3ecc3d56c06083 tx_index: 0 type_hashes: ~ - capacity: 1000.0 (CKB) data_bytes: 0 index: output_index: 0 tx_index: 1 lock_hash: 0xb686ca47423942cd64f4d091829450ad3b968b485d922b773a4672616a084c0c mature: true

• https://docs.nervos.org

• https://github.com/nervosnetwork/ckb/blob/develop/docs/get-ckb.md

• https://github.com/nervosnetwork/ckb/tree/develop/docs

• RFC: Simple UDT Draft Spec

• RFC: anyone-can-pay lock

• https://gist.github.com/WilfredTA/bb1d2a80c420f0910d40ca881c2bd5d4

• https://github.com/nervosnetwork/ckb-cli/blob/develop/README.md

• https://github.com/nervosnetwork/ckb/wiki/Chains

• https://github.com/nervosnetwork/ckb/wiki

• Introducing Godwoken - A missing piece of the cell model

• SECBIT Labs - zkp-toolkit-ckb - a Zero-Knowledge Proof toolkit for CKB

• https://github.com/nervosnetwork/wasm-secp256k1-test

• Summa - Bitcoin SPV Utils

• https://github.com/nervosnetwork/ckb-cli/wiki/Sub-Commands

The above is the first experience of Brian, a heavyweight Rust developer, in building and running CKB. Let’s summarize Brian’s development ideas:

Let's summarize Brian's points:

1. I can’t find any instructions on how to build my own CKB chain in the documentation.
2. There is no instruction on how to build CKB in the Getting Start document.
3. The version numbers of ckb-cli and ckb do not match, maybe one is v0.29.0 and the other is v0.27.1, but there is no special description
4. The documentation for Aggron test is still from December
5. There is no guide telling you how long it takes to sync the testnet or where to go to the block explorer, etc.
6. There is not enough explanation for the special meaning of the fields, for example, what does lock_arg \ lock_hash \ hash_type do? They are not explained in the document (ps, maybe some content like rfc19 will be helpful)
7. Dev chain can give you some guidance to let you know that it is here to build your own CKB chain
8. ckb-cli does not provide any immediate feedback or prompts after entering the password
9. It doesn't seem to explain why you need to set a password.
10. The testnet was reset and there wasn’t enough guidance for developers on what they should do.
11. A thousand-word comment suddenly appeared in [block_assembler] of ckb.toml, but some parts of the comment itself are difficult to understand
12. The examples in the document are collapsed by default and may be accidentally overlooked.
13. It seems that the documentation does not clearly explain why many instructions are done this way, such as why lock_arg must be prefixed with 0x, why massage must be filled with 0x, etc.
14. What is capacity in a wallet? It seems that many of the technical terms seem to be known to developers.