Native Only Substrate Without Wasm

Posted on In Views: Disqus:

Native Only Substrate Without Wasm

Although Wasm execution is a main feature of Substrate, you may want to disable it in some use cases. In this article, I will show you how to build a native-only Substrate chain without wasm.

Warning: Think twice in your design!

This is not a recommended way to build a Substrate chain.

The Parity team is removing native execution: The road to the native runtime free world #62. We are in the opposite direction.

If you want to use some native libraries which do not support no-std inside runtime pallet, you should consider using:

  1. offchain workers
  2. runtime_interface

Both of them are in the outer node, not in runtime, so they can use std libraries and not limited by wasm.

Environment Setup

Environment: substrate-node-template | tag: polkadot-v0.9.40 | commit: 700c3a1

Suppose you want to import an std Rust library called rust-bert into your runtime pallet. rust-bert is a machine learning library including large language models (LLMs) such as GPT2.

First, download substrate-node-template

git clone https://github.com/substrate-developer-hub/substrate-node-template
cd substrate-node-template
git checkout polkadot-v0.9.40

Build

Add rust-bert as a dependency in pallets/template/Cargo.toml.

You also need to specify getrandom as a dependency. Otherwise it will throw an error error: the wasm32-unknown-unknown target is not supported by default, you may need to enable the "js" feature. For more information see: https://docs.rs/getrandom/#webassembly-support

In runtime pallets, all your dependencies must:

  1. Support no-std.
  2. std is not enabled by default. (that’s what default-features = false accomplishes)

Otherwise, you will get an error error[E0152]: found duplicate lang item panic_impl when building. The reason is that std is leaking into the runtime code.

You can check this stackoverflow question for more details.

In pallets/template/Cargo.toml:

[dependencies]
rust-bert = { version = "0.21.0", default-features = false, features = ["remote", "download-libtorch"] }
getrandom = { version = "0.2", default-features = false, features = ["js"] }

However, rust-bert do no support no-std. Even if you add default-feature = false in Cargo.toml, it will still throw an error error[E0152]: found duplicate lang item panic_impl when running cargo build.

To fix this error, you should skip building wasm code by adding env SKIP_WASM_BUILD=1.

https://github.com/paritytech/substrate/blob/master/utils/wasm-builder/README.md#environment-variables

SKIP_WASM_BUILD=1 cargo build

Run

By now, You should successfully build a native-only Substrate target without wasm.

But running the target binary is not easy.

--execution native specify the execution strategy to native-first.

./target/debug/node-template --dev --execution native
Error: Input("Development wasm not available")

Search Development wasm not available in the codebase, you will find that it is thrown by node/src/chain_spec.rs.

pub fn development_config() -> Result<ChainSpec, String> {
    let wasm_binary = WASM_BINARY.ok_or_else(|| "Development wasm not available".to_string())?;
  // ...
}

Since we don’t have wasm, we should remove this check.

let wasm_binary = WASM_BINARY.unwrap_or(&[] as &[u8]);

Rebuild and run again, new error occurs:

./target/debug/node-template --dev --execution native
2023-08-31 18:10:07 Substrate Node    
2023-08-31 18:10:07 ✌️  version 4.0.0-dev-700c3a186e5    
2023-08-31 18:10:07 ❤️  by Substrate DevHub <https://github.com/substrate-developer-hub>, 2017-2023    
2023-08-31 18:10:07 📋 Chain specification: Development    
2023-08-31 18:10:07 🏷  Node name: evanescent-agreement-4299    
2023-08-31 18:10:07 👤 Role: AUTHORITY    
2023-08-31 18:10:07 💾 Database: RocksDb at /tmp/substrate8yJbyt/chains/dev/db/full    
2023-08-31 18:10:07 ⛓  Native runtime: node-template-100 (node-template-1.tx1.au1)    
Error: Service(Client(VersionInvalid("cannot deserialize module: HeapOther(\"I/O Error: UnexpectedEof\")")))
2023-08-31 18:10:08 Cannot create a runtime error=Other("cannot deserialize module: HeapOther(\"I/O Error: UnexpectedEof\")")

This error is much harder to debug. I spend a lot of time to find the root cause.

Debug Process

Search online only finds one similar issue: https://github.com/paritytech/substrate/issues/7675. The core developer @bkchr suggests to create a dummy wasm binary to bypass the check.

I cannot find relevent information about constructing a dummy wasm binary. So I decide to debug the code step by step. Finally, I find the root cause is in one system dependency native_executor.rs.

There are 2 executors implementations: WasmExecutor and NativeElseWasmExecutor. When --execution native is specified, NativeElseWasmExecutor will be used.

NativeElseWasmExecutor wrap WasmExecutor as its field.

/// A generic `CodeExecutor` implementation that uses a delegate to determine wasm code equivalence
/// and dispatch to native code when possible, falling back on `WasmExecutor` when not.
pub struct NativeElseWasmExecutor<D: NativeExecutionDispatch> {
    /// Native runtime version info.
    native_version: NativeVersion,
    /// Fallback wasm executor.
    wasm:
        WasmExecutor<ExtendedHostFunctions<sp_io::SubstrateHostFunctions, D::ExtendHostFunctions>>, 
}

During substrate node running, even if NativeElseWasmExecutor is used, it will still try to load wasm binary in 2 methods: runtime_version and call.

Let’s look at runtime_version at first:

impl<D: NativeExecutionDispatch> RuntimeVersionOf for NativeElseWasmExecutor<D> {
    fn runtime_version(
        &self,
        ext: &mut dyn Externalities,
        runtime_code: &RuntimeCode,
    ) -> Result<RuntimeVersion> {
    Ok(self.native_version.runtime_version.clone()) // <--- Edit: We should return native version 

        self.wasm.runtime_version(ext, runtime_code) // <--- Original: it will try to load wasm binary
    }
}

Then, let’s look at call.

It will first check if the native version is compatible with the on-chain version. If it is compatible, it will call native executor. Otherwise, it will call wasm executor.

However, we don’t have valid wasm binary, so we should always call native executor. I just use the code inside this branch if use_native && can_call_with {}.

impl<D: NativeExecutionDispatch + 'static> CodeExecutor for NativeElseWasmExecutor<D> {
    type Error = Error;

    fn call(
        &self,
        ext: &mut dyn Externalities,
        runtime_code: &RuntimeCode,
        method: &str,
        data: &[u8],
        use_native: bool,
        context: CallContext,
    ) -> (Result<Vec<u8>>, bool) {
        // Edit
        // Do not check wasm since it is dummy, use native execution directly
        let used_native = true;
        let mut ext = AssertUnwindSafe(ext);
        let result = match with_externalities_safe(&mut **ext, move || D::dispatch(method, data)) {
            Ok(Some(value)) => Ok(value),
            Ok(None) => Err(Error::MethodNotFound(method.to_owned())),
            Err(err) => Err(err),
        };
    (result, used_native)
    
        // Original
        tracing::trace!(
            target: "executor",
            function = %method,
            "Executing function",
        );

        let on_chain_heap_alloc_strategy = runtime_code
            .heap_pages
            .map(|h| HeapAllocStrategy::Static { extra_pages: h as _ })
            .unwrap_or_else(|| self.wasm.default_onchain_heap_alloc_strategy);

        let heap_alloc_strategy = match context {
            CallContext::Offchain => self.wasm.default_offchain_heap_alloc_strategy,
            CallContext::Onchain => on_chain_heap_alloc_strategy,
        };

        let mut used_native = false;
        let result = self.wasm.with_instance(
            runtime_code,
            ext,
            heap_alloc_strategy,
            |_, mut instance, onchain_version, mut ext| {
                let onchain_version =
                    onchain_version.ok_or_else(|| Error::ApiError("Unknown version".into()))?;

                let can_call_with =
                    onchain_version.can_call_with(&self.native_version.runtime_version);

                if use_native && can_call_with {
                      // call native executor
                    tracing::trace!(
                        target: "executor",
                        native = %self.native_version.runtime_version,
                        chain = %onchain_version,
                        "Request for native execution succeeded",
                    );

                    used_native = true;
                    Ok(with_externalities_safe(&mut **ext, move || D::dispatch(method, data))?
                        .ok_or_else(|| Error::MethodNotFound(method.to_owned())))
                } else {
                      // call wasm executor
                    if !can_call_with {
                        tracing::trace!(
                            target: "executor",
                            native = %self.native_version.runtime_version,
                            chain = %onchain_version,
                            "Request for native execution failed",
                        );
                    }

                    with_externalities_safe(&mut **ext, move || instance.call_export(method, data))
                }
            },
        );
        (result, used_native)
    }

Finish!

You can use my substrate forked repo, which has already fixed the issue.

You should replace dependencies url in 3 Cargo.toml files:

git = "https://github.com/paritytech/substrate.git" -> git = "https://github.com/doutv/substrate.git"

replace

Build and run again, it should work now!

SKIP_WASM_BUILD=1 cargo build
./target/debug/node-template --dev --execution native

We successfully build a native-only Substrate chain without wasm!

Final code: substrate-node-template forked repo in native-only-polkadot-v0.9.40 branch, https://github.com/doutv/substrate-node-template/tree/native-only-polkadot-v0.9.40

Further Improvement

Modify NativeElseWasmExecutor is not the best solution, you may add an new executor implementation NativeOnlyExecutor.