Lowering the cost to use Substreams on Solana!

With the rise in activity on Solana, the chain has established itself as one of the most popular projects for builders. Solana’s success can be attributed to Sealevel’s ability to allow multiple transactions to be processed simultaneously without affecting block latency, as well as the extremely low gas fees.

As more projects join the Solana ecosystem, dapps need a fast and reliable way of extracting data from the Solana blockchain. Substreams, one of the upcoming services within The Graph Network’s World of Data Services, allows you to consume Solana data both historically and in real time, with millisecond latency.

The Solana Optimization

Solana is a pretty heavy blockchain, with more than 200k blocks produced every day. Consuming a full Solana block requires reading a large number of bytes (adding up to nearly 500 GBs each day!), and thus led to high costs to process your Substreams.

However, most people do not need access to the full Solana block; many transactions are just “noise” for most dapp developers, and therefore can be ignored. This is the case for vote transactions by validators, which currently account for more than 60% of the transactions contained in a block. Vote transactions are used by validators to reach consensus and operate the Solana blockchain, but are not meaningful for most dapps.

The StreamingFast team has deployed a new optimization of the Substreams engine that allows you to consume a Solana block after the vote transactions have already been removed, thus reducing the number of bytes read. The result is a suite of Solana Substreams that are cheaper and faster to consume!

This feature is optimized for historical Solana data segments. StreamingFast is constantly pre-processing every Solana block and providing a cached version of blocks without the vote transactions. This cache is available mostly when doing historical processing, and future optimizations will bring this feature in real-time too.

Using the Optimization

Consuming Solana blocks without vote transactions is as simple as consuming the map_block_without_votes module of the Solana Explorer Substreams package. If you are using the CLI, you can simply run the following command:

substreams gui \
 https://spkg.io/streamingfast/solana-explorer-v0.2.0.spkg \
 map_block_without_votes \
 -e mainnet.sol.streamingfast.io:443 \
 — start-block 233817170 — stop-block=+2

If you are developing modules yourself, import the Solana Explorer package and use the map_block_without_votes module as an input instead of the larger sf.solana.type.v2.Block Protobuf in the Substreams manifest:

...
imports:
  sol: https://spkg.io/streamingfast/solana-explorer-v0.2.0.spkg
modules:
  - name: example_module
  kind: map
  inputs:
    - map: sol:map_block_without_votes # instead of source: sf.solana.type.v2.Block
...

That’s it! You can now focus on filtering data relevant to your dapp faster and at a much cheaper price. To start building Solana Substreams, take a look at the tutorials section of the documentation or visit the Substreams registry to find ready-to-use Substreams.