# Carbon 2.0.0 documentation (all pages)

Source: https://carbon.sevenlabs.org. Per-program decoder references are not included here; they are listed with their crates and program IDs in https://carbon.sevenlabs.org/llms.txt, and the MCP tool `find_decoder` looks one up.

## Important: common misconceptions

- These docs are for Carbon 2.0.0. Do not rely on training data: Carbon's APIs changed in 1.0 and 2.0. Use `carbon-core = "2.0.0"` and the matching `carbon-*` crates at 2.0.0; never mix Carbon 1.x and 2.x crates in one dependency graph.
- Carbon 2.0 needs Rust 1.96.1 or newer and the Agave 4.2-compatible Solana crates. If you add Solana crates directly, use versions that resolve to a single copy of each Solana type crate. Mixing Carbon 1 and 2 crates fails with "the trait `InstructionDecoder` is not implemented" and a note about multiple versions of `carbon_core`; mismatched Solana crates fail with errors like "expected `Account`, found a different `Account`".
- `Processor::process` is a native `async fn` (`impl Processor<T> for P { async fn process(&mut self, input: &T) -> CarbonResult<()> }`). Do not add `#[async_trait]` to processors. The `Datasource` trait does use `#[async_trait]`.
- Decoded instruction enums use struct variants: `MyProgramInstruction::Swap { program_id, data, accounts }`. Match with `MyProgramInstruction::Swap { data, .. }`.
- Solana's public endpoint `https://api.mainnet-beta.solana.com` rejects `blockSubscribe` ("Method not found"). On the public endpoint use the RPC transaction crawler, `getProgramAccounts` or `programSubscribe` datasources; use a provider that enables block subscriptions, or a Geyser gRPC stream, for `RpcBlockSubscribe`.
- RPC block and transaction configs must request a binary encoding (`UiTransactionEncoding::Base64`) and `max_supported_transaction_version: Some(1)`. Carbon cannot rebuild transactions from `Json` or `JsonParsed` payloads.
- Zero-lamport account updates arrive as `Update::AccountDeletion` (handled by `.account_deletions(...)`), not as account updates. Custom datasources should build an `AccountUpdate` and call `.into_update()`.
- Decoder crates are named `carbon-<program>-decoder`, export `PROGRAM_ID` and a `<Program>Decoder` unit struct, and are added with `.instruction(Decoder, processor)` or `.account(Decoder, processor)` on `Pipeline::builder()`. Generate one for any Anchor or Codama IDL with `npx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl <file-or-program-id> --out-dir <dir>` (for Codama IDLs also pass `--standard codama --program-id <address>`).


---

# Introduction (Carbon 2.0)

> What Carbon is, how a pipeline moves Solana data from datasources through decoders to your processors, what you can build and when to use something else.

Carbon is an open-source Rust framework for building Solana indexers and data pipelines. You choose where the data comes from, Carbon decodes it into typed Rust values, and your code decides what to do with them. Carbon is maintained by [SevenLabs](https://github.com/sevenlabs-hq) with the Solana Foundation and released under the MIT license.

## How a pipeline works

A Carbon program builds one `Pipeline` from three kinds of parts:

- **Datasources** read updates from Solana: account writes, transactions, account closures and block metadata. A datasource can be a Geyser gRPC stream, a Solana RPC subscription, a crawler over history, a validator snapshot or a provider API.
- **Decoders** turn raw bytes into Rust types. Carbon publishes decoders for 60+ programs, and the Carbon CLI generates one from any Anchor or Codama IDL.
- **Processors** are your code. Each one receives decoded data together with its metadata: slot, signature, the instruction's position in the transaction and so on.

![How a Carbon pipeline moves data](https://carbon.sevenlabs.org/diagrams/pipeline-overview.svg)

**Diagram: How a Carbon pipeline moves data.** Datasources such as Yellowstone gRPC, an RPC crawler or a snapshot each send updates into one channel. The pipeline takes updates off the channel one at a time and offers each one to the pipes for its type: account pipes and instruction pipes filter and decode, transaction pipes filter and decode every instruction with a decoder set, and account-deletion and block-details pipes only filter. Every pipe then calls its processor, which is your code.

Every datasource runs in its own task and sends updates into one channel. The pipeline takes updates off the channel one at a time and offers each one to the pipes that match its type. A pipe applies its filters, decodes the update and calls its processor. [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md) explains the exact order and guarantees.

The same processor works with any datasource. You can develop against the free public RPC endpoint and switch to a Geyser stream in production without changing your processing code.

## What you can build

- **Real-time indexers** that write decoded instructions and account state to Postgres or another store as blocks land, from a Geyser gRPC stream such as [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) or [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md).
- **Backfills** that walk a program's transaction history with the [transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) or a block range with the [block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md).
- **State loaders** that read every account of a program at once from `getProgramAccounts` or a [validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md), then keep it current with a stream.
- **Bots and alerting** that react to swaps, liquidations or account changes within the same slot.
- **Analytics pipelines** that aggregate trades, fees or balances across several programs, using [transaction processors](https://carbon.sevenlabs.org/concepts/transactions.md) to see every instruction of a transaction together.

Carbon also ships optional [Postgres](https://carbon.sevenlabs.org/guides/postgres.md) and [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md) layers, [Prometheus metrics](https://carbon.sevenlabs.org/guides/prometheus.md) and test utilities.

## When not to use Carbon

- **You need ad-hoc queries over all of Solana's history.** Carbon processes the updates you subscribe to; it isn't a hosted warehouse. Use a data warehouse or a provider's historical API, and use Carbon when you want your own tables for specific programs.
- **You only need a few RPC calls.** Reading one account or sending transactions is simpler with `solana-client` directly.
- **You can't use Rust.** Carbon is a Rust library. The Carbon CLI is a Node.js tool, but it only generates Rust code.
- **You need a datasource that isn't on the Carbon 2 line yet.** Jito Shredstream and Jetstreamer have no 2.0 crates. See [Datasources](https://carbon.sevenlabs.org/datasources.md) for what's available, or [write your own](https://carbon.sevenlabs.org/guides/custom-datasource.md).

## Map of the docs

| Section                                                                                                                                                                          | Read it when you want to                                                                      |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [Quickstart](https://carbon.sevenlabs.org/quickstart.md)                                                                                                                         | Run a working indexer against mainnet in five minutes.                                        |
| [Installation](https://carbon.sevenlabs.org/installation.md)                                                                                                                     | Add Carbon to a project and pick crates and features.                                         |
| Concepts, starting with [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md)                                                                                    | Understand updates, datasources, decoders, processors, filters, metrics, shutdown and errors. |
| Guides, such as [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md)                                                                                  | Follow a task end to end: streaming, backfills, Postgres, GraphQL, testing, production.       |
| Reference, starting with [`PipelineBuilder`](https://carbon.sevenlabs.org/reference/pipeline-builder.md)                                                                         | Look up a builder method, trait, filter, metric, error, feature flag or CLI flag.             |
| [Datasources](https://carbon.sevenlabs.org/datasources.md) and [Decoders](https://carbon.sevenlabs.org/decoders.md)                                                              | Configure a specific datasource or decoder crate.                                             |
| [Examples](https://carbon.sevenlabs.org/examples.md), [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md), [Changelog](https://carbon.sevenlabs.org/changelog.md) | Copy a full project, fix an error, or upgrade.                                                |

Using a coding agent? Every page is available as Markdown; see [For AI agents](https://carbon.sevenlabs.org/agents.md).

## Next steps

- [Quickstart](https://carbon.sevenlabs.org/quickstart.md): Decode live Meteora swaps from mainnet. No account needed.
- [Installation](https://carbon.sevenlabs.org/installation.md): Requirements, crates and feature flags.
- [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md): How Pipeline::run moves an update from datasource to processor.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/introduction · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Quickstart (Carbon 2.0)

> Build an indexer that decodes live Meteora DLMM swaps from Solana mainnet in about five minutes. No account or API key needed.

You'll build a small Rust program that reads a Solana program's transactions, decodes each instruction into a typed Rust struct, and prints swaps as they happen. It uses Solana's public RPC endpoint, so you don't need an account or an API key.

**You need:** Rust 1.96.1 or newer, an internet connection, and about 3 GB of free disk space for the first build.

### Step 1: Check your Rust version

```sh
rustc --version
```

Output:

```text
rustc 1.99.0 (b940084d7 2026-09-28)
```

Any version from 1.96.1 up works. If yours is older, run `rustup update stable`. If you don't have Rust, install it from [rustup.rs](https://rustup.rs).

### Step 2: Create a project

```sh
cargo new carbon-quickstart
cd carbon-quickstart
```

### Step 3: Add Carbon

Add the core crate, a datasource, a decoder for the program you want to read, and a metrics exporter. All Carbon crates in one project must have the same version.

```sh
cargo add carbon-core@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-log-metrics@2.0.0
cargo add tokio --features full
cargo add env_logger log
```

Your `[dependencies]` now look like this (patch versions of `tokio`, `env_logger` and `log` may be newer):

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 4: Write the pipeline

Replace `src/main.rs` with this program. It connects three parts: a **datasource** that fetches the program's transactions, a **decoder** that turns Meteora DLMM instructions into Rust types, and a **processor**, `SwapLogger`, that prints them.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // Solana's public endpoint works for this example. Set RPC_URL to use your own.
    let rpc_url = std::env::var("RPC_URL")
        .unwrap_or_else(|_| "https://api.mainnet-beta.solana.com".to_string());

    // Fetches the program's recent transactions, newest first, then keeps polling for new ones.
    let datasource = RpcTransactionCrawler::new(
        rpc_url,
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let signature = input.metadata.transaction_metadata.signature;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, accounts, .. } => println!(
                "swap  pool={} amount_in={} min_amount_out={}",
                accounts.lb_pair, data.amount_in, data.min_amount_out,
            ),
            MeteoraDlmmInstruction::Swap2 { data, accounts, .. } => println!(
                "swap2 pool={} amount_in={} min_amount_out={}",
                accounts.lb_pair, data.amount_in, data.min_amount_out,
            ),
            // Every other instruction is decoded too; print its name.
            other => println!("{} in {signature}", variant_name(other)),
        }
        Ok(())
    }
}

/// The variant name of a decoded instruction, e.g. "AddLiquidityByStrategy2".
fn variant_name(instruction: &MeteoraDlmmInstruction) -> String {
    format!("{instruction:?}").split([' ', '{', '(']).next().unwrap_or_default().to_string()
}
```

### Step 5: Run it

```sh
cargo run
```

The first build compiles the Solana dependencies and takes one to three minutes. Then decoded instructions start printing, newest transactions first. On Solana's public endpoint the first line can take a minute or more, because it rate-limits the crawler; see [Nothing prints for a minute or more](#nothing-prints-for-a-minute-or-more).

Output:

```text
swap2 pool=6e7V9eegCHw997T72MxgwwJipZ6GJyZF8NvjkzT1rvpN amount_in=84225603 min_amount_out=0
CpiEvent in 5oc7rbKmGEti7jEfvCt77cWWTG8gUvQn5breYsMwpA396gn3FJAozKjUvKE6Nqma3SHZbKg4kXSi8G9pzSfTL8cA
CpiEvent in 5oc7rbKmGEti7jEfvCt77cWWTG8gUvQn5breYsMwpA396gn3FJAozKjUvKE6Nqma3SHZbKg4kXSi8G9pzSfTL8cA
swap  pool=GMgh4NtWrGaUf1RR2kcXD7LY1jou1qFAuSsQeKp5ow4a amount_in=14968688 min_amount_out=0
swap2 pool=4ju8phDxUU4WSJP1wWPLZ8R2KRCqwP3VZFXAqhT9cfVy amount_in=56765035 min_amount_out=1
…
```

`CpiEvent` lines are the events Meteora emits through self-CPI after each swap; Carbon decodes them as instructions too. Press `Ctrl+C` to stop.

## What you built

- **The datasource**, [`RpcTransactionCrawler`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), asks the RPC node for the program's signatures (`getSignaturesForAddress`), fetches each transaction (`getTransaction`), skips failed ones, and sends the rest into the pipeline. It keeps polling every five seconds for new transactions.
- **The decoder**, `MeteoraDlmmDecoder` from [`carbon-meteora-dlmm-decoder`](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md), matches each instruction's discriminator and returns a `MeteoraDlmmInstruction` with typed `data` and named `accounts`. It decodes instructions nested inside other programs' calls too, such as a Jupiter route that swaps through a DLMM pool.
- **The processor**, `SwapLogger`, implements [`Processor`](https://carbon.sevenlabs.org/concepts/processors.md) for the decoder's output. Carbon calls `process` once per decoded instruction, with the instruction's [metadata](https://carbon.sevenlabs.org/reference/processor-inputs.md) (signature, slot, position in the transaction).
- **The pipeline** wires them together and runs until you stop it. See [how a pipeline runs](https://carbon.sevenlabs.org/concepts/architecture.md).

## See the pipeline's metrics

`LogMetrics` writes throughput and queue depth to the log every five seconds. Turn the log on with `RUST_LOG`:

```sh
RUST_LOG=info cargo run
```

Every five seconds you get a summary line followed by each metric's value. An excerpt:

Output:

```text
[2026-11-09T00:02:47Z INFO  carbon_log_metrics] 00:00:50 (+5.006587213s) | 7 processed (100%), 7 successful, 0 failed (0%), 0 in queue
[2026-11-09T00:02:47Z INFO  carbon_log_metrics]   carbon_updates_received_total: 7
[2026-11-09T00:02:47Z INFO  carbon_log_metrics]   transaction_crawler_signatures_fetched_total: 1000
[2026-11-09T00:02:47Z INFO  carbon_log_metrics]   transaction_crawler_transactions_fetched_total: 7
…
```

The [metrics reference](https://carbon.sevenlabs.org/reference/metrics.md) lists every metric.

## Use your own RPC endpoint

The public endpoint is rate limited, so the example fetches only a few transactions per second. Any Solana RPC provider's URL works in its place:

```sh
RPC_URL="https://your-rpc-provider.example/?api-key=…" cargo run
```

For production, stream with Geyser gRPC instead of polling: see [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md). The processor doesn't change.

## Troubleshooting

### `rustc 1.95.0 is not supported by the following packages`

Output:

```text
error: rustc 1.95.0 is not supported by the following packages:
  carbon-core@2.0.0 requires rustc 1.96.1
```

Your Rust toolchain is older than Carbon 2.0 needs. Run `rustup update stable`, then `cargo run` again.

### Nothing prints for a minute or more

The public RPC endpoint limits how many requests you can make, and the crawler waits and retries when it hits the limit. It also skips failed transactions, so a burst of failed bot transactions prints nothing. To see whether it's working, run with metrics:

```sh
RUST_LOG=info cargo run
```

If `transaction_crawler_signatures_fetched_total` grows while `transaction_crawler_transactions_fetched_total` stays low, the crawler has found transactions and is being rate limited while it fetches them. Output follows; use your own endpoint with `RPC_URL` for steady throughput.

### `Failed to fetch transaction … after 3 retries` in the log

The RPC node kept rejecting requests, usually with HTTP 429 (Too Many Requests). The crawler logs the error, skips that transaction and continues. Lower the request rate with a custom [`ConnectionConfig`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md#configuration), or use your own endpoint.

### `the trait InstructionDecoder is not implemented` for a decoder

A Carbon 1 crate is mixed with Carbon 2, so two versions of `carbon-core` are compiled; the compiler adds the note `there are multiple different versions of crate carbon_core`. Make every `carbon-*` dependency `2.0.0`, then run `cargo update`. See [Installation](https://carbon.sevenlabs.org/installation.md).

More problems and fixes are in [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md).

## Next steps

- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): Decode your own program from its Anchor or Codama IDL.
- [Choose a datasource](https://carbon.sevenlabs.org/datasources.md): Real-time streams, backfills, snapshots and current state.
- [Persist to Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Store decoded instructions and accounts in typed tables.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/quickstart · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Installation (Carbon 2.0)

> Requirements, the Carbon crates to add for each job, core and decoder feature flags, and how to install and run the Carbon CLI.

This page lists what Carbon 2.0 needs, which crates to add for the indexer you're building, and how to run the Carbon CLI. For a guided first project, follow the [quickstart](https://carbon.sevenlabs.org/quickstart.md) instead.

## Requirements

| Requirement   | Version          | Needed for                                                                                                                                     |
| ------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Rust          | 1.96.1 or newer  | Every Carbon crate (`rust-version = "1.96.1"`).                                                                                                |
| Node.js       | 20.18.0 or newer | Only the [Carbon CLI](#install-and-run-the-cli), which generates decoders and scaffolds projects. Your indexer doesn't need Node.js.           |
| Solana crates | Agave 4.2 cohort | Carbon 2.0 is built on `solana-client 4.2.2`, `solana-transaction-status 4.2.2`, `solana-pubkey 4.2`, `solana-account 4.3` and related crates. |

Check your toolchain:

```sh
rustc --version
```

Output:

```text
rustc 1.99.0 (b940084d7 2026-09-28)
```

If it's older than 1.96.1, run `rustup update stable`.

### Keep one Carbon version and one Solana cohort

All `carbon-*` crates in a project must be version 2.0.0. Don't mix Carbon 2 crates with Carbon 1 decoders or datasources: Cargo builds both versions of `carbon-core`, and their types don't match. The same goes for Solana crates you depend on directly: pick versions from the Agave 4.2 cohort so that one version of `solana-pubkey`, `solana-account` and the other type crates ends up in your build.

When you add a Solana crate yourself, use the version Carbon uses:

| Crate                                                                  | Version  |
| ---------------------------------------------------------------------- | -------- |
| `solana-pubkey`                                                        | `4.2.1`  |
| `solana-account`                                                       | `4.3.2`  |
| `solana-signature`                                                     | `3.4.1`  |
| `solana-instruction`                                                   | `3.4.1`  |
| `solana-commitment-config`                                             | `3.1.1`  |
| `solana-client`, `solana-account-decoder`, `solana-transaction-status` | `=4.2.2` |

## Add Carbon to a project

Every indexer needs `carbon-core`, at least one datasource, the decoders for the programs you read, and an async runtime. A metrics exporter is optional but recommended.

```sh
cargo add carbon-core@2.0.0
cargo add carbon-rpc-transaction-crawler-datasource@2.0.0
cargo add carbon-meteora-dlmm-decoder@2.0.0
cargo add carbon-log-metrics@2.0.0
cargo add tokio --features full
cargo add env_logger log
```

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
tokio = { version = "1.54.1", features = ["full"] }
```

Replace the datasource and decoder with the ones you need. `env_logger` and `log` aren't required, but Carbon reports datasource and processor errors through the `log` crate, so install a logger to see them.

## Which crates you need

| You want to                                                       | Add                                                                                                                                                           |
| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Build any pipeline                                                | `carbon-core` and `tokio`                                                                                                                                     |
| Stream from Yellowstone gRPC                                      | `carbon-yellowstone-grpc-datasource`                                                                                                                          |
| Stream from Helius LaserStream                                    | `carbon-helius-laserstream-datasource`                                                                                                                        |
| Subscribe over Solana RPC websockets                              | `carbon-rpc-block-subscribe-datasource` or `carbon-rpc-program-subscribe-datasource`                                                                          |
| Subscribe over Helius enhanced websockets                         | `carbon-helius-atlas-ws-datasource`                                                                                                                           |
| Backfill transaction history                                      | `carbon-rpc-transaction-crawler-datasource`, `carbon-rpc-block-crawler-datasource` or `carbon-helius-gtfa-datasource`                                         |
| Load a program's current accounts                                 | `carbon-rpc-gpa-datasource` or `carbon-helius-gpa-v2-datasource`                                                                                              |
| Load accounts from a validator snapshot                           | `carbon-validator-snapshot-datasource`                                                                                                                        |
| Feed updates from your own source (Kafka, files, a Geyser plugin) | `carbon-stream-message-datasource`, or implement [`Datasource`](https://carbon.sevenlabs.org/concepts/datasources.md) yourself with `async-trait`             |
| Decode a program                                                  | Its decoder crate, such as `carbon-token-program-decoder` or `carbon-pump-swap-decoder`. See the [decoder catalog](https://carbon.sevenlabs.org/decoders.md). |
| Decode a program without a published decoder                      | Generate one with the [CLI](#install-and-run-the-cli). See [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md).                   |
| Log metrics                                                       | `carbon-log-metrics`                                                                                                                                          |
| Expose metrics to Prometheus                                      | `carbon-prometheus-metrics`                                                                                                                                   |
| Test decoders against fixtures                                    | `carbon-test-utils` (as a dev-dependency)                                                                                                                     |

The [datasource index](https://carbon.sevenlabs.org/datasources.md) compares datasources in detail.

> **Note.** `carbon-jito-shredstream-grpc-datasource` and `carbon-jetstreamer-datasource` are not published for Carbon 2.0. Their 1.x versions depend on Carbon 1 and the Solana 3 crates, so they can't be used in a Carbon 2 project.

## `carbon-core` feature flags

| Feature       | Default | What it enables                                                                                                                                                                                                                                       |
| ------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `macros`      | Yes     | Re-exports the `try_decode_instructions!` and `instruction_decoder_collection!` macros and the `InstructionType` derive from `carbon_core`. Decoder crates turn it on themselves.                                                                     |
| `postgres`    | No      | The `carbon_core::postgres` module: `sqlx`-backed processors, row types and `Pubkey`/integer column wrappers. See [Postgres](https://carbon.sevenlabs.org/guides/postgres.md).                                                                        |
| `graphql`     | No      | The `carbon_core::graphql` module: Juniper scalars for Solana types and an Axum router helper. See [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md).                                                                                         |
| `yellowstone` | No      | `carbon_core::transformers::yellowstone`, which converts Yellowstone protobuf transactions to Solana types. The Yellowstone and LaserStream datasources turn it on for you; enable it yourself only to convert Yellowstone messages in your own code. |

```sh
cargo add carbon-core@2.0.0 --features postgres,graphql
```

The [feature flag reference](https://carbon.sevenlabs.org/reference/feature-flags.md) covers every crate.

## Decoder feature flags

Every published decoder crate has the same features, all off by default:

| Feature    | What it enables                                                                                                             |
| ---------- | --------------------------------------------------------------------------------------------------------------------------- |
| `serde`    | `Serialize` and `Deserialize` on the decoder's accounts, instructions, events and types.                                    |
| `postgres` | Postgres row types and migrations for the decoder's accounts and instructions. Turns on `serde` and `carbon-core/postgres`. |
| `graphql`  | GraphQL schema types for the decoder. Turns on `serde` and `carbon-core/graphql`.                                           |

```sh
cargo add carbon-meteora-dlmm-decoder@2.0.0 --features serde
```

Decoders you generate with the CLI get the same features, controlled by `--with-postgres` and `--with-graphql` (both on by default) and `--with-serde`. They also get a `base58` feature when you pass `--with-base58 true`; it turns on `serde` and serializes `Pubkey` fields as base58 strings.

## Install and run the CLI

The Carbon CLI, `@sevenlabs-hq/carbon-cli`, generates decoder crates from Anchor or Codama IDLs (`parse`) and scaffolds complete indexer projects (`scaffold`). It needs Node.js 20.18.0 or newer. Run it without installing:

**npx**

```sh
npx @sevenlabs-hq/carbon-cli@2.0.0 --version
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 --version
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 --version
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 --version
```

Output:

```text
2.0.0
```

To install it globally instead:

```sh
npm install -g @sevenlabs-hq/carbon-cli@2.0.0
carbon-cli --version
```

The package installs two equivalent commands, `carbon-cli` and `carbon`. Run `carbon-cli parse` or `carbon-cli scaffold` without options to be prompted for each one. CLI 2.0.0 generates code that depends on Carbon 2.0.0 crates. See the [CLI reference](https://carbon.sevenlabs.org/reference/cli.md) for every command and flag.

## Verify the install

Put this program in `src/main.rs`. A pipeline with no datasources starts and returns straight away, so it checks that Carbon compiles and runs without touching the network.

`src/main.rs`:

```rust
use carbon_core::{error::CarbonResult, pipeline::Pipeline};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    // A pipeline with no datasources returns as soon as it starts.
    Pipeline::builder().build()?.run().await?;
    println!("carbon-core is installed and the pipeline runs");
    Ok(())
}
```

```sh
cargo run
```

Output:

```text
carbon-core is installed and the pipeline runs
```

Then check that only one version of `carbon-core` is in your build:

```sh
cargo tree -i carbon-core --depth 1
```

Output:

```text
carbon-core v2.0.0
├── carbon-log-metrics v2.0.0
├── carbon-meteora-dlmm-decoder v2.0.0
├── carbon-rpc-transaction-crawler-datasource v2.0.0
└── carbon-quickstart v0.1.0 (/home/you/carbon-quickstart)
```

If Cargo reports that there are multiple `carbon-core` packages, one of your `carbon-*` dependencies isn't 2.0.0. Set every one to `2.0.0` and run `cargo update`.

## Next steps

- [Quickstart](https://carbon.sevenlabs.org/quickstart.md): Build and run your first indexer against mainnet.
- [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md): How a pipeline moves updates from datasources to processors.
- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): Use the CLI to decode your own program.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/installation · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# For AI agents (Carbon 2.0)

> Connect Claude Code, Cursor or any coding agent to these docs through MCP, Markdown pages and llms.txt, and give it the rules it needs to write correct Carbon 2.0 code.

Coding agents write a lot of Carbon code, and most of them learned Carbon from versions before 2.0. This page shows how to give your agent the current docs and the rules that keep it from writing outdated code.

## Connect the docs MCP server

The docs run an MCP server at `https://carbon.sevenlabs.org/mcp`. It uses Streamable HTTP and needs no login. It has four tools:

| Tool           | What it does                                                                                                                                                                                    |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `search_docs`  | Searches every page and returns the best sections with their URLs.                                                                                                                              |
| `get_page`     | Returns one page as Markdown, from a path such as `/concepts/filters`.                                                                                                                          |
| `list_pages`   | Lists every page by section with a one-line description.                                                                                                                                        |
| `find_decoder` | Finds the decoder crate for a Solana program from its program ID, crate or name, and returns the install command, decoder struct, instruction, account and event names, and the reference page. |

**Claude Code**

```sh
claude mcp add --transport http carbon-docs https://carbon.sevenlabs.org/mcp
```

**Cursor**

`.cursor/mcp.json`:

```json
{
  "mcpServers": {
    "carbon-docs": { "url": "https://carbon.sevenlabs.org/mcp" }
  }
}
```

**VS Code**

`.vscode/mcp.json`:

```json
{
  "servers": {
    "carbon-docs": { "type": "http", "url": "https://carbon.sevenlabs.org/mcp" }
  }
}
```

Any other client that supports Streamable HTTP works with the same URL.

## Read pages as Markdown

Every page has a Markdown version with no navigation or scripts:

- Add `.md` to any page URL: `https://carbon.sevenlabs.org/concepts/filters.md`. The home page is `https://carbon.sevenlabs.org/index.md`.
- Or request the normal URL with `Accept: text/markdown`.
- [`/llms.txt`](https://carbon.sevenlabs.org/llms.txt) lists every page with a one-line description, plus the mistakes agents make most often.
- [`/llms-full.txt`](https://carbon.sevenlabs.org/llms-full.txt) has every guide, concept and reference page in one file. The per-program decoder references are left out because they're large; `llms.txt` links each one.

Each Markdown page starts with its title and the Carbon version, so an agent can tell which release it's reading. A path that doesn't exist returns HTTP 404 with a list of the closest pages instead of an empty error.

On the web, the **Copy page** button at the top of every page copies its Markdown, or opens the page in Claude or ChatGPT.

## Add an AGENTS.md to your project

Agents follow the instruction file in your repository more reliably than anything they fetch. Paste this into `AGENTS.md` (or `CLAUDE.md`) at the root of a project that uses Carbon:

`AGENTS.md`:

```md
## Carbon

This project indexes Solana data with Carbon 2.0 (Rust). Docs: https://carbon.sevenlabs.org (append `.md` to any page URL for Markdown; index at https://carbon.sevenlabs.org/llms.txt).

- Do not rely on training data for Carbon APIs: they changed in 1.0 and 2.0. Check the docs before writing Carbon code.
- All `carbon-*` crates must be version 2.0.0. Never mix Carbon 1.x and 2.x crates. Rust 1.96.1+.
- Processors implement `Processor<T>` with a native `async fn process(&mut self, input: &T) -> CarbonResult<()>`. No `#[async_trait]` on processors; `Datasource` impls do use `#[async_trait]`.
- Instruction enums use struct variants: match `MyInstruction::Swap { data, accounts, .. }`.
- RPC transaction configs need `UiTransactionEncoding::Base64` and `max_supported_transaction_version: Some(1)`.
- The public endpoint api.mainnet-beta.solana.com does not support `blockSubscribe`.
- Generate decoders with `npx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl <file-or-program-id> --out-dir <dir>` (Codama IDLs: add `--standard codama --program-id <address>`). `parse` deletes `--out-dir` first, so use a dedicated directory.
- Check work with `cargo check` and `cargo clippy`.
```

## Rules for agents writing Carbon code

These are the facts agents most often get wrong. The same list is at the top of `llms.txt`.

1. These docs are for Carbon 2.0.0. Use `carbon-core = "2.0.0"` and matching `carbon-*` crates at 2.0.0. Never mix Carbon 1.x and 2.x crates in one dependency graph.
2. Carbon 2.0 needs Rust 1.96.1 or newer and the Agave 4.2-compatible Solana crates. If you add Solana crates directly, pick versions that resolve to a single copy of each. Mixing Carbon 1 and 2 crates fails with ``the trait `InstructionDecoder` is not implemented`` and a note about multiple versions of `carbon_core`.
3. `Processor::process` is a native `async fn`. Don't add `#[async_trait]` to processors. The `Datasource` trait does use `#[async_trait]`.
4. Instruction enums use struct variants: `MyProgramInstruction::Swap { program_id, data, accounts }`.
5. `https://api.mainnet-beta.solana.com` rejects `blockSubscribe`. On the public endpoint, use the [transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), [`getProgramAccounts`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md) or [`programSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md) datasources.
6. RPC block and transaction configs must request `UiTransactionEncoding::Base64` and `max_supported_transaction_version: Some(1)`. Carbon can't rebuild transactions from `Json` or `JsonParsed`.
7. Zero-lamport account updates arrive as `Update::AccountDeletion` and go to `.account_deletions(...)` processors, not account processors.
8. Decoder crates are named `carbon-<program>-decoder` and export `PROGRAM_ID` and a `<Program>Decoder` struct. Check the [decoder catalog](https://carbon.sevenlabs.org/decoders.md) before generating one.

## Instructions for agents: create a Carbon indexer

When a user asks for a new indexer, follow these steps in order.

1. Run `rustc --version`. If it's older than 1.96.1, tell the user to run `rustup update stable`.
2. Find the program's decoder in the [catalog](https://carbon.sevenlabs.org/decoders.md). If there isn't one, generate it from the IDL with the [CLI](https://carbon.sevenlabs.org/guides/generate-a-decoder.md) and add it as a path dependency.
3. Choose the datasource from the [chooser](https://carbon.sevenlabs.org/datasources.md): Yellowstone gRPC or Helius LaserStream for real-time production; the RPC transaction crawler for history or for a no-account start; `getProgramAccounts` for current account state.
4. Add every crate at version `2.0.0` with `cargo add`, plus `tokio` with the `full` feature.
5. Write `main.rs` following the [quickstart](https://carbon.sevenlabs.org/quickstart.md): build the datasource, then `Pipeline::builder().datasource(…).metrics(…).instruction(Decoder, Processor).build()?.run().await`.
6. Implement one `Processor` per decoder output type you need. Match only the variants you handle and ignore the rest with `_ => {}`.
7. Run `cargo check`, fix errors using the [errors reference](https://carbon.sevenlabs.org/reference/errors.md), then `cargo run` and show the user the first output.
8. Before calling it done, read [Run in production](https://carbon.sevenlabs.org/guides/production.md) and tell the user which items apply.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/agents · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Architecture (Carbon 2.0)

> How Pipeline::run moves an update from datasource to processor, with the exact ordering, concurrency, error, metrics and shutdown rules.

This page describes exactly what `Pipeline::run` does in Carbon 2.0.0, from the moment you call it until it returns. Read it when you need to know in which order processors run, what happens when one is slow or fails, and when the pipeline stops. The behavior described here comes from [`crates/core/src/pipeline.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/pipeline.rs).

## In short

Each datasource runs in its own task and sends updates into one shared channel. The pipeline takes updates off that channel one at a time and hands each one to every pipe registered for its type; a pipe filters it, decodes it and awaits your processor before the next update starts. A processor error is logged and counted, and processing carries on. The pipeline stops when you cancel its token, when you press Ctrl-C, or when every datasource has finished.

The rest of this page is the exact behaviour behind each of those sentences.

## The parts of a pipeline

You build a pipeline with [`Pipeline::builder()`](https://carbon.sevenlabs.org/reference/pipeline-builder.md). The builder collects:

- **Datasources**, each paired with a [`DatasourceId`](https://carbon.sevenlabs.org/concepts/datasources.md#datasource-ids). `datasource()` assigns a random UUID; `datasource_with_id()` uses the ID you give it.
- **Pipes**, one per `account`, `instruction`, `transaction`, `account_deletions` or `block_details` call. A pipe bundles a decoder (where the update type needs one), your [processor](https://carbon.sevenlabs.org/concepts/processors.md) and its [filters](https://carbon.sevenlabs.org/concepts/filters.md).
- **Metrics exporters** added with `metrics()`.
- **Settings**: the [shutdown strategy](https://carbon.sevenlabs.org/concepts/shutdown.md) (default `ProcessPending`), an optional cancellation token, and `channel_buffer_size` (default `DEFAULT_CHANNEL_BUFFER_SIZE`, 1,000).

`build()` registers the pipeline's built-in metrics in the global [metrics registry](https://carbon.sevenlabs.org/concepts/metrics.md) and returns the `Pipeline`. In 2.0.0 it doesn't validate anything and always returns `Ok`.

![What the pipeline does with each update](https://carbon.sevenlabs.org/diagrams/pipeline-loop.svg)

**Diagram: What the pipeline does with each update.** Each datasource runs in its own Tokio task and sends (Update, DatasourceId) pairs into one bounded channel. The pipeline loop takes one update at a time and routes it by type. Update::Account goes to the account pipes, which filter, decode and call the processor. Update::Transaction goes first to the instruction pipes, which filter, decode and process each instruction, then to the transaction pipes, which filter, decode every instruction with a decoder collection and call the processor once. Update::AccountDeletion and Update::BlockDetails go to the deletion and block-details pipes, which filter and call the processor without a decoder.

## What `run` does

1. **Logs a start line** with the number of datasources, exporters and pipes, at `info` level.
2. **Initializes exporters.** It calls `MetricsExporter::initialize` on each exporter in order. [`LogMetrics`](https://carbon.sevenlabs.org/concepts/metrics.md#log-metrics) starts its five-second flush task here, and `PrometheusMetrics::with_server` starts its HTTP server. If an exporter returns an error, `run` returns that error before any datasource starts.
3. **Creates the channel**, a bounded Tokio MPSC channel of `(Update, DatasourceId)` with room for `channel_buffer_size` updates.
4. **Picks the cancellation token**: the one you passed to `datasource_cancellation_token()`, or a new one.
5. **Starts every datasource** in its own Tokio task with `tokio::spawn`, calling `consume(id, sender, token)` with a clone of the channel's sender and of the token. If `consume` returns an error, the task logs `error consuming datasource: …` and ends. The pipeline isn't told and doesn't retry; the other datasources keep running.
6. **Drops its own sender**, so the channel closes once every datasource has dropped its copy.
7. **Runs the loop.** Each iteration waits for whichever comes first:
   - **The token is cancelled**: exports metrics once, shuts the exporters down and returns `Ok(())`.
   - **Ctrl-C**: cancels the token. With `ShutdownStrategy::Immediate` it exports metrics, shuts the exporters down and returns. With `ProcessPending` it logs a message and continues the loop. The loop notices Ctrl-C only while it's waiting for an update; see [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md#ctrl-c) for what that means in practice.
   - **An update arrives**: processes it as described below, then starts the next iteration.
   - **The channel is closed and empty** (every datasource finished): logs `update_receiver closed, shutting down.`, exports metrics, shuts the exporters down and returns `Ok(())`.

## How each update is processed

For every update it receives, the loop increments `carbon_updates_received_total`, processes the update through every matching pipe, records the time it took in two histograms, and increments either `carbon_updates_successful_total` or `carbon_updates_failed_total`, then `carbon_updates_processed_total`. It then sets `carbon_updates_queued` to the number of updates still waiting in the channel.

The update's variant decides which pipes see it. For each pipe, the pipeline runs the pipe's filters in order and stops at the first `Reject`. A pipe runs only if every filter returns `Accept`.

### Account updates

The pipeline builds an `AccountMetadata` (slot, pubkey, transaction signature) and visits each account pipe in the order you registered them. A pipe whose filters accept calls its decoder's `decode_account`. If the decoder returns `None`, the account isn't for this pipe and the processor isn't called. Otherwise the processor receives an [`AccountProcessorInputType`](https://carbon.sevenlabs.org/concepts/processors.md#account-pipes) with the metadata, the decoded account and the raw account.

### Transactions

1. The pipeline converts the `TransactionUpdate` into a `TransactionMetadata`, shared through an `Arc`. The fee payer is the first static account key; a transaction without one fails with `MissingFeePayer` before any pipe runs.
2. It builds the instruction list: each top-level instruction, followed by its inner instructions in execution order, with account keys resolved from the message and, for v0 transactions, the loaded address lookup tables. Legacy, v0 and V1 messages are supported. An inner instruction without a valid stack height (2 to 5) is dropped, together with the rest of its group, and a warning is logged.
3. It rebuilds that list into a tree of [`NestedInstructions`](https://carbon.sevenlabs.org/concepts/instructions.md), and flattens the tree depth-first.
4. **Instruction pipes run first.** For each instruction pipe, in registration order, the pipeline walks every instruction of the transaction in depth-first order: top-level instruction 0, its inner instructions, top-level instruction 1, and so on. Each instruction that passes the pipe's filters goes to the pipe's decoder; when the decoder returns `Some`, the processor is called with an [`InstructionProcessorInputType`](https://carbon.sevenlabs.org/concepts/processors.md#instruction-pipes). The first pipe sees all instructions before the second pipe sees any.
5. **Transaction pipes run next**, in registration order. Their filters receive the transaction metadata and the instruction tree. A pipe that accepts decodes every instruction with its [`InstructionDecoderCollection`](https://carbon.sevenlabs.org/concepts/transactions.md) and calls the processor once with the decoded list, even if the list is empty.

### Account deletions and block details

`AccountDeletion` and `BlockDetails` updates go to the `account_deletions` and `block_details` pipes, in registration order, after filtering. There is no decoder: the processor receives the update itself.

## Ordering and concurrency guarantees

- **One update at a time.** The loop processes an update completely, through every matching pipe, before it takes the next one off the channel. Processors never run concurrently with each other, and a processor's `process` calls never overlap, which is why `process` takes `&mut self`.
- **Registration order within an update.** Pipes of the same kind run in the order you added them. For a transaction, every instruction pipe runs before any transaction pipe.
- **Execution order within a transaction.** Instruction pipes see instructions in depth-first order, which is the order in which they executed.
- **Channel order across updates.** Updates are processed in the order they entered the channel. With one datasource, that's the order it sends them in. With several, their updates interleave in whatever order they arrive. Carbon doesn't sort by slot or remove duplicates unless you add a [filter](https://carbon.sevenlabs.org/concepts/filters.md) for it.
- **No parallelism inside the pipeline.** To use more cores, make `process` cheap and hand heavy work to tasks or channels you own. See [Sharing state](https://carbon.sevenlabs.org/concepts/processors.md#share-state-with-the-rest-of-your-program). Once you do, ordering across that work is up to you.

### When processing falls behind

A slow processor slows the whole loop, and updates accumulate in the channel up to `channel_buffer_size`. What happens when the channel is full depends on the datasource:

- Datasources that send with `send().await` wait until there's room. Back-pressure reaches the source and nothing is lost. The RPC and Helius `getProgramAccounts` datasources, the Helius GTFA datasource, the validator snapshot datasource and the stream message datasource behave this way, as does the RPC transaction crawler with `blocking_send: true`.
- Datasources that send with `try_send` drop the update and log an error. Yellowstone gRPC, Helius LaserStream, Helius Atlas websockets, RPC block subscribe, RPC program subscribe, the RPC block crawler, and the transaction crawler with its default `blocking_send: false` behave this way.

Watch `carbon_updates_queued`. If it stays near `channel_buffer_size`, raise the buffer with `.channel_buffer_size(…)` to absorb bursts, and make the processors faster for sustained load.

## Errors

- **A processor returns `Err`.** The pipe's `?` stops processing of that update: pipes after it don't run for this update, and for a transaction the remaining instructions and the transaction pipes are skipped too. The loop logs `error processing update ({update:?}): {error:?}` at `error` level, counts the update in `carbon_updates_failed_total` and moves on to the next update. Nothing is retried.
- **A datasource's `consume` returns `Err`.** The error is logged and that datasource stops. The pipeline continues with the others; if none are left, the channel closes and `run` returns `Ok(())`.
- **A processor panics.** The pipeline doesn't catch panics. The panic propagates out of `run`, so with `#[tokio::main]` the program exits.

[Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md) covers patterns for these cases.

## Metrics flushing

The pipeline updates its counters, gauge and histograms on every update, but it calls `MetricsExporter::export` only once, when it shuts down, followed by `shutdown` on each exporter. Periodic export is the exporter's job: `LogMetrics` logs a snapshot every five seconds from its own task, and Prometheus reads the registry each time it scrapes `/metrics`. See [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md).

## Example

This program feeds three hand-built account updates through the [stream message datasource](https://carbon.sevenlabs.org/datasources/stream-message.md), registers two account pipes and a deletion pipe, and prints what each processor sees. The source is finite, so the pipeline returns once it has processed all three.

```sh
cargo add carbon-core@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::{AccountDeletion, AccountUpdate},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
};

/// Accepts every account and exposes the length of its data.
struct AnyAccount;

impl<'a> AccountDecoder<'a> for AnyAccount {
    type AccountType = usize;

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<usize>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: account.data.len(),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

struct Print(&'static str);

impl Processor<AccountProcessorInputType<'_, usize>> for Print {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, usize>) -> CarbonResult<()> {
        println!(
            "{}: slot {} lamports {}",
            self.0, input.metadata.slot, input.decoded_account.lamports
        );
        Ok(())
    }
}

impl Processor<AccountDeletion> for Print {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        println!("{}: slot {} account closed", self.0, deletion.slot);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    // Hand-built updates stand in for a real datasource.
    let (tx, rx) = tokio::sync::mpsc::channel(16);
    for (slot, lamports) in [(100, 2_039_280), (101, 1_461_600), (102, 0)] {
        let update = AccountUpdate {
            pubkey: Pubkey::new_unique(),
            account: Account { lamports, ..Account::default() },
            slot,
            transaction_signature: None,
        };
        tx.send(UnifiedMessage::Account(update)).await.unwrap();
    }
    drop(tx); // The source is finite: the pipeline returns once it drains.

    Pipeline::builder()
        .datasource(StreamMessageClient::new(rx))
        .account(AnyAccount, Print("pipe A"))
        .account(AnyAccount, Print("pipe B"))
        .account_deletions(Print("deletions"))
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
pipe A: slot 100 lamports 2039280
pipe B: slot 100 lamports 2039280
pipe A: slot 101 lamports 1461600
pipe B: slot 101 lamports 1461600
deletions: slot 102 account closed
```

Both account pipes see each update, in registration order, before the next update starts. The zero-lamport account arrives as an `AccountDeletion` (see [Updates](https://carbon.sevenlabs.org/concepts/updates.md#account-deletions)), so only the deletion pipe sees it.

## Next steps

- [Updates](https://carbon.sevenlabs.org/concepts/updates.md): The four update types a datasource can send.
- [Processors](https://carbon.sevenlabs.org/concepts/processors.md): What each pipe passes to your code, and how to keep state.
- [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md): How the pipeline stops, and what happens to queued updates.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/architecture · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Updates (Carbon 2.0)

> The Update enum that datasources send into a pipeline, its four variants, UpdateType, zero-lamport account deletions, Transaction V1 and block details.

Every piece of data a datasource produces enters the pipeline as a `carbon_core::datasource::Update`. You meet this type when you write a [custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md), and its fields show up again in the metadata your processors receive. Full field-by-field details are in the [updates reference](https://carbon.sevenlabs.org/reference/updates.md).

## The `Update` enum

```rust
pub enum Update {
    Account(AccountUpdate),
    Transaction(Box<TransactionUpdate>),
    AccountDeletion(AccountDeletion),
    BlockDetails(BlockDetails),
}
```

| Variant           | Means                                                                                     | Delivered to                                                                   |
| ----------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `Account`         | An account's state at a slot: a write, or a read from a snapshot or `getProgramAccounts`. | `account` pipes, through their decoder                                         |
| `Transaction`     | A transaction with its execution metadata.                                                | `instruction` pipes (one call per decoded instruction) and `transaction` pipes |
| `AccountDeletion` | An account that was closed: its lamports reached zero.                                    | `account_deletions` pipes                                                      |
| `BlockDetails`    | Metadata about a block.                                                                   | `block_details` pipes                                                          |

The pipeline routes each variant only to pipes of the matching kind; [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md#how-each-update-is-processed) gives the exact order.

## Account updates

```rust
pub struct AccountUpdate {
    pub pubkey: Pubkey,
    pub account: Account,
    pub slot: u64,
    pub transaction_signature: Option<Signature>,
}
```

`account` is a `solana_account::Account` with `lamports`, `data`, `owner`, `executable` and `rent_epoch`. `transaction_signature` is the transaction that wrote the account, when the source knows it: Geyser streams usually set it; `getProgramAccounts` and snapshots don't.

## Account deletions

Solana has no explicit "account closed" event. A program closes an account by moving all its lamports out, and the runtime then removes it. Carbon 2.0 reports every account write with zero lamports as an `AccountDeletion`:

```rust
pub struct AccountDeletion {
    pub pubkey: Pubkey,
    pub account: Account, // the final state the source reported
    pub slot: u64,
    pub transaction_signature: Option<Signature>,
}
```

Datasources do this conversion with `AccountUpdate::into_update`, which returns `Update::AccountDeletion` when `account.lamports == 0` and `Update::Account` otherwise. The Yellowstone gRPC, Helius LaserStream, Helius Atlas websocket, RPC program subscribe and stream message datasources all use it. Call it in your own datasources too, so that deletions are classified in one place:

```rust
let update = AccountUpdate { pubkey, account, slot, transaction_signature }.into_update();
```

Because deletions are a separate variant, an `account` pipe never sees a zero-lamport account from these datasources. Register an `account_deletions` processor to handle closures. The final `account` keeps the owner and data the source reported, so you can tell which program the account belonged to.

## Transactions

```rust
pub struct TransactionUpdate {
    pub signature: Signature,
    pub transaction: VersionedTransaction,
    pub meta: TransactionStatusMeta,
    pub is_vote: bool,
    pub slot: u64,
    pub index: Option<u64>,
    pub block_time: Option<i64>,
    pub block_hash: Option<Hash>,
}
```

- `transaction` is the signed transaction; `meta` is its execution result: status, fee, balances, log messages, inner instructions, loaded addresses and so on.
- `index` is the transaction's position in its block. Yellowstone, LaserStream, RPC block subscribe, the block crawler and Helius GTFA set it; the transaction crawler and Helius Atlas websockets leave it `None`.
- `block_time` is a Unix timestamp in seconds, when the source provides it.

The pipeline turns each `TransactionUpdate` into a shared [`TransactionMetadata`](https://carbon.sevenlabs.org/concepts/transactions.md#transaction-metadata) and the list of its instructions before any processor runs.

### Transaction V1

The Agave 4.2 Solana crates add the V1 transaction format alongside legacy and v0 transactions. `transaction.message` is a `VersionedMessage`, and Carbon 2.0 handles all three variants (`Legacy`, `V0` and `V1`) when it extracts instructions and their accounts.

Datasources must request V1 transactions and a binary encoding, because Carbon can't rebuild a transaction from a `Json` or `JsonParsed` payload. The RPC transaction crawler and the Helius GTFA datasource set this internally. When you pass your own `RpcBlockSubscribeConfig` or `RpcBlockConfig`, set it yourself:

```rust
RpcBlockSubscribeConfig {
    encoding: Some(UiTransactionEncoding::Base64),
    transaction_details: Some(TransactionDetails::Full),
    max_supported_transaction_version: Some(1),
    ..RpcBlockSubscribeConfig::default()
}
```

## Block details

```rust
pub struct BlockDetails {
    pub slot: u64,
    pub block_hash: Option<Hash>,
    pub previous_block_hash: Option<Hash>,
    pub rewards: Option<Rewards>,
    pub num_reward_partitions: Option<u64>,
    pub block_time: Option<i64>,
    pub block_height: Option<u64>,
}
```

Among the Carbon 2.0 datasources, only [RPC block subscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md) sends `BlockDetails`: one per block, before that block's transactions. A [custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md) can send them too.

## `UpdateType`

`UpdateType` names the variants without their data:

```rust
pub enum UpdateType {
    AccountUpdate,
    Transaction,
    AccountDeletion,
    BlockDetails,
}
```

`Update::update_type()` returns the `UpdateType` of an update, and `Datasource::update_types()` returns the types a datasource declares it sends. In 2.0.0 the pipeline doesn't read `update_types()` or check updates against it, so treat it as documentation. It isn't always complete: RPC block subscribe declares only `Transaction` but also sends `BlockDetails`.

## Example

This program builds the same token account twice, once funded and once with zero lamports, and shows how `into_update` classifies each.

```sh
cargo add carbon-core@2.0.0 solana-account@4.3.2 solana-pubkey@4.2.1
```

`src/main.rs`:

```rust
use {
    carbon_core::datasource::{AccountUpdate, Update},
    solana_account::Account,
    solana_pubkey::Pubkey,
};

fn main() {
    let token_program = Pubkey::from_str_const("TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA");

    for lamports in [2_039_280, 0] {
        let update = AccountUpdate {
            pubkey: Pubkey::new_unique(),
            account: Account {
                lamports,
                data: vec![0; 165],
                owner: token_program,
                executable: false,
                rent_epoch: u64::MAX,
            },
            slot: 371_000_000,
            transaction_signature: None,
        };

        match update.into_update() {
            Update::Account(update) => {
                println!("account update: {} lamports", update.account.lamports)
            }
            Update::AccountDeletion(deletion) => println!(
                "account deletion: owner {}, {} bytes of final data",
                deletion.account.owner,
                deletion.account.data.len()
            ),
            other => println!("{:?}", other.update_type()),
        }
    }
}
```

```sh
cargo run
```

Output:

```text
account update: 2039280 lamports
account deletion: owner TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA, 165 bytes of final data
```

## Next steps

- [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md): Where updates come from, and how to choose a source.
- [Processors](https://carbon.sevenlabs.org/concepts/processors.md): What each pipe passes to your code.
- [Updates reference](https://carbon.sevenlabs.org/reference/updates.md): Every field of every update type.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/updates · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Datasources (Carbon 2.0)

> The Datasource trait, datasource IDs and declared update types, and how the Geyser, RPC, crawler, snapshot, provider and adapter datasources compare.

A datasource reads updates from somewhere (a Geyser stream, an RPC node, a snapshot file, your own queue) and sends them into the pipeline. This page explains the trait every datasource implements, how datasources are identified, and how the published datasources compare. Each datasource's configuration is on its own page under [Datasources](https://carbon.sevenlabs.org/datasources.md).

## The `Datasource` trait

```rust
#[async_trait]
pub trait Datasource: Send + Sync {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: tokio::sync::mpsc::Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()>;

    fn update_types(&self) -> Vec<UpdateType>;
}
```

`Datasource` uses the `async_trait` macro, so implementations need the `async-trait` crate. The pipeline calls `consume` once, in a Tokio task of its own, when [`run`](https://carbon.sevenlabs.org/concepts/architecture.md#what-run-does) starts. The contract:

- **Send each update with its ID.** Wrap your data in an [`Update`](https://carbon.sevenlabs.org/concepts/updates.md) and send `(update, id.clone())` on `sender`. Build account updates with `AccountUpdate { … }.into_update()` so zero-lamport accounts become deletions.
- **Choose how to handle a full channel.** `sender.send(…).await` waits for room, which slows your source down to the pipeline's pace. `sender.try_send(…)` fails immediately, so you drop the update or buffer it yourself. Most streaming datasources use `try_send`; see [When processing falls behind](https://carbon.sevenlabs.org/concepts/architecture.md#when-processing-falls-behind).
- **Stop when cancelled.** Watch `cancellation_token` and return when it's cancelled. See [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).
- **Drop the sender when you're done.** The pipeline stops once every datasource has dropped its sender and the channel is empty. A finite source, such as a crawler with an end or a snapshot, returns from `consume` or ends its tasks; a stream keeps going until cancelled.
- **Return errors only for fatal problems.** If `consume` returns `Err`, the pipeline logs it and that datasource is gone; nothing retries it. Reconnect and retry inside `consume` if your source can recover.

`consume` may also spawn tasks that hold clones of `sender` and return `Ok(())` straight away. Many published datasources do this. The pipeline keeps running for as long as any clone of the sender is alive.

`update_types` lists the [`UpdateType`](https://carbon.sevenlabs.org/concepts/updates.md#updatetype)s the datasource can send. In 2.0.0 the pipeline doesn't check it, but keep it accurate for readers of your code.

## Datasource IDs

Every datasource in a pipeline has a `DatasourceId`, which the pipeline passes along with each of its updates. [Filters](https://carbon.sevenlabs.org/concepts/filters.md) read it from `FilterContext` to route updates by source.

| Constructor                          | Gives                                                           | Use it when                                                |
| ------------------------------------ | --------------------------------------------------------------- | ---------------------------------------------------------- |
| `DatasourceId::new_unique()`         | A random UUID v4. `.datasource(ds)` calls this for you.         | You never need to refer to the source.                     |
| `DatasourceId::new_named("primary")` | The ID `"primary"`. Pass it with `.datasource_with_id(ds, id)`. | You filter by source, for example with `DatasourceFilter`. |

IDs compare by their string value, and the builder doesn't check that they're unique. Two datasources registered with the same name look like one source to every filter.

```rust
let primary = DatasourceId::new_named("primary");

Pipeline::builder()
    .datasource_with_id(yellowstone, primary.clone())
    .datasource_with_id(laserstream, DatasourceId::new_named("backup"))
    .instruction_with_filters(decoder, processor, vec![Box::new(DatasourceFilter::new(primary))])
```

[Use multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md) shows a full redundant setup.

## Kinds of datasources

The published datasources fall into six families:

- **Geyser gRPC streams** push account writes and transactions as a validator processes them. Lowest latency; you need a Yellowstone-compatible endpoint from a provider, or your own node with the plugin.
- **RPC websocket subscriptions** use the standard Solana `blockSubscribe` and `programSubscribe` methods. They work with many RPC providers, but the public endpoint doesn't support `blockSubscribe`.
- **RPC crawlers** poll RPC methods to walk history: by address with `getSignaturesForAddress` and `getTransaction`, or by slot with `getBlock`. Use them for backfills; they work against the public endpoint, slowly.
- **Account state loaders** read every account a program owns at one point in time, with `getProgramAccounts` or from a validator snapshot.
- **Hosted provider APIs** use Helius-specific methods for websockets, history and paginated account reads.
- **Adapters** turn your own channel of updates into a datasource, for queues, files or a custom Geyser plugin.

### Comparison

| Datasource                                                                                     | Family        | Latency                              | History                                                        | You need                                           | Update types                          |
| ---------------------------------------------------------------------------------------------- | ------------- | ------------------------------------ | -------------------------------------------------------------- | -------------------------------------------------- | ------------------------------------- |
| [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md)               | Geyser gRPC   | Real time                            | From now on                                                    | A Yellowstone gRPC endpoint                        | Account, Transaction, AccountDeletion |
| [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)           | Geyser gRPC   | Real time                            | Optional replay from a recent `from_slot`                      | A Helius plan with LaserStream                     | Account, Transaction, AccountDeletion |
| [RPC block subscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)         | RPC websocket | One block at a time                  | From now on                                                    | An RPC websocket with `blockSubscribe` enabled     | Transaction, BlockDetails             |
| [RPC program subscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md)     | RPC websocket | Real time                            | From now on                                                    | Any RPC websocket, including the public one        | Account, AccountDeletion              |
| [Helius Atlas websockets](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md)         | Provider API  | Real time                            | From now on                                                    | A Helius API key                                   | Account, Transaction, AccountDeletion |
| [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) | RPC crawler   | Seconds (polls every 5 s by default) | An address's full history, newest first, then new transactions | Any RPC, including the public one                  | Transaction                           |
| [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)             | RPC crawler   | Seconds                              | A slot range, or from a slot onward                            | An RPC node that serves `getBlock` for those slots | Transaction                           |
| [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                         | Provider API  | One pass, then stops                 | An address's history, with filters and sorting                 | A Helius API key                                   | Transaction                           |
| [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md)                  | State loader  | One-shot                             | Current state only                                             | Any RPC; the public one with filters               | Account                               |
| [Helius GPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md)                     | Provider API  | One-shot                             | Current state, or changes since a slot                         | A Helius API key                                   | Account                               |
| [Validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md)           | State loader  | One-shot                             | State at the snapshot's slot                                   | A local snapshot or a download URL, and disk space | Account                               |
| [Stream message](https://carbon.sevenlabs.org/datasources/stream-message.md)                   | Adapter       | Whatever you feed it                 | Whatever you feed it                                           | Your own producer                                  | Account, Transaction, AccountDeletion |

Common combinations:

- **Real-time indexer**: a Geyser stream, plus a second one with a [`DatasourceFilter`](https://carbon.sevenlabs.org/concepts/filters.md#datasourcefilter) or [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) for redundancy.
- **Backfill then follow**: the transaction or block crawler for history, then a stream for new data. See [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md).
- **Current state then follow**: `getProgramAccounts` or a snapshot to load accounts, then program subscribe or a Geyser stream for changes. See [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md).

The [datasource index](https://carbon.sevenlabs.org/datasources.md) helps you choose. If none fits, [write your own](https://carbon.sevenlabs.org/guides/custom-datasource.md).

## Example

This custom datasource replays a fixed list of accounts and then finishes, which ends the pipeline. It sends with `send().await`, respects cancellation and uses `into_update`. The pipeline registers it under a named ID.

```sh
cargo add carbon-core@2.0.0 async-trait tokio-util
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::{AccountUpdate, Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    solana_account::Account,
    solana_pubkey::Pubkey,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Replays a fixed list of accounts, then finishes.
struct Fixtures {
    accounts: Vec<(Pubkey, Account)>,
    slot: u64,
}

#[async_trait]
impl Datasource for Fixtures {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        for (pubkey, account) in &self.accounts {
            if cancellation_token.is_cancelled() {
                break;
            }
            let update = AccountUpdate {
                pubkey: *pubkey,
                account: account.clone(),
                slot: self.slot,
                transaction_signature: None,
            }
            .into_update(); // zero-lamport accounts become AccountDeletion

            // `send` waits when the pipeline's channel is full (back-pressure).
            if sender.send((update, id.clone())).await.is_err() {
                break; // the pipeline has stopped
            }
        }
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::AccountUpdate, UpdateType::AccountDeletion]
    }
}

struct AnyAccount;

impl<'a> AccountDecoder<'a> for AnyAccount {
    type AccountType = ();

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<()>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: (),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

struct PrintAccount;

impl Processor<AccountProcessorInputType<'_, ()>> for PrintAccount {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, ()>) -> CarbonResult<()> {
        println!("{} has {} lamports", input.metadata.pubkey, input.raw_account.lamports);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let fixtures = Fixtures {
        accounts: vec![
            (Pubkey::new_from_array([1; 32]), Account { lamports: 1_000_000, ..Account::default() }),
            (Pubkey::new_from_array([2; 32]), Account { lamports: 2_000_000, ..Account::default() }),
        ],
        slot: 1,
    };

    Pipeline::builder()
        .datasource_with_id(fixtures, DatasourceId::new_named("fixtures"))
        .account(AnyAccount, PrintAccount)
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
4vJ9JU1bJJE96FWSJKvHsmmFADCg4gpZQff4P3bkLKi has 1000000 lamports
8qbHbw2BbbTHBW1sbeqakYXVKRQM8Ne7pLK7m6CVfeR has 2000000 lamports
```

[Write a custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md) builds a networked one with reconnects.

## Next steps

- [Choose a datasource](https://carbon.sevenlabs.org/datasources.md): Configuration for every published datasource.
- [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md): How raw accounts and instructions become Rust types.
- [Use multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Run redundant or complementary sources in one pipeline.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/datasources · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Decoders (Carbon 2.0)

> How AccountDecoder and InstructionDecoder turn raw bytes into Rust types, how discriminators and ArrangeAccounts work, and how decoder crates are laid out.

A decoder recognizes the accounts or instructions of one program and turns their bytes into Rust types. Every account and instruction pipe has one. This page covers the two decoder traits, how decoding works, how a generated decoder crate is organized and where decoders come from.

## The decoder traits

```rust
pub trait AccountDecoder<'a> {
    type AccountType;

    fn decode_account(
        &self,
        account: &'a solana_account::Account,
    ) -> Option<DecodedAccount<Self::AccountType>>;
}

pub trait InstructionDecoder<'a> {
    type InstructionType;

    fn decode_instruction(
        &self,
        instruction: &'a solana_instruction::Instruction,
    ) -> Option<Self::InstructionType>;
}
```

Both return `None` for data that isn't theirs. The pipe then skips the update and the processor isn't called, so a pipe with a Meteora decoder ignores every non-Meteora instruction in a transaction. A decoder that returns `Some` for everything sees every update of its kind.

`DecodedAccount<T>` carries the decoded data plus the account header:

```rust
pub struct DecodedAccount<T> {
    pub lamports: u64,
    pub data: T,
    pub owner: Pubkey,
    pub executable: bool,
    pub rent_epoch: u64,
}
```

The `instruction` passed to `decode_instruction` is a `solana_instruction::Instruction` with the program ID, the account metas (pubkey, signer, writable) and the data. Carbon builds it from the transaction and resolves address lookup tables first, so account keys are complete. Pass a decoder to the pipeline by value: `.instruction(MeteoraDlmmDecoder, processor)` or `.account(MeteoraDlmmDecoder, processor)`.

## How decoding works

### Discriminators

Programs put a discriminator at the start of their data to say what it is:

- **Anchor programs** use 8 bytes: the first 8 bytes of the SHA-256 of `global:<instruction_name>` for instructions, `account:<AccountName>` for accounts and `event:<EventName>` for events. Meteora DLMM's `swap` instruction starts with `[248, 198, 158, 145, 225, 117, 135, 200]`.
- **Native and SPL programs** use their own layouts. SPL Token instructions start with a one-byte tag (`Transfer` is `3`), and token accounts have no discriminator: the decoder recognizes them by size.

A generated decoder gives each account, instruction and event type a `decode(data: &[u8]) -> Option<Self>` method that checks the discriminator and deserializes the rest with Borsh, ignoring trailing bytes. The program's `decode_instruction` first checks `instruction.program_id == PROGRAM_ID`, then tries each instruction type in turn; `decode_account` first checks `account.owner == PROGRAM_ID`, then tries each account type. The first match wins.

### `CarbonDeserialize`

`carbon_core::deserialize::CarbonDeserialize` is the trait form of the same idea, for types you write yourself:

```rust
pub trait CarbonDeserialize
where
    Self: Sized + borsh::BorshDeserialize,
{
    const DISCRIMINATOR: &'static [u8];

    fn deserialize(data: &[u8]) -> Option<Self>;
}
```

[`decode_log_events`](https://carbon.sevenlabs.org/concepts/instructions.md#log-events) needs it to decode events from transaction logs. Generated decoders in 2.0.0 don't implement it; they use the inherent `decode` methods above. The same module provides `extract_discriminator(length, data)`, which splits data into `(discriminator, rest)`.

### `ArrangeAccounts`

An instruction's accounts arrive as a list in the order the program expects them. `ArrangeAccounts` maps that list to a struct with named fields:

```rust
pub trait ArrangeAccounts {
    type ArrangedAccounts: Clone + Send + Sync + std::fmt::Debug;

    fn arrange_accounts(accounts: &[solana_instruction::AccountMeta]) -> Option<Self::ArrangedAccounts>;
}
```

Generated instruction types implement it. Optional accounts in the IDL become `Option<Pubkey>`, and any accounts beyond the IDL's list land in a `remaining: Vec<AccountMeta>` field. If the instruction has fewer accounts than it requires, `arrange_accounts` returns `None` and the instruction isn't decoded.

## Generated decoder crates

Every published decoder, and every decoder the CLI generates, has the same layout. For Meteora DLMM (`carbon-meteora-dlmm-decoder`):

```text
src/
├── lib.rs           MeteoraDlmmDecoder, PROGRAM_ID
├── accounts/        one module per account type, MeteoraDlmmAccount, AccountDecoder impl
├── instructions/    one module per instruction, MeteoraDlmmInstruction, InstructionDecoder impl
├── events/          one module per event type, each with decode()
├── types/           structs and enums the IDL defines and the others use
└── graphql/         behind the graphql feature
```

| Item                                           | What it is                                                                                                                                                                                                        |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MeteoraDlmmDecoder`                           | A unit struct that implements both `AccountDecoder` and `InstructionDecoder`.                                                                                                                                     |
| `PROGRAM_ID`                                   | The program's address, a `const Pubkey`. Use it to configure datasources.                                                                                                                                         |
| `accounts::MeteoraDlmmAccount`                 | One variant per account type, such as `LbPair(Box<LbPair>)`.                                                                                                                                                      |
| `instructions::MeteoraDlmmInstruction`         | One struct variant per instruction: `Swap { program_id, data, accounts }`, where `data` holds the arguments (`Swap { amount_in, min_amount_out }`) and `accounts` the named accounts (`SwapInstructionAccounts`). |
| `events::swap::SwapEvent`                      | An event payload, with its own `decode`.                                                                                                                                                                          |
| `accounts::postgres`, `instructions::postgres` | Row types and migrations, behind the `postgres` feature.                                                                                                                                                          |

The account and instruction enums derive `Debug`, `Clone` and `PartialEq`, plus `Serialize` and `Deserialize` with the `serde` feature. Decoders generated by CLI 2.0.0 for programs with Anchor events also export `EVENT_CPI_DISCRIMINATOR`; the published 2.0.0 crates predate that and don't. The [decoder catalog](https://carbon.sevenlabs.org/decoders.md) documents each published crate.

## Events and CPI events

Anchor programs publish events in one of two ways, and Carbon decodes each differently:

- **Event CPI** (`emit_cpi!`): the program calls itself with an instruction whose data is the 8-byte event-CPI tag `[228, 69, 165, 46, 81, 203, 154, 29]`, the event's discriminator and the event. A generated decoder decodes this inner instruction as a `CpiEvent` variant: `MeteoraDlmmInstruction::CpiEvent { data: CpiEvent::Swap(SwapEvent { .. }), .. }`. It reaches your instruction processor like any other instruction, and also appears among the parent instruction's nested instructions.
- **Log events** (`emit!`): the program writes a `Program data: <base64>` log line. Carbon doesn't turn these into instructions; decode them from the instruction's metadata with `decode_log_events`.

[Instructions](https://carbon.sevenlabs.org/concepts/instructions.md#event-cpi-instructions) shows both in a processor.

## Where decoders come from

- **Published crates.** Carbon 2.0 publishes 63 decoder crates, `carbon-<program>-decoder`, for SPL Token, Token-2022, System, Jupiter, Raydium, Orca, Meteora, Pump.fun, Kamino and more. See the [catalog](https://carbon.sevenlabs.org/decoders.md).
- **Generated with the CLI.** `carbon-cli parse` turns an Anchor or Codama IDL, or an on-chain IDL fetched by program address, into a decoder crate with this layout. See [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md).
- **Written by hand.** For programs without an IDL, or when you only need one or two types, implement the traits yourself. See [Write a custom decoder](https://carbon.sevenlabs.org/guides/custom-decoder.md) and the second example below.

## Examples

### Decode an instruction with a generated decoder

This program builds a Meteora DLMM `swap` instruction by hand and decodes it. In a pipeline, the instruction comes from a transaction instead.

```sh
cargo add carbon-core@2.0.0 carbon-meteora-dlmm-decoder@2.0.0
cargo add solana-instruction@3.4.1 solana-pubkey@4.2.1
```

`src/main.rs`:

```rust
use {
    carbon_core::instruction::InstructionDecoder,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    solana_instruction::{AccountMeta, Instruction},
    solana_pubkey::Pubkey,
};

fn main() {
    // Anchor discriminator of `swap`, then the Borsh-encoded arguments.
    let mut data = vec![248, 198, 158, 145, 225, 117, 135, 200];
    data.extend_from_slice(&1_000_000u64.to_le_bytes()); // amount_in
    data.extend_from_slice(&990_000u64.to_le_bytes()); // min_amount_out

    let accounts = (0..15)
        .map(|_| AccountMeta::new(Pubkey::new_unique(), false))
        .collect();
    let instruction = Instruction { program_id: PROGRAM_ID, accounts, data };

    match MeteoraDlmmDecoder.decode_instruction(&instruction) {
        Some(MeteoraDlmmInstruction::Swap { data, accounts, .. }) => println!(
            "swap: amount_in={} min_amount_out={} lb_pair={}",
            data.amount_in, data.min_amount_out, accounts.lb_pair
        ),
        Some(_) => println!("another DLMM instruction"),
        None => println!("not a DLMM instruction"),
    }
}
```

Output:

```text
swap: amount_in=1000000 min_amount_out=990000 lb_pair=11157t3sqMV725NVRLrVQbAu98Jjfk1uCKehJnXXQs
```

### Write an account decoder by hand

This decoder reads a `CounterState` account of a hypothetical Anchor program: an 8-byte discriminator followed by a Borsh-encoded authority and count.

```sh
cargo add carbon-core@2.0.0 solana-account@4.3.2 solana-pubkey@4.2.1
cargo add borsh --features derive
```

`src/main.rs`:

```rust
use {
    borsh::BorshDeserialize,
    carbon_core::{
        account::{AccountDecoder, DecodedAccount},
        deserialize::CarbonDeserialize,
    },
    solana_account::Account,
    solana_pubkey::Pubkey,
};

const PROGRAM_ID: Pubkey = Pubkey::from_str_const("Counter111111111111111111111111111111111111");

/// On-chain layout: 8-byte discriminator, then Borsh fields.
#[derive(Debug, BorshDeserialize)]
pub struct CounterState {
    pub authority: Pubkey,
    pub count: u64,
}

impl CarbonDeserialize for CounterState {
    // The first 8 bytes of sha256("account:CounterState")
    const DISCRIMINATOR: &'static [u8] = &[98, 23, 205, 159, 14, 202, 79, 139];

    fn deserialize(data: &[u8]) -> Option<Self> {
        let mut body = data.strip_prefix(Self::DISCRIMINATOR)?;
        // `deserialize` (not `try_from_slice`) tolerates trailing padding bytes.
        <Self as BorshDeserialize>::deserialize(&mut body).ok()
    }
}

pub struct CounterDecoder;

impl<'a> AccountDecoder<'a> for CounterDecoder {
    type AccountType = CounterState;

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<CounterState>> {
        if account.owner != PROGRAM_ID {
            return None;
        }
        Some(DecodedAccount {
            lamports: account.lamports,
            data: <CounterState as CarbonDeserialize>::deserialize(&account.data)?,
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

fn main() {
    let mut data = CounterState::DISCRIMINATOR.to_vec();
    data.extend_from_slice(&[7; 32]); // authority
    data.extend_from_slice(&42u64.to_le_bytes()); // count
    data.extend_from_slice(&[0; 16]); // padding

    let account = Account {
        lamports: 1_224_960,
        data,
        owner: PROGRAM_ID,
        executable: false,
        rent_epoch: u64::MAX,
    };

    let decoded = CounterDecoder.decode_account(&account).expect("a CounterState account");
    println!("{:?}", decoded.data);
}
```

Output:

```text
CounterState { authority: US517G5965aydkZ46HS38QLi7UQiSojurfbQfKCELFx, count: 42 }
```

`CounterDecoder` works in a pipeline like a generated one: `.account(CounterDecoder, processor)`.

## Next steps

- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): Create a decoder crate from your program's IDL.
- [Processors](https://carbon.sevenlabs.org/concepts/processors.md): Receive decoded accounts and instructions in your code.
- [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md): Nested instructions, CPIs and events.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/decoders · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Processors (Carbon 2.0)

> The Processor trait, the input each pipe passes to it, keeping state with &mut self, sharing state through Arc and channels, and what happens on errors.

A processor is your code: Carbon calls it with each decoded update that reaches its pipe. This page covers the trait, what each kind of pipe passes in, and how to keep and share state. The [processor inputs reference](https://carbon.sevenlabs.org/reference/processor-inputs.md) lists every field.

## The `Processor` trait

```rust
pub trait Processor<T>
where
    T: Sync,
{
    fn process(&mut self, data: &T) -> impl Future<Output = CarbonResult<()>> + Send;
}
```

Implement it with a plain `async fn`; you don't need `#[async_trait]`:

```rust
impl Processor<AccountDeletion> for MyProcessor {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        println!("closed {}", deletion.pubkey);
        Ok(())
    }
}
```

The builder requires processors to be `Send + Sync + 'static`, and the future `process` returns must be `Send`. In practice: don't hold a `std::sync::MutexGuard`, an `Rc` or another non-`Send` value across an `.await` inside `process`.

The pipeline awaits each `process` call before it moves on, and never calls two processors at once (see [ordering guarantees](https://carbon.sevenlabs.org/concepts/architecture.md#ordering-and-concurrency-guarantees)). A slow `process` slows the whole pipeline.

## What each pipe passes in

The input type depends on the builder method you register the processor with. One type can implement `Processor` for several inputs, and you register an instance for each.

| Builder method                     | Your processor implements                                                                         |
| ---------------------------------- | ------------------------------------------------------------------------------------------------- |
| `.account(decoder, processor)`     | `Processor<AccountProcessorInputType<'_, T>>`, where `T` is the decoder's `AccountType`           |
| `.instruction(decoder, processor)` | `Processor<InstructionProcessorInputType<'_, T>>`, where `T` is the decoder's `InstructionType`   |
| `.transaction::<C, _>(processor)`  | `Processor<TransactionProcessorInputType<'_, C>>`, where `C` is an `InstructionDecoderCollection` |
| `.account_deletions(processor)`    | `Processor<AccountDeletion>`                                                                      |
| `.block_details(processor)`        | `Processor<BlockDetails>`                                                                         |

The `*_with_filters` variants take the same processors.

### Account pipes

```rust
pub struct AccountProcessorInputType<'a, T> {
    pub metadata: &'a AccountMetadata,        // slot, pubkey, transaction_signature
    pub decoded_account: &'a DecodedAccount<T>, // data: T, plus lamports, owner, executable, rent_epoch
    pub raw_account: &'a solana_account::Account,
}
```

The processor runs once per account update the decoder recognizes. `raw_account` gives you the original bytes if you need fields the decoder doesn't expose.

### Instruction pipes

```rust
pub struct InstructionProcessorInputType<'a, T> {
    pub metadata: &'a InstructionMetadata,
    pub decoded_instruction: &'a T,
    pub nested_instructions: &'a NestedInstructions,
    pub raw_instruction: &'a solana_instruction::Instruction,
}
```

The processor runs once per decoded instruction, including instructions called through CPI. `metadata` holds the instruction's position (`stack_height`, `index`, `absolute_path`) and an `Arc<TransactionMetadata>` with the transaction's signature, slot, fee payer, message and execution meta. `nested_instructions` are the instructions this one called. See [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md).

### Transaction pipes

```rust
pub struct TransactionProcessorInputType<'a, T> {
    pub metadata: &'a Arc<TransactionMetadata>,
    pub instructions: &'a [(InstructionMetadata, T)],
}
```

The processor runs once per transaction that passes the pipe's filters, with every instruction its collection decoded, top-level and nested, in execution order. It runs even when nothing decoded, with an empty slice. See [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md).

### Account deletion and block details pipes

These processors receive the update itself: an [`AccountDeletion`](https://carbon.sevenlabs.org/concepts/updates.md#account-deletions) (pubkey, final account, slot, transaction signature) or a [`BlockDetails`](https://carbon.sevenlabs.org/concepts/updates.md#block-details). There's no decoder, so an `account_deletions` processor sees closures for every program the datasource covers; check `deletion.account.owner` if you only care about one.

## Keep state in the processor

`process` takes `&mut self`, and each registered processor is owned by its pipe. Keep counters, caches and batches in plain fields: nothing else can access them while `process` runs, so they need no locks.

Each registration owns a separate instance. If you register the same processor type twice, each copy has its own state.

## Share state with the rest of your program

To read a processor's state from elsewhere, or to share it between processors, put it behind an `Arc`:

- **`Arc<Mutex<T>>` or atomics** for small shared values. A `std::sync::Mutex` is fine if you never `.await` while holding the guard; use `tokio::sync::Mutex` if you must.
- **A channel to a task you own** for I/O. The processor sends a message and returns; a separate task writes to the database or calls an API. This keeps slow I/O out of the pipeline loop. With a bounded channel, a slow writer still slows the pipeline down instead of using unbounded memory.

This processor forwards Meteora DLMM swaps to a writer task. It goes with a pipeline that registers `.instruction(MeteoraDlmmDecoder, Forward { tx })`:

```rust
pub struct SwapRow {
    pub signature: String,
    pub amount_in: u64,
}

pub struct Forward {
    tx: tokio::sync::mpsc::Sender<SwapRow>,
}

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for Forward {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmInstruction::Swap { data, .. } = input.decoded_instruction {
            let row = SwapRow {
                signature: input.metadata.transaction_metadata.signature.to_string(),
                amount_in: data.amount_in,
            };
            self.tx
                .send(row)
                .await
                .map_err(|_| CarbonError::Custom("writer task stopped".to_string()))?;
        }
        Ok(())
    }
}
```

`CarbonError` is `carbon_core::error::Error`. Once work leaves the processor, the pipeline no longer sees its errors or its order: the writer task has to handle both.

## Errors

If `process` returns `Err`, the pipeline logs the error with the update, counts the update in `carbon_updates_failed_total`, skips the remaining pipes for that update and continues with the next one. It doesn't retry and doesn't stop. Retry inside `process` if the operation can succeed on a second attempt. See [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md).

## Example

This processor counts updates in a private field and adds lamports per owner to a shared map, which `main` reads after the pipeline finishes. Three hand-built account updates stand in for a datasource.

```sh
cargo add carbon-core@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::AccountUpdate,
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
    std::{
        collections::HashMap,
        sync::{Arc, Mutex},
    },
};

struct AnyAccount;

impl<'a> AccountDecoder<'a> for AnyAccount {
    type AccountType = ();

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<()>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: (),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

/// Keeps private state in `self` and publishes totals to shared state.
struct LamportsByOwner {
    seen: u64,
    totals: Arc<Mutex<HashMap<Pubkey, u64>>>,
}

impl Processor<AccountProcessorInputType<'_, ()>> for LamportsByOwner {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, ()>) -> CarbonResult<()> {
        self.seen += 1; // `&mut self`: no lock needed for state only this processor uses

        let mut totals = self.totals.lock().unwrap(); // never hold a std lock across `.await`
        *totals.entry(input.decoded_account.owner).or_default() += input.decoded_account.lamports;

        println!("update #{} at slot {}", self.seen, input.metadata.slot);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let owner_a = Pubkey::new_from_array([1; 32]);
    let owner_b = Pubkey::new_from_array([2; 32]);

    let (tx, rx) = tokio::sync::mpsc::channel(16);
    for (slot, owner, lamports) in [(1, owner_a, 10), (2, owner_b, 5), (3, owner_a, 7)] {
        let update = AccountUpdate {
            pubkey: Pubkey::new_unique(),
            account: Account { lamports, owner, ..Account::default() },
            slot,
            transaction_signature: None,
        };
        tx.send(UnifiedMessage::Account(update)).await.unwrap();
    }
    drop(tx);

    let totals = Arc::new(Mutex::new(HashMap::new()));

    Pipeline::builder()
        .datasource(StreamMessageClient::new(rx))
        .account(AnyAccount, LamportsByOwner { seen: 0, totals: totals.clone() })
        .build()?
        .run()
        .await?;

    let totals = totals.lock().unwrap();
    println!("owner A: {} lamports, owner B: {} lamports", totals[&owner_a], totals[&owner_b]);
    Ok(())
}
```

```sh
cargo run
```

Output:

```text
update #1 at slot 1
update #2 at slot 2
update #3 at slot 3
owner A: 17 lamports, owner B: 5 lamports
```

## Next steps

- [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md): Instruction metadata, nested instructions and events.
- [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md): Process every instruction of a transaction together.
- [Persist to Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Write decoded data to typed tables.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/processors · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Instructions (Carbon 2.0)

> InstructionMetadata, stack heights and absolute paths, nested instructions and CPIs, event CPI instructions, and decoding log events with decode_log_events.

A Solana transaction runs top-level instructions, and each of those can call other programs through cross-program invocation (CPI). Carbon decodes all of them, nested ones included, and tells your processor where each instruction sits. This page explains that position data, the nested instruction tree, and the two ways programs emit events.

## Instruction metadata

Every [instruction processor](https://carbon.sevenlabs.org/concepts/processors.md#instruction-pipes) receives an `InstructionMetadata`:

```rust
pub struct InstructionMetadata {
    pub transaction_metadata: Arc<TransactionMetadata>,
    pub stack_height: u32,
    pub index: u32,
    pub absolute_path: Vec<u8>,
}
```

| Field                  | Meaning                                                                                                                                                                                                                                                                                            |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `transaction_metadata` | The transaction this instruction belongs to: signature, slot, fee payer, message, execution meta (logs, balances, inner instructions), block time. Shared by every instruction of the transaction. See [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md#transaction-metadata). |
| `stack_height`         | 1 for a top-level instruction, 2 for an instruction it called, and so on up to 5 (`MAX_INSTRUCTION_STACK_DEPTH`).                                                                                                                                                                                  |
| `index`                | The index of the top-level instruction this one belongs to. An inner instruction has the same `index` as its top-level ancestor.                                                                                                                                                                   |
| `absolute_path`        | The position at each level, starting with the top-level index. `[3]` is top-level instruction 3; `[3, 0, 2]` is the third instruction called by the first instruction called by top-level instruction 3.                                                                                           |

`absolute_path.len()` equals `stack_height`, and `absolute_path[0]` equals `index`. The pair (transaction signature, `absolute_path`) identifies an instruction uniquely, which is how [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) recognizes repeats.

A Jupiter route that swaps through Meteora DLMM looks like this (simplified):

![Instructions nested by stack height](https://carbon.sevenlabs.org/diagrams/instruction-tree.svg)

**Diagram: Instructions nested by stack height.** A Jupiter route at absolute_path \[2] has stack_height 1. It calls Token: transfer at \[2, 0] and Meteora DLMM: swap at \[2, 3], both stack_height 2, followed by more inner instructions. The Meteora DLMM swap calls Token: transferChecked at \[2, 3, 0] and \[2, 3, 1] and emits its event CPI at \[2, 3, 2], all stack_height 3.

## How Carbon builds the tree

Solana records inner instructions in the transaction's metadata as one flat group per top-level instruction, each with its stack height. For every transaction, Carbon:

1. Walks the top-level instructions in order and, after each one, its group of inner instructions.
2. Derives each inner instruction's `absolute_path` from the sequence of stack heights.
3. Resolves account keys, including those loaded from address lookup tables, and builds a `solana_instruction::Instruction` for each.
4. Nests the list into `NestedInstructions`, a tree of `NestedInstruction` nodes.

An inner instruction without a stack height, or with one outside 2 to 5, can't be placed in the tree. Carbon logs a warning and drops it together with the rest of its group, so transactions whose metadata lacks stack heights reach your processors without inner instructions.

## Nested instructions

```rust
pub struct NestedInstruction {
    pub metadata: InstructionMetadata,
    pub instruction: solana_instruction::Instruction,
    pub inner_instructions: NestedInstructions,
}

pub struct NestedInstructions(pub Vec<NestedInstruction>);
```

`NestedInstructions` dereferences to `&[NestedInstruction]`, so you can index it, call `.iter()` and `.len()`, and loop over it. In an instruction processor, `input.nested_instructions` holds the direct children of the current instruction; each child's `inner_instructions` holds its own children. The children are raw instructions: decode them with any decoder by calling `decode_instruction(&child.instruction)`.

An instruction pipe runs its processor for every instruction its decoder recognizes, at any depth. A CPI therefore reaches you twice: once as its own processor call and once as a child of its caller. Pick one view when you count things, or you'll count CPIs twice.

## Event CPI instructions

Anchor's `emit_cpi!` publishes an event by having the program call itself with the event as instruction data. Generated decoders decode these self-calls as a `CpiEvent` variant, such as `MeteoraDlmmInstruction::CpiEvent { data: CpiEvent::Swap(event), .. }`. Because the event is a child of the instruction that emitted it, you can pair them: look for the `CpiEvent` among a swap's `nested_instructions` to get the swap's actual output amount.

## Log events

Anchor's `emit!` publishes an event as a `Program data: <base64>` line in the transaction's log. `InstructionMetadata::decode_log_events` returns the events logged by one instruction:

```rust
pub fn decode_log_events<T: CarbonDeserialize>(&self) -> Vec<T>
```

It follows the `Program … invoke [n]` and `Program … success` lines to find the log lines that belong to this instruction's `absolute_path` (excluding its CPIs, and accounting for precompile instructions, which don't log), base64-decodes them, and keeps those that `T::deserialize` accepts. It returns an empty list if the transaction has no log messages, and it can't recover events from logs the runtime truncated.

`T` must implement [`CarbonDeserialize`](https://carbon.sevenlabs.org/concepts/decoders.md#carbondeserialize). Generated event types don't, so wrap them. This newtype decodes Meteora DLMM's `SwapEvent` from log lines; its `deserialize` calls the generated `decode`, which checks the discriminator:

```rust
use {
    borsh::BorshDeserialize,
    carbon_core::deserialize::CarbonDeserialize,
    carbon_meteora_dlmm_decoder::events::swap::SwapEvent,
};

/// A `SwapEvent` emitted with Anchor's `emit!` (a `Program data:` log line).
#[derive(Debug, BorshDeserialize)]
pub struct LoggedSwap(pub SwapEvent);

impl CarbonDeserialize for LoggedSwap {
    const DISCRIMINATOR: &'static [u8] = &[81, 108, 227, 190, 205, 208, 10, 196];

    fn deserialize(data: &[u8]) -> Option<Self> {
        SwapEvent::decode(data).map(LoggedSwap)
    }
}
```

Then, in an instruction processor:

```rust
for LoggedSwap(event) in input.metadata.decode_log_events::<LoggedSwap>() {
    println!("logged swap: amount_out={}", event.amount_out);
}
```

Meteora DLMM itself emits through event CPI, so in practice you use this pattern for programs that log their events, such as Raydium CLMM.

## Example

This program crawls Meteora DLMM transactions from the public RPC endpoint. For each swap it prints the swap's position, then finds the swap's event CPI among its children and prints the output amount and fee from it.

```sh
cargo add carbon-core@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0
cargo add tokio --features full
cargo add env_logger
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult,
        instruction::{InstructionDecoder, InstructionProcessorInputType},
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::{cpi_event::CpiEvent, MeteoraDlmmInstruction},
        MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
};

struct SwapTree;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapTree {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let MeteoraDlmmInstruction::Swap { data, .. } = input.decoded_instruction else {
            return Ok(());
        };
        let meta = input.metadata;
        println!(
            "swap {} path={:?} stack_height={} amount_in={}",
            meta.transaction_metadata.signature,
            meta.absolute_path,
            meta.stack_height,
            data.amount_in
        );

        // Children of this instruction: token transfers and the program's event CPI.
        for child in input.nested_instructions.iter() {
            if let Some(MeteoraDlmmInstruction::CpiEvent {
                data: CpiEvent::Swap(event),
                ..
            }) = MeteoraDlmmDecoder.decode_instruction(&child.instruction)
            {
                println!(
                    "  event path={:?} amount_out={} fee={}",
                    child.metadata.absolute_path, event.amount_out, event.fee
                );
            }
        }
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let crawler = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(crawler)
        .instruction(MeteoraDlmmDecoder, SwapTree)
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
swap 4qSjUGPcKNNcEUxcK2uaTUYKoTrUBYCYEFLbZt8yqiWxrH7Z2E8nXCj4zMQjJ1TxNYztjWC83qg2WXKhSjUTc1gw path=[3, 0, 2] stack_height=3 amount_in=1679907
  event path=[3, 0, 2, 2] amount_out=19337053 fee=233
swap 24cQfJCNzD67i83gUk4jV8Z6sNz9TrPXZV8kdZjKK3sNeUHLXAxddb3JmKoETAEgsFrpJjWzyqg5qr2fuj8nzZXc path=[0, 0] stack_height=2 amount_in=4254885455
  event path=[0, 0, 2] amount_out=11300348 fee=65622
swap 4pr7pw96PGJSiXYcGT3Vman8v1rgbjEn8XASQmkdURRayQHsJZut88G9mwyzYhWH1XksEtBAdALj5dD13VYoRgM5 path=[2, 3] stack_height=2 amount_in=854429039
  event path=[2, 3, 2] amount_out=318436120 fee=349280
…
```

Most DLMM swaps arrive through an aggregator, so their paths have two or more levels. The public endpoint is rate limited; output can pause for a minute or more, and you may see `429` warnings in the log. Press `Ctrl+C` to stop.

## Next steps

- [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md): See every instruction of a transaction in one processor.
- [Filters](https://carbon.sevenlabs.org/concepts/filters.md): Deduplicate instructions and restrict them by slot or source.
- [Processor inputs reference](https://carbon.sevenlabs.org/reference/processor-inputs.md): Every field of InstructionMetadata and the input types.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/instructions · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Transactions (Carbon 2.0)

> Transaction processors, TransactionMetadata, InstructionDecoderCollection and the instruction_decoder_collection! macro, and when to use each pipe type.

An instruction processor sees one instruction at a time. A transaction processor sees a whole transaction: its metadata and every instruction in it, decoded across several programs at once. This page explains transaction pipes and the decoder collections they need.

## When to use a transaction processor

| Use an instruction processor when                                                  | Use a transaction processor when                                                                                     |
| ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Each instruction stands on its own: record every swap, every mint, every transfer. | You need several instructions together: a swap and the token transfers it caused, or a route through three programs. |
| One program's decoder is enough.                                                   | You decode instructions from several programs and combine them.                                                      |
| You want the generated decoder's full types directly.                              | You aggregate per transaction: one row per transaction, totals, classification.                                      |

Both can run in the same pipeline. For each transaction, every instruction pipe runs before any transaction pipe (see [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md#transactions)).

## Transaction pipes

Register a transaction processor with the collection type it decodes with:

```rust
Pipeline::builder()
    .datasource(datasource)
    .transaction::<AllInstructions, _>(SwapSummary)
```

The processor implements `Processor<TransactionProcessorInputType<'_, AllInstructions>>` and receives:

```rust
pub struct TransactionProcessorInputType<'a, T> {
    pub metadata: &'a Arc<TransactionMetadata>,
    pub instructions: &'a [(InstructionMetadata, T)],
}
```

`instructions` holds every instruction the collection decoded, top-level and nested, in execution order, each with its [`InstructionMetadata`](https://carbon.sevenlabs.org/concepts/instructions.md#instruction-metadata). Instructions the collection doesn't recognize are left out. The processor runs once for every transaction that passes the pipe's filters, **including transactions where nothing decoded**: check for an empty slice if that matters to you.

`transaction_with_filters` takes filters too. Their `filter_transaction` method receives the transaction metadata and the full [`NestedInstructions`](https://carbon.sevenlabs.org/concepts/instructions.md#nested-instructions) tree before anything is decoded.

## Transaction metadata

```rust
pub struct TransactionMetadata {
    pub slot: u64,
    pub signature: Signature,
    pub fee_payer: Pubkey,
    pub meta: solana_transaction_status::TransactionStatusMeta,
    pub message: solana_message::VersionedMessage,
    pub index: Option<u64>,
    pub block_time: Option<i64>,
    pub block_hash: Option<Hash>,
}
```

`fee_payer` is the first account key of the message. `meta` has the execution results: `status`, `fee`, `pre_balances` and `post_balances`, `pre_token_balances` and `post_token_balances`, `log_messages`, `inner_instructions`, `loaded_addresses` and `compute_units_consumed`. `index`, `block_time` and `block_hash` are set only when the datasource provides them (see [Updates](https://carbon.sevenlabs.org/concepts/updates.md#transactions)). Instruction processors reach the same struct through `input.metadata.transaction_metadata`.

## Decoder collections

A collection is an enum with one variant per program, which knows how to decode an instruction from any of them. It implements `InstructionDecoderCollection`:

```rust
pub trait InstructionDecoderCollection:
    Clone + std::fmt::Debug + Send + Sync + Eq + std::hash::Hash + Serialize + 'static
{
    type InstructionType: Clone + std::fmt::Debug + PartialEq + Eq + Send + Sync + 'static;

    fn parse_instruction(instruction: &solana_instruction::Instruction) -> Option<Self>;

    fn get_type(&self) -> Self::InstructionType;
}
```

- `parse_instruction` returns the decoded instruction wrapped in the right variant, or `None` if no program matches.
- `get_type` returns a lightweight tag, such as `AllInstructionTypes::Dlmm(DlmmType::Swap)`, which you can compare without matching on the payload.

The transaction pipe calls `carbon_core::transaction::parse_instructions_flat::<T>(instructions)`, which runs `parse_instruction` over the transaction's flat instruction list and keeps the matches with their metadata. It's public, so you can call it yourself in tests.

### The `instruction_decoder_collection!` macro

Writing the enum and the trait by hand is repetitive, so `carbon_core` provides a macro (with the default `macros` feature):

```rust
instruction_decoder_collection!(
    AllInstructions, AllInstructionTypes, AllPrograms,
    Dlmm => carbon_meteora_dlmm_decoder::PROGRAM_ID => DlmmDecoder => Dlmm,
    Token => carbon_token_program_decoder::PROGRAM_ID => TokenDecoder => Token,
);
```

The first three names are the enums it generates: the collection (`AllInstructions`), its type tags (`AllInstructionTypes`) and a plain list of programs (`AllPrograms`). Each following entry is `Variant => PROGRAM_ID => decoder expression => instruction type`. `parse_instruction` matches `instruction.program_id` against each `PROGRAM_ID` and calls that decoder. You can omit the program ID (`Variant => decoder => type`); those entries are tried in order, for instructions whose program ID matched no other entry.

Each instruction type you list must:

- derive `Debug`, `Clone`, `PartialEq`, `Eq`, `Hash`, `serde::Serialize` and `serde::Deserialize`;
- derive `InstructionType` (from `carbon_core`), which generates a sibling enum named `<Type>Type` with one unit variant per variant, plus a `get_instruction_type()` method;
- be in scope by its bare name, together with its `<Type>Type` enum, because the macro refers to them by their last path segment.

The code the macro generates also needs the `InstructionDecoder` trait in scope and `serde` (with `derive`) and `solana-instruction` as direct dependencies of your crate.

> **Warning: Generated instruction enums can't be listed directly.** In 2.0.0 the instruction enums of generated decoders, such as `MeteoraDlmmInstruction`, derive only `Debug`, `Clone` and `PartialEq` (and serde behind the `serde` feature). They lack `Eq`, `Hash` and `InstructionType`, so they can't be collection variants. Wrap each generated decoder in a small decoder that maps its output to your own enum, as in the example below. This also keeps only the fields you use.

## Example

This program decodes Meteora DLMM and SPL Token instructions together. For every transaction with a DLMM swap, it prints the fee payer and the amounts of the token transfers in the same transaction.

```sh
cargo add carbon-core@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-token-program-decoder@2.0.0
cargo add carbon-rpc-transaction-crawler-datasource@2.0.0
cargo add serde --features derive
cargo add solana-instruction@3.4.1
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        collection::InstructionDecoderCollection,
        error::CarbonResult,
        instruction::InstructionDecoder,
        instruction_decoder_collection,
        pipeline::Pipeline,
        processor::Processor,
        transaction::TransactionProcessorInputType,
        InstructionType,
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    carbon_token_program_decoder::{instructions::TokenProgramInstruction, TokenProgramDecoder},
    serde::{Deserialize, Serialize},
};

// Slim per-program enums with the derives a collection needs.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, InstructionType)]
pub enum Dlmm {
    Swap { amount_in: u64, min_amount_out: u64 },
    Other,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, InstructionType)]
pub enum Token {
    Transfer { amount: u64 },
    TransferChecked { amount: u64, decimals: u8 },
    Other,
}

pub struct DlmmDecoder;

impl<'a> InstructionDecoder<'a> for DlmmDecoder {
    type InstructionType = Dlmm;

    fn decode_instruction(&self, ix: &'a solana_instruction::Instruction) -> Option<Dlmm> {
        Some(match MeteoraDlmmDecoder.decode_instruction(ix)? {
            MeteoraDlmmInstruction::Swap { data, .. } => Dlmm::Swap {
                amount_in: data.amount_in,
                min_amount_out: data.min_amount_out,
            },
            _ => Dlmm::Other,
        })
    }
}

pub struct TokenDecoder;

impl<'a> InstructionDecoder<'a> for TokenDecoder {
    type InstructionType = Token;

    fn decode_instruction(&self, ix: &'a solana_instruction::Instruction) -> Option<Token> {
        Some(match TokenProgramDecoder.decode_instruction(ix)? {
            TokenProgramInstruction::Transfer { data, .. } => Token::Transfer { amount: data.amount },
            TokenProgramInstruction::TransferChecked { data, .. } => Token::TransferChecked {
                amount: data.amount,
                decimals: data.decimals,
            },
            _ => Token::Other,
        })
    }
}

instruction_decoder_collection!(
    AllInstructions, AllInstructionTypes, AllPrograms,
    Dlmm => carbon_meteora_dlmm_decoder::PROGRAM_ID => DlmmDecoder => Dlmm,
    Token => carbon_token_program_decoder::PROGRAM_ID => TokenDecoder => Token,
);

struct SwapSummary;

impl Processor<TransactionProcessorInputType<'_, AllInstructions>> for SwapSummary {
    async fn process(
        &mut self,
        input: &TransactionProcessorInputType<'_, AllInstructions>,
    ) -> CarbonResult<()> {
        let swaps = input
            .instructions
            .iter()
            .filter(|(_, ix)| ix.get_type() == AllInstructionTypes::Dlmm(DlmmType::Swap))
            .count();
        if swaps == 0 {
            return Ok(()); // the processor runs for every transaction, even with no matches
        }

        let transfers: Vec<u64> = input
            .instructions
            .iter()
            .filter_map(|(_, ix)| match ix {
                AllInstructions::Token(Token::Transfer { amount })
                | AllInstructions::Token(Token::TransferChecked { amount, .. }) => Some(*amount),
                _ => None,
            })
            .collect();

        println!(
            "{} fee_payer={} swaps={} token_transfers={:?}",
            input.metadata.signature, input.metadata.fee_payer, swaps, transfers
        );
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let crawler = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        carbon_meteora_dlmm_decoder::PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(crawler)
        .transaction::<AllInstructions, _>(SwapSummary)
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
2HNnxMudUcYCYXcRuV2SejC4p1fcY2kiLqXfqgZeNgPZ6pZjLUkxCT4QzXJruDXp6rXeu7SHwYr2aa22j1YtmSTt fee_payer=HYe4vSaEGqQKnDrxWDrk3o5H2gznv7qtij5G6NNG8WHd swaps=1 token_transfers=[51427757, 115, 4468209]
hX9jqFeYAkVk7rnAYATuRBxsrDQ36TBgHHfPNhP1uHGjbcdt5YR9JXMJTkX4L5K7UZNAMkHUEAM2iLEHcuBtaxK fee_payer=9nXDunV8eNvYSVys7JSBkm2hz9Q79Q3EYeko19srVeMC swaps=1 token_transfers=[98400000, 345227222]
fAQWC4yoDc5QT542V8YqKf1CncpVeC2ST9Kvoi5F38vQKjgkxmdzG8g3v1rULNnPpvYpdT2Mae6BzzcGPJjC6Uw fee_payer=GoGoGo6N99mpyB7rfzhw2R4fXmaFctURXHaHMoGCyoLD swaps=1 token_transfers=[20000000, 230210607, 230210607, 20000688]
…
```

The crawler only fetches transactions that mention the DLMM program, but the collection decodes the token program's instructions in them too. Token-2022 transfers don't appear because the collection has no Token-2022 entry; add `carbon-token-2022-decoder` the same way to include them. The public endpoint is rate limited, so output can pause for a minute or more.

## Next steps

- [Decode multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md): Combine decoders in one pipeline, with instruction or transaction pipes.
- [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md): Instruction metadata, nested instructions and events.
- [Macros reference](https://carbon.sevenlabs.org/reference/macros.md): Full syntax of instruction_decoder_collection! and InstructionType.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/transactions · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Filters (Carbon 2.0)

> Route, deduplicate and window updates per pipe with the Filter trait, DeduplicationFilter, DatasourceFilter, SlotRangeFilter and your own filters.

A filter decides, per pipe, whether an update reaches the pipe's decoder and processor. Use filters to send each datasource's updates to different processors, to drop duplicates when two sources overlap, or to process only a slot range. Filters belong to one pipe; other pipes still see everything.

## Adding filters to a pipe

Every pipe method has a `*_with_filters` variant that takes a `Vec<Box<dyn Filter>>`:

| Without filters                   | With filters                                            |
| --------------------------------- | ------------------------------------------------------- |
| `account(decoder, processor)`     | `account_with_filters(decoder, processor, filters)`     |
| `instruction(decoder, processor)` | `instruction_with_filters(decoder, processor, filters)` |
| `transaction::<C, _>(processor)`  | `transaction_with_filters::<C, _>(processor, filters)`  |
| `account_deletions(processor)`    | `account_deletions_with_filters(processor, filters)`    |
| `block_details(processor)`        | `block_details_with_filters(processor, filters)`        |

```rust
Pipeline::builder()
    .instruction_with_filters(
        MeteoraDlmmDecoder,
        SwapLogger,
        vec![
            Box::new(DatasourceFilter::new(DatasourceId::new_named("primary"))),
            Box::new(SlotRangeFilter::from(370_000_000, None)),
        ],
    )
```

The pipeline runs a pipe's filters in order, before decoding, and stops at the first `Reject`. The pipe runs only if every filter accepts. Because evaluation stops early, a filter later in the list never sees updates an earlier one rejected; this matters for stateful filters such as `DeduplicationFilter`.

## The `Filter` trait

```rust
pub trait Filter: Send + Sync {
    fn filter_account(
        &self,
        _context: &FilterContext,
        _account_metadata: &AccountMetadata,
        _account: &solana_account::Account,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_instruction(
        &self,
        _context: &FilterContext,
        _nested_instruction: &NestedInstruction,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_transaction(
        &self,
        _context: &FilterContext,
        _transaction_metadata: &TransactionMetadata,
        _nested_instructions: &NestedInstructions,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_account_deletion(
        &self,
        _context: &FilterContext,
        _account_deletion: &AccountDeletion,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_block_details(
        &self,
        _context: &FilterContext,
        _block_details: &BlockDetails,
    ) -> FilterResult {
        FilterResult::Accept
    }
}
```

Every method defaults to `Accept`, so implement only the ones for the pipes you attach the filter to. Each pipe calls one method: account pipes call `filter_account`, instruction pipes call `filter_instruction` once per instruction (nested ones included), transaction pipes call `filter_transaction` once per transaction, and so on.

`FilterContext` carries the [`DatasourceId`](https://carbon.sevenlabs.org/concepts/datasources.md#datasource-ids) of the source the update came from:

```rust
pub struct FilterContext<'a> {
    pub datasource_id: &'a DatasourceId,
}

pub enum FilterResult {
    Accept,
    Reject,
}
```

Filter methods take `&self` and are synchronous. Keep them fast: they run in the pipeline loop for every update. A filter that keeps state needs interior mutability, such as a `Mutex` or atomics.

## Built-in filters

### `DatasourceFilter`

Accepts updates only from the listed datasources, for every update type.

```rust
DatasourceFilter::new(DatasourceId::new_named("primary"))
DatasourceFilter::new_many(vec![DatasourceId::new_named("a"), DatasourceId::new_named("b")])
```

Register the datasources with [`datasource_with_id`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) so that you know their IDs. Use it to send a backfill crawler's transactions to one processor and a live stream's to another.

### `DeduplicationFilter`

Drops updates it has already seen within a time window:

```rust
DeduplicationFilter::new(Duration::from_secs(60))
```

| Pipe                                 | Key                                      | Notes                                                           |
| ------------------------------------ | ---------------------------------------- | --------------------------------------------------------------- |
| Instruction                          | (transaction signature, `absolute_path`) | Each instruction of a transaction is tracked separately.        |
| Account                              | (transaction signature, pubkey)          | Accounts without a `transaction_signature` are always accepted. |
| Transaction, deletion, block details | none                                     | Always accepted; the filter doesn't implement these methods.    |

Entries older than the TTL are purged, at most once a minute, when the filter is called. The filter is in memory and per instance: each pipe needs its own instance, and nothing survives a restart. Put it first in the list so that it records every update, and give it a TTL longer than the maximum delay between your redundant sources.

### `SlotRangeFilter`

Accepts updates whose slot is inside a range:

```rust
SlotRangeFilter::from(370_000_000, None)               // slot ≥ 370,000,000
SlotRangeFilter::to(371_000_000, None)                 // slot ≤ 371,000,000
SlotRangeFilter::between(370_000_000, None, 371_000_000, None)
```

Both ends are inclusive when you pass `None` for the transaction index. The optional index narrows the boundary slots for transactions and instructions, using the transaction's position in its block: at `from_slot`, transactions with an index below `from_transaction_index` are rejected; at `to_slot`, transactions with an index at or above `to_transaction_index` are rejected. Updates without an index, such as account updates and transactions from the RPC transaction crawler, are compared by slot alone. The filter applies to every update type.

## Filter by program

You rarely need a filter to keep one program's data: decoders already do it. A generated decoder returns nothing for an instruction whose `program_id` isn't its `PROGRAM_ID`, and nothing for an account whose `owner` isn't, so its processor only ever sees that program. That includes the program's instructions nested inside other programs' calls.

What decides how much data reaches the pipeline is the **datasource**. Narrow it at the source so you don't download and discard the rest of Solana:

| Datasource                                                                                                                                                                                                                                           | Where you choose the program                                                                    |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md), [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)                                                                               | `account_include` / `account_required` in the transaction filter, `owner` in the account filter |
| [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                                                                               | The address you pass to the constructor                                                         |
| [RPC blockSubscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)                                                                                                                                                                | `RpcBlockSubscribeFilter::MentionsAccountOrProgram(program_id)`                                 |
| [RPC programSubscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md), [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md), [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md) | The program ID in `Filters`                                                                     |
| [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md), [validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md)                                                                             | No program filter: every transaction or account in range arrives, and decoders pick out yours   |

Use a [`Filter`](#write-a-custom-filter) only for what neither can express, such as routing one datasource's updates to one processor ([`DatasourceFilter`](#datasourcefilter)).

## Decode or skip failed transactions

Solana records transactions that fail during execution, with their instructions and logs. Whether your processors see them depends on the datasource:

| Datasource                                                                                 | Failed transactions                                                                                                                                  |
| ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| RPC transaction crawler, RPC block crawler, RPC blockSubscribe, Helius enhanced WebSockets | Always skipped                                                                                                                                       |
| Yellowstone gRPC, Helius LaserStream                                                       | Included unless your transaction filter sets `failed: Some(false)`; for block subscriptions, `BlockFilters { failed_transactions: Some(false), .. }` |
| Helius GTFA                                                                                | Chosen with `TransactionStatusFilter` (`Succeeded`, `Failed` or `Any`)                                                                               |

A failed transaction's instructions decode normally, because decoding reads the instruction bytes, not the result. To ignore failures from a source that includes them, check the status in your processor:

```rust
if input.metadata.transaction_metadata.meta.status.is_err() {
    return Ok(()); // the transaction failed on chain; nothing it did took effect
}
```

"Decode failed" is a different problem: an instruction from your program that the decoder doesn't recognize, usually after a program upgrade changed its IDL. See [Versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md) and [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md).

## Write a custom filter

Implement `Filter` on your own type and override the methods you need. This filter keeps only accounts above a balance:

```rust
struct MinLamports {
    min: u64,
}

impl Filter for MinLamports {
    fn filter_account(
        &self,
        _context: &FilterContext,
        _metadata: &AccountMetadata,
        account: &Account,
    ) -> FilterResult {
        if account.lamports >= self.min {
            FilterResult::Accept
        } else {
            FilterResult::Reject
        }
    }
}
```

A filter sees raw data, before decoding. To filter on decoded fields, check them in the processor instead.

## Example

Two redundant datasources deliver the same three account writes. The pipe drops the duplicates, then the writes outside slots 0 to 25, then the accounts with less than 1,000 lamports.

```sh
cargo add carbon-core@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1 solana-signature@3.4.1
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::{AccountDecoder, AccountMetadata, AccountProcessorInputType, DecodedAccount},
        datasource::{AccountUpdate, DatasourceId},
        error::CarbonResult,
        filter::{DeduplicationFilter, Filter, FilterContext, FilterResult, SlotRangeFilter},
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
    solana_signature::Signature,
    std::time::Duration,
};

/// Custom filter: only accounts holding at least `min` lamports.
struct MinLamports {
    min: u64,
}

impl Filter for MinLamports {
    fn filter_account(
        &self,
        _context: &FilterContext,
        _metadata: &AccountMetadata,
        account: &Account,
    ) -> FilterResult {
        if account.lamports >= self.min {
            FilterResult::Accept
        } else {
            FilterResult::Reject
        }
    }
}

struct AnyAccount;

impl<'a> AccountDecoder<'a> for AnyAccount {
    type AccountType = ();

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<()>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: (),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

struct Print;

impl Processor<AccountProcessorInputType<'_, ()>> for Print {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, ()>) -> CarbonResult<()> {
        println!("slot {} lamports {}", input.metadata.slot, input.raw_account.lamports);
        Ok(())
    }
}

/// Builds a finite source that replays the same three account writes.
async fn replica() -> StreamMessageClient {
    let (tx, rx) = tokio::sync::mpsc::channel(16);
    for (i, (slot, lamports)) in [(10, 5_000), (20, 50), (30, 7_000)].into_iter().enumerate() {
        let update = AccountUpdate {
            pubkey: Pubkey::new_from_array([i as u8; 32]),
            account: Account { lamports, ..Account::default() },
            slot,
            transaction_signature: Some(Signature::from([i as u8 + 1; 64])),
        };
        tx.send(UnifiedMessage::Account(update)).await.unwrap();
    }
    StreamMessageClient::new(rx)
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    Pipeline::builder()
        // Two redundant sources deliver the same writes.
        .datasource_with_id(replica().await, DatasourceId::new_named("primary"))
        .datasource_with_id(replica().await, DatasourceId::new_named("backup"))
        .account_with_filters(
            AnyAccount,
            Print,
            vec![
                Box::new(DeduplicationFilter::new(Duration::from_secs(60))),
                Box::new(SlotRangeFilter::between(0, None, 25, None)),
                Box::new(MinLamports { min: 1_000 }),
            ],
        )
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
slot 10 lamports 5000
```

Six updates arrive. `DeduplicationFilter` rejects the second copy of each write, `SlotRangeFilter` rejects slot 30, and `MinLamports` rejects the 50-lamport account.

## Next steps

- [Use multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Redundant and complementary sources with routing.
- [Filters reference](https://carbon.sevenlabs.org/reference/filters.md): Every filter constructor and method.
- [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md): Count what your pipeline and filters let through.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/filters · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Metrics (Carbon 2.0)

> Carbon's metrics model (Counter, Gauge, Histogram, MetricsRegistry, MetricsExporter), LogMetrics versus Prometheus, and registering your own metrics.

Carbon counts what flows through a pipeline: updates received, processed and failed, queue depth and processing time, plus each datasource's own numbers. You add your own metrics the same way and choose an exporter to publish them. The [metrics reference](https://carbon.sevenlabs.org/reference/metrics.md) lists every built-in metric name.

## The model

Metrics live in one process-wide registry, `MetricsRegistry::global()`. The registry holds `&'static` references to three kinds of metric, all lock-free:

| Type        | Constructor                                | Methods                         | Use for                                                                            |
| ----------- | ------------------------------------------ | ------------------------------- | ---------------------------------------------------------------------------------- |
| `Counter`   | `const fn new(name, help)`                 | `inc()`, `inc_by(u64)`, `get()` | Totals that only go up: updates, errors, rows written.                             |
| `Gauge`     | `const fn new(name, help)`                 | `set(f64)`, `add(f64)`, `get()` | Values that go up and down: queue depth, last slot seen.                           |
| `Histogram` | `fn new(name, help, boundaries: Vec<f64>)` | `record(f64)`, `get()`          | Distributions: latencies, sizes. Counts values per bucket and keeps a running sum. |

Metrics have a name and a help string, and no labels. Updating one is a single relaxed atomic operation, so it's cheap enough for the hot path.

An exporter publishes the registry. Exporters implement `MetricsExporter`:

```rust
pub trait MetricsExporter: Send + Sync {
    fn initialize(self: Arc<Self>) -> CarbonResult<()> { Ok(()) }
    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()>;
    fn shutdown(&self) -> CarbonResult<()> { Ok(()) }
}
```

`MetricsRegistry::global().snapshot()` returns a `MetricsSnapshot` with the current value of every registered metric: `counters` and `gauges` as `(name, help, value)` tuples, and `histograms` with their bucket counts, boundaries and sum.

## How the pipeline uses metrics

- `build()` registers the pipeline's metrics: `carbon_updates_received_total`, `carbon_updates_processed_total`, `carbon_updates_successful_total`, `carbon_updates_failed_total`, `carbon_updates_queued`, two processing-time histograms and one counter per update type. Datasources register theirs when they start, such as the transaction crawler's `transaction_crawler_*` metrics.
- `run()` calls `initialize` on each exporter you added with `.metrics(…)`, updates the pipeline metrics on every update, and calls `export` with a final snapshot followed by `shutdown` when it stops.
- The pipeline doesn't export periodically. Each exporter decides when to publish.

Because the registry is global, every pipeline in a process shares the same metrics.

## Exporters

### Log metrics

`carbon-log-metrics` writes the registry to the `log` crate at `info` level: a summary line, then every metric.

```rust
.metrics(Arc::new(LogMetrics::new()))                       // every 5 seconds
.metrics(Arc::new(LogMetrics::new_with_flush_interval(30))) // every 30 seconds
```

`initialize` starts a task that exports on that interval; `shutdown` stops it. You need a logger, such as `env_logger` with `RUST_LOG=info`, to see the output. Use it in development and for small jobs.

### Prometheus

`carbon-prometheus-metrics` serves the registry in the Prometheus text format:

```rust
use carbon_prometheus_metrics::{PrometheusMetrics, PrometheusServerConfig};

.metrics(Arc::new(PrometheusMetrics::with_server(PrometheusServerConfig::default())))
```

`with_server` starts an HTTP server during `initialize`, on `0.0.0.0:9464` by default, with the metrics at `/metrics`. Each scrape reads the registry directly, so values are always current and `export` does nothing. `PrometheusMetrics::new()` doesn't start a server; pair it with `PrometheusMetrics::start_server(config)` or call `PrometheusMetrics::prometheus_text()` to serve the text yourself. See [Monitor with Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md).

### Your own exporter

Implement `MetricsExporter` to send metrics elsewhere. Spawn a periodic task in `initialize` if you want regular exports, as `LogMetrics` does.

```rust
pub struct StdoutExporter;

impl MetricsExporter for StdoutExporter {
    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()> {
        for (name, _help, value) in &snapshot.counters {
            println!("{name} {value}");
        }
        Ok(())
    }
}
```

You can add several exporters to one pipeline.

## Register your own metrics

1. Declare each metric as a `static`. `Counter` and `Gauge` have `const` constructors; wrap a `Histogram` in `LazyLock`.
2. Register each one once, before the pipeline runs, with `MetricsRegistry::global().register_counter(&METRIC)` (or `register_gauge`, `register_histogram`). Registering a metric twice makes it appear twice in snapshots.
3. Update it anywhere: in processors, filters or your own tasks.

Prefix names with your application's name so they don't collide with Carbon's.

## Example

This pipeline records two custom metrics in its processor and logs every metric when it finishes. Three hand-built account updates stand in for a datasource.

```sh
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio --features full
cargo add env_logger
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::AccountUpdate,
        error::CarbonResult,
        metrics::{Counter, Histogram, MetricsRegistry},
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
    std::sync::{Arc, LazyLock},
};

static LARGE_ACCOUNTS: Counter =
    Counter::new("myapp_large_accounts_total", "Accounts holding at least 1 SOL");

static ACCOUNT_SOL: LazyLock<Histogram> = LazyLock::new(|| {
    Histogram::new("myapp_account_sol", "Account balances in SOL", vec![0.1, 1.0, 10.0])
});

fn register_metrics() {
    let registry = MetricsRegistry::global();
    registry.register_counter(&LARGE_ACCOUNTS);
    registry.register_histogram(&ACCOUNT_SOL);
}

struct AnyAccount;

impl<'a> AccountDecoder<'a> for AnyAccount {
    type AccountType = ();

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<()>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: (),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

struct Measure;

impl Processor<AccountProcessorInputType<'_, ()>> for Measure {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, ()>) -> CarbonResult<()> {
        let sol = input.raw_account.lamports as f64 / 1e9;
        ACCOUNT_SOL.record(sol);
        if sol >= 1.0 {
            LARGE_ACCOUNTS.inc();
        }
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();
    register_metrics();

    let (tx, rx) = tokio::sync::mpsc::channel(16);
    for lamports in [50_000_000, 2_000_000_000, 30_000_000_000] {
        let update = AccountUpdate {
            pubkey: Pubkey::new_unique(),
            account: Account { lamports, ..Account::default() },
            slot: 1,
            transaction_signature: None,
        };
        tx.send(UnifiedMessage::Account(update)).await.unwrap();
    }
    drop(tx);

    Pipeline::builder()
        .datasource(StreamMessageClient::new(rx))
        .account(AnyAccount, Measure)
        .metrics(Arc::new(LogMetrics::new()))
        .build()?
        .run()
        .await
}
```

```sh
RUST_LOG=info cargo run
```

Output:

```text
[2026-11-09T00:08:19Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 1, num_account_deletion_pipes: 0, num_instruction_pipes: 0, num_transaction_pipes: 0
[2026-11-09T00:08:19Z WARN  carbon_stream_message_datasource] Receiver closed
[2026-11-09T00:08:19Z INFO  carbon_stream_message_datasource] Geyser message loop exited
[2026-11-09T00:08:19Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:08:19Z INFO  carbon_log_metrics] 00:00:00 (+616.231µs) | 3 processed (100%), 3 successful, 0 failed (0%), 0 in queue
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   myapp_large_accounts_total: 2
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   carbon_updates_received_total: 3
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   carbon_updates_processed_total: 3
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   carbon_updates_successful_total: 3
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   carbon_updates_failed_total: 0
…
[2026-11-09T00:08:19Z INFO  carbon_log_metrics]   myapp_account_sol: count=3
…
[2026-11-09T00:08:19Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

Two of the three accounts hold at least 1 SOL. The `WARN … Receiver closed` line is the stream message datasource noticing that its input channel closed, which is what ends this finite example. `LogMetrics` prints only the sample count of each histogram; use Prometheus to see the buckets.

## Next steps

- [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md): Every built-in metric and what it measures.
- [Monitor with Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md): Scrape a pipeline and build dashboards and alerts.
- [Run in production](https://carbon.sevenlabs.org/guides/production.md): What to watch and how to size the pipeline.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/metrics · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Shutdown (Carbon 2.0)

> How a pipeline stops, ShutdownStrategy, datasource_cancellation_token, Ctrl-C handling, in-flight and queued updates, and finite datasources.

A pipeline stops in one of three ways: its datasources finish, its cancellation token is cancelled, or the process receives Ctrl-C. This page explains each, what happens to updates that are in flight or queued, and how to stop a pipeline reliably.

## How a pipeline stops

| Trigger                                          | What `run` does                                                                                        | Queued updates       |
| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------ | -------------------- |
| Every datasource finished and dropped its sender | Processes everything left in the channel, exports metrics, shuts the exporters down, returns `Ok(())`. | All processed.       |
| The cancellation token is cancelled              | Finishes the update it's processing, exports metrics, shuts the exporters down, returns `Ok(())`.      | Dropped.             |
| Ctrl-C                                           | Cancels the token, then behaves according to the `ShutdownStrategy`.                                   | Dropped (see below). |

In every case the update being processed when the stop arrives runs to completion: the pipeline never interrupts a processor. When updates are waiting in the channel, the loop may process one or two more before it notices the cancellation, because it picks at random among the events that are ready. `run` doesn't wait for datasource tasks to finish. They see the cancelled token and stop on their own, or are dropped with the Tokio runtime when `main` returns.

## `ShutdownStrategy`

```rust
pub enum ShutdownStrategy {
    Immediate,
    #[default]
    ProcessPending,
}
```

Set it with `.shutdown_strategy(ShutdownStrategy::Immediate)`. The strategy only affects Ctrl-C:

- **`Immediate`**: on Ctrl-C the pipeline cancels the token, exports metrics and returns.
- **`ProcessPending`** (default): on Ctrl-C the pipeline cancels the token, logs `shutting down the pipeline after processing pending updates.` and keeps looping. The intent is to drain the channel, but in 2.0.0 the loop also watches the token it has cancelled and exits as soon as it sees it, so at most a few more updates are processed. Don't rely on `ProcessPending` to drain the queue.

Neither strategy affects a cancellation that comes from your own token: the pipeline returns after the current update.

## Stop with a cancellation token

Pass a `tokio_util::sync::CancellationToken` to the builder and cancel it when you want to stop:

```rust
let shutdown = CancellationToken::new();

let pipeline = Pipeline::builder()
    .datasource(datasource)
    .instruction(decoder, processor)
    .datasource_cancellation_token(shutdown.clone())
    .build()?;
```

The pipeline hands the same token to every datasource, so cancelling it stops the sources and the processing loop together. Cancel it from a signal handler, a timer, an admin endpoint or another part of your program. A cancelled token stays cancelled, so the pipeline sees it at its next iteration whenever you cancel it.

To run for a fixed time, for example in a test:

```rust
let stop = shutdown.clone();
tokio::spawn(async move {
    tokio::time::sleep(Duration::from_secs(30)).await;
    stop.cancel();
});
```

## Ctrl-C

The pipeline listens for Ctrl-C itself, but only while it waits for the next update. In 2.0.0 a Ctrl-C that arrives while a processor is running is lost, and because Tokio has replaced the default signal handler, the process doesn't exit either. A busy pipeline is almost always running a processor, so Ctrl-C can appear to do nothing.

Handle Ctrl-C in a task of your own and cancel the token from there, as the example below does. Your task waits for the signal the whole time, so it never misses it.

## Finite datasources

Some datasources end on their own: `getProgramAccounts` after one fetch, the validator snapshot after its last account, Helius GTFA after the last page, the block crawler at its `end_slot`, and custom datasources that return. When the last finite datasource drops its sender, the pipeline processes every remaining update and returns `Ok(())`. This is the only way to drain the channel completely.

Others never end: Geyser streams, websocket subscriptions, the block crawler without an `end_slot`, and the transaction crawler, which keeps polling for new transactions after it reaches the end of history. A pipeline with any of these runs until you cancel it.

A datasource whose `consume` returns early but leaves spawned tasks holding the sender (as many published ones do) still counts as running until those tasks end.

## Example

The `Ticker` datasource below sends a `BlockDetails` update every 400 ms until it's cancelled. `main` handles Ctrl-C in its own task and cancels the pipeline's token.

```sh
cargo add carbon-core@2.0.0 async-trait tokio-util
cargo add tokio --features full
```

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{BlockDetails, Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    std::time::Duration,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Emits one `BlockDetails` per 400 ms until cancelled.
struct Ticker;

#[async_trait]
impl Datasource for Ticker {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        let mut interval = tokio::time::interval(Duration::from_millis(400));
        for slot in 0.. {
            tokio::select! {
                _ = cancellation_token.cancelled() => break,
                _ = interval.tick() => {
                    let update = Update::BlockDetails(BlockDetails {
                        slot,
                        block_hash: None,
                        previous_block_hash: None,
                        rewards: None,
                        num_reward_partitions: None,
                        block_time: None,
                        block_height: None,
                    });
                    if sender.send((update, id.clone())).await.is_err() {
                        break;
                    }
                }
            }
        }
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::BlockDetails]
    }
}

struct PrintSlot;

impl Processor<BlockDetails> for PrintSlot {
    async fn process(&mut self, block: &BlockDetails) -> CarbonResult<()> {
        println!("slot {}", block.slot);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let shutdown = CancellationToken::new();

    // Listen for Ctrl-C in a task of its own so the signal is never missed.
    let on_ctrl_c = shutdown.clone();
    tokio::spawn(async move {
        tokio::signal::ctrl_c().await.ok();
        println!("Ctrl-C received, stopping");
        on_ctrl_c.cancel();
    });

    Pipeline::builder()
        .datasource(Ticker)
        .block_details(PrintSlot)
        .datasource_cancellation_token(shutdown)
        .build()?
        .run()
        .await?;

    println!("pipeline stopped");
    Ok(())
}
```

Run it and press Ctrl-C after a second or two:

```sh
cargo run
```

Output:

```text
slot 0
slot 1
slot 2
slot 3
Ctrl-C received, stopping
pipeline stopped
```

## Next steps

- [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md): What the pipeline does when processors and datasources fail.
- [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md): The full processing loop, step by step.
- [Run in production](https://carbon.sevenlabs.org/guides/production.md): Deploy, monitor and restart pipelines safely.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/shutdown · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Error handling (Carbon 2.0)

> CarbonError and CarbonResult, what the pipeline does with processor, datasource and exporter errors, and patterns for context, retries and failing fast.

Carbon uses one error type across the framework, and the pipeline treats errors differently depending on where they come from. This page explains both, and shows how to add context, retry and stop on failure. Each error variant is described in the [errors reference](https://carbon.sevenlabs.org/reference/errors.md).

## `CarbonError` and `CarbonResult`

Carbon's error type is `carbon_core::error::Error`, usually imported as `CarbonError`, and `CarbonResult<T>` is `Result<T, Error>`. Processors, datasources, exporters and the builder all return `CarbonResult`.

```rust
pub enum Error {
    MissingUpdateTypeInDatasource(UpdateType),
    FailedToReceiveUpdates(String),
    MissingFeePayer,
    MissingInnerInstructions,
    MissingAccountInTransaction,
    MissingInstructionData,
    FailedToConsumeDatasource(String),
    Custom(String),
}
```

In 2.0.0 Carbon itself produces only a few of these:

- `MissingFeePayer` when a transaction has no account keys;
- `FailedToConsumeDatasource` from datasources that can't start, such as a failed `getProgramAccounts` call or an invalid gRPC endpoint;
- `Custom` from datasources and the Postgres layer, wrapping errors from other crates;
- `MissingAccountInTransaction` from the public `transformers::extract_account_metas` helper.

Use `Custom(String)` for your own errors. `Error` implements `std::error::Error` and `Display` through `thiserror`, so `?` works in a `main` that returns `CarbonResult<()>`.

## What the pipeline does with errors

| Where the error happens                                            | What happens                                                                                                                                                                                                          |
| ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| A processor returns `Err`                                          | The pipeline logs `error processing update ({update:?}): {error:?}` at `error` level, increments `carbon_updates_failed_total` and skips the remaining pipes for that update. It then continues with the next update. |
| Converting a transaction fails (`MissingFeePayer`)                 | Same as a processor error, before any pipe runs.                                                                                                                                                                      |
| A datasource's `consume` returns `Err`                             | The pipeline logs `error consuming datasource: {error:?}`. That datasource stops; the others continue. If it was the last one, the pipeline stops.                                                                    |
| An exporter's `initialize` returns `Err`                           | `run` returns the error before any datasource starts.                                                                                                                                                                 |
| An exporter's `export` or `shutdown` returns `Err` during shutdown | `run` returns the error.                                                                                                                                                                                              |
| A processor panics                                                 | The panic isn't caught. It unwinds out of `run` and, in a `#[tokio::main]` program, ends the process.                                                                                                                 |

Nothing is retried, and in particular a failed update isn't re-delivered. "Skips the remaining pipes" means pipes registered after the failing one don't see that update. For a transaction, the failing instruction pipe also stops at that instruction, so later instructions, later instruction pipes and all transaction pipes are skipped for that transaction. See [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md#errors).

Network errors inside datasources usually never reach the pipeline. Datasources retry, back off and reconnect on their own, logging warnings as they go; the RPC transaction crawler, for example, retries each request three times with exponential backoff by default. Check each datasource's page for its retry settings.

## Patterns

### Add context to errors

The pipeline's log line includes the whole update in `Debug` form, which for a transaction is very long, and your error message is at the end. Put the identifiers you need in the error itself:

```rust
let label = std::str::from_utf8(&input.decoded_account.data).map_err(|e| {
    CarbonError::Custom(format!(
        "account {} at slot {}: {e}",
        input.metadata.pubkey, input.metadata.slot
    ))
})?;
```

For instruction processors, include `input.metadata.transaction_metadata.signature` and `input.metadata.absolute_path`. If you'd rather keep the log short, log the error yourself with that context and return `Ok(())`; the update then counts as successful in the metrics.

### Retry transient failures

The pipeline won't retry for you, so retry inside `process` when an operation can succeed on a later attempt, such as a database write during a failover. This helper retries an async operation three times with a doubling delay:

```rust
pub async fn with_retries<T, E, F, Fut>(mut operation: F) -> CarbonResult<T>
where
    E: std::fmt::Display,
    F: FnMut() -> Fut,
    Fut: std::future::Future<Output = Result<T, E>>,
{
    let mut delay = std::time::Duration::from_millis(200);
    let mut attempt = 1;
    loop {
        match operation().await {
            Ok(value) => return Ok(value),
            Err(e) if attempt < 3 => {
                log::warn!("attempt {attempt} failed, retrying in {delay:?}: {e}");
                tokio::time::sleep(delay).await;
                delay *= 2;
                attempt += 1;
            }
            Err(e) => return Err(CarbonError::Custom(format!("gave up after {attempt} attempts: {e}"))),
        }
    }
}
```

While `process` retries, the pipeline waits: no other update is processed. Keep retry budgets short, or hand the work to a task you own (see [Processors](https://carbon.sevenlabs.org/concepts/processors.md#share-state-with-the-rest-of-your-program)).

### Stop the pipeline on an error

By default a pipeline keeps going after errors. When a failure means the indexed data is no longer trustworthy, stop instead: give the processor a clone of the pipeline's [cancellation token](https://carbon.sevenlabs.org/concepts/shutdown.md#stop-with-a-cancellation-token) and cancel it.

```rust
pub struct StopOnError {
    shutdown: tokio_util::sync::CancellationToken,
}

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for StopOnError {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let signature = input.metadata.transaction_metadata.signature;
        let result = with_retries(|| async { Ok::<_, std::io::Error>(()) }).await;
        if let Err(error) = result {
            log::error!("tx {signature} path {:?}: {error}", input.metadata.absolute_path);
            self.shutdown.cancel(); // the pipeline stops after this update
            return Err(error);
        }
        Ok(())
    }
}
```

Replace the placeholder operation with your own write. After the pipeline stops, `run` returns `Ok(())`; record the failure somewhere your `main` can check if the process should exit with an error.

### Watch the failure rate

`carbon_updates_failed_total` counts failed updates, and `LogMetrics` prints the failure percentage on every flush. Alert on it in production; see [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md).

## Example

This pipeline parses each account's data as a UTF-8 label. The second account's data isn't valid UTF-8, so `ParseLabel` returns an error with context. The pipeline logs it, skips the `Audit` pipe for that update and carries on.

```sh
cargo add carbon-core@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio --features full
cargo add env_logger
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::AccountUpdate,
        error::{CarbonResult, Error as CarbonError},
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
};

struct RawData;

impl<'a> AccountDecoder<'a> for RawData {
    type AccountType = Vec<u8>;

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<Vec<u8>>> {
        Some(DecodedAccount {
            lamports: account.lamports,
            data: account.data.clone(),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

/// Parses the account data as UTF-8 and fails on anything else.
struct ParseLabel;

impl Processor<AccountProcessorInputType<'_, Vec<u8>>> for ParseLabel {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, Vec<u8>>) -> CarbonResult<()> {
        let label = std::str::from_utf8(&input.decoded_account.data).map_err(|e| {
            CarbonError::Custom(format!(
                "account {} at slot {}: {e}",
                input.metadata.pubkey, input.metadata.slot
            ))
        })?;
        println!("slot {}: label {label:?}", input.metadata.slot);
        Ok(())
    }
}

struct Audit;

impl Processor<AccountProcessorInputType<'_, Vec<u8>>> for Audit {
    async fn process(&mut self, input: &AccountProcessorInputType<'_, Vec<u8>>) -> CarbonResult<()> {
        println!("slot {}: audited", input.metadata.slot);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let (tx, rx) = tokio::sync::mpsc::channel(16);
    for (slot, data) in [(1, b"alpha".to_vec()), (2, vec![0xff, 0xfe]), (3, b"gamma".to_vec())] {
        let update = AccountUpdate {
            pubkey: Pubkey::new_from_array([slot as u8; 32]),
            account: Account { lamports: 1, data, ..Account::default() },
            slot,
            transaction_signature: None,
        };
        tx.send(UnifiedMessage::Account(update)).await.unwrap();
    }
    drop(tx);

    Pipeline::builder()
        .datasource(StreamMessageClient::new(rx))
        .account(RawData, ParseLabel) // runs first
        .account(RawData, Audit) // skipped for an update when ParseLabel fails
        .build()?
        .run()
        .await
}
```

```sh
RUST_LOG=error cargo run
```

Output:

```text
slot 1: label "alpha"
slot 1: audited
[2026-11-09T00:08:56Z ERROR carbon_core::pipeline] error processing update (Account(AccountUpdate { pubkey: 8qbHbw2BbbTHBW1sbeqakYXVKRQM8Ne7pLK7m6CVfeR, account: Account { lamports: 1, data.len: 2, owner: 11111111111111111111111111111111, executable: false, rent_epoch: 0, data: fffe }, slot: 2, transaction_signature: None })): Custom("account 8qbHbw2BbbTHBW1sbeqakYXVKRQM8Ne7pLK7m6CVfeR at slot 2: invalid utf-8 sequence of 1 bytes from index 0")
slot 3: label "gamma"
slot 3: audited
```

## Next steps

- [Errors reference](https://carbon.sevenlabs.org/reference/errors.md): Every error variant: message, cause and fix.
- [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md): Fixes for common problems, by symptom.
- [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md): Stop a pipeline cleanly, on a signal or on failure.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/concepts/error-handling · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Generate a decoder (Carbon 2.0)

> Generate a Carbon decoder crate from an Anchor IDL, a Codama IDL or a program address with carbon-cli parse, then use it in a pipeline.

When Carbon has no [published decoder](https://carbon.sevenlabs.org/decoders.md) for the program you want to index, generate one from the program's IDL with `carbon-cli parse`. You get a Rust crate with typed instructions, accounts and events that plugs into any pipeline. This guide generates a decoder for the Pump.fun AMM program and runs it against mainnet.

**You need:**

- Node.js 20.18.0 or newer, to run the CLI with `npx`, `pnpm`, `bun` or `yarn`.
- Rust 1.96.1 or newer, with `rustfmt` (`rustup component add rustfmt`). The CLI runs `cargo fmt` on the generated code and only warns if it's missing.
- The program's IDL: a program address whose Anchor IDL is stored on chain (as in this guide), an Anchor IDL file, or a Codama IDL file. No account or API key is needed.

### Step 1: Create a project

```sh
cargo new my-indexer
cd my-indexer
```

### Step 2: Generate the decoder crate

Pump.fun AMM stores its Anchor IDL on chain, so the CLI can read it from the program address. `--url` tells it which RPC endpoint to read from; `mainnet-beta` is short for Solana's public endpoint. `--with-postgres false --with-graphql false` leaves out the Postgres and GraphQL modules, which you don't need to decode.

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --url mainnet-beta --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --url mainnet-beta --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --url mainnet-beta --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --url mainnet-beta --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

After a banner, the CLI prints:

Output:

```text
✔ Decoder generated
✔ Code formatted successfully

✨ Decoder Generated Successfully! ✨
  • Output: /home/you/my-indexer/decoders/pump-amm
  • Source: program-address (anchor) @ https://api.mainnet-beta.solana.com
```

> **Warning: parse deletes the output directory.** The CLI deletes `--out-dir` and recreates it on every run. Point it at a directory that holds only generated code, never `.`, or pass `--no-clean`.

The crate is named `carbon-<name>-decoder`, where `<name>` is the IDL's program name in kebab case (`pump_amm` gives `carbon-pump-amm-decoder`):

```text
decoders/pump-amm/
├── Cargo.toml
└── src/
    ├── lib.rs           # PumpAmmDecoder, PROGRAM_ID, EVENT_CPI_DISCRIMINATOR
    ├── accounts/        # PumpAmmAccount enum, one file per account type
    ├── instructions/    # PumpAmmInstruction enum, one file per instruction, cpi_event.rs
    ├── events/          # one file per event
    └── types/           # the IDL's defined types
```

### Step 3: Check that it compiles

```sh
cargo check --manifest-path decoders/pump-amm/Cargo.toml
```

Output:

```text
    Checking carbon-core v2.0.0
    Checking carbon-pump-amm-decoder v0.1.0 (/home/you/my-indexer/decoders/pump-amm)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.56s
```

### Step 4: Add it to the project

Depend on the crate by path, next to the Carbon crates. The generated `Cargo.toml` has its own empty `[workspace]` table, so it builds as a separate package. That works in a single-package project like this one; inside a Cargo workspace, see [Generate into a Cargo workspace](#generate-into-a-cargo-workspace).

`Cargo.toml`:

```toml
[package]
name = "my-indexer"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-pump-amm-decoder = { path = "decoders/pump-amm" }
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 5: Use the decoder in a pipeline

The generated `PumpAmmDecoder` implements `InstructionDecoder` (and `AccountDecoder`), so it goes straight into `.instruction()`. Each instruction is a struct variant of `PumpAmmInstruction` with typed `data` and named `accounts`. Events that the program emits through a self-CPI arrive as the `CpiEvent` variant.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_pump_amm_decoder::{
        instructions::{CpiEvent, PumpAmmInstruction},
        PumpAmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(PumpAmmDecoder, PumpAmmLogger)
        .build()?
        .run()
        .await
}

struct PumpAmmLogger;

impl Processor<InstructionProcessorInputType<'_, PumpAmmInstruction>> for PumpAmmLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, PumpAmmInstruction>,
    ) -> CarbonResult<()> {
        let signature = input.metadata.transaction_metadata.signature;
        match input.decoded_instruction {
            PumpAmmInstruction::Buy { data, .. } => {
                println!("buy  {signature}: base_amount_out={}", data.base_amount_out);
            }
            PumpAmmInstruction::Sell { data, .. } => {
                println!("sell {signature}: base_amount_in={}", data.base_amount_in);
            }
            PumpAmmInstruction::CpiEvent { data: CpiEvent::BuyEvent(event), .. } => {
                println!("  buy event: quote_amount_in={}", event.quote_amount_in);
            }
            PumpAmmInstruction::CpiEvent { data: CpiEvent::SellEvent(event), .. } => {
                println!("  sell event: quote_amount_out={}", event.quote_amount_out);
            }
            _ => {}
        }
        Ok(())
    }
}
```

### Step 6: Run it

```sh
cargo run
```

The first build takes one to three minutes. Then decoded trades print, newest transactions first:

Output:

```text
sell QKcP663tvCWRqXgndwxYz5TZ5Vq42QW5g9wvLGeht4LhAwd7fY9Nj2RnsX4f6q8pdzxQsLHzswbyFaTu4PasJdU: base_amount_in=7038649829
  sell event: quote_amount_out=79229627543477
buy  QKcP663tvCWRqXgndwxYz5TZ5Vq42QW5g9wvLGeht4LhAwd7fY9Nj2RnsX4f6q8pdzxQsLHzswbyFaTu4PasJdU: base_amount_out=7000317342
  buy event: quote_amount_in=78796291039434
sell 52CfxhHXHoqx62yFsNy4AeXz8TxjGn8Zo6CJKJdcieJZBBAaRXUcZ5GcEVQrWPvAT6c9tFmR53zxoTgNfCvQGRzP: base_amount_in=10549842158
  sell event: quote_amount_out=339272578
…
```

An event line without a `buy` or `sell` line above it comes from another instruction that emits the same event, such as `BuyExactQuoteIn`. Press `Ctrl+C` to stop.

## Verify the decoder

Run the pipeline against the program and compare a few decoded values with a block explorer: pick a signature from the output, open the transaction, and check the instruction's arguments and accounts. Accounts decode the same way: register the decoder with `.account(PumpAmmDecoder, processor)` and a datasource that emits account updates, such as [`rpc-gpa`](https://carbon.sevenlabs.org/guides/load-account-state.md).

If an instruction never shows up, the IDL may not match the deployed program: an outdated IDL decodes old instructions and returns `None` for new ones, which the pipeline skips silently. Regenerate from the IDL that matches the deployed version.

## From an Anchor IDL file

Pass a path ending in `.json` instead of an address. Anchor writes your own program's IDL to `target/idl/<program>.json` on `anchor build`; for other programs, use the IDL their authors publish. The CLI reads the program ID from the IDL's `address` field:

`pump_amm.json`:

```json
{
  "address": "pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA",
  "metadata": {
    "name": "pump_amm",
    "version": "0.1.0",
    "spec": "0.1.0",
    "description": "Created with Anchor"
  },
  "instructions": [
    …
  ]
}
```

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./pump_amm.json --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./pump_amm.json --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./pump_amm.json --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./pump_amm.json --out-dir decoders/pump-amm --with-postgres false --with-graphql false
```

Output:

```text
✨ Decoder Generated Successfully! ✨
  • Output: /home/you/my-indexer/decoders/pump-amm
  • Source: file (anchor)
```

The crate is identical to the one generated from the address. Program addresses work only for Anchor programs that stored their IDL on chain with `anchor idl init`; `--url` is required with them and takes a full URL or the aliases `mainnet-beta` and `devnet`.

## From a Codama IDL

Pass `-s codama` for a Codama IDL (a JSON file whose root has `"kind": "rootNode"`). The CLI looks for the program ID only in a top-level `address` field, which Codama IDLs don't have, so pass it with `--program-id`:

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./codama.json --standard codama --program-id pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --out-dir decoders/pump-amm
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./codama.json --standard codama --program-id pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --out-dir decoders/pump-amm
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./codama.json --standard codama --program-id pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --out-dir decoders/pump-amm
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./codama.json --standard codama --program-id pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --out-dir decoders/pump-amm
```

Events in a Codama IDL come from its `eventNode`s (`program.events`). The generator decodes events whose discriminators are an event-CPI prefix followed by the event discriminator, or a single discriminator repeated as a hidden prefix on the event data. For the hidden-prefix form, `--event-cpi-discriminator <hex>` sets the CPI envelope; it defaults to Anchor's `e445a52e51cb9a1d`. An IDL without `eventNode`s produces a decoder without events.

> **Note: --event-hints has no effect in 2.0.0.** The CLI accepts `--event-hints "BuyEvent,SellEvent"` with `-s codama` (and rejects it with `anchor`), but CLI 2.0.0 doesn't pass the hints to the generator: the output is identical with or without them. Declare events as `eventNode`s in the IDL instead.

## Options

| Flag                                | What it changes                                                                                                                                                                       |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--name <name>`                     | Crate and type names. `--name my-amm` gives the crate `carbon-my-amm-decoder` with `MyAmmDecoder`, `MyAmmInstruction` and `MyAmmAccount`. Without it, the IDL's program name is used. |
| `--with-postgres`, `--with-graphql` | `true` by default. Each adds generated modules and a Cargo feature of the same name.                                                                                                  |
| `--with-serde`                      | Adds the `serde` feature (derives `Serialize` and `Deserialize` on every type). On by default when both flags above are `false`.                                                      |
| `--with-base58`                     | Adds a `base58` feature that serializes `Pubkey` fields as base58 strings.                                                                                                            |
| `--standalone false`                | Removes the `[workspace]` table and writes every dependency as `{ workspace = true }`, for a crate that lives in a Cargo workspace. See below.                                        |
| `--version-name <v>`                | Appends `-<v>` to the crate name, for [versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md).                                                                |
| `--no-clean`                        | Writes over `--out-dir` instead of deleting it first.                                                                                                                                 |
| `-c`, `--as-crate`                  | Accepted, but has no effect: the output is always a crate.                                                                                                                            |

Boolean flags take a value: `--with-postgres false`. Every flag is in the [CLI reference](https://carbon.sevenlabs.org/reference/cli.md#parse).

### Feature flags

The generated crate has `default = []`, and each feature is opt-in from the crate that depends on it:

`Cargo.toml`:

```toml
[dependencies]
carbon-pump-amm-decoder = { path = "decoders/pump-amm", features = ["serde"] }
```

| Feature    | Enables                                                                                                                                           | Exists when generated with                |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| `serde`    | `Serialize` and `Deserialize` on instructions, accounts, events and types                                                                         | `--with-serde true`, or any of the others |
| `postgres` | Row types, migrations and processors for [Postgres](https://carbon.sevenlabs.org/guides/postgres.md); turns on `serde` and `carbon-core/postgres` | `--with-postgres true` (default)          |
| `graphql`  | GraphQL schema types for [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md); turns on `serde` and `carbon-core/graphql`                    | `--with-graphql true` (default)           |
| `base58`   | Base58 strings for `Pubkey` fields when serialized; turns on `serde`                                                                              | `--with-base58 true`                      |

See [Feature flags](https://carbon.sevenlabs.org/reference/feature-flags.md) for the Carbon crates' own features.

### Generate into a Cargo workspace

With `--standalone false`, the crate inherits `edition`, `rust-version` and its dependencies from the workspace. Add it to `members`, and define every dependency it uses in the workspace root:

`Cargo.toml`:

```toml
[workspace]
members = ["indexer", "decoders/pump-amm"]
resolver = "2"

[workspace.package]
edition = "2021"
rust-version = "1.96.1"

[workspace.dependencies]
carbon-core = { version = "2.0.0", default-features = false }
carbon-test-utils = "2.0.0"
borsh = { version = "1.5.1", features = ["derive"] }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.145"
solana-account = "4.3.2"
solana-instruction = { version = "3.4.1", default-features = false }
solana-pubkey = "4.2.1"
```

A crate generated with Postgres or GraphQL also needs `sqlx`, `sqlx_migrator`, `async-trait` and `juniper` there.

To keep a standalone crate (the default) inside a workspace directory instead, add it to the workspace's `exclude` list, and depend on it by path from your members:

`Cargo.toml`:

```toml
[workspace]
members = ["indexer"]
exclude = ["decoders/pump-amm"]
resolver = "2"
```

## Scaffold a whole project

`carbon-cli scaffold` generates a Cargo workspace with the decoder in `decoder/` and a runnable indexer in `indexer/`, wired to one of five datasources (`rpc-transaction-crawler`, `rpc-block-subscribe`, `rpc-program-subscribe`, `yellowstone-grpc`, `helius-laserstream`):

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 scaffold --name pump-indexer --out-dir . --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --idl-url mainnet-beta --idl-standard anchor --data-source rpc-transaction-crawler --with-postgres false --with-graphql false
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 scaffold --name pump-indexer --out-dir . --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --idl-url mainnet-beta --idl-standard anchor --data-source rpc-transaction-crawler --with-postgres false --with-graphql false
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 scaffold --name pump-indexer --out-dir . --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --idl-url mainnet-beta --idl-standard anchor --data-source rpc-transaction-crawler --with-postgres false --with-graphql false
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 scaffold --name pump-indexer --out-dir . --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA --idl-url mainnet-beta --idl-standard anchor --data-source rpc-transaction-crawler --with-postgres false --with-graphql false
```

```text
pump-indexer/
├── Cargo.toml        # workspace: indexer, decoder
├── .env              # RPC_URL=your-rpc-url-here
├── decoder/          # carbon-pump-amm-decoder
└── indexer/
    └── src/main.rs   # pipeline with a stub processor
```

Set the variables in `.env`, then `cargo run` in `pump-indexer/`. The [CLI reference](https://carbon.sevenlabs.org/reference/cli.md#scaffold) lists the flags, what each datasource template needs and the known issues.

## Common problems

### `Error: RPC URL is required when using a program address`

`--idl` is a program address and `--url` is missing. Add `--url mainnet-beta`, `--url devnet` or a full RPC URL.

### `Error: No Anchor IDL found for program address.`

The program never stored its IDL on chain with `anchor idl init`, or it isn't an Anchor program. Get the IDL file from the program's repository or authors and pass its path.

### `Error: Failed to fetch Anchor IDL: … 429` or `… 413 … You have used your data allowance`

The public RPC endpoint rate-limited your IP. Wait a minute, or pass your own RPC URL with `--url`.

### `Error: Program ID is required. IDL missing or invalid address field`

The IDL has no top-level `address` (all Codama IDLs, and old Anchor IDLs that keep it elsewhere). Pass `--program-id <address>`.

### `Error: Event hints can only be used with Codama standard`

`--event-hints` was passed without `-s codama`. Anchor IDLs carry their events; remove the flag.

### `Failed to format code` warning

`cargo fmt` isn't installed. The crate is still generated, unformatted. Run `rustup component add rustfmt` and generate again.

### `error: multiple workspace roots found in the same workspace`

A standalone decoder crate (with its own `[workspace]` table) is inside a Cargo workspace's directory, either in `members` or as a path dependency. Add it to the workspace's `exclude` list, or regenerate it with `--standalone false` and add it to `members`. See [Generate into a Cargo workspace](#generate-into-a-cargo-workspace).

### Instructions you expect never reach the processor

The decoder returns `None` for instructions whose discriminator or account count doesn't match the IDL, and the pipeline skips them. Check that the IDL matches the deployed program, and that your datasource delivers that program's transactions.

## Next steps

- [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md): How generated decoders match discriminators and arrange accounts.
- [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md): Feed the decoder real-time transactions.
- [Multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md): Run your decoder next to others in one pipeline.
- [CLI reference](https://carbon.sevenlabs.org/reference/cli.md): Every parse and scaffold flag.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/generate-a-decoder · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Stream with Yellowstone gRPC (Carbon 2.0)

> Index transactions and account changes in real time from a Yellowstone (Dragon's Mouth) Geyser gRPC endpoint, with filters, commitment and reconnects.

Yellowstone gRPC streams transactions and account writes from a validator's Geyser plugin as they happen. It's the lowest-latency way to feed a Carbon pipeline. This guide builds an indexer that decodes Meteora DLMM swaps from a Yellowstone stream, then covers filters, commitment and reconnect handling.

**You need:**

- Rust 1.96.1 or newer.
- A Yellowstone gRPC endpoint URL and, for most providers, an access token (sent as the `x-token` header). Most Solana RPC providers sell Yellowstone (also called Dragon's Mouth) access; you can also run the plugin on your own node. The public Solana endpoint doesn't offer it.

### Step 1: Create a project

```sh
cargo new dlmm-stream
cd dlmm-stream
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-stream"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-yellowstone-grpc-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
rustls = { version = "0.23", default-features = false, features = ["std", "ring"] }
tokio = { version = "1.54.1", features = ["full"] }
yellowstone-grpc-proto = { version = "=12.7.0", default-features = false }
```

You build the subscription filters with types from `yellowstone-grpc-proto`, so pin it to `=12.7.0`, the version `carbon-yellowstone-grpc-datasource` 2.0.0 uses. A different version compiles a second copy of the types and the constructor rejects them. `rustls` is there to install a TLS crypto provider; see the next step.

### Step 3: Write the pipeline

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_yellowstone_grpc_datasource::{
        BlockFilters, YellowstoneGrpcClientConfig, YellowstoneGrpcGeyserClient,
    },
    std::{collections::HashMap, env, sync::Arc},
    yellowstone_grpc_proto::geyser::{CommitmentLevel, SubscribeRequestFilterTransactions},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // The gRPC client uses rustls; pick its crypto provider before connecting.
    rustls::crypto::ring::default_provider()
        .install_default()
        .expect("failed to install the rustls crypto provider");

    // Successful, non-vote transactions that invoke Meteora DLMM.
    let mut transaction_filters = HashMap::new();
    transaction_filters.insert(
        "meteora_dlmm".to_string(),
        SubscribeRequestFilterTransactions {
            vote: Some(false),
            failed: Some(false),
            account_required: vec![PROGRAM_ID.to_string()],
            ..Default::default()
        },
    );

    let datasource = YellowstoneGrpcGeyserClient::new(
        env::var("GEYSER_URL").expect("GEYSER_URL must be set"),
        env::var("X_TOKEN").ok(),
        Some(CommitmentLevel::Confirmed),
        HashMap::new(), // account filters
        transaction_filters,
        BlockFilters::default(),
        YellowstoneGrpcClientConfig::default(),
        None, // disconnect notifier
        None, // stream timeout (default 30 s)
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => {
                println!("slot {} swap  {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            MeteoraDlmmInstruction::Swap2 { data, .. } => {
                println!("slot {} swap2 {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}
```

The constructor's arguments, in order: endpoint, token, commitment, account filters, transaction filters, block filters, client config, a channel for disconnect events, and the stream timeout.

### Step 4: Run it

Pass your provider's endpoint and token. Use the full URL with its scheme and port, as your provider gives it:

```sh
GEYSER_URL="https://your-yellowstone-endpoint:443" X_TOKEN="your-x-token" cargo run
```

Each decoded swap prints one line, a second or so after the transaction lands:

Output:

```text
slot 467110307 swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
slot 467110307 swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
slot 467110307 swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
slot 467110308 swap2 2miT4n9yFXZMdUTVF4My1RVvHrbbZMwD64vYyppzdkxNg5qVtYJgsm4dL8MPYtBRrJm2V3EzP4ZfYJKjEc61Wzr1: amount_in=28328938705735
…
```

The stream runs until you press `Ctrl+C`.

## Verify it's streaming

Run with `RUST_LOG=info`. Every five seconds `LogMetrics` logs the pipeline counters, and the datasource's own counter, `yellowstone_grpc_transaction_updates_received_total`, should keep growing:

```sh
RUST_LOG=info GEYSER_URL="https://your-yellowstone-endpoint:443" X_TOKEN="your-x-token" cargo run
```

If the counter stays at 0, the subscription is accepted but nothing matches your filters. If you see `Geyser stream error` lines instead, see [Common problems](#common-problems).

## Filter transactions

The transaction filter map's keys are labels of your choice; the server sends a transaction if it matches any entry. Within one `SubscribeRequestFilterTransactions`, all the fields you set must match:

| Field              | Matches                                                                                             |
| ------------------ | --------------------------------------------------------------------------------------------------- |
| `vote`             | `Some(false)` excludes vote transactions, `Some(true)` sends only votes, `None` sends both.         |
| `failed`           | `Some(false)` excludes failed transactions, `Some(true)` sends only failed ones, `None` sends both. |
| `account_include`  | Transactions that mention **any** of these accounts or programs.                                    |
| `account_required` | Transactions that mention **all** of these accounts or programs.                                    |
| `account_exclude`  | Drops transactions that mention any of these.                                                       |
| `signature`        | One transaction by signature.                                                                       |

"Mention" includes accounts loaded through address lookup tables, so it catches a program called through a CPI, such as a Jupiter route through a DLMM pool. Carbon then decodes nested instructions too (see [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md)). To follow several programs with one subscription, list them in `account_include`: see [Multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md#choose-a-datasource-filter).

Transaction updates carry the slot and the transaction's index in the block, but no block time: `transaction_metadata.block_time` is `None` for them.

## Stream account changes

Account filters send an account's new state every time a transaction writes to it. This filter streams every Meteora DLMM `LbPair` account (the pools), selected by owner and by the account's 8-byte discriminator:

```rust
use yellowstone_grpc_proto::geyser::{
    subscribe_request_filter_accounts_filter::Filter as AccountsFilter,
    subscribe_request_filter_accounts_filter_memcmp::Data as MemcmpData,
    SubscribeRequestFilterAccounts, SubscribeRequestFilterAccountsFilter,
    SubscribeRequestFilterAccountsFilterMemcmp,
};

let mut account_filters = HashMap::new();
account_filters.insert(
    "dlmm_pairs".to_string(),
    SubscribeRequestFilterAccounts {
        owner: vec![PROGRAM_ID.to_string()],
        filters: vec![SubscribeRequestFilterAccountsFilter {
            filter: Some(AccountsFilter::Memcmp(SubscribeRequestFilterAccountsFilterMemcmp {
                offset: 0,
                data: Some(MemcmpData::Bytes(vec![33, 11, 49, 98, 181, 101, 177, 13])),
            })),
        }],
        ..Default::default()
    },
);
```

Pass `account_filters` as the fourth argument and register an account pipe, `.account(MeteoraDlmmDecoder, processor)`, whose processor takes `AccountProcessorInputType<'_, MeteoraDlmmAccount>`. `account` selects accounts by address, `owner` by owning program, and `filters` narrows further with `Memcmp`, `Datasize`, `Lamports` or `TokenAccountState`. Set `nonempty_txn_signature: Some(true)` to skip writes that don't come from a transaction.

Account updates written by a transaction carry its signature, so a [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) can drop repeats. An update with zero lamports reaches your pipeline as an [account deletion](https://carbon.sevenlabs.org/concepts/updates.md#account-deletions), so add `.account_deletions(processor)` if you track closures.

## Stream whole blocks

`BlockFilters` subscribes to blocks instead. Each block's transactions and accounts become ordinary transaction and account updates, with the block's time set:

```rust
use yellowstone_grpc_proto::geyser::SubscribeRequestFilterBlocks;

let mut filters = HashMap::new();
filters.insert(
    "dlmm_blocks".to_string(),
    SubscribeRequestFilterBlocks {
        account_include: vec![PROGRAM_ID.to_string()],
        include_transactions: Some(true),
        ..Default::default()
    },
);
let block_filters = BlockFilters { filters, failed_transactions: Some(false) };
```

`failed_transactions: Some(false)` drops failed transactions; `None` or `Some(true)` keeps them. Not every provider enables block subscriptions.

## Choose a commitment

| `CommitmentLevel` | When updates arrive                                                             | Trade-off                                                                                |
| ----------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `Processed`       | As soon as the node executes the transaction                                    | Fastest. The slot can still be skipped, so you may see transactions that never finalize. |
| `Confirmed`       | After a supermajority of stake voted on the block, usually under a second later | Rarely rolled back. A good default for indexers.                                         |
| `Finalized`       | When 31 more blocks are confirmed on top, roughly 13 seconds later              | Never rolled back.                                                                       |

`None` uses the server's default.

## Handle disconnects

By default, when the stream ends, errors or goes silent for `stream_timeout` (30 seconds unless you pass another `Duration`), the datasource subscribes again with the same filters. A failed subscribe attempt is retried after 100 ms, doubling up to 3 s. Updates produced while disconnected are lost.

To find out about gaps, pass a channel as the disconnect notifier. After each reconnect the datasource sends a `DatasourceDisconnection` with the last slot before and the first slot after the outage:

```rust
use {carbon_core::datasource::DatasourceDisconnection, tokio::sync::mpsc};

let (disconnect_tx, mut disconnect_rx) = mpsc::channel::<DatasourceDisconnection>(16);
tokio::spawn(async move {
    while let Some(gap) = disconnect_rx.recv().await {
        log::warn!(
            "missed slots {}..{} ({} slots)",
            gap.last_slot_before_disconnect, gap.first_slot_after_reconnect, gap.missed_slots
        );
    }
});
// pass Some(disconnect_tx) as the eighth argument
```

Backfill the reported range with the [block crawler](https://carbon.sevenlabs.org/guides/backfill-transactions.md#crawl-a-slot-range).

### Recover missed data with the client's reconnect

Carbon 2.0 can also turn on the Yellowstone client's own reconnect, which is off by default. With `ReconnectionPolicy::RecoverMissedData`, the client reconnects inside the stream, asks the server to replay from the last slot it saw, and drops the duplicates the replay produces. The server must support replay (`from_slot`) and still hold those slots. Add the client crate at the version the datasource uses:

`Cargo.toml`:

```toml
[dependencies]
yellowstone-grpc-client = "=13.5.0"
```

```rust
use {
    std::time::Duration,
    yellowstone_grpc_client::{Backoff, ReconnectConfig, ReconnectionPolicy},
};

let config = YellowstoneGrpcClientConfig::default().with_reconnect(ReconnectConfig {
    backoff: Backoff::new(Duration::from_millis(100), 2.0, 5), // 5 attempts per outage
    policy: ReconnectionPolicy::RecoverMissedData { slot_retention: 250 },
});
// pass config as the seventh argument
```

`ReconnectConfig::default()` uses a 10 ms backoff with 3 retries and `RecoverMissedData` with 250 slots of deduplication state. `ReconnectionPolicy::SkipMissedData` reconnects at the tip without replay. If the client gives up, the stream ends and the datasource's own resubscribe loop takes over.

## Other client settings

`YellowstoneGrpcClientConfig::default()` sets 15 s connect and request timeouts, TLS with the system's root certificates, and no compression. Its public fields:

| Field                        | Default                 | Description                                                                                        |
| ---------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------- |
| `compression`                | `None`                  | `Some(CompressionEncoding::Gzip)` or `Zstd` compresses both directions, if the server supports it. |
| `connect_timeout`, `timeout` | 15 s                    | Connection and request timeouts.                                                                   |
| `max_decoding_message_size`  | tonic's default (4 MiB) | Raise it if large blocks or accounts fail with a message-size error.                               |
| `tls_config`                 | system roots            | A custom `ClientTlsConfig`.                                                                        |
| `tcp_nodelay`                | `None`                  | Sets `TCP_NODELAY`.                                                                                |
| `reconnect`                  | `None`                  | See above.                                                                                         |

## Keep up with the stream

The datasource never waits for the pipeline: if the pipeline's channel is full, the update is dropped and logged as `Failed to send transaction update …` or `Failed to send account event …`. The channel holds 1,000 updates by default. If you see those lines, make processors faster (move slow I/O to a background task) and raise the buffer:

```rust
Pipeline::builder()
    .datasource(datasource)
    .channel_buffer_size(10_000)
```

## Common problems

### Panic: `Could not automatically determine the process-level CryptoProvider from Rustls crate features`

The gRPC client uses rustls, and no crypto provider was selected: either none is enabled, or both `ring` and `aws-lc-rs` are. Add the `rustls` dependency from the steps above and call `rustls::crypto::ring::default_provider().install_default()` at the top of `main`.

### Pipeline exits immediately with `error consuming datasource: FailedToConsumeDatasource("gRPC transport error: transport error")`

The client couldn't connect: the host or port is wrong, the URL has no `https://` scheme, or a firewall blocks the port. The pipeline stops when its only datasource fails, and the process exits with status 0. Check the URL with your provider.

### `Geyser stream error: Status { code: Unauthenticated, … }` repeated

The token is missing or wrong. Set `X_TOKEN`. Some providers put the token in the URL instead; then leave `X_TOKEN` unset.

### Many `Geyser stream error` lines per second

The connection works but the server rejects the subscription, for example `405 Method Not Allowed` when the URL points at an HTTP RPC endpoint instead of the gRPC one, or `PermissionDenied` when your plan doesn't include a filter you used. The datasource resubscribes immediately after a stream error, so it repeats quickly. Fix the endpoint or the filter.

### `mismatched types` errors on `SubscribeRequestFilterTransactions` or `CommitmentLevel`

Your `yellowstone-grpc-proto` isn't `=12.7.0`, so two copies of the types are in the build. Pin the version shown in the steps and run `cargo update`.

### Nothing prints

No transaction matched. `account_required` needs every listed key in the transaction; use `account_include` for "any of". `failed: Some(false)` and `vote: Some(false)` drop those transactions. Also check that the pipe's decoder matches the program you subscribed to.

## Next steps

- [Stream with Helius LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md): The same stream with replay from a past slot.
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Two streams for redundancy, deduplicated.
- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md): Fill history and disconnect gaps.
- [Yellowstone gRPC datasource](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md): The datasource's full reference.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/stream-with-yellowstone · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Stream with Helius LaserStream (Carbon 2.0)

> Stream transactions and accounts from Helius LaserStream, start from a past slot with from_slot, and resume after disconnects with slot replay.

Helius LaserStream is a hosted, Yellowstone-compatible gRPC stream that can replay recent slots. Carbon's LaserStream datasource uses that to start from a past slot and to resume where it left off after a disconnect. This guide streams Meteora DLMM swaps from LaserStream and shows how `from_slot` and replay behave.

**You need:**

- Rust 1.96.1 or newer.
- A Helius account on a plan that includes LaserStream, and its API key.
- The LaserStream endpoint for your region, such as `https://laserstream-mainnet-ewr.helius-rpc.com`. Helius's dashboard and docs list the regional endpoints.

If you already use [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md), the filters are identical; only the datasource type and its config differ.

### Step 1: Create a project

```sh
cargo new dlmm-laserstream
cd dlmm-laserstream
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-laserstream"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-helius-laserstream-datasource = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
rustls = { version = "0.23", default-features = false, features = ["std", "ring"] }
tokio = { version = "1.54.1", features = ["full"] }
yellowstone-grpc-proto = { version = "=12.7.0", default-features = false }
```

The filters are `yellowstone-grpc-proto` types, so pin it to `=12.7.0`, the version the datasource uses. `rustls` lets you choose the TLS crypto provider the gRPC client needs.

### Step 3: Write the pipeline

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_laserstream_datasource::{
        BlockFilters, LaserStreamClientConfig, LaserStreamGeyserClient,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    std::{collections::HashMap, env, sync::Arc},
    yellowstone_grpc_proto::geyser::{CommitmentLevel, SubscribeRequestFilterTransactions},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // The gRPC client uses rustls; pick its crypto provider before connecting.
    rustls::crypto::ring::default_provider()
        .install_default()
        .expect("failed to install the rustls crypto provider");

    // Successful, non-vote transactions that invoke Meteora DLMM.
    let mut transaction_filters = HashMap::new();
    transaction_filters.insert(
        "meteora_dlmm".to_string(),
        SubscribeRequestFilterTransactions {
            vote: Some(false),
            failed: Some(false),
            account_required: vec![PROGRAM_ID.to_string()],
            ..Default::default()
        },
    );

    // Start from a past slot if FROM_SLOT is set; otherwise start at the tip.
    let from_slot = env::var("FROM_SLOT")
        .ok()
        .map(|slot| slot.parse().expect("FROM_SLOT must be a slot number"));

    let config = LaserStreamClientConfig {
        from_slot,
        ..LaserStreamClientConfig::default() // replay_enabled: true
    };

    let datasource = LaserStreamGeyserClient::new(
        env::var("LASERSTREAM_URL").expect("LASERSTREAM_URL must be set"),
        Some(env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set")),
        Some(CommitmentLevel::Confirmed),
        HashMap::new(), // account filters
        transaction_filters,
        BlockFilters::default(),
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => {
                println!("slot {} swap  {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            MeteoraDlmmInstruction::Swap2 { data, .. } => {
                println!("slot {} swap2 {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}
```

The API key goes in the second argument; the datasource sends it as the `x-token` header.

### Step 4: Run it

```sh
LASERSTREAM_URL="https://laserstream-mainnet-ewr.helius-rpc.com" HELIUS_API_KEY="your-helius-api-key" cargo run
```

Each decoded swap prints one line:

Output:

```text
slot 467110307 swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
slot 467110307 swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
slot 467110307 swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
…
```

Press `Ctrl+C` to stop.

## Verify it's streaming

Run with `RUST_LOG=info`. `LogMetrics` logs counters every five seconds; `laserstream_transaction_updates_received_total` should grow steadily. To check replay, note a slot from the output, stop the program, and start it again with `FROM_SLOT` set a few hundred slots earlier: the first lines printed should have slots at or after `FROM_SLOT`, and then catch up to the tip.

```sh
FROM_SLOT=467110000 LASERSTREAM_URL="https://laserstream-mainnet-ewr.helius-rpc.com" HELIUS_API_KEY="your-helius-api-key" cargo run
```

## Start from a past slot

`from_slot` asks LaserStream to replay from that slot before streaming live data. The slot must still be in Helius's replay window; check Helius's LaserStream docs for how far back your plan goes. Use it to:

- **Resume after a restart.** Store the last slot your processor handled (in your database, next to the data) and pass it as `from_slot` on startup. You'll see that slot's updates again, so make writes idempotent or add a [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter).
- **Cover a deploy gap.** Start the new version from a slot before the old one stopped.

For history beyond the replay window, use a [backfill](https://carbon.sevenlabs.org/guides/backfill-transactions.md) instead.

> **Warning: from_slot needs replay_enabled.** With `replay_enabled: false`, the datasource clears `from_slot` before every subscribe, including the first, so the stream always starts at the tip.

## How replay handles disconnects

`LaserStreamClientConfig::default()` has `replay_enabled: true`. The datasource then adds an internal slot subscription to the request and records the latest slot it reports. When the stream ends or errors, it waits 3 seconds and subscribes again with `from_slot` set to:

| Commitment               | Resumes from                                                            |
| ------------------------ | ----------------------------------------------------------------------- |
| `Confirmed`, `Finalized` | The last slot seen                                                      |
| `Processed`, or `None`   | The last slot seen minus 31, since processed slots can still be dropped |

Updates from the resumed slots arrive a second time. Instruction and account pipes can drop them with a `DeduplicationFilter`; key your storage on signature and instruction path (or pubkey and slot) so a repeat is harmless.

After 10 consecutive failed subscribe attempts, the datasource stops. If it's your pipeline's only datasource, the pipeline then exits. Run a second stream for redundancy ([Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md)) or restart the process from a supervisor with `from_slot`.

Unlike the Yellowstone datasource, the LaserStream datasource has no stream timeout and no disconnect notifier. A connection that stays open but stops sending data isn't detected; watch `laserstream_transaction_updates_received_total` with [Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md) and alert when it stops growing.

## Filters and settings

The account, transaction and block filters are the same Yellowstone types as in [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md#filter-transactions), and the datasource turns them into the same updates. `LaserStreamClientConfig` has these public fields:

| Field                        | Default                 | Description                                                                             |
| ---------------------------- | ----------------------- | --------------------------------------------------------------------------------------- |
| `replay_enabled`             | `true`                  | Tracks slots and resumes from the last one after a reconnect. Required for `from_slot`. |
| `from_slot`                  | `None`                  | Slot to start the first subscription from.                                              |
| `connect_timeout`, `timeout` | 15 s                    | Connection and request timeouts.                                                        |
| `compression`                | `None`                  | `Some(CompressionEncoding::Gzip)` or `Zstd`.                                            |
| `max_decoding_message_size`  | tonic's default (4 MiB) | Raise it for large blocks or accounts.                                                  |
| `tls_config`                 | system roots            | A custom `ClientTlsConfig`.                                                             |
| `tcp_nodelay`                | `None`                  | Sets `TCP_NODELAY`.                                                                     |

`LaserStreamClientConfig::new` takes the same values positionally, ending with `replay_enabled` and `from_slot`.

Like the Yellowstone datasource, it never waits for the pipeline: when the pipeline's channel is full, updates are dropped with `Failed to send transaction update …`. A replay from an old `from_slot` delivers a burst, so raise `.channel_buffer_size(…)` on the builder (default 1,000) when you replay.

## Common problems

### Panic: `Could not automatically determine the process-level CryptoProvider from Rustls crate features`

No rustls crypto provider is selected. Add the `rustls` dependency from the steps and call `rustls::crypto::ring::default_provider().install_default()` first thing in `main`.

### Pipeline exits immediately with `error consuming datasource: FailedToConsumeDatasource("gRPC transport error: …")`

The client couldn't connect. Check that `LASERSTREAM_URL` is a LaserStream gRPC endpoint with `https://`, not an RPC URL.

### `Geyser stream error, will reconnect: Status { … }` every 3 seconds

The server accepted the connection but rejected the stream: `Unauthenticated` or `PermissionDenied` for a wrong API key or a plan without LaserStream, or `405 Method Not Allowed` when the URL points at an HTTP RPC endpoint. These retries don't count toward the reconnection limit, so the datasource keeps retrying until you fix the cause.

### `Failed to subscribe, will retry after 5s delay` repeated, then `Max reconnection attempts reached: 10`

The subscribe request itself failed ten times in a row, for example because the endpoint is unreachable or `from_slot` is outside the replay window. The retries are 3 seconds apart, despite the message. After the tenth, the datasource stops.

### The stream starts at the tip although `FROM_SLOT` is set

`replay_enabled` is `false`. Leave it at the default `true`.

### Every swap appears twice after a reconnect

That's the replay resuming from the last slot. Add a `DeduplicationFilter` to the pipe, or make your writes idempotent.

## Next steps

- [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md): Filters, commitment and reconnect options in detail.
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Combine LaserStream with a second stream or a backfill.
- [Helius LaserStream datasource](https://carbon.sevenlabs.org/datasources/helius-laserstream.md): The datasource's full reference.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/stream-with-laserstream · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Stream with RPC websockets (Carbon 2.0)

> Stream account changes with programSubscribe on any Solana RPC, including the public one, and whole blocks with blockSubscribe on providers that enable it.

Solana RPC nodes push updates over websockets. Carbon has two websocket datasources: `rpc-program-subscribe` streams every change to a program's accounts and works on any RPC, the public endpoint included; `rpc-block-subscribe` streams confirmed blocks with their transactions, but only on RPC providers that enable `blockSubscribe`. Use them when you don't have a [Geyser gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) endpoint.

**You need:** Rust 1.96.1 or newer. For account updates, nothing else: this guide runs on `wss://api.mainnet-beta.solana.com`. For block updates, a websocket URL from an RPC provider that supports `blockSubscribe`.

## Stream account changes

This program streams every Meteora DLMM pool (`LbPair` account) as swaps and liquidity changes write to it.

### Step 1: Create a project

```sh
cargo new dlmm-accounts
cd dlmm-accounts
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-accounts"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-program-subscribe-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

`solana-client` provides the subscription config types. Pin it to `=4.2.2`, the version Carbon 2.0.0 uses, so one copy of the Solana types is in your build.

### Step 3: Write the pipeline

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{accounts::MeteoraDlmmAccount, MeteoraDlmmDecoder, PROGRAM_ID},
    carbon_rpc_program_subscribe_datasource::{Filters, RpcProgramSubscribe},
    solana_client::{
        rpc_config::{
            CommitmentConfig, RpcAccountInfoConfig, RpcProgramAccountsConfig, UiAccountEncoding,
        },
        rpc_filter::{Memcmp, RpcFilterType},
    },
};

// First 8 bytes of every Meteora DLMM `LbPair` account.
const LB_PAIR_DISCRIMINATOR: [u8; 8] = [33, 11, 49, 98, 181, 101, 177, 13];

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let config = RpcProgramAccountsConfig {
        filters: Some(vec![RpcFilterType::Memcmp(Memcmp::new_raw_bytes(
            0,
            LB_PAIR_DISCRIMINATOR.to_vec(),
        ))]),
        account_config: RpcAccountInfoConfig {
            encoding: Some(UiAccountEncoding::Base64),
            commitment: Some(CommitmentConfig::confirmed()),
            ..Default::default()
        },
        ..Default::default()
    };

    let datasource = RpcProgramSubscribe::new(
        "wss://api.mainnet-beta.solana.com".to_string(),
        Filters::new(PROGRAM_ID, Some(config)),
    );

    Pipeline::builder()
        .datasource(datasource)
        .account(MeteoraDlmmDecoder, PairLogger)
        .build()?
        .run()
        .await
}

struct PairLogger;

impl Processor<AccountProcessorInputType<'_, MeteoraDlmmAccount>> for PairLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, MeteoraDlmmAccount>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmAccount::LbPair(pair) = &input.decoded_account.data {
            println!(
                "slot {} pair {}: active_id={} bin_step={}",
                input.metadata.slot, input.metadata.pubkey, pair.active_id, pair.bin_step
            );
        }
        Ok(())
    }
}
```

The filters run on the RPC node, so only `LbPair` accounts cross the network. `encoding: Some(UiAccountEncoding::Base64)` is required: the datasource can't rebuild accounts from the node's default encoding.

### Step 4: Run it

```sh
cargo run
```

A pool prints each time it changes, with its active bin:

Output:

```text
slot 467109944 pair 8obGpjiUu7QTJHK58YHCoz5HxobmrVP2x5zpMZu3c4BT: active_id=-78 bin_step=80
slot 467109944 pair EXXfbVutbuexDZtPw8pkSvWxLFoUMBHd544CAGkmtPZm: active_id=-2912 bin_step=10
slot 467109944 pair 54Vp27uLaw4wNLo5n7r4fcC6zLamoQc28xBARjss4EUJ: active_id=-2625 bin_step=16
slot 467109945 pair 2tFYCTqYJ2pyrb7CeA824o2Ybq3jq7agvr3Qpv2uVDmL: active_id=-543 bin_step=80
slot 467109945 pair GMgh4NtWrGaUf1RR2kcXD7LY1jou1qFAuSsQeKp5ow4a: active_id=4903 bin_step=5
slot 467109946 pair GMgh4NtWrGaUf1RR2kcXD7LY1jou1qFAuSsQeKp5ow4a: active_id=4904 bin_step=5
…
```

Press `Ctrl+C` to stop.

### How the account stream behaves

- **Only changes.** The subscription sends an account when it's written, not its current state. Load the starting state with [`getProgramAccounts`](https://carbon.sevenlabs.org/guides/load-account-state.md) and combine the two as in [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md#load-state-then-follow-changes).
- **No transaction signature.** `input.metadata.transaction_signature` is always `None`, so a [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) lets every update through. `input.metadata.slot` is the slot of the notification.
- **Closed accounts.** An update with zero lamports arrives as an [account deletion](https://carbon.sevenlabs.org/concepts/updates.md#account-deletions), not an account update: add `.account_deletions(processor)` to handle it.
- **Reconnects.** When the websocket closes, the datasource waits 3 seconds and subscribes again. After 10 failed connection attempts in a row it stops with an error. Changes made while disconnected aren't replayed.
- **Backpressure.** If the pipeline's channel is full (1,000 updates by default), the datasource logs `Error sending account update`, drops the update and reconnects. Raise `.channel_buffer_size(…)` on the builder if your processor is slow.

The public endpoint limits subscriptions per IP address, and busy programs send a lot of data. Narrow the filters (by discriminator, `DataSize`, or a `Memcmp` on a field such as a mint) and use a provider's websocket for production.

## Stream blocks

`rpc-block-subscribe` sends each block's transactions as transaction updates and the block itself as a `BlockDetails` update. It needs an RPC node started with `--rpc-pubsub-enable-block-subscription`. Most providers enable it on paid plans; the public endpoint rejects it with `Method not found`.

Add the datasource to a project set up like the one above:

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-block-subscribe-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        datasource::BlockDetails, error::CarbonResult,
        instruction::InstructionProcessorInputType, pipeline::Pipeline, processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_block_subscribe_datasource::{Filters, RpcBlockSubscribe},
    solana_client::rpc_config::{
        CommitmentConfig, RpcBlockSubscribeConfig, RpcBlockSubscribeFilter, TransactionDetails,
        UiTransactionEncoding,
    },
    std::env,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let filters = Filters::new(
        // Only transactions that mention Meteora DLMM.
        RpcBlockSubscribeFilter::MentionsAccountOrProgram(PROGRAM_ID.to_string()),
        Some(RpcBlockSubscribeConfig {
            commitment: Some(CommitmentConfig::confirmed()),
            encoding: Some(UiTransactionEncoding::Base64),
            transaction_details: Some(TransactionDetails::Full),
            show_rewards: Some(false),
            max_supported_transaction_version: Some(1),
        }),
    );

    let datasource = RpcBlockSubscribe::new(
        env::var("RPC_WS_URL").expect("RPC_WS_URL must be set"),
        filters,
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .block_details(BlockLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => {
                println!("slot {} swap  {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            MeteoraDlmmInstruction::Swap2 { data, .. } => {
                println!("slot {} swap2 {}: amount_in={}", meta.slot, meta.signature, data.amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}

struct BlockLogger;

impl Processor<BlockDetails> for BlockLogger {
    async fn process(&mut self, block: &BlockDetails) -> CarbonResult<()> {
        println!("block {} time={:?}", block.slot, block.block_time);
        Ok(())
    }
}
```

```sh
RPC_WS_URL="wss://your-provider.example/?api-key=…" cargo run
```

Each block prints first, followed by its swaps:

Output:

```text
block 467110307 time=Some(1794182812)
slot 467110307 swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
slot 467110307 swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
slot 467110307 swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
block 467110308 time=Some(1794182812)
slot 467110308 swap2 2miT4n9yFXZMdUTVF4My1RVvHrbbZMwD64vYyppzdkxNg5qVtYJgsm4dL8MPYtBRrJm2V3EzP4ZfYJKjEc61Wzr1: amount_in=28328938705735
…
```

> **Warning: Base64, full details and version 1 are required.** Set `encoding: Some(UiTransactionEncoding::Base64)`, `transaction_details: Some(TransactionDetails::Full)` and `max_supported_transaction_version: Some(1)`. Carbon can't rebuild a transaction from `Json` or `JsonParsed` payloads, and without version `1` the node refuses blocks that contain versioned transactions.

### How the block stream behaves

- **Filter.** `MentionsAccountOrProgram` takes one address. To follow several programs, use `RpcBlockSubscribeFilter::All` (every transaction on the network, which is a lot of data) or one datasource per program with a [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter).
- **Failed transactions are skipped.** Only successful ones reach your pipes.
- **Metadata.** Each transaction gets the block's slot, time and hash. `transaction_metadata.index` is the transaction's position in the list the node sent; with `MentionsAccountOrProgram` that list holds only the matching transactions, so it isn't the position in the full block.
- **Block details.** The `BlockDetails` update carries `slot`, `block_hash`, `previous_block_hash`, `block_time`, `block_height` and, when you request them, `rewards`.
- **Stale connections.** If no block arrives for 30 seconds, the datasource reconnects. Change it with `RpcBlockSubscribe::new(url, filters).with_stream_timeout(Duration::from_secs(60))`.
- **Gaps.** Blocks produced while disconnected aren't replayed. Construct the datasource with `RpcBlockSubscribe::with_disconnect_notifier(url, filters, sender)` to receive a `DatasourceDisconnection` with the missed slot range after each reconnect, then fill it with the [block crawler](https://carbon.sevenlabs.org/guides/backfill-transactions.md#crawl-a-slot-range).
- **Reconnects.** After a disconnect it waits 3 seconds and subscribes again. It gives up after 10 failed connection attempts, or 11 rejected subscriptions, in a row.

## Common problems

### `Failed to subscribe to block updates: SubscribeFailed { reason: "Method not found (-32601)", … }`

The node doesn't allow `blockSubscribe`. The public endpoint never does. Use a provider whose websocket enables it, or stream with [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md). After 11 rejections the datasource stops and the pipeline exits.

### `Error decoding account event` for every update

The program subscription uses the node's default encoding. Set `encoding: Some(UiAccountEncoding::Base64)` in `RpcAccountInfoConfig`, as in the example. Passing `None` as the config has the same problem.

### `Failed to decode transaction` for every transaction

The block subscription uses `Json` or `JsonParsed` encoding. Set `encoding: Some(UiTransactionEncoding::Base64)`.

### `Failed to create RPC subscribe client`

The websocket URL is wrong or unreachable. It must start with `wss://` (or `ws://`), not `https://`. Providers usually give a separate websocket URL.

### Nothing prints

The filters match nothing, or the decoder doesn't match the program you subscribed to. Remove the `Memcmp` filter to test, and check that the account discriminator bytes are those of the account type you want.

## Next steps

- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md): Fetch every matching account before streaming changes.
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Load state with gPA and follow it with programSubscribe.
- [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md): Transactions and accounts in real time, with filters on both.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/stream-with-rpc · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Backfill transactions (Carbon 2.0)

> Load a program's transaction history with the RPC transaction crawler, the RPC block crawler or Helius GTFA, bounded by signatures or slots, and resume when interrupted.

A backfill runs your decoders and processors over past transactions, to build an index from a program's launch or to fill a gap in a live stream. Carbon has three history datasources. This guide runs the two RPC ones against the public endpoint and shows how to bound, resume and speed up a backfill.

**You need:** Rust 1.96.1 or newer. The RPC crawlers work with any Solana RPC endpoint, including `https://api.mainnet-beta.solana.com`, which is enough for small ranges. For large backfills, use a paid RPC endpoint or Helius GTFA, which needs a Helius API key.

## Choose a source

| Datasource                                                                                       | Walks                                                                                          | Order                                    | Stops by itself                                      | Best for                                                        |
| ------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- | ---------------------------------------- | ---------------------------------------------------- | --------------------------------------------------------------- |
| [`rpc-transaction-crawler`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) | One address's signatures (`getSignaturesForAddress`), then each transaction (`getTransaction`) | Newest first, a few out of order         | No: after the history, it polls for new transactions | A program's or account's full history, on any RPC               |
| [`rpc-block-crawler`](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)             | Every block in a slot range (`getBlock`)                                                       | Out of order across blocks               | Yes, at `end_slot`                                   | Exact slot ranges, gaps from a stream, several programs at once |
| [`helius-gtfa`](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                         | One address's transactions in pages (`getTransactionsForAddress`)                              | Oldest or newest first, strictly ordered | Yes, after the last page                             | Fast, ordered backfills with slot, time and status filters      |

All three skip failed transactions unless you configure GTFA to keep them.

## Backfill an address's history

This program crawls Meteora DLMM's transactions between two signatures and prints its swaps.

### Step 1: Create a project

```sh
cargo new dlmm-backfill
cd dlmm-backfill
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-backfill"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
solana-signature = "3.4.1"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 3: Write the pipeline

`before` and `until` bound the crawl. Both are exclusive: the crawler starts with the transaction just older than `before` and stops just before `until`. The `SlotRangeFilter` drops anything outside the range's slots; you need it because the crawler doesn't stop at `until` (see [Bound the range](#bound-the-range)).

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, filter::SlotRangeFilter, instruction::InstructionProcessorInputType,
        pipeline::Pipeline, processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    solana_client::rpc_config::CommitmentConfig,
    solana_signature::Signature,
    std::str::FromStr,
};

// The newest transaction to stop before (exclusive) and its slot.
const BEFORE: &str =
    "54hSq76xk8noTVuNQaEyCM9m7j9t7LWEbu2tRRBwAFmFPHXC54jYLJqUyexrGGBaDEC8PokZKN7jUQeZnBejP9fc";
const BEFORE_SLOT: u64 = 467_110_309;
// The oldest transaction to stop at (exclusive) and its slot.
const UNTIL: &str =
    "2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W";
const UNTIL_SLOT: u64 = 467_110_307;

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let filters = Filters::new(
        None,
        Some(Signature::from_str(BEFORE).expect("valid signature")),
        Some(Signature::from_str(UNTIL).expect("valid signature")),
    );

    let connection = ConnectionConfig {
        max_concurrent_requests: 2, // stay under the public endpoint's limits
        blocking_send: true,        // wait for the pipeline instead of dropping updates
        ..ConnectionConfig::default()
    };

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        connection,
        filters,
        Some(CommitmentConfig::finalized()),
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction_with_filters(
            MeteoraDlmmDecoder,
            SwapLogger,
            vec![Box::new(SlotRangeFilter::between(UNTIL_SLOT, None, BEFORE_SLOT, None))],
        )
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => println!(
                "slot {} swap  {}: amount_in={}",
                meta.slot, meta.signature, data.amount_in
            ),
            MeteoraDlmmInstruction::Swap2 { data, .. } => println!(
                "slot {} swap2 {}: amount_in={}",
                meta.slot, meta.signature, data.amount_in
            ),
            _ => {}
        }
        Ok(())
    }
}
```

To choose your own bounds, list signatures with `getSignaturesForAddress` (or a block explorer) and copy two of them with their slots.

### Step 4: Run it

```sh
cargo run
```

Swaps print newest first. On the public endpoint, expect long pauses: it allows only a handful of `getTransaction` calls per IP every few seconds.

Output:

```text
slot 467110309 swap2 cXStywP9DDr2Cgk4RBnCkhKTVwwRq9XaYYebcTUd1Uk97xZiVSVnc814B3G9J1xAJufVJaRGQiPeVZaX2tRKfdL: amount_in=516957831
slot 467110309 swap2 57o62AmoqKSKqznKJQ4yd5fwcKi7KezErbVS1tFDy5xexHmVeKmTgyGHtfsGY184W7zHvmjypBBXT4XDnyAympuZ: amount_in=512968
slot 467110309 swap2 2f5k2YKnopfhtGG9ro2Zd4KLXUznr1U2ege2GbrQdUbwDeNSX4cSDcMAadMk3TVjG9Ggc9TpbojRyeVDZhQHzoi3: amount_in=381964223
slot 467110309 swap2 4jvbKVzCJ1SYVDWh6pftBtUnrub7QDq3nGzQGVNAaybRV1jhjvoXhDkUSU6wK7BoNCNoERondJSHC9uxEhGgiTHC: amount_in=2002103
…
```

When the range is done, the program keeps running (the crawler starts polling for new transactions, which the filter drops). Press `Ctrl+C` to stop.

## Verify the backfill

Count what your processor handled and compare it with the source. For a signature range, call `getSignaturesForAddress` with the same `before` and `until`: the signatures with `"err": null` are the transactions the crawler delivers, since it skips failed ones. For a slot range, a block explorer shows each block's transactions for the program. Add a [`LogMetrics`](https://carbon.sevenlabs.org/concepts/metrics.md) exporter and run with `RUST_LOG=info` to watch `transaction_crawler_signatures_fetched_total` and `transaction_crawler_transactions_fetched_total` (both count failed transactions too) while the crawl runs.

## Bound the range

**RPC**

The transaction crawler takes signatures, not slots:

```rust
Filters::new(
    None,                 // accounts: keep only transactions that also mention one of these
    Some(before),         // start just older than this signature; None = the newest
    Some(until),          // stop just before this signature; None = the address's first transaction
)
```

The crawler doesn't exit when it reaches `until`. It then polls every `polling_interval` (5 s by default) for transactions newer than the newest one it has seen, fetching from the tip down. With `before` set, that also fetches everything between `before` and the tip. A `SlotRangeFilter` on the pipe drops those, but they still cost RPC calls. To stop, press `Ctrl+C` or cancel a [`CancellationToken`](https://carbon.sevenlabs.org/concepts/shutdown.md#stop-with-a-cancellation-token). For a range that ends by itself, use the [block crawler](#crawl-a-slot-range) or GTFA.

The `accounts` filter is applied after fetching: the crawler still downloads every transaction of the crawled address, then drops those that don't mention one of the accounts. To crawl a single pool's history, pass the pool's address as the crawled address instead.

**Helius**

GTFA filters on the server, by slot, block time, signature or status:

```rust
use carbon_helius_gtfa_datasource::{HeliusGtfaFilters, SlotFilter, TransactionStatusFilter};

HeliusGtfaFilters {
    slot: Some(SlotFilter { gte: Some(467_110_307), lte: Some(467_110_309), gt: None, lt: None }),
    block_time: None, // BlockTimeFilter { gte, gt, lte, lt, eq } in Unix seconds
    signature: None,
    status: Some(TransactionStatusFilter::Succeeded),
}
```

The datasource stops after the last page. See [Backfill with Helius GTFA](#backfill-with-helius-gtfa).

Any datasource can also be bounded on the pipe with [`SlotRangeFilter`](https://carbon.sevenlabs.org/concepts/filters.md#slotrangefilter): `from(slot, None)`, `to(slot, None)` or `between(from, None, to, None)`, inclusive at both ends.

## Crawl a slot range

The block crawler fetches every block from `start_slot` to `end_slot` (inclusive), turns each successful transaction into an update, and exits when it's done. Each block holds all programs' transactions, so the decoders pick what they understand. This program prints DLMM swaps from three blocks on the public endpoint.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-block-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_rpc_block_crawler_datasource::{RpcBlockConfig, RpcBlockCrawler},
    solana_client::rpc_config::{CommitmentConfig, TransactionDetails, UiTransactionEncoding},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let block_config = RpcBlockConfig {
        encoding: Some(UiTransactionEncoding::Base64),
        transaction_details: Some(TransactionDetails::Full),
        rewards: Some(false),
        commitment: Some(CommitmentConfig::finalized()),
        max_supported_transaction_version: Some(1),
    };

    let datasource = RpcBlockCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        467_110_307,       // start slot (inclusive)
        Some(467_110_309), // end slot (inclusive); None keeps following the tip
        None,              // block interval (default 100 ms)
        block_config,
        Some(1), // concurrent getBlock requests (default 10)
        None,    // internal channel size (default 1000)
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .channel_buffer_size(10_000)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => println!(
                "slot {} tx {} swap  {}: amount_in={}",
                meta.slot, meta.index.unwrap_or_default(), meta.signature, data.amount_in
            ),
            MeteoraDlmmInstruction::Swap2 { data, .. } => println!(
                "slot {} tx {} swap2 {}: amount_in={}",
                meta.slot, meta.index.unwrap_or_default(), meta.signature, data.amount_in
            ),
            _ => {}
        }
        Ok(())
    }
}
```

```sh
RUST_LOG=info cargo run
```

Output:

```text
[2026-11-09T00:10:31Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 0, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
slot 467110307 tx 171 swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
slot 467110307 tx 475 swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
slot 467110307 tx 623 swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
slot 467110308 tx 192 swap2 2miT4n9yFXZMdUTVF4My1RVvHrbbZMwD64vYyppzdkxNg5qVtYJgsm4dL8MPYtBRrJm2V3EzP4ZfYJKjEc61Wzr1: amount_in=28328938705735
slot 467110308 tx 411 swap  36sp8szy2wPdGGFWXnqJLh2Y4yFhd5SRxE7PR1UYcbJf9Kjd7TuksxUDt26wWHTcfhqg3mU4UquJAGFeVEwvpdQM: amount_in=23315968
slot 467110309 tx 47 swap  5XMR5d7vnGdiNoicdAPXhcSrB62d6z4rmM3vqyG2TSxg9wzDfKUMp8PSmt7awJhbhAujdG4Pq8BuvhxHBioS31vf: amount_in=1231056
…
[2026-11-09T00:10:34Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:10:34Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

Unlike the transaction crawler, the block crawler sets `index` (the transaction's position in the block), `block_time` and `block_hash`. Slots without a block are skipped silently. With several concurrent requests, blocks finish out of order; transactions within a block stay in order.

> **Warning: Raise the pipeline's channel buffer.** The block crawler doesn't wait for the pipeline. If the pipeline's channel is full, it logs `Error sending transaction update` and drops the rest of that block. A mainnet block holds over a thousand transactions and the default buffer is 1,000, so set `.channel_buffer_size(10_000)` or more, as above.

`RpcBlockConfig` must request `Base64` encoding, `Full` transaction details and `max_supported_transaction_version: Some(1)`; with the default `Json` encoding, every transaction fails with `Failed to decode transaction`.

## Backfill with Helius GTFA

Helius's `getTransactionsForAddress` returns full transactions for an address in pages of up to 100, filtered and sorted on the server, so it needs one request per 100 transactions instead of one per transaction. It needs a Helius RPC URL with an API key, on a plan that includes the method.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-helius-gtfa-datasource = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_gtfa_datasource::{
        HeliusGtfaConfig, HeliusGtfaDatasource, HeliusGtfaFilters, SlotFilter, SortOrder,
        TransactionStatusFilter,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    solana_client::rpc_config::CommitmentConfig,
    std::env,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let filters = HeliusGtfaFilters {
        slot: Some(SlotFilter {
            gte: Some(467_110_307),
            lte: Some(467_110_309),
            gt: None,
            lt: None,
        }),
        block_time: None,
        signature: None,
        status: Some(TransactionStatusFilter::Succeeded),
    };

    let config = HeliusGtfaConfig::new(
        Some(SortOrder::Asc), // oldest first
        Some(100),            // page size (max 100)
        Some(CommitmentConfig::finalized()),
        Some(filters),
        None, // min_context_slot
    );

    let datasource = HeliusGtfaDatasource::new_with_config(
        env::var("HELIUS_RPC_URL").expect("HELIUS_RPC_URL must be set"),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => println!(
                "slot {} swap  {}: amount_in={}",
                meta.slot, meta.signature, data.amount_in
            ),
            MeteoraDlmmInstruction::Swap2 { data, .. } => println!(
                "slot {} swap2 {}: amount_in={}",
                meta.slot, meta.signature, data.amount_in
            ),
            _ => {}
        }
        Ok(())
    }
}
```

```sh
HELIUS_RPC_URL="https://mainnet.helius-rpc.com/?api-key=your-helius-api-key" cargo run
```

It prints the same swaps as the crawlers, oldest first, and exits after `Reached end of pagination`.

- `HeliusGtfaDatasource::new(url, address)` uses the defaults: oldest first, 100 per page, `confirmed`, no filters.
- `status`: `Succeeded` (and `None`) skip failed transactions, `Failed` keeps only failed ones, `Any` keeps both.
- Each transaction carries its `index` in the block and its `block_time`.
- The datasource waits for the pipeline, so nothing is dropped. It doesn't retry: the first HTTP or RPC error stops it with `FailedToConsumeDatasource`. Resume as described below.

## Resume an interrupted backfill

Store your progress next to the data your processor writes, then restart from it:

- **GTFA, oldest first:** store the last slot you finished. Restart with `slot: Some(SlotFilter { gt: Some(last_slot), .. })`. Its transactions arrive in order, so the last slot is a clean checkpoint.
- **Block crawler:** store finished slots. Blocks complete out of order, so restart from the lowest slot you haven't finished, not the highest you have.
- **Transaction crawler:** store the oldest signature you processed and restart with it as `before`. Requests run concurrently, so a few transactions just newer than it may run again.

Every source can deliver a transaction twice across a restart. Make writes idempotent: key rows on `(signature, instruction path)` for instructions, or `(pubkey, slot)` for accounts, and upsert. See [Persist to Postgres](https://carbon.sevenlabs.org/guides/postgres.md).

## Rate limits and throughput

Throughput depends on the RPC endpoint far more than on Carbon. The public endpoint limits each IP per method (the response headers `x-ratelimit-method-limit` and `x-ratelimit-method-remaining` show the current budget) and the total data you download. Solana's RPC client retries HTTP 429 responses on its own, so on a rate-limited endpoint the crawler slows down instead of failing. To see the retries:

```sh
RUST_LOG=info,solana_rpc_client=debug cargo run
```

Output:

```text
[2026-11-09T00:14:13Z DEBUG solana_rpc_client::http_sender] Too many requests: server responded with Response { url: "https://api.mainnet-beta.solana.com/", status: 429, headers: {"retry-after": "10", …
```

The transaction crawler's `ConnectionConfig` controls its request rate. Its fields are public, so set the ones you need on top of `ConnectionConfig::default()`:

| Field                                                        | Default                         | Description                                                                                                                                                                |
| ------------------------------------------------------------ | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `batch_limit`                                                | `100`                           | Signatures per `getSignaturesForAddress` call (the RPC maximum is 1,000).                                                                                                  |
| `polling_interval`                                           | 5 s                             | Wait between polls once the history is done.                                                                                                                               |
| `max_concurrent_requests`                                    | `5`                             | Parallel `getTransaction` calls. Lower it on rate-limited endpoints; raise it on a paid one.                                                                               |
| `retry_config`                                               | 3 retries, 1 s doubling to 10 s | `RetryConfig::new(max_retries, initial_backoff_ms, max_backoff_ms, multiplier)` or `RetryConfig::no_retry()`. After the last retry, the transaction is skipped and logged. |
| `max_signature_channel_size`, `max_transaction_channel_size` | `None` (1,000)                  | Internal buffers between the fetch stages.                                                                                                                                 |
| `blocking_send`                                              | `false`                         | `true` waits when the pipeline is busy. `false` drops the update and logs `Failed to send update`. Use `true` for backfills.                                               |

For the block crawler, the `max_concurrent_requests` argument (default 10) sets parallel `getBlock` calls and `block_interval` (default 100 ms) how often it checks for new slots when following the tip.

## Backfill, then follow live data

Run a backfill datasource and a stream in the same pipeline: for example the block crawler from a stored slot to the current slot, plus [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) from now on, with a [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) for the overlap. [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md) shows how to combine and route them.

## Common problems

### Long pauses, or nothing for minutes, on the public endpoint

You're being rate-limited, possibly by other programs on the same IP. Run with `RUST_LOG=info,solana_rpc_client=debug` to see the 429 responses. Lower `max_concurrent_requests`, or use your own RPC endpoint.

### `Failed to fetch transaction … after 3 retries`

The endpoint kept failing for that transaction, and the crawler skipped it. Raise `RetryConfig::max_retries` or the backoff, or lower the request rate. Re-crawl the range later to pick up skipped transactions, with idempotent writes.

### `Failed to send update` warnings and missing transactions

`blocking_send` is `false` and the pipeline was busy, so updates were dropped. Set `blocking_send: true`.

### `Error sending transaction update` from the block crawler

The pipeline's channel was full and the rest of that block was dropped. Raise `.channel_buffer_size(…)` on the builder.

### The backfill never finishes

The transaction crawler polls for new transactions forever after its history; stop it with `Ctrl+C` or a cancellation token. The block crawler with `end_slot: None` follows the tip forever.

### `Failed to decode transaction` for every transaction

`RpcBlockConfig` doesn't request `Base64` encoding. Set `encoding: Some(UiTransactionEncoding::Base64)`.

Looking for transactions that **failed on chain** rather than ones Carbon couldn't decode? See [Decode or skip failed transactions](https://carbon.sevenlabs.org/concepts/filters.md#decode-or-skip-failed-transactions).

### `RPC error: … (code: …)` and the GTFA backfill stops

Helius rejected a request: a wrong API key, a plan without `getTransactionsForAddress`, or an invalid filter. The datasource doesn't retry; fix the cause and resume from your last slot.

## Next steps

- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Combine a backfill with a live stream.
- [Filters](https://carbon.sevenlabs.org/concepts/filters.md): SlotRangeFilter, DeduplicationFilter and your own.
- [Persist to Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Idempotent writes for restartable backfills.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/backfill-transactions · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Load account state (Carbon 2.0)

> Load every account a program owns at one point in time with getProgramAccounts, or page through it with Helius gPA v2 and sync only what changed since a slot.

Streams tell you what changes; to index a program's accounts you first need their current state. Carbon's state loaders read every account a program owns, decode it with your account decoder, and stop. This guide loads Meteora DLMM pools with `getProgramAccounts` on the public endpoint, then shows Helius gPA v2 for large programs and incremental syncs.

**You need:** Rust 1.96.1 or newer. `rpc-gpa` works with any RPC endpoint, including `https://api.mainnet-beta.solana.com` when you filter the request. Helius gPA v2 needs a Helius API key.

### Step 1: Create a project

```sh
cargo new dlmm-state
cd dlmm-state
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-state"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-gpa-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
solana-pubkey = "4.2.1"
tokio = { version = "1.54.1", features = ["full"] }
```

`solana-client` provides the request config and filter types; `solana-pubkey` the `pubkey!` macro. Both versions match Carbon 2.0.0's.

### Step 3: Write the pipeline

Meteora DLMM owns more than 160,000 pool accounts, too many for the public endpoint in one response. The `Memcmp` filters narrow the request to SOL/USDC pools: the account type's 8-byte discriminator at offset 0, the token X mint at byte 88 and the token Y mint at byte 120 of the `LbPair` layout.

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{accounts::MeteoraDlmmAccount, MeteoraDlmmDecoder, PROGRAM_ID},
    carbon_rpc_gpa_datasource::GpaDatasource,
    solana_client::{
        rpc_config::{CommitmentConfig, RpcAccountInfoConfig, RpcProgramAccountsConfig},
        rpc_filter::{Memcmp, RpcFilterType},
    },
    solana_pubkey::{pubkey, Pubkey},
};

const LB_PAIR_DISCRIMINATOR: [u8; 8] = [33, 11, 49, 98, 181, 101, 177, 13];
const WSOL: Pubkey = pubkey!("So11111111111111111111111111111111111111112");
const USDC: Pubkey = pubkey!("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // SOL/USDC `LbPair` accounts: the account discriminator at byte 0,
    // token X mint at byte 88 and token Y mint at byte 120.
    let config = RpcProgramAccountsConfig {
        filters: Some(vec![
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(0, LB_PAIR_DISCRIMINATOR.to_vec())),
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(88, WSOL.to_bytes().to_vec())),
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(120, USDC.to_bytes().to_vec())),
        ]),
        account_config: RpcAccountInfoConfig {
            commitment: Some(CommitmentConfig::confirmed()),
            ..Default::default()
        },
        ..Default::default()
    };

    let datasource = GpaDatasource::new_with_config(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .account(MeteoraDlmmDecoder, PairLogger)
        .build()?
        .run()
        .await
}

struct PairLogger;

impl Processor<AccountProcessorInputType<'_, MeteoraDlmmAccount>> for PairLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, MeteoraDlmmAccount>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmAccount::LbPair(pair) = &input.decoded_account.data {
            println!(
                "slot {} pair {}: bin_step={} active_id={}",
                input.metadata.slot, input.metadata.pubkey, pair.bin_step, pair.active_id
            );
        }
        Ok(())
    }
}
```

### Step 4: Run it

```sh
RUST_LOG=info cargo run
```

The request takes about a second. Every pool prints with the same slot, the slot the RPC node read them at, and the program exits on its own:

Output:

```text
[2026-11-09T00:13:42Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 0, num_account_pipes: 1, num_account_deletion_pipes: 0, num_instruction_pipes: 0, num_transaction_pipes: 0
[2026-11-09T00:13:43Z INFO  carbon_rpc_gpa_datasource] Fetched 113 accounts for program LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo (slot: 467112209)
slot 467112209 pair 1jw5fDodwGEGBVqNXsx2eqiLgNmgMDEeXWSbrTreLCM: bin_step=10 active_id=-2056
slot 467112209 pair 5rCf1DM8LjKTw4YqhnoLcngyZYeNnQqztScTogYHAS6: bin_step=4 active_id=-6110
slot 467112209 pair BGm1tav58oGcsQJehL9WXBFXF7D27vZsKefj4xJKD5Y: bin_step=10 active_id=-2445
…
slot 467112209 pair Hk96GjGpo1VMotNCC5WAZ6CcNfvgPVaD4zv1mDVx47nt: bin_step=250 active_id=-90
[2026-11-09T00:13:43Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:13:43Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

## Verify the load

The datasource logs how many accounts the node returned (`Fetched 113 accounts …`). Count the accounts your processor handles: if it's lower, some accounts didn't decode, usually because the filter also matches other account types or the decoder is outdated for the program. Any account's current data can be checked with `getAccountInfo` or a block explorer.

## How `rpc-gpa` works

- **One request.** `GpaDatasource` sends a single `getProgramAccounts` call and sends every account into the pipeline. When it's done, the datasource finishes; a pipeline with no other datasource then exits.
- **One slot.** All accounts carry the response's context slot in `input.metadata.slot`. `transaction_signature` is `None`.
- **Encoding and context are set for you.** The datasource always requests `Base64` and `withContext`, whatever your config says. `filters`, `commitment` and `min_context_slot` are passed through.
- **Everything is in memory.** The node builds the whole response and the datasource holds it until it's sent. Filter as tightly as you can.
- **Don't use `data_slice`.** Decoders need each account's full data. A sliced account fails to decode and is skipped without a message.

`GpaDatasource::new(url, program)` sends no filters at all. That works for programs with few accounts; on large programs the public endpoint refuses or times out.

### Write filters

| Filter                                                        | Matches                                                                                                                   |
| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `RpcFilterType::DataSize(165)`                                | Accounts of exactly that many bytes (165 is an SPL token account). Cheap for the node; good for fixed-size account types. |
| `RpcFilterType::Memcmp(Memcmp::new_raw_bytes(offset, bytes))` | Accounts whose data at `offset` equals `bytes`. Use offset 0 with the account's discriminator to pick one account type.   |
| `RpcFilterType::TokenAccountState`                            | Valid SPL token accounts (Token program only).                                                                            |

All filters must match. The discriminator of each account type is in the generated decoder's `accounts/<type>.rs` (`decode` compares `data[0..8]` with it). Field offsets follow from the account's layout: 8 bytes of discriminator, then each field in order with its Borsh size.

## Large programs with Helius gPA v2

Helius's `getProgramAccountsV2` returns accounts in pages and can return only the accounts that changed since a slot. The datasource fetches page after page and stops after the last one.

**RPC**

```rust
let datasource = GpaDatasource::new_with_config(
    "https://api.mainnet-beta.solana.com".to_string(),
    PROGRAM_ID,
    config, // RpcProgramAccountsConfig with your filters
);
```

One request, any RPC endpoint.

**Helius**

`Cargo.toml`:

```toml
[dependencies]
carbon-helius-gpa-v2-datasource = "2.0.0"
```

```rust
use carbon_helius_gpa_v2_datasource::{HeliusGpaV2Config, HeliusGpaV2Datasource};

let datasource = HeliusGpaV2Datasource::new_with_config(
    std::env::var("HELIUS_RPC_URL").expect("HELIUS_RPC_URL must be set"),
    PROGRAM_ID,
    HeliusGpaV2Config::new(
        Some(config), // the same RpcProgramAccountsConfig
        None,         // changed_since_slot
        Some(5_000),  // accounts per page (default 1,000, max 10,000)
    ),
);
```

Pages until the last one, on a Helius endpoint such as `https://mainnet.helius-rpc.com/?api-key=…`.

With Helius you can drop the mint filters and page through every DLMM pool. Everything else in the program stays the same, and the processor sees the same `AccountProcessorInputType`. Differences from `rpc-gpa`:

- **Each page has its own slot.** `input.metadata.slot` is the context slot of the page the account came in, so slots increase during a long load.
- **It waits for the pipeline** between accounts, so memory stays bounded to one page.
- **No retries.** An HTTP or RPC error stops the datasource with `FailedToConsumeDatasource("RPC error: …")`. Restart the load, or use incremental syncs below so a restart costs less.

### Sync only what changed

`changed_since_slot` returns only the accounts modified since that slot. Run a full load once, store the highest slot you saw, then run incremental loads from it, for example on a schedule:

```rust
let last_synced_slot: u64 = 467_112_209; // read it from your database

let config = HeliusGpaV2Config::new(
    Some(lb_pair_filters),
    Some(last_synced_slot),
    Some(5_000),
);
```

After each run, store the highest `input.metadata.slot` your processor handled. Accounts closed since the last sync aren't returned, because they no longer exist; detect closures with a stream's [account deletions](https://carbon.sevenlabs.org/concepts/updates.md#account-deletions) or a periodic full load.

## Keep the state current

A load is a point-in-time copy. To keep it current, run it together with a stream of changes in the same pipeline, `rpc-program-subscribe` on any RPC or a [Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md#stream-account-changes) account filter, and apply updates in slot order per account. [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md#load-state-then-follow-changes) has a complete, runnable example. For a whole program at a past slot without any RPC load, use a [validator snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md).

## Common problems

### `error consuming datasource: FailedToConsumeDatasource("Failed to fetch program accounts")`

The RPC node rejected or dropped the request, and the datasource doesn't log why. Usual causes: no filters on a large program (the public endpoint refuses some programs entirely), a response too large to finish before the timeout, or rate limiting. Add filters, or use a paid endpoint or Helius gPA v2. To see the node's answer, send the same request with `curl`.

### The processor sees fewer accounts than `Fetched N accounts`

Some accounts didn't decode: other account types match the filter, a `data_slice` truncated the data, or the decoder doesn't match the program's current layout. Filter on the discriminator of the type you want.

### The pipeline exits right after loading

That's expected: the state loader is finite. Add a stream as a second datasource to keep running (see [Keep the state current](#keep-the-state-current)).

### `RPC error: … (code: …)` from Helius gPA v2

The API key or plan doesn't allow `getProgramAccountsV2`, or a filter is invalid. The load stops at that page; fix the cause and run it again.

## Next steps

- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Load state, then follow changes in one pipeline.
- [Load a snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md): A whole program's accounts at a snapshot's slot.
- [RPC getProgramAccounts datasource](https://carbon.sevenlabs.org/datasources/rpc-gpa.md): The datasource's full reference.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/load-account-state · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Load a snapshot (Carbon 2.0)

> Read every account owned by your programs from a Solana validator snapshot, local or downloaded, with carbon-validator-snapshot-datasource.

A validator snapshot is a full copy of every account on the network at one slot. `carbon-validator-snapshot-datasource` loads one, scans it for the accounts you ask for, and sends them through your pipeline. Use it to bootstrap state for programs too large for [`getProgramAccounts`](https://carbon.sevenlabs.org/guides/load-account-state.md), without any RPC load.

**You need:**

- Rust 1.96.1 or newer, a C++ compiler and libclang (on Debian or Ubuntu: `sudo apt install build-essential clang libclang-dev`). The datasource builds Agave's ledger and runtime crates, including RocksDB.
- A snapshot: a mainnet full snapshot archive was 126 GB in November 2026. Plan for several times that in free disk space, since the accounts are unpacked next to the archive, and for a machine sized like a Solana RPC node: loading builds the full accounts index in memory.
- No account or API key: you can download the snapshot from any RPC node that serves it.

## What the datasource reads

`SnapshotDatasource::new(source, owners, accounts)` takes a source and two lists. It sends every account whose owner is in `owners`, plus every account whose address is in `accounts`. If both lists are empty, it logs `No owners or accounts specified, skipping account scan` and sends nothing.

The source is one of:

| `SnapshotSource`  | Reads                                                                                                                                                                                         |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `LocalPath(path)` | A ledger directory you prepared: `genesis.tar.bz2` at its root and the snapshot archive in `snapshots/`.                                                                                      |
| `Remote { url }`  | A node's base URL. The datasource downloads `<url>/genesis.tar.bz2` and `<url>/snapshot.tar.bz2` into a `ledger/` directory in the current working directory, then loads it as a `LocalPath`. |

A local ledger directory looks like this:

```text
ledger/
├── genesis.tar.bz2
└── snapshots/
    └── snapshot-467081864-Hc57Ndf3cWvzMJmCgAmRyqHNeqLbrWhXXQWQaid7NCsA.tar.zst
```

Keep the archive's original file name: Agave reads the slot and hash from it. The datasource creates `ledger/accounts/` and unpacks the accounts there. If `snapshots/` holds several archives, the one with the highest slot is used.

## Get a snapshot

Agave RPC nodes serve their latest full snapshot at `/snapshot.tar.bz2`, which redirects to the real archive name, and the genesis at `/genesis.tar.bz2`. Solana's public endpoint serves both. Look up the archive's name first:

```sh
NODE=https://api.mainnet-beta.solana.com
SNAPSHOT=$(curl -sI "$NODE/snapshot.tar.bz2" | grep -i '^location:' | sed 's|.*/||; s/\r//')
echo "$SNAPSHOT"
```

Output:

```text
snapshot-467081864-Hc57Ndf3cWvzMJmCgAmRyqHNeqLbrWhXXQWQaid7NCsA.tar.zst
```

Then download the genesis and the archive under that name:

```sh
mkdir -p ledger/snapshots
curl -L -o ledger/genesis.tar.bz2 "$NODE/genesis.tar.bz2"
curl -L -o "ledger/snapshots/$SNAPSHOT" "$NODE/$SNAPSHOT"
```

Any other Agave node that exposes its RPC port works the same way, and is often faster than the rate-limited public endpoint. Snapshots taken by your own validator are already in its ledger directory.

## Load the accounts

This program loads every Meteora DLMM account from a snapshot and prints its pools.

### Step 1: Create a project

```sh
cargo new snapshot-loader
cd snapshot-loader
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "snapshot-loader"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-validator-snapshot-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 3: Write the pipeline

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{accounts::MeteoraDlmmAccount, MeteoraDlmmDecoder, PROGRAM_ID},
    carbon_validator_snapshot_datasource::{SnapshotDatasource, SnapshotSource},
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let source = match env::var("LEDGER_DIR") {
        // A directory with genesis.tar.bz2 and snapshots/snapshot-<slot>-<hash>.tar.zst
        Ok(dir) => SnapshotSource::LocalPath(dir.into()),
        // A node that serves /genesis.tar.bz2 and /snapshot.tar.bz2
        Err(_) => SnapshotSource::Remote {
            url: env::var("SNAPSHOT_URL").expect("set LEDGER_DIR or SNAPSHOT_URL"),
        },
    };

    let datasource = SnapshotDatasource::new(
        source,
        vec![PROGRAM_ID], // load every account owned by these programs
        vec![],           // plus these specific accounts
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(MeteoraDlmmDecoder, PairCounter { pairs: 0 })
        .build()?
        .run()
        .await
}

struct PairCounter {
    pairs: u64,
}

impl Processor<AccountProcessorInputType<'_, MeteoraDlmmAccount>> for PairCounter {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, MeteoraDlmmAccount>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmAccount::LbPair(pair) = &input.decoded_account.data {
            self.pairs += 1;
            println!(
                "#{} slot {} pair {}: {} / {}",
                self.pairs, input.metadata.slot, input.metadata.pubkey, pair.token_x_mint, pair.token_y_mint
            );
        }
        Ok(())
    }
}
```

### Step 4: Build it

```sh
cargo build --release
```

The first build compiles Agave's runtime and RocksDB and takes far longer than a typical Carbon project, often 10 minutes or more. Use `--release`: loading a snapshot with a debug build is much slower.

### Step 5: Run it

Point it at a prepared ledger directory:

```sh
LEDGER_DIR=./ledger RUST_LOG=info cargo run --release
```

Or let it download the snapshot first:

```sh
SNAPSHOT_URL=https://api.mainnet-beta.solana.com RUST_LOG=info cargo run --release
```

With `RUST_LOG=info` the datasource logs each phase: the download progress (`Download: 12.50% | …`), `Snapshot initialization complete: loaded bank at slot …`, `Scanning accounts for 1 owner(s) and 0 specific account(s) at slot …`, then `Account scan completed`. Your processor prints each pool as the scan finds it, every one with the snapshot's slot:

```text
#1 slot 467081864 pair <pool address>: <token X mint> / <token Y mint>
```

When the scan is done, the pipeline exits.

## How the scan behaves

- **One slot.** Every account carries the snapshot's slot in `input.metadata.slot`, and `transaction_signature` is `None`.
- **Only live accounts.** Zero-lamport accounts aren't loadable and are skipped, so the datasource never emits account deletions.
- **Order.** Accounts arrive in the order the accounts database stores them, not sorted by address.
- **Backpressure.** The scan waits for the pipeline, so a slow processor slows the scan but loses nothing.
- **Cancellation.** `Ctrl+C` is checked only before each download attempt and during the scan. A download or snapshot load in progress runs to completion first; kill the process to stop it immediately.
- **Downloads aren't cached.** Each run with `Remote` downloads both files again into `./ledger`. Download once, then switch to `LocalPath("./ledger")`. A failed download is retried 3 times, 2 seconds apart.

## Follow the snapshot with live data

A snapshot is often hours old by the time it's loaded. To keep the state current, add a stream of account changes as a second datasource, a [Yellowstone account filter](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md#stream-account-changes) or [`rpc-program-subscribe`](https://carbon.sevenlabs.org/guides/stream-with-rpc.md#stream-account-changes), and apply updates only when their slot is newer than what you stored for that account. Changes between the snapshot's slot and the moment the stream starts aren't covered by either: replay them with [LaserStream's `from_slot`](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md#start-from-a-past-slot) if the snapshot is recent enough, or reload the changed accounts with [`getProgramAccounts`](https://carbon.sevenlabs.org/guides/load-account-state.md). [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md#load-state-then-follow-changes) shows the pattern with `getProgramAccounts`; the snapshot datasource drops into the same place.

## Common problems

### Build fails: `Unable to find libclang` or `couldn't find any valid shared libraries matching: ['libclang.so', …]`

RocksDB's bindings need libclang at build time. Install it (`sudo apt install clang libclang-dev`, or `brew install llvm` on macOS), or set `LIBCLANG_PATH` to the directory that contains `libclang.so`.

### Build fails with `No space left on device`

Agave's crates and RocksDB need several gigabytes in `target/`. Free space, or point `CARGO_TARGET_DIR` at a larger disk.

### `FailedToConsumeDatasource("Ledger directory does not exist: …")`

`LEDGER_DIR` points to a missing directory. Use the directory that contains `genesis.tar.bz2`, not the `snapshots/` directory itself.

### `Genesis archive not found: … Expected genesis.tar.bz2`

Download `genesis.tar.bz2` from the same cluster as the snapshot and put it at the root of the ledger directory.

### `No loadable full snapshot archive found in …/snapshots`

The `snapshots/` directory has no archive with a name Agave recognizes. Keep the `snapshot-<slot>-<hash>.tar.zst` name the server gave it.

### `Failed to download …/snapshot.tar.bz2 after 3 retries: HTTP error: …`

`SNAPSHOT_URL` must be the node's base URL (`https://api.mainnet-beta.solana.com`), not the snapshot file's URL: the datasource appends `/genesis.tar.bz2` and `/snapshot.tar.bz2` itself. A `429` or `413` status means the node rate-limited you; download from another node.

### The process is killed while loading

The machine ran out of memory building the accounts index. Use a machine with more RAM.

### Nothing is printed and the log says `No owners or accounts specified`

Both lists passed to `SnapshotDatasource::new` are empty. Add your program ID to `owners`.

## Next steps

- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md): getProgramAccounts and Helius gPA v2 for smaller programs.
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Combine a state load with a live stream.
- [Validator snapshot datasource](https://carbon.sevenlabs.org/datasources/validator-snapshot.md): The datasource's full reference.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/load-a-snapshot · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Use multiple datasources (Carbon 2.0)

> Combine datasources in one pipeline, such as a state load plus a live stream or two streams for redundancy, and route or deduplicate their updates.

A pipeline can read from several datasources at once. Use it to load current state and then follow changes, to backfill history while streaming new data, or to run two streams for redundancy. This guide builds the first case and runs it on the public endpoint, then shows redundancy with `DeduplicationFilter`.

**You need:** Rust 1.96.1 or newer. The first two examples run on `https://api.mainnet-beta.solana.com`. The production redundancy setup needs two Yellowstone gRPC endpoints.

## How several datasources run

Each datasource runs in its own task, and all of them send into the pipeline's single channel, so their updates interleave in arrival order. Every update carries the `DatasourceId` of the datasource that produced it, which [filters](https://carbon.sevenlabs.org/concepts/filters.md) can read. Processors don't see the ID; route with a filter instead.

The pipeline runs until every datasource has finished or you press `Ctrl+C`. A finite datasource, such as `rpc-gpa` or a bounded crawler, ends and the others keep going. A datasource that fails logs `error consuming datasource: …` and stops without affecting the others.

## Load state, then follow changes

This program loads the current state of every SOL/USDC Meteora DLMM pool with `getProgramAccounts`, and follows their changes with `programSubscribe`. Two pipes share one decoder type, each limited to one datasource with a `DatasourceFilter`, and a shared map applies an update only if it's at least as new as what's stored, so a late snapshot row can't overwrite a newer live one.

### Step 1: Create a project

```sh
cargo new dlmm-sync
cd dlmm-sync
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dlmm-sync"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-gpa-datasource = "2.0.0"
carbon-rpc-program-subscribe-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
solana-pubkey = "4.2.1"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 3: Write the pipeline

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType,
        datasource::DatasourceId,
        error::CarbonResult,
        filter::DatasourceFilter,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{accounts::MeteoraDlmmAccount, MeteoraDlmmDecoder, PROGRAM_ID},
    carbon_rpc_gpa_datasource::GpaDatasource,
    carbon_rpc_program_subscribe_datasource::{Filters, RpcProgramSubscribe},
    solana_client::{
        rpc_config::{
            CommitmentConfig, RpcAccountInfoConfig, RpcProgramAccountsConfig, UiAccountEncoding,
        },
        rpc_filter::{Memcmp, RpcFilterType},
    },
    solana_pubkey::{pubkey, Pubkey},
    std::{
        collections::HashMap,
        sync::{Arc, Mutex},
    },
};

const LB_PAIR_DISCRIMINATOR: [u8; 8] = [33, 11, 49, 98, 181, 101, 177, 13];
const WSOL: Pubkey = pubkey!("So11111111111111111111111111111111111111112");
const USDC: Pubkey = pubkey!("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");

/// SOL/USDC `LbPair` accounts: discriminator, then token X mint at byte 88
/// and token Y mint at byte 120.
fn sol_usdc_pairs() -> RpcProgramAccountsConfig {
    RpcProgramAccountsConfig {
        filters: Some(vec![
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(0, LB_PAIR_DISCRIMINATOR.to_vec())),
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(88, WSOL.to_bytes().to_vec())),
            RpcFilterType::Memcmp(Memcmp::new_raw_bytes(120, USDC.to_bytes().to_vec())),
        ]),
        account_config: RpcAccountInfoConfig {
            encoding: Some(UiAccountEncoding::Base64),
            commitment: Some(CommitmentConfig::confirmed()),
            ..Default::default()
        },
        ..Default::default()
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let snapshot_id = DatasourceId::new_named("snapshot");
    let live_id = DatasourceId::new_named("live");

    let snapshot = GpaDatasource::new_with_config(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        sol_usdc_pairs(),
    );
    let live = RpcProgramSubscribe::new(
        "wss://api.mainnet-beta.solana.com".to_string(),
        Filters::new(PROGRAM_ID, Some(sol_usdc_pairs())),
    );

    // Shared by both processors: the newest slot applied for each pair.
    let state = Arc::new(Mutex::new(HashMap::new()));

    Pipeline::builder()
        .datasource_with_id(snapshot, snapshot_id.clone())
        .datasource_with_id(live, live_id.clone())
        .account_with_filters(
            MeteoraDlmmDecoder,
            PairWriter { source: "snapshot", state: state.clone() },
            vec![Box::new(DatasourceFilter::new(snapshot_id))],
        )
        .account_with_filters(
            MeteoraDlmmDecoder,
            PairWriter { source: "live", state },
            vec![Box::new(DatasourceFilter::new(live_id))],
        )
        .build()?
        .run()
        .await
}

struct PairWriter {
    source: &'static str,
    state: Arc<Mutex<HashMap<Pubkey, u64>>>,
}

impl Processor<AccountProcessorInputType<'_, MeteoraDlmmAccount>> for PairWriter {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, MeteoraDlmmAccount>,
    ) -> CarbonResult<()> {
        let MeteoraDlmmAccount::LbPair(pair) = &input.decoded_account.data else {
            return Ok(());
        };
        let (pubkey, slot) = (input.metadata.pubkey, input.metadata.slot);

        let mut state = self.state.lock().unwrap();
        if state.get(&pubkey).is_some_and(|&seen| seen > slot) {
            return Ok(()); // older than what we already have
        }
        state.insert(pubkey, slot);

        println!("{:<8} slot {slot} pair {pubkey}: active_id={}", self.source, pair.active_id);
        Ok(())
    }
}
```

In a real indexer, the map is your database: upsert each account with its slot and skip writes whose slot is older than the stored one.

### Step 4: Run it

```sh
RUST_LOG=info cargo run
```

The snapshot's 113 pools print first, all at the same slot, then live updates follow as swaps move the pools:

Output:

```text
[2026-11-09T00:13:01Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 2, num_exporters: 0, num_account_pipes: 2, num_account_deletion_pipes: 0, num_instruction_pipes: 0, num_transaction_pipes: 0
[2026-11-09T00:13:03Z INFO  carbon_rpc_gpa_datasource] Fetched 113 accounts for program LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo (slot: 467112022)
snapshot slot 467112022 pair 1jw5fDodwGEGBVqNXsx2eqiLgNmgMDEeXWSbrTreLCM: active_id=-2056
snapshot slot 467112022 pair 5rCf1DM8LjKTw4YqhnoLcngyZYeNnQqztScTogYHAS6: active_id=-6112
snapshot slot 467112022 pair BGm1tav58oGcsQJehL9WXBFXF7D27vZsKefj4xJKD5Y: active_id=-2446
…
live     slot 467112036 pair 2L47gbUzGsnJZqRzfny4b3VWvKBnMRTMeNH3uzFrj6ko: active_id=-307
live     slot 467112051 pair 5rCf1DM8LjKTw4YqhnoLcngyZYeNnQqztScTogYHAS6: active_id=-6112
live     slot 467112059 pair HTvjzsfX3yU6BUodCjZ5vZkUrAxMDTrBs3CJaq43ashR: active_id=-24438
live     slot 467112059 pair 5rCf1DM8LjKTw4YqhnoLcngyZYeNnQqztScTogYHAS6: active_id=-6111
…
```

The `getProgramAccounts` datasource finishes after the load; the subscription keeps the pipeline running until `Ctrl+C`.

The subscription starts at the same time as the load, so changes made while the load is in flight still arrive, and the slot check discards any that the load already covers. To follow changes with a gRPC stream instead, replace the `live` datasource:

**RPC**

```rust
let live = RpcProgramSubscribe::new(
    "wss://api.mainnet-beta.solana.com".to_string(),
    Filters::new(PROGRAM_ID, Some(sol_usdc_pairs())),
);
```

**Yellowstone gRPC**

```rust
// account_filters: a SubscribeRequestFilterAccounts with owner = PROGRAM_ID and
// the three memcmp filters, as in /guides/stream-with-yellowstone#stream-account-changes
let live = YellowstoneGrpcGeyserClient::new(
    env::var("GEYSER_URL").expect("GEYSER_URL must be set"),
    env::var("X_TOKEN").ok(),
    Some(CommitmentLevel::Confirmed),
    account_filters,
    HashMap::new(),
    BlockFilters::default(),
    YellowstoneGrpcClientConfig::default(),
    None,
    None,
);
```

**Helius LaserStream**

```rust
// account_filters: the same SubscribeRequestFilterAccounts as for Yellowstone
let live = LaserStreamGeyserClient::new(
    env::var("LASERSTREAM_URL").expect("LASERSTREAM_URL must be set"),
    Some(env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set")),
    Some(CommitmentLevel::Confirmed),
    account_filters,
    HashMap::new(),
    BlockFilters::default(),
    LaserStreamClientConfig::default(),
);
```

The gRPC datasources need the extra dependencies and the rustls provider setup from their guides ([Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md), [LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md)). For programs too large for `getProgramAccounts`, load the state from [Helius gPA v2](https://carbon.sevenlabs.org/guides/load-account-state.md#large-programs-with-helius-gpa-v2) or a [validator snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md) instead; the pipeline is the same.

## Deduplicate two redundant streams

Two datasources that deliver the same transactions, such as two Yellowstone endpoints from different providers, keep the pipeline fed when one of them drops. Each transaction then arrives twice; a `DeduplicationFilter` on the pipe lets only the first copy through.

This program demonstrates it on the public endpoint, with two block crawlers reading the same block:

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-block-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult,
        filter::{DeduplicationFilter, Filter},
        instruction::InstructionProcessorInputType,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_rpc_block_crawler_datasource::{RpcBlockConfig, RpcBlockCrawler},
    solana_client::rpc_config::{CommitmentConfig, TransactionDetails, UiTransactionEncoding},
    std::{env, time::Duration},
};

/// Reads one block, 467110307.
fn crawler() -> RpcBlockCrawler {
    RpcBlockCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        467_110_307,
        Some(467_110_307),
        None,
        RpcBlockConfig {
            encoding: Some(UiTransactionEncoding::Base64),
            transaction_details: Some(TransactionDetails::Full),
            rewards: Some(false),
            commitment: Some(CommitmentConfig::finalized()),
            max_supported_transaction_version: Some(1),
        },
        Some(1),
        None,
    )
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // Set NO_DEDUP=1 to see every duplicate.
    let filters: Vec<Box<dyn Filter>> = if env::var("NO_DEDUP").is_err() {
        vec![Box::new(DeduplicationFilter::new(Duration::from_secs(60)))]
    } else {
        vec![]
    };

    Pipeline::builder()
        .datasource(crawler())
        .datasource(crawler())
        .instruction_with_filters(MeteoraDlmmDecoder, SwapLogger, filters)
        .channel_buffer_size(10_000)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let meta = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => {
                println!("swap  {}: amount_in={}", meta.signature, data.amount_in)
            }
            MeteoraDlmmInstruction::Swap2 { data, .. } => {
                println!("swap2 {}: amount_in={}", meta.signature, data.amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}
```

```sh
cargo run
```

Each swap prints once, although both crawlers delivered it:

Output:

```text
swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
```

Without the filter, every line appears twice:

```sh
NO_DEDUP=1 cargo run
```

Output:

```text
swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
swap  ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
swap  48kbDBkFMweb1bVYk9o3uGSR9CQS5pkgm8YCGB9Qg9ymui5dXmwjjoiSAHfF4eQ8mt6j688pkrG8n33GnZX4cffd: amount_in=1880866
swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
swap2 2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
```

### In production: two gRPC streams

Swap the crawlers for two streams with the same filters. With the dependencies from [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md), the datasource part of the program becomes:

```rust
fn geyser(endpoint: String, x_token: Option<String>) -> YellowstoneGrpcGeyserClient {
    let mut transaction_filters = HashMap::new();
    transaction_filters.insert(
        "meteora_dlmm".to_string(),
        SubscribeRequestFilterTransactions {
            vote: Some(false),
            failed: Some(false),
            account_required: vec![PROGRAM_ID.to_string()],
            ..Default::default()
        },
    );

    YellowstoneGrpcGeyserClient::new(
        endpoint,
        x_token,
        Some(CommitmentLevel::Confirmed),
        HashMap::new(),
        transaction_filters,
        BlockFilters::default(),
        YellowstoneGrpcClientConfig::default(),
        None,
        None,
    )
}

// in main, after installing the rustls provider:
let primary = geyser(env::var("GEYSER_URL_A").unwrap(), env::var("X_TOKEN_A").ok());
let backup = geyser(env::var("GEYSER_URL_B").unwrap(), env::var("X_TOKEN_B").ok());

Pipeline::builder()
    .datasource(primary)
    .datasource(backup)
    .instruction_with_filters(
        MeteoraDlmmDecoder,
        SwapLogger,
        vec![Box::new(DeduplicationFilter::new(Duration::from_secs(60)))],
    )
```

A Yellowstone stream and a [LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md) stream combine the same way.

### What `DeduplicationFilter` covers

| Pipe                                         | Considered a duplicate when                                          | Not deduplicated                                                                                  |
| -------------------------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| Instruction                                  | Same transaction signature and same position in the instruction tree | —                                                                                                 |
| Account                                      | Same transaction signature and same account                          | Account updates without a signature: `rpc-program-subscribe`, `rpc-gpa`, Helius gPA v2, snapshots |
| Transaction, account deletion, block details | Never                                                                | Every update                                                                                      |

Keep these limits in mind:

- **One instance per pipe.** Each `DeduplicationFilter` remembers only what passed through its own pipe. Give every pipe that needs it its own.
- **The TTL is a window.** It forgets an entry after its TTL (entries are cleaned up at most once a minute). Set it longer than the worst lag between your sources: 60 seconds covers two healthy live streams; raise it to a few minutes if a source can lag further, for example while it reconnects.
- **In memory only.** A restart forgets everything, so storage writes should still be idempotent.

For transaction processors and signature-less account sources, deduplicate in your processor or database, for example with a unique key on the signature.

## Route updates by source

`datasource_with_id` names a datasource; `DatasourceFilter::new(id)` (or `new_many(vec![…])`) limits a pipe to the named ones. Use it when sources need different processing:

```rust
let backfill = DatasourceId::new_named("backfill");
let live = DatasourceId::new_named("live");

Pipeline::builder()
    .datasource_with_id(block_crawler, backfill.clone())
    .datasource_with_id(yellowstone, live.clone())
    .instruction_with_filters(MeteoraDlmmDecoder, BulkWriter, vec![Box::new(DatasourceFilter::new(backfill))])
    .instruction_with_filters(MeteoraDlmmDecoder, LiveWriter, vec![Box::new(DatasourceFilter::new(live))])
```

Here a [block crawler](https://carbon.sevenlabs.org/guides/backfill-transactions.md#crawl-a-slot-range) fills history from a stored slot while Yellowstone streams from now on: the backfill can batch its writes, and the live writer can publish notifications that make no sense for old data. `DatasourceId::new_named` compares by name, so build each ID once and clone it, and never give two datasources the same name. `.datasource(…)` assigns a random ID you can't filter on.

## Common problems

### Every update appears twice

Two datasources deliver the same data and the pipe has no `DeduplicationFilter`, or the updates have no signature to deduplicate on (see the table above). Add the filter, or make writes idempotent.

### A pipe with a `DatasourceFilter` receives nothing

The datasource was added with `.datasource(…)` instead of `.datasource_with_id(…)`, or the IDs differ in spelling. Build the `DatasourceId` once and pass clones to both places.

### Old values overwrite newer ones

A finite source (a load or a backfill) delivered an account after the stream had already delivered a newer version. Compare `input.metadata.slot` with the stored slot before writing, as in the first example.

### The pipeline keeps running after the load finished

That's expected while a stream is still running. The pipeline stops when every datasource has finished, or on `Ctrl+C`.

### One datasource failed and the pipeline kept going without it

Each datasource fails independently and the pipeline only logs `error consuming datasource: …`. Watch the per-datasource counters with [metrics](https://carbon.sevenlabs.org/guides/prometheus.md), and alert when one of them stops growing.

## Next steps

- [Filters](https://carbon.sevenlabs.org/concepts/filters.md): DatasourceFilter, DeduplicationFilter, SlotRangeFilter and custom filters.
- [Multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md): Several decoders in one pipeline.
- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md): History sources to combine with a stream.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/multiple-datasources · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Index multiple programs (Carbon 2.0)

> Decode several programs in one pipeline with one instruction pipe per decoder, or a decoder collection and a transaction processor, and pick a datasource filter that covers them all.

One pipeline can decode any number of programs. Register one pipe per decoder when each program's instructions stand on their own, or one transaction pipe with a decoder collection when you need to see several programs' instructions together. Either way, the datasource must deliver every program's transactions. This guide runs both approaches on the public endpoint with the Meteora DLMM and Pump.fun AMM decoders.

**You need:** Rust 1.96.1 or newer. The examples read two mainnet blocks from `https://api.mainnet-beta.solana.com`; no account is needed.

## One pipe per decoder

Each `.instruction(decoder, processor)` call adds a pipe. For every transaction, the pipeline runs each pipe over every instruction, top-level and nested. A decoder returns `None` for other programs' instructions, so each processor only sees its own program, with the decoder's full types.

### Step 1: Create a project

```sh
cargo new dex-indexer
cd dex-indexer
```

### Step 2: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "dex-indexer"
version = "0.1.0"
edition = "2024"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-pump-swap-decoder = "2.0.0"
carbon-rpc-block-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
```

### Step 3: Write the pipeline

The block crawler delivers every transaction in a block, whichever program it calls, so it covers both decoders without any filter.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_pump_swap_decoder::{instructions::PumpSwapInstruction, PumpSwapDecoder},
    carbon_rpc_block_crawler_datasource::{RpcBlockConfig, RpcBlockCrawler},
    solana_client::rpc_config::{CommitmentConfig, TransactionDetails, UiTransactionEncoding},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // Every transaction in two blocks, whichever programs it calls.
    let datasource = RpcBlockCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        467_110_307,
        Some(467_110_308),
        None,
        RpcBlockConfig {
            encoding: Some(UiTransactionEncoding::Base64),
            transaction_details: Some(TransactionDetails::Full),
            rewards: Some(false),
            commitment: Some(CommitmentConfig::finalized()),
            max_supported_transaction_version: Some(1),
        },
        Some(1),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, DlmmLogger)
        .instruction(PumpSwapDecoder, PumpSwapLogger)
        .channel_buffer_size(10_000)
        .build()?
        .run()
        .await
}

struct DlmmLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for DlmmLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let signature = input.metadata.transaction_metadata.signature;
        match input.decoded_instruction {
            MeteoraDlmmInstruction::Swap { data, .. } => {
                println!("dlmm swap       {signature}: amount_in={}", data.amount_in)
            }
            MeteoraDlmmInstruction::Swap2 { data, .. } => {
                println!("dlmm swap2      {signature}: amount_in={}", data.amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}

struct PumpSwapLogger;

impl Processor<InstructionProcessorInputType<'_, PumpSwapInstruction>> for PumpSwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, PumpSwapInstruction>,
    ) -> CarbonResult<()> {
        let signature = input.metadata.transaction_metadata.signature;
        match input.decoded_instruction {
            PumpSwapInstruction::Buy { data, .. } => {
                println!("pump_swap buy   {signature}: base_amount_out={}", data.base_amount_out)
            }
            PumpSwapInstruction::Sell { data, .. } => {
                println!("pump_swap sell  {signature}: base_amount_in={}", data.base_amount_in)
            }
            _ => {}
        }
        Ok(())
    }
}
```

### Step 4: Run it

```sh
cargo run
```

Both processors print, interleaved in block order, and the program exits after the second block:

Output:

```text
pump_swap sell  5UzcVzyLK7uM1ttjjcN98zLV1s9b7xJ3e572bthzEorbiLCbcEXzDPNaPXnkv8AV1zw6Pbfs7cfddFaov54e3oaH: base_amount_in=28426915529
pump_swap sell  4wwzVEoKvEnqL98QXEEE9ftRUgYnLCDHLt5WiYiYdG9xzvYFFKPNEq9pHCvYURw7vsSHhYa5vBLvc8cU66oYJujT: base_amount_in=14949087
pump_swap sell  4e4qr8vA7t39w5TCMygkGk4DHC6RWo1BnwBgiQ6BfhaqDoxHuTqcNSmkWe8dszozPFeBrVbHyRapfAd9juR49S1P: base_amount_in=3680531
dlmm swap       ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: amount_in=10298583
pump_swap sell  36N7ynNwzHfkaa2gKNPRdj9AnFyLdx8U1qudgC2UKUziR4uL8sCRYkoK4jM8B9q3DoE8ebN8AaqmHiMPR6Xjiqn8: base_amount_in=208155812
…
dlmm swap2      2d6GNQzSzxgV85mnzD4ed6JyxdLYdwBtyCH8LujdYaxySSKfD8x96yJLtxmuVYt6XGRjy8m3AutxyxqiumscEb4W: amount_in=381756
pump_swap buy   2i5pWH6S4KZP6Q232XiMWPxS7MXwJ9JHqB38TvLtmJPkusy7AziWhbrVZXFEBF9pcdVFUScAQx5V5diptWaQc9H8: base_amount_out=1110999784
…
```

Within one transaction, the pipes run in the order you registered them: all of the DLMM pipe's instructions, then all of the Pump.fun AMM pipe's. Pipes don't share state; if two processors need the same data, share it with an `Arc<Mutex<…>>` or write to the same database.

Accounts work the same way: add `.account(MeteoraDlmmDecoder, …)` and `.account(PumpSwapDecoder, …)`. Each generated account decoder checks the account's owner, so it ignores the other program's accounts.

## One transaction processor for several programs

A transaction processor sees a whole transaction at once: its metadata and every instruction that any of your programs' decoders understands, in execution order. Use it when one row depends on several programs, such as an aggregator route through two DEXes, or when you classify transactions by what they contain. See [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md) for how transaction pipes work.

A transaction pipe needs a type that implements `InstructionDecoderCollection`: an enum with one variant per program. In 2.0.0, the `instruction_decoder_collection!` macro can't take generated decoders' instruction enums directly, because they don't derive `Eq`, `Hash` or `InstructionType`. You can either wrap each decoder in a slim enum and use the macro, as shown in [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md#example), or implement the trait by hand over the full generated types, as here.

Add `serde` and `solana-instruction`, and turn on the decoders' `serde` feature (the trait requires `Serialize`):

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["serde"] }
carbon-pump-swap-decoder = { version = "2.0.0", features = ["serde"] }
carbon-rpc-block-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
serde = { version = "1.0.228", features = ["derive"] }
solana-client = "=4.2.2"
solana-instruction = "3.4.1"
tokio = { version = "1.54.1", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        collection::InstructionDecoderCollection,
        error::CarbonResult,
        instruction::InstructionDecoder,
        pipeline::Pipeline,
        processor::Processor,
        transaction::TransactionProcessorInputType,
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_pump_swap_decoder::{instructions::PumpSwapInstruction, PumpSwapDecoder},
    carbon_rpc_block_crawler_datasource::{RpcBlockConfig, RpcBlockCrawler},
    solana_client::rpc_config::{CommitmentConfig, TransactionDetails, UiTransactionEncoding},
    std::hash::{Hash, Hasher},
};

/// One enum over every program the transaction processor cares about.
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
pub enum AllInstructions {
    MeteoraDlmm(MeteoraDlmmInstruction),
    PumpSwap(PumpSwapInstruction),
}

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AllPrograms {
    MeteoraDlmm,
    PumpSwap,
}

// `InstructionDecoderCollection` requires `Eq + Hash`; the decoded
// instruction types only implement `PartialEq`, so implement both by hand.
impl Eq for AllInstructions {}

impl Hash for AllInstructions {
    fn hash<H: Hasher>(&self, state: &mut H) {
        std::mem::discriminant(self).hash(state);
    }
}

impl InstructionDecoderCollection for AllInstructions {
    type InstructionType = AllPrograms;

    fn parse_instruction(instruction: &solana_instruction::Instruction) -> Option<Self> {
        match instruction.program_id {
            carbon_meteora_dlmm_decoder::PROGRAM_ID => MeteoraDlmmDecoder
                .decode_instruction(instruction)
                .map(AllInstructions::MeteoraDlmm),
            carbon_pump_swap_decoder::PROGRAM_ID => PumpSwapDecoder
                .decode_instruction(instruction)
                .map(AllInstructions::PumpSwap),
            _ => None,
        }
    }

    fn get_type(&self) -> Self::InstructionType {
        match self {
            AllInstructions::MeteoraDlmm(_) => AllPrograms::MeteoraDlmm,
            AllInstructions::PumpSwap(_) => AllPrograms::PumpSwap,
        }
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let datasource = RpcBlockCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        467_110_307,
        Some(467_110_308),
        None,
        RpcBlockConfig {
            encoding: Some(UiTransactionEncoding::Base64),
            transaction_details: Some(TransactionDetails::Full),
            rewards: Some(false),
            commitment: Some(CommitmentConfig::finalized()),
            max_supported_transaction_version: Some(1),
        },
        Some(1),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .transaction::<AllInstructions, _>(RouteLogger)
        .channel_buffer_size(10_000)
        .build()?
        .run()
        .await
}

struct RouteLogger;

impl Processor<TransactionProcessorInputType<'_, AllInstructions>> for RouteLogger {
    async fn process(
        &mut self,
        input: &TransactionProcessorInputType<'_, AllInstructions>,
    ) -> CarbonResult<()> {
        let programs: Vec<AllPrograms> =
            input.instructions.iter().map(|(_, ix)| ix.get_type()).collect();

        if !programs.is_empty() {
            println!("{}: {programs:?}", input.metadata.signature);
        }
        Ok(())
    }
}
```

```sh
cargo run
```

Each transaction with at least one decoded instruction prints once, with every decoded instruction's program. A DLMM swap shows three entries, the swap and the two events Meteora emits through self-CPI:

Output:

```text
5UzcVzyLK7uM1ttjjcN98zLV1s9b7xJ3e572bthzEorbiLCbcEXzDPNaPXnkv8AV1zw6Pbfs7cfddFaov54e3oaH: [PumpSwap, PumpSwap]
4wwzVEoKvEnqL98QXEEE9ftRUgYnLCDHLt5WiYiYdG9xzvYFFKPNEq9pHCvYURw7vsSHhYa5vBLvc8cU66oYJujT: [PumpSwap, PumpSwap]
4e4qr8vA7t39w5TCMygkGk4DHC6RWo1BnwBgiQ6BfhaqDoxHuTqcNSmkWe8dszozPFeBrVbHyRapfAd9juR49S1P: [PumpSwap, PumpSwap]
ue82HW7rk3DcWiCTzRrnMzMeKDYDLWUiyeMBtMi4tLtqiKaRGgbr7EYigU9RbpftsFsgEC7P6kgxkNWJgZhvKZs: [MeteoraDlmm, MeteoraDlmm, MeteoraDlmm]
36N7ynNwzHfkaa2gKNPRdj9AnFyLdx8U1qudgC2UKUziR4uL8sCRYkoK4jM8B9q3DoE8ebN8AaqmHiMPR6Xjiqn8: [PumpSwap, PumpSwap]
…
```

To work with the payloads, match on them: `AllInstructions::MeteoraDlmm(MeteoraDlmmInstruction::Swap2 { data, .. })`. Each entry of `input.instructions` also carries its [`InstructionMetadata`](https://carbon.sevenlabs.org/concepts/instructions.md#instruction-metadata), including its position in the instruction tree.

The transaction processor runs for **every** transaction the datasource delivers, including those where nothing decoded, so check for an empty list. It can run next to instruction pipes in the same pipeline; for each transaction, the instruction pipes run first.

### Which approach to use

|                                                                                               | One pipe per decoder                  | Transaction processor                            |
| --------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------ |
| Sees                                                                                          | One instruction at a time             | A whole transaction                              |
| Types                                                                                         | The decoder's own types               | Your collection enum                             |
| Setup                                                                                         | One `.instruction()` call per program | A collection type, then `.transaction::<T, _>()` |
| Runs                                                                                          | Once per decoded instruction          | Once per transaction, decoded or not             |
| [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter) | Supported                             | Not applied to transaction pipes                 |

## Choose a datasource filter

Every datasource must deliver the transactions of all your programs. How you widen its filter depends on the datasource:

**RPC**

- **Block crawler:** delivers every transaction in each block. No change needed.
- **Block subscribe:** `MentionsAccountOrProgram` takes one address. Use `RpcBlockSubscribeFilter::All` (the whole network's transactions) or one datasource per program.
- **Transaction crawler:** crawls one address. Add one crawler per program. A transaction that calls both programs is then delivered twice, so add a `DeduplicationFilter` to each instruction pipe:

```rust
Pipeline::builder()
    .datasource(RpcTransactionCrawler::new(rpc_url.clone(), carbon_meteora_dlmm_decoder::PROGRAM_ID, ConnectionConfig::default(), Filters::new(None, None, None), None))
    .datasource(RpcTransactionCrawler::new(rpc_url, carbon_pump_swap_decoder::PROGRAM_ID, ConnectionConfig::default(), Filters::new(None, None, None), None))
    .instruction_with_filters(MeteoraDlmmDecoder, DlmmLogger, vec![Box::new(DeduplicationFilter::new(Duration::from_secs(300)))])
    .instruction_with_filters(PumpSwapDecoder, PumpSwapLogger, vec![Box::new(DeduplicationFilter::new(Duration::from_secs(300)))])
```

- **getProgramAccounts and program subscribe:** one program per datasource; add one datasource per program.

**Yellowstone gRPC**

List every program in `account_include`, which matches transactions that mention **any** of them. `account_required` would require **all** of them in the same transaction.

```rust
SubscribeRequestFilterTransactions {
    vote: Some(false),
    failed: Some(false),
    account_include: vec![
        carbon_meteora_dlmm_decoder::PROGRAM_ID.to_string(),
        carbon_pump_swap_decoder::PROGRAM_ID.to_string(),
    ],
    ..Default::default()
}
```

Several entries in the filter map also work: the server sends a transaction once if it matches any entry. For accounts, list every program in the account filter's `owner`.

**Helius LaserStream**

LaserStream takes the same Yellowstone filters. List every program in `account_include` (any of them), not `account_required` (all of them):

```rust
SubscribeRequestFilterTransactions {
    vote: Some(false),
    failed: Some(false),
    account_include: vec![
        carbon_meteora_dlmm_decoder::PROGRAM_ID.to_string(),
        carbon_pump_swap_decoder::PROGRAM_ID.to_string(),
    ],
    ..Default::default()
}
```

**Helius**

GTFA and gPA v2 read one address each. Add one datasource per program, with a `DeduplicationFilter` on each instruction pipe for GTFA, since a transaction that calls both programs is returned for both addresses.

Transaction pipes aren't deduplicated, so when two datasources can deliver the same transaction, prefer a single datasource that covers every program, or deduplicate by signature in your processor.

## Common problems

### One program's processor never prints

The datasource doesn't deliver that program's transactions: a Yellowstone filter with `account_required` instead of `account_include`, a crawler for only one address, or a `MentionsAccountOrProgram` filter for the other program. Widen the filter as shown above.

### Instructions are processed twice

Two datasources delivered the same transaction. Add a `DeduplicationFilter` to each instruction pipe, or use one datasource for all programs.

### ``the trait bound `MeteoraDlmmInstruction: Eq` is not satisfied`` (or `Hash`, or ``cannot find type `MeteoraDlmmInstructionType` ``)

You passed a generated instruction enum to `instruction_decoder_collection!`. Implement `InstructionDecoderCollection` by hand as above, or wrap the decoder as in [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md#example).

### ``the trait bound `AllInstructions: serde::Serialize` is not satisfied``

The collection must implement `Serialize`. Turn on each decoder's `serde` feature and derive `serde::Serialize`, with `serde` as a direct dependency.

### The transaction processor prints nothing for transactions you expect

`parse_instruction` returned `None` for all their instructions. Check the `match` on `instruction.program_id` lists each program's `PROGRAM_ID`, and that the datasource delivers those transactions.

## Next steps

- [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md): Transaction pipes, collections and the macro in depth.
- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): Add a program that has no published decoder.
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Combine sources and deduplicate their updates.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/multiple-programs · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Index token transfers (Carbon 2.0)

> Decode SPL Token and Token-2022 transfers of one mint with the token decoders, and turn raw amounts into token amounts.

Every token movement on Solana is an instruction of the SPL Token program or of Token-2022, often nested inside a swap or another program's call. This guide indexes the transfers of one mint from both programs with the published decoders, and prints each one with its token amount.

**You need:** Rust 1.96.1 or newer. The example uses Solana's public RPC, so no account is needed.

## Which instructions move tokens

| Instruction                       | In                    | Has the mint?        | Notes                                                                                                                                                                        |
| --------------------------------- | --------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TransferChecked`                 | SPL Token, Token-2022 | Yes, plus `decimals` | What wallets and most programs send today. Use it to attribute a transfer to a mint.                                                                                         |
| `Transfer`                        | SPL Token, Token-2022 | No                   | Legacy. The mint is only known from the token accounts; see [Plain `Transfer` has no mint](#plain-transfer-has-no-mint).                                                     |
| `TransferCheckedWithFee`          | Token-2022            | Yes                  | For mints with a transfer fee. Its fields decode wrongly in 2.0.0; see the [Token-2022 decoder](https://carbon.sevenlabs.org/decoders/token-2022.md#transfercheckedwithfee). |
| `MintTo`, `Burn` (and `*Checked`) | Both                  | Yes                  | Supply changes rather than transfers; handle them the same way if you need balances.                                                                                         |

A mint belongs to one program: classic tokens such as USDC and JUP use SPL Token, newer ones may use Token-2022. Register both decoders and your indexer works for any mint.

### Step 1: Create a project

```sh
cargo new token-transfers
cd token-transfers
cargo add carbon-core@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0 carbon-token-program-decoder@2.0.0 carbon-token-2022-decoder@2.0.0 carbon-log-metrics@2.0.0 solana-pubkey@4.2.1
cargo add tokio --features full
cargo add env_logger
```

### Step 2: Write the indexer

The [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) fetches every transaction that references the mint's address. Both decoders feed the same processor, which keeps `TransferChecked` instructions for the mint and converts the raw `amount` with the instruction's own `decimals`.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    carbon_token_2022_decoder::{instructions::Token2022Instruction, Token2022Decoder},
    carbon_token_program_decoder::{instructions::TokenProgramInstruction, TokenProgramDecoder},
    solana_pubkey::{pubkey, Pubkey},
    std::sync::Arc,
};

/// The mint whose transfers we index: JUP.
const MINT: Pubkey = pubkey!("JUPyiwrYJFskUPiHa7hkeR8VUtAeFoSYbKedZNsDvCN");

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let rpc_url = std::env::var("RPC_URL")
        .unwrap_or_else(|_| "https://api.mainnet-beta.solana.com".to_string());

    // Every transaction that references the mint, newest first, then new ones as they land.
    let datasource = RpcTransactionCrawler::new(
        rpc_url,
        MINT,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        // A mint belongs to one token program; register both so the code works for any mint.
        .instruction(TokenProgramDecoder, TransferLogger)
        .instruction(Token2022Decoder, TransferLogger)
        .build()?
        .run()
        .await
}

struct TransferLogger;

fn print_transfer(amount: u64, decimals: u8, mint: Pubkey, from: Pubkey, to: Pubkey) {
    if mint != MINT {
        return; // a transfer of another token in the same transaction
    }
    let ui_amount = amount as f64 / 10f64.powi(decimals as i32);
    println!("{ui_amount:>14.6} from {from} to {to}");
}

impl Processor<InstructionProcessorInputType<'_, TokenProgramInstruction>> for TransferLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, TokenProgramInstruction>,
    ) -> CarbonResult<()> {
        if let TokenProgramInstruction::TransferChecked { data, accounts, .. } = input.decoded_instruction {
            print_transfer(data.amount, data.decimals, accounts.mint, accounts.source, accounts.destination);
        }
        Ok(())
    }
}

impl Processor<InstructionProcessorInputType<'_, Token2022Instruction>> for TransferLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, Token2022Instruction>,
    ) -> CarbonResult<()> {
        if let Token2022Instruction::TransferChecked { data, accounts, .. } = input.decoded_instruction {
            print_transfer(data.amount, data.decimals, accounts.mint, accounts.source, accounts.destination);
        }
        Ok(())
    }
}
```

### Step 3: Run it

```sh
cargo run
```

Output:

```text
     15.817487 from CcXcW9Ad7DpvcWjv7iDz6RUUdmBSP8dXZS9EbQktQzXq to DqHAep7hD2VJ4rsWSxkLoBSRMiLkdw7y8DLx1htrryts
     15.817487 from DqHAep7hD2VJ4rsWSxkLoBSRMiLkdw7y8DLx1htrryts to 4gPSQHK96TttxoK3AmAVKk6mJvcDDNhAiRmbZh1sDkyA
     13.616322 from CcXcW9Ad7DpvcWjv7iDz6RUUdmBSP8dXZS9EbQktQzXq to CugEgUFKcEE8CayShReXpkQjfYVJXEGqasidZr99ViiJ
     13.616322 from CugEgUFKcEE8CayShReXpkQjfYVJXEGqasidZr99ViiJ to 4gPSQHK96TttxoK3AmAVKk6mJvcDDNhAiRmbZh1sDkyA
    210.276046 from CcXcW9Ad7DpvcWjv7iDz6RUUdmBSP8dXZS9EbQktQzXq to 47cA3idUSGNbJ6tuKEx8UwPLmPkPZRvBkVjeqTbtDYkZ
…
```

Transfers often come in pairs or chains: a swap routed through an aggregator moves the tokens from the user to an intermediate account and on to the pool. Each hop is its own instruction, nested inside the aggregator's call, and Carbon decodes [nested instructions](https://carbon.sevenlabs.org/concepts/instructions.md) like top-level ones. Use `input.metadata.stack_height` and `input.metadata.absolute_path` to tell a direct transfer from one inside another program's call.

## Index a different mint

Change `MINT`. For a Token-2022 mint nothing else changes. For very busy addresses, such as the USDC mint, Solana's public endpoint can return months-old signatures; use your own RPC endpoint with `RPC_URL`, or stream instead of crawling (below).

## Stream transfers in real time

The crawler polls. For production, subscribe to transactions that include the mint with a Geyser stream, using the same processor:

- [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md): a transaction filter with `account_include: vec![MINT.to_string()]`.
- [Helius LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md): the same filter.

## Plain `Transfer` has no mint

`Transfer` lists only the source account, the destination and the authority. To attribute one to a mint, look up the source token account's mint: from the transaction's token balances (`input.metadata.transaction_metadata.meta.pre_token_balances`, which give each token account's mint by account index), or from your own table of token accounts filled by an [account processor](https://carbon.sevenlabs.org/concepts/processors.md). `TransferChecked` avoids the lookup, which is why most programs send it.

## Common problems

### Nothing prints for a while

The public endpoint is rate limited, and the crawler skips failed transactions. Busy mints print within a few seconds; quiet mints may have no transfers in the last minutes. Set `RPC_URL` to your own endpoint for steady throughput.

### The same transfer prints twice

If you also run a second datasource, deduplicate with [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md#deduplicationfilter). Two lines with different accounts are two hops of a routed swap, not duplicates.

## Next steps

- [SPL Token decoder](https://carbon.sevenlabs.org/decoders/token-program.md): Every instruction and account of the token program.
- [Store transfers in Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Typed tables and idempotent writes.
- [Index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md): Several decoders in one pipeline.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/index-token-transfers · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Index swaps across DEXs (Carbon 2.0)

> Turn Raydium CPMM, PumpSwap and Meteora DLMM swaps into one record with the amounts that executed, read from each program's events, in one pipeline.

Every DEX describes a swap in its own types, but each one also records the amounts that executed in an event. This guide indexes swaps from Raydium CPMM, PumpSwap and Meteora DLMM into one `Swap` record in a single pipeline, using each program's events. The same pattern extends to any program in the [DEX and AMM decoders](https://carbon.sevenlabs.org/decoders/dex-and-amm.md).

**You need:** Rust 1.96.1 or newer. The example uses Solana's public RPC, so no account is needed.

## Where each DEX records a swap

| DEX                                                                   | Event                                                                                                                                                  | How it is emitted       | Pool      | Amount in                                        | Amount out                                         |
| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | --------- | ------------------------------------------------ | -------------------------------------------------- |
| [Raydium CPMM](https://carbon.sevenlabs.org/decoders/raydium-cpmm.md) | [`SwapEvent`](https://carbon.sevenlabs.org/decoders/raydium-cpmm.md#swapevent)                                                                         | `emit!`, a log line     | `pool_id` | `input_amount`                                   | `output_amount`                                    |
| [PumpSwap](https://carbon.sevenlabs.org/decoders/pump-swap.md)        | [`BuyEvent`](https://carbon.sevenlabs.org/decoders/pump-swap.md#buyevent), [`SellEvent`](https://carbon.sevenlabs.org/decoders/pump-swap.md#sellevent) | `emit_cpi!`, a self-CPI | `pool`    | `quote_amount_in` (buy), `base_amount_in` (sell) | `base_amount_out` (buy), `quote_amount_out` (sell) |
| [Meteora DLMM](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md) | [`Swap`](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md#swap-1)                                                                                 | `emit_cpi!`, a self-CPI | `lb_pair` | `amount_in`                                      | `amount_out`                                       |

Read events rather than instruction arguments. The arguments are limits the trader set: Raydium CPMM's `SwapBaseInput` carries `amount_in` and `minimum_amount_out`, and `SwapBaseOutput` carries `max_amount_in` and `amount_out`. The events hold what actually moved.

Anchor programs emit events in one of two ways, and you read them differently:

- **`emit_cpi!`** makes the program call itself with the event as data. The decoder returns it as the `CpiEvent` instruction, so it reaches your processor like any other instruction. PumpSwap and Meteora DLMM do this.
- **`emit!`** writes the event to the transaction's log as a `Program data:` line. Nothing calls your processor for it: decode it from the swap instruction's logs with [`decode_log_events`](https://carbon.sevenlabs.org/concepts/instructions.md#log-events). Raydium CPMM does this, so its `CpiEvent` never arrives.

If a program's `CpiEvent` never shows up, look for `Program data:` lines in one of its transactions: it logs its events.

### Step 1: Create a project

```sh
cargo new dex-swaps
cd dex-swaps
cargo add carbon-core@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0 carbon-raydium-cpmm-decoder@2.0.0 carbon-pump-swap-decoder@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-log-metrics@2.0.0 solana-pubkey@4.2.1
cargo add tokio --features full
cargo add env_logger borsh
```

`borsh` is for the small wrapper that lets `decode_log_events` read Raydium CPMM's logged event.

### Step 2: Write the indexer

Three [RPC transaction crawlers](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), one per program, feed one pipeline. One `SwapIndexer` implements a processor for each program's instructions and turns that program's swap event into the shared `Swap` record.

`src/main.rs`:

```rust
use {
    borsh::BorshDeserialize,
    carbon_core::{
        deserialize::CarbonDeserialize,
        error::CarbonResult,
        filter::{DeduplicationFilter, Filter},
        instruction::InstructionProcessorInputType,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        MeteoraDlmmDecoder,
        instructions::{CpiEvent as DlmmEvent, MeteoraDlmmInstruction},
    },
    carbon_pump_swap_decoder::{
        PumpSwapDecoder,
        instructions::{CpiEvent as PumpSwapEvent, PumpSwapInstruction},
    },
    carbon_raydium_cpmm_decoder::{
        RaydiumCpmmDecoder, events::swap_event::SwapEventEvent,
        instructions::RaydiumCpmmInstruction,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    solana_pubkey::Pubkey,
    std::{sync::Arc, time::Duration},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let rpc_url = std::env::var("RPC_URL")
        .unwrap_or_else(|_| "https://api.mainnet-beta.solana.com".to_string());
    // One crawler per program, all feeding the same pipeline. One request in flight each
    // keeps the three of them under the public endpoint's per-IP rate limit.
    let crawler = |program: Pubkey| {
        RpcTransactionCrawler::new(
            rpc_url.clone(),
            program,
            ConnectionConfig {
                max_concurrent_requests: 1,
                ..ConnectionConfig::default()
            },
            Filters::new(None, None, None),
            None,
        )
    };

    Pipeline::builder()
        .datasource(crawler(carbon_raydium_cpmm_decoder::PROGRAM_ID))
        .datasource(crawler(carbon_pump_swap_decoder::PROGRAM_ID))
        .datasource(crawler(carbon_meteora_dlmm_decoder::PROGRAM_ID))
        .metrics(Arc::new(LogMetrics::new()))
        .instruction_with_filters(RaydiumCpmmDecoder, SwapIndexer, dedupe())
        .instruction_with_filters(PumpSwapDecoder, SwapIndexer, dedupe())
        .instruction_with_filters(MeteoraDlmmDecoder, SwapIndexer, dedupe())
        .build()?
        .run()
        .await
}

/// A transaction that calls two of the programs is fetched by both crawlers. This drops the
/// second copy of each instruction, keyed by signature and position in the transaction.
fn dedupe() -> Vec<Box<dyn Filter>> {
    vec![Box::new(DeduplicationFilter::new(Duration::from_secs(600)))]
}

/// One executed swap, the same shape for every DEX. Amounts are raw token units.
struct Swap {
    dex: &'static str,
    pool: Pubkey,
    amount_in: u64,
    amount_out: u64,
}

struct SwapIndexer;

impl SwapIndexer {
    fn record(&self, signature: impl std::fmt::Display, swap: Swap) {
        // Replace with a database write; see /guides/postgres.
        println!(
            "{:<12} pool={} in={} out={} tx={signature}",
            swap.dex, swap.pool, swap.amount_in, swap.amount_out,
        );
    }
}

/// Raydium CPMM publishes `SwapEvent` with Anchor's `emit!`, as a `Program data:` log line,
/// not as a self-CPI. Wrapping the generated type lets `decode_log_events` read it.
#[derive(BorshDeserialize)]
struct LoggedCpmmSwap(SwapEventEvent);

impl CarbonDeserialize for LoggedCpmmSwap {
    const DISCRIMINATOR: &'static [u8] = &[64, 198, 205, 232, 38, 8, 113, 226];

    fn deserialize(data: &[u8]) -> Option<Self> {
        SwapEventEvent::decode(data).map(LoggedCpmmSwap)
    }
}

impl Processor<InstructionProcessorInputType<'_, RaydiumCpmmInstruction>> for SwapIndexer {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, RaydiumCpmmInstruction>,
    ) -> CarbonResult<()> {
        if !matches!(
            input.decoded_instruction,
            RaydiumCpmmInstruction::SwapBaseInput { .. }
                | RaydiumCpmmInstruction::SwapBaseOutput { .. }
        ) {
            return Ok(());
        }
        // The swap instruction's own log lines hold its SwapEvent.
        for LoggedCpmmSwap(e) in input.metadata.decode_log_events::<LoggedCpmmSwap>() {
            let swap = Swap {
                dex: "raydium-cpmm",
                pool: e.pool_id,
                amount_in: e.input_amount,
                amount_out: e.output_amount,
            };
            self.record(input.metadata.transaction_metadata.signature, swap);
        }
        Ok(())
    }
}

impl Processor<InstructionProcessorInputType<'_, PumpSwapInstruction>> for SwapIndexer {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, PumpSwapInstruction>,
    ) -> CarbonResult<()> {
        let PumpSwapInstruction::CpiEvent { data, .. } = input.decoded_instruction else {
            return Ok(());
        };
        // A buy pays the quote token (usually SOL) for the base token; a sell is the reverse.
        let swap = match data {
            PumpSwapEvent::BuyEvent(e) => Swap {
                dex: "pump-swap",
                pool: e.pool,
                amount_in: e.quote_amount_in,
                amount_out: e.base_amount_out,
            },
            PumpSwapEvent::SellEvent(e) => Swap {
                dex: "pump-swap",
                pool: e.pool,
                amount_in: e.base_amount_in,
                amount_out: e.quote_amount_out,
            },
            _ => return Ok(()),
        };
        self.record(input.metadata.transaction_metadata.signature, swap);
        Ok(())
    }
}

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapIndexer {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let MeteoraDlmmInstruction::CpiEvent { data, .. } = input.decoded_instruction else {
            return Ok(());
        };
        // DLMM emits both `Swap` and `Swap2Evt` for each swap, with the same amounts: read one.
        let DlmmEvent::Swap(e) = data else {
            return Ok(());
        };
        let swap = Swap {
            dex: "meteora-dlmm",
            pool: e.lb_pair,
            amount_in: e.amount_in,
            amount_out: e.amount_out,
        };
        self.record(input.metadata.transaction_metadata.signature, swap);
        Ok(())
    }
}
```

### Step 3: Run it

```sh
cargo run
```

The first build takes one to three minutes. Then swaps start printing, newest first, from whichever crawler gets a response. On the public endpoint the first line can take a minute or more.

Output:

```text
raydium-cpmm pool=hzZaxBF4J6sKNhGtDt2TK1sgEDY31YUzfnkthWA5LpB in=8963567785194 out=196216404 tx=2hx82t435iDNNV6SzYCmh6Hn6MKSdpCKeM69Kxvv6zDm3Dg1Z3pCg6MscmyLmZpNQNB3iDPfZvLjxxmDRraB9GrR
pump-swap    pool=47MMNWgNHKUZFJGKqECZRCpzFKSLMq5YaFXxrieiJQv1 in=7049236 out=15289158360 tx=2Yand83ERTbY9qGjkYKUCrTJCxQnsJeJDkJX4u4LcjDQSPh8jfLBeHAPJig5j9fTYdFfvRraG11KxmfrT2yjL3q7
meteora-dlmm pool=GMgh4NtWrGaUf1RR2kcXD7LY1jou1qFAuSsQeKp5ow4a in=3404263 out=38912302 tx=5Tv7HWwmTxw2avMdHwJGapG932BuYB6ARiJsqxhBc4ejPooMaR7bGfJeNTXtt8wJYtZkwRgwgu6VLtFkCN5VN79W
pump-swap    pool=8zVwLfWw8KpbKoTGdufn6BwTg6Z12SFREHBuZM6UUrVs in=698486361 out=277606557301 tx=2rA17BYRFf4NLP3uGyuxb91bnchdW1rzZfgJLK6M2n5BaoZH1gJoaGjQuEBJ1GNYn1fNyhDQJewX4hFdmZeWZWqk
meteora-dlmm pool=8BD8x5Ms3f1unL9YiDGG4XQYxEcot1UqCkqkyTjniUNZ in=274040325 out=165191368923 tx=2rA17BYRFf4NLP3uGyuxb91bnchdW1rzZfgJLK6M2n5BaoZH1gJoaGjQuEBJ1GNYn1fNyhDQJewX4hFdmZeWZWqk
meteora-dlmm pool=6e7V9eegCHw997T72MxgwwJipZ6GJyZF8NvjkzT1rvpN in=465609675 out=721187169 tx=2rA17BYRFf4NLP3uGyuxb91bnchdW1rzZfgJLK6M2n5BaoZH1gJoaGjQuEBJ1GNYn1fNyhDQJewX4hFdmZeWZWqk
…
```

The last three lines are one transaction that routed through a PumpSwap pool and two Meteora DLMM pools. Each hop is its own swap, so a multi-hop route prints one line per pool. Amounts are raw token units. Press `Ctrl+C` to stop.

## Why each pipe has a deduplication filter

Each crawler fetches every transaction that mentions its program. A transaction that routes through two of the three DEXs is fetched by both crawlers, so without a filter its swaps print twice. [`DeduplicationFilter`](https://carbon.sevenlabs.org/reference/filters.md#deduplicationfilter) keys each instruction by transaction signature and position in the transaction, and drops the second copy.

With a single datasource that follows all three programs, such as a [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) subscription with the three program IDs in `account_include`, every transaction arrives once. The filter is then harmless, and it still protects you if you add a second datasource later.

## Meteora DLMM emits two events per swap

Recent DLMM swaps emit both `Swap` and `Swap2Evt`, with the same pool and amounts. Matching both prints every swap twice, so the indexer reads `Swap` only. `Swap2Evt` adds `amount_left` and a breakdown of the fees, including limit-order fees; read it instead if you need those.

## Add another DEX

1. Find the program on the [DEX and AMM decoders](https://carbon.sevenlabs.org/decoders/dex-and-amm.md) page, add its crate at 2.0.0, and add a crawler for its `PROGRAM_ID`.
2. Find its swap event in the decoder page's **Events** section. For example, Orca Whirlpools has [`Traded`](https://carbon.sevenlabs.org/decoders/orca-whirlpool.md#traded), with `whirlpool`, `input_amount` and `output_amount`.
3. Implement `Processor` for its instruction input on `SwapIndexer`, read the event as a `CpiEvent` or from the logs (see above), and build a `Swap`.
4. Register it with `.instruction_with_filters(Decoder, SwapIndexer, dedupe())`.

Some programs have no events at all: the [Raydium AMM v4 decoder](https://carbon.sevenlabs.org/decoders/raydium-amm-v4.md) lists none, and its `SwapBaseIn` carries only `amount_in` and `minimum_amount_out`. For those, compute the executed amounts from the transaction's token balance changes in `input.metadata.transaction_metadata.meta`.

## Turn raw amounts into token amounts

The events report raw units of each mint. To show token amounts, divide by 10 to the power of the mint's decimals. Look up each mint's decimals once and cache them: a mint's decimals never change. PumpSwap's quote token is usually wrapped SOL, which has 9 decimals.

## Common problems

### Nothing prints for a minute or more

The public endpoint rate-limits `getTransaction` per IP, and the three crawlers share that limit. This is why each crawler keeps one request in flight. Run with `RUST_LOG=info` to see the crawler counters and 429 retries, and set `RPC_URL` to your own endpoint for more than a test.

### The same swap prints twice

Either a pipe has no deduplication filter, or you matched both of DLMM's events. See [why each pipe has a deduplication filter](#why-each-pipe-has-a-deduplication-filter) and [Meteora DLMM emits two events per swap](#meteora-dlmm-emits-two-events-per-swap).

### Raydium CPMM swaps never print

You matched `RaydiumCpmmInstruction::CpiEvent`. Raydium CPMM logs its events instead, so read them from `SwapBaseInput` and `SwapBaseOutput` with `decode_log_events`, as in the example.

## Next steps

- [Track Pump.fun launches and trades](https://carbon.sevenlabs.org/guides/track-pumpfun-launches.md): Follow tokens from launch through their bonding curve, before they reach PumpSwap.
- [Store decoded data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Write each swap to a table instead of printing it.
- [Index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md): Datasource filters for several programs, and one processor over a whole transaction.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/index-dex-swaps · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Track Pump.fun launches and trades (Carbon 2.0)

> Decode Pump.fun token launches, bonding-curve buys and sells, and completed curves from the program's events with the Pump.fun decoder.

Pump.fun records every token launch, every trade on a bonding curve and every curve that fills in an Anchor event, which it emits through a self-CPI. The [Pump.fun decoder](https://carbon.sevenlabs.org/decoders/pumpfun.md) decodes those events as `PumpfunInstruction::CpiEvent`, so your processor reads what actually executed. This guide prints launches, trades and completed curves as they happen on mainnet.

**You need:** Rust 1.96.1 or newer. The example uses Solana's public RPC, so no account is needed.

## Read events, not instruction arguments

| You want           | Read                                                                              | Why                                                                                                           |
| ------------------ | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| New tokens         | [`CreateEvent`](https://carbon.sevenlabs.org/decoders/pumpfun.md#createevent)     | Name, symbol, metadata URI, mint, bonding curve and creator in one place.                                     |
| Buys and sells     | [`TradeEvent`](https://carbon.sevenlabs.org/decoders/pumpfun.md#tradeevent)       | The SOL and token amounts that executed, `is_buy`, the trader, fees and the curve's reserves after the trade. |
| Curves that filled | [`CompleteEvent`](https://carbon.sevenlabs.org/decoders/pumpfun.md#completeevent) | The mint and bonding curve; trading moves to an AMM after this.                                               |

The trade instructions carry limits, not results: `Buy` has `amount` and `max_sol_cost`, `Sell` has `amount` and `min_sol_output`, and newer variants such as `BuyExactSolIn` take `spendable_sol_in` and `min_tokens_out`. Each emits the same `TradeEvent`, whose `ix_name` field names the instruction that ran, so one match arm covers all of them.

### Step 1: Create a project

```sh
cargo new pumpfun-tracker
cd pumpfun-tracker
cargo add carbon-core@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0 carbon-pumpfun-decoder@2.0.0 carbon-log-metrics@2.0.0
cargo add tokio --features full
cargo add env_logger
```

### Step 2: Write the tracker

The [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) fetches every transaction that calls Pump.fun, newest first, then keeps polling for new ones. The processor ignores every decoded instruction except the events, and prints the three it cares about.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_pumpfun_decoder::{
        PROGRAM_ID, PumpfunDecoder,
        instructions::{CpiEvent, PumpfunInstruction},
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let rpc_url = std::env::var("RPC_URL")
        .unwrap_or_else(|_| "https://api.mainnet-beta.solana.com".to_string());

    // Every transaction that calls Pump.fun, newest first, then new ones as they land.
    let datasource = RpcTransactionCrawler::new(
        rpc_url,
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(PumpfunDecoder, LaunchTracker)
        .build()?
        .run()
        .await
}

/// SOL has 9 decimals; Pump.fun tokens have 6.
const LAMPORTS_PER_SOL: f64 = 1e9;
const TOKEN_UNITS: f64 = 1e6;

struct LaunchTracker;

impl Processor<InstructionProcessorInputType<'_, PumpfunInstruction>> for LaunchTracker {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, PumpfunInstruction>,
    ) -> CarbonResult<()> {
        // Pump.fun reports what happened in events, emitted as a self-CPI after each action.
        let PumpfunInstruction::CpiEvent { data, .. } = input.decoded_instruction else {
            return Ok(());
        };
        match data {
            CpiEvent::CreateEvent(e) => {
                println!(
                    "launch   {} ({}) mint={} creator={}",
                    e.name, e.symbol, e.mint, e.creator
                )
            }
            CpiEvent::TradeEvent(e) => println!(
                "{} {:>12.4} SOL {:>16.2} tokens mint={}",
                if e.is_buy { "buy     " } else { "sell    " },
                e.sol_amount as f64 / LAMPORTS_PER_SOL,
                e.token_amount as f64 / TOKEN_UNITS,
                e.mint,
            ),
            CpiEvent::CompleteEvent(e) => println!("complete mint={} bonding curve filled", e.mint),
            _ => {}
        }
        Ok(())
    }
}
```

### Step 3: Run it

```sh
cargo run
```

The first build takes one to three minutes. Then trades start printing, newest first. On the public endpoint the first line can take a minute or more, because it rate-limits `getTransaction`.

Output:

```text
buy            0.0059 SOL        201272.73 tokens mint=7TUpdfVds47vFqwiDubJjAq51y9nMhU5mmCwEmkUpump
sell           0.2325 SOL       1385083.79 tokens mint=EaCgYCrKyGDfquYhUapfw33Xh65U4ZQshPgQnKt1pump
sell           0.0063 SOL      26635028.55 tokens mint=Fqptwy5FV7y89yawECyK9xiCNKmcLHWhLGAoJxL4pump
sell           0.0259 SOL      39136997.32 tokens mint=3sTmKFwVV6PRB5GD8M88VapRa524Mozu3LAwdB4Ypump
sell           0.0028 SOL         28738.62 tokens mint=GdMjsTtrn3AKFHD8x3u15Hg6QLCWfXbdvPsofd83pump
sell           0.0067 SOL      12869465.83 tokens mint=3DvhrxjakLL5s8aJJcHUuCtaUMVrCHAj68KBYiLfpump
…
```

A `launch` line prints when a token is created, and a `complete` line when its bonding curve fills. Both are much rarer than trades. Press `Ctrl+C` to stop.

## Amounts and decimals

- `sol_amount` is in lamports: divide by 10⁹ for SOL.
- `token_amount` is in the token's base units. Pump.fun mints have 6 decimals, so divide by 10⁶.
- `fee` and `creator_fee` are in lamports too, with their rates in `fee_basis_points` and `creator_fee_basis_points`.
- The event also has `quote_mint` and `quote_amount` fields. Check `quote_mint` before you treat `sol_amount` as the price if you index curves quoted in another token.

Every field is listed on the decoder page under [`TradeEvent`](https://carbon.sevenlabs.org/decoders/pumpfun.md#tradeevent).

## After a curve completes

Once `CompleteEvent` fires, the token leaves the bonding curve and trades on an AMM. Pump.fun moves completed tokens to PumpSwap, and `CompletePumpAmmMigrationEvent` records the move. To keep following a token after it graduates, add the [PumpSwap decoder](https://carbon.sevenlabs.org/decoders/pump-swap.md) and read its `BuyEvent` and `SellEvent`: [index swaps across DEXs](https://carbon.sevenlabs.org/guides/index-dex-swaps.md) shows how. Other launchpads work the same way; see [launchpad decoders](https://carbon.sevenlabs.org/decoders/launchpads.md).

## Stream launches in real time

The crawler polls every few seconds and the public endpoint is slow. For launches within a second of landing, swap the datasource for a Geyser stream that follows Pump.fun's program ID: [stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) or [stream with Helius LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md). The processor doesn't change.

## Common problems

### Nothing prints for a minute or more

The public endpoint rate-limits `getTransaction` per IP, and Pump.fun has more transactions than it lets through. Run with `RUST_LOG=info` to see the crawler's counters and 429 retries, and set `RPC_URL` to your own endpoint for anything beyond a test.

### Trades print out of order

The crawler fetches transactions concurrently, so they complete in any order. Sort by `input.metadata.transaction_metadata.slot`, or set `max_concurrent_requests: 1` in [`ConnectionConfig`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md#connectionconfig) for newest-first order.

## Next steps

- [Index swaps across DEXs](https://carbon.sevenlabs.org/guides/index-dex-swaps.md): Follow graduated tokens on PumpSwap, Raydium and Meteora in one record.
- [Store decoded data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md): Write launches and trades to tables instead of printing them.
- [Pump.fun decoder](https://carbon.sevenlabs.org/decoders/pumpfun.md): Every instruction, account and event, with all fields.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/track-pumpfun-launches · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Index a program across an IDL upgrade (Carbon 2.0)

> Decode a program before and after a breaking IDL change by registering one generated decoder per version, each limited to its slot range.

When a program upgrade changes an instruction's layout, one decoder can't read both the old and the new data. You generate one decoder crate per IDL version, register both with the pipeline, and give each a [`SlotRangeFilter`](https://carbon.sevenlabs.org/reference/filters.md) so it only sees the slots where its layout applied. A single processor turns both versions into one output type.

This guide uses a two-version program, `simple_dex`, whose `swap` instruction gains a `fee_tier: u8` argument in V2. The pipeline replays four swaps around the upgrade from memory, so it runs without any network access. To index a real program, swap the replay datasource for a real one.

**You need:**

- Rust 1.96.1 or newer.
- Node.js 20.18.0 or newer, to run the [Carbon CLI](https://carbon.sevenlabs.org/reference/cli.md) (`@sevenlabs-hq/carbon-cli@2.0.0`).
- The IDL of each program version. Keep the old one when you upgrade; you can't fetch it from chain afterwards, because the on-chain IDL account only holds the latest version.
- The **last slot that ran the old program**. See [Find the upgrade slot](#find-the-upgrade-slot).

## How it works

![Routing instructions to the decoder for their program version](https://carbon.sevenlabs.org/diagrams/versioned-decoders.svg)

**Diagram: Routing instructions to the decoder for their program version.** Every instruction of a transaction reaches two instruction pipes. The V1 pipe's filter, SlotRangeFilter::to(500, None), accepts slots up to 500 and passes them to SimpleDexDecoder v1. The V2 pipe's filter, SlotRangeFilter::from(501, None), accepts slots from 501 and passes them to SimpleDexDecoder v2. Both pipes call the same SwapProcessor.

Every instruction reaches both pipes. Each pipe's filter runs before its decoder, so an instruction from slot 500 is decoded only as V1 and one from slot 501 only as V2. The filter is required, not an optimisation: generated decoders ignore trailing bytes, so the V1 decoder also decodes V2 data and returns a `Swap` without `fee_tier`. Without filters, every V2 swap would reach your processor twice.

### Step 1: Create the project and save both IDLs

```sh
cargo new simple-dex-indexer
cd simple-dex-indexer
mkdir idls
```

Save the IDL of each version. These are Anchor IDLs; the only change is the new `fee_tier` argument.

`idls/simple_dex_v1.json`:

```json
{
  "address": "Dex1111111111111111111111111111111111111111",
  "metadata": { "name": "simple_dex", "version": "0.1.0", "spec": "0.1.0" },
  "instructions": [
    {
      "name": "swap",
      "discriminator": [248, 198, 158, 145, 225, 117, 135, 200],
      "accounts": [
        { "name": "pool", "writable": true },
        { "name": "user", "signer": true }
      ],
      "args": [
        { "name": "amount_in", "type": "u64" },
        { "name": "min_amount_out", "type": "u64" }
      ]
    }
  ]
}
```

`idls/simple_dex_v2.json`:

```json
{
  "address": "Dex1111111111111111111111111111111111111111",
  "metadata": { "name": "simple_dex", "version": "0.2.0", "spec": "0.1.0" },
  "instructions": [
    {
      "name": "swap",
      "discriminator": [248, 198, 158, 145, 225, 117, 135, 200],
      "accounts": [
        { "name": "pool", "writable": true },
        { "name": "user", "signer": true }
      ],
      "args": [
        { "name": "amount_in", "type": "u64" },
        { "name": "min_amount_out", "type": "u64" },
        { "name": "fee_tier", "type": "u8" }
      ]
    }
  ]
}
```

### Step 2: Generate one decoder per version

Run `parse` once per IDL. `--version-name` appends `-v1` or `-v2` to the crate name, so both crates can live in one dependency graph. The output directory is deleted before generation, so give each version its own directory.

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v1.json --out-dir decoders/simple-dex-v1 --name simple-dex --version-name v1 --with-postgres false --with-graphql false
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v2.json --out-dir decoders/simple-dex-v2 --name simple-dex --version-name v2 --with-postgres false --with-graphql false
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v1.json --out-dir decoders/simple-dex-v1 --name simple-dex --version-name v1 --with-postgres false --with-graphql false
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v2.json --out-dir decoders/simple-dex-v2 --name simple-dex --version-name v2 --with-postgres false --with-graphql false
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v1.json --out-dir decoders/simple-dex-v1 --name simple-dex --version-name v1 --with-postgres false --with-graphql false
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v2.json --out-dir decoders/simple-dex-v2 --name simple-dex --version-name v2 --with-postgres false --with-graphql false
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v1.json --out-dir decoders/simple-dex-v1 --name simple-dex --version-name v1 --with-postgres false --with-graphql false
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl idls/simple_dex_v2.json --out-dir decoders/simple-dex-v2 --name simple-dex --version-name v2 --with-postgres false --with-graphql false
```

Output:

```text
✔ Decoder generated
✔ Code formatted successfully

✨ Decoder Generated Successfully! ✨
  • Output: /home/you/simple-dex-indexer/decoders/simple-dex-v1
  • Source: file (anchor)
…
```

Each crate is named `carbon-simple-dex-decoder-v1` or `-v2` (Rust paths `carbon_simple_dex_decoder_v1` and `_v2`). Only the crate name changes: both contain the same type names, `SimpleDexDecoder`, `SimpleDexInstruction` and `Swap`, so you rename them on import. Each generated `Cargo.toml` has its own empty `[workspace]` table, which lets you use it as a path dependency as-is.

### Step 3: Add the dependencies

`Cargo.toml`:

```toml
[package]
name = "simple-dex-indexer"
version = "0.1.0"
edition = "2021"

[dependencies]
carbon-core = "2.0.0"
carbon-simple-dex-decoder-v1 = { path = "decoders/simple-dex-v1" }
carbon-simple-dex-decoder-v2 = { path = "decoders/simple-dex-v2" }

async-trait = "0.1.89"
env_logger = "0.11.8"
log = "0.4.28"
solana-hash = "4.5.0"
solana-message = "4.4.1"
solana-pubkey = "4.2.1"
solana-signature = { version = "3.4.1", features = ["rand"] }
solana-transaction = "4.1.6"
solana-transaction-status = "=4.2.2"
tokio = { version = "1.48.0", features = ["full"] }
tokio-util = "0.7.17"
```

The `solana-*` crates and `async-trait` are only needed by the replay datasource at the bottom of `main.rs`.

### Step 4: Write the pipeline

`LAST_V1_SLOT` is the boundary. The V1 pipe accepts slots up to and including it, the V2 pipe every slot after it. `SwapProcessor` implements `Processor` once per decoder's instruction type and converts both into the same `Swap` struct, with `fee_tier: None` for V1.

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{Datasource, DatasourceId, TransactionUpdate, Update, UpdateType},
        error::{CarbonResult, Error},
        filter::SlotRangeFilter,
        instruction::InstructionProcessorInputType,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_simple_dex_decoder_v1::{
        instructions::SimpleDexInstruction as V1Instruction, SimpleDexDecoder as V1Decoder,
        PROGRAM_ID,
    },
    carbon_simple_dex_decoder_v2::{
        instructions::SimpleDexInstruction as V2Instruction, SimpleDexDecoder as V2Decoder,
    },
    solana_hash::Hash,
    solana_message::{
        compiled_instruction::CompiledInstruction, legacy::Message, MessageHeader, VersionedMessage,
    },
    solana_pubkey::Pubkey,
    solana_signature::Signature,
    solana_transaction::versioned::VersionedTransaction,
    solana_transaction_status::TransactionStatusMeta,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Last slot that ran the V1 program. Every transaction from the next slot on uses V2.
const LAST_V1_SLOT: u64 = 500;

/// One swap, whichever program version produced it.
#[derive(Debug)]
struct Swap {
    signature: Signature,
    slot: u64,
    pool: Pubkey,
    amount_in: u64,
    min_amount_out: u64,
    fee_tier: Option<u8>,
}

struct SwapProcessor;

impl SwapProcessor {
    fn handle(&mut self, swap: Swap) -> CarbonResult<()> {
        log::info!(
            "slot={} sig={}… pool={}… amount_in={} min_out={} fee_tier={:?}",
            swap.slot,
            &swap.signature.to_string()[..8],
            &swap.pool.to_string()[..8],
            swap.amount_in,
            swap.min_amount_out,
            swap.fee_tier,
        );
        Ok(())
    }
}

impl Processor<InstructionProcessorInputType<'_, V1Instruction>> for SwapProcessor {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, V1Instruction>,
    ) -> CarbonResult<()> {
        let V1Instruction::Swap { data, accounts, .. } = input.decoded_instruction;
        self.handle(Swap {
            signature: input.metadata.transaction_metadata.signature,
            slot: input.metadata.transaction_metadata.slot,
            pool: accounts.pool,
            amount_in: data.amount_in,
            min_amount_out: data.min_amount_out,
            fee_tier: None,
        })
    }
}

impl Processor<InstructionProcessorInputType<'_, V2Instruction>> for SwapProcessor {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, V2Instruction>,
    ) -> CarbonResult<()> {
        let V2Instruction::Swap { data, accounts, .. } = input.decoded_instruction;
        self.handle(Swap {
            signature: input.metadata.transaction_metadata.signature,
            slot: input.metadata.transaction_metadata.slot,
            pool: accounts.pool,
            amount_in: data.amount_in,
            min_amount_out: data.min_amount_out,
            fee_tier: Some(data.fee_tier),
        })
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    Pipeline::builder()
        .datasource(ReplayDatasource::new())
        .instruction_with_filters(
            V1Decoder,
            SwapProcessor,
            vec![Box::new(SlotRangeFilter::to(LAST_V1_SLOT, None))],
        )
        .instruction_with_filters(
            V2Decoder,
            SwapProcessor,
            vec![Box::new(SlotRangeFilter::from(LAST_V1_SLOT + 1, None))],
        )
        .build()?
        .run()
        .await
}

/// Replays four swaps around the upgrade, then stops. Replace it with a real datasource.
struct ReplayDatasource {
    updates: Vec<Update>,
}

impl ReplayDatasource {
    fn new() -> Self {
        let pool = Pubkey::new_unique();
        Self {
            updates: vec![
                swap_tx(499, swap_data(1_000_000, 990_000, None), pool),
                swap_tx(500, swap_data(2_000_000, 1_980_000, None), pool),
                swap_tx(501, swap_data(3_000_000, 2_970_000, Some(5)), pool),
                swap_tx(502, swap_data(4_000_000, 3_960_000, Some(10)), pool),
            ],
        }
    }
}

#[async_trait]
impl Datasource for ReplayDatasource {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        _cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        for update in &self.updates {
            sender
                .send((update.clone(), id.clone()))
                .await
                .map_err(|e| Error::Custom(e.to_string()))?;
        }
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::Transaction]
    }
}

const SWAP_DISCRIMINATOR: [u8; 8] = [248, 198, 158, 145, 225, 117, 135, 200];

fn swap_data(amount_in: u64, min_amount_out: u64, fee_tier: Option<u8>) -> Vec<u8> {
    let mut data = SWAP_DISCRIMINATOR.to_vec();
    data.extend_from_slice(&amount_in.to_le_bytes());
    data.extend_from_slice(&min_amount_out.to_le_bytes());
    data.extend(fee_tier);
    data
}

fn swap_tx(slot: u64, data: Vec<u8>, pool: Pubkey) -> Update {
    let user = Pubkey::new_unique();
    let signature = Signature::new_unique();
    let message = VersionedMessage::Legacy(Message {
        header: MessageHeader {
            num_required_signatures: 1,
            num_readonly_signed_accounts: 0,
            num_readonly_unsigned_accounts: 1,
        },
        account_keys: vec![user, pool, PROGRAM_ID],
        recent_blockhash: Hash::default(),
        instructions: vec![CompiledInstruction {
            program_id_index: 2,
            accounts: vec![1, 0],
            data,
        }],
    });

    Update::Transaction(Box::new(TransactionUpdate {
        signature,
        transaction: VersionedTransaction {
            signatures: vec![signature],
            message,
        },
        meta: TransactionStatusMeta::default(),
        is_vote: false,
        slot,
        index: None,
        block_time: None,
        block_hash: None,
    }))
}
```

### Step 5: Run it

```sh
RUST_LOG=info cargo run
```

Output:

```text
[… INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 0, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 2, num_transaction_pipes: 0
[… INFO  simple_dex_indexer] slot=499 sig=u3p7abzh… pool=11157t3s… amount_in=1000000 min_out=990000 fee_tier=None
[… INFO  simple_dex_indexer] slot=500 sig=2QXH3Ngt… pool=11157t3s… amount_in=2000000 min_out=1980000 fee_tier=None
[… INFO  simple_dex_indexer] slot=501 sig=45pyVgCR… pool=11157t3s… amount_in=3000000 min_out=2970000 fee_tier=Some(5)
[… INFO  simple_dex_indexer] slot=502 sig=5dJJo4Ni… pool=11157t3s… amount_in=4000000 min_out=3960000 fee_tier=Some(10)
[… INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[… INFO  carbon_core::pipeline] pipeline shutdown complete.
```

Signatures and pubkeys differ on each run. The replay datasource returns after sending its four updates, so the pipeline stops on its own.

## Verify the routing

Each swap must appear exactly once. Slots up to 500 print `fee_tier=None` (V1), later slots print `fee_tier=Some(..)` (V2). If you see a slot twice, both filters accept it: check that the V2 filter starts at `LAST_V1_SLOT + 1`.

To check against real data, run the indexer over a range that spans the upgrade and compare the number of swaps per slot with a block explorer.

## Find the upgrade slot

A program's ProgramData account records the slot of its **most recent** deployment. The Solana CLI prints it as `Last Deployed In Slot`:

```sh
solana program show <PROGRAM_ID> --url https://api.mainnet-beta.solana.com
```

That is only the latest upgrade. For an older one, find the upgrade transaction in the ProgramData account's history (its signatures are listed by `getSignaturesForAddress` on that address) and take its slot.

Use the upgrade transaction's slot as `LAST_V1_SLOT`. Agave makes upgraded program code visible from the slot after the upgrade, so every successful transaction in the upgrade slot itself still ran the old program.

## Choose slot or transaction-index boundaries

`SlotRangeFilter::to(slot, index)` and `SlotRangeFilter::from(slot, index)` also take a transaction index inside the slot. At the boundary slot, `from` accepts `index >= from_index` and `to` accepts `index < to_index`; outside the boundary slot only the slot matters. When an update has no index (`None`), both filters accept every transaction in the boundary slot.

Datasources differ in what they provide:

| Datasource                                                                                                                                                                     | `TransactionUpdate::index`                                                                                                                |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md), [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)         | Position in the block                                                                                                                     |
| [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)                                                                                             | Position in the block                                                                                                                     |
| [RPC block subscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)                                                                                         | Position in the list the RPC returned. With a `mentionsAccountOrProgram` filter that list is filtered, so this is not the block position. |
| [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                                                                                                         | `transactionIndex` from the API                                                                                                           |
| [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), [Helius Atlas WS](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md) | Always `None`                                                                                                                             |

Because upgraded code takes effect at a slot boundary, a slot boundary with `None` is exact and works with every datasource. Use a transaction index only if you need to split inside a slot for another reason, and only with a datasource that provides one.

## Version accounts too

Account layouts change in upgrades as well. If both versions keep the same discriminator, register both account decoders with the same pair of filters (`account_with_filters`). `SlotRangeFilter` compares the **slot of the account update**, not the slot the account was last written in the old layout, so an account that the program migrates lazily can still hold V1 bytes after the boundary. In that case, decide by content instead: try the V2 decoder first and fall back to V1, or compare `account.data.len()` with each layout's size in a custom [`AccountDecoder`](https://carbon.sevenlabs.org/guides/custom-decoder.md).

## Common problems

### Every V2 swap is printed twice

The V1 pipe has no filter, or its range overlaps V2. Generated decoders stop reading after the last field and ignore extra bytes, so V1 decodes V2 data. Give both pipes a `SlotRangeFilter` with ranges that don't overlap.

### V2 swaps are missing for the first slots after the upgrade

The V2 decoder rejects V1 data (it is one byte short), so this is a boundary that is too late: `LAST_V1_SLOT` is after the real upgrade slot. Recheck the upgrade transaction's slot.

### ``no matching package named `carbon-simple-dex-decoder` found``

The dependency name must include the version suffix: `carbon-simple-dex-decoder-v1`. In Rust code the crate is `carbon_simple_dex_decoder_v1`.

### ``the name `SimpleDexDecoder` is defined multiple times``

Both crates export the same names. Import them with `as V1Decoder` and `as V2Decoder`, as in `main.rs` above.

### The output directory lost files

`parse` deletes `--out-dir` before writing. Keep each decoder in a directory that holds nothing else, or pass `--no-clean`.

## Next steps

- [Filters](https://carbon.sevenlabs.org/concepts/filters.md): SlotRangeFilter, DatasourceFilter and writing your own filter.
- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): All carbon parse options, from Anchor and Codama IDLs.
- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md): Replace the replay datasource with history from RPC.
- [Write a decoder by hand](https://carbon.sevenlabs.org/guides/custom-decoder.md): Decide between layouts in your own decoder when slots aren't enough.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/versioned-decoders · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Store decoded data in Postgres (Carbon 2.0)

> Persist decoded instructions and accounts to PostgreSQL with the postgres feature: generated typed tables or generic JSONB, and sqlx_migrator migrations.

Carbon's `postgres` feature gives you processors that write every decoded instruction or account to PostgreSQL, the migrations that create the tables, and row types for reading them back. This guide stores Meteora DLMM instructions from mainnet, first in typed tables (one table per instruction, one column per field), then in a single generic JSONB table, and shows what each looks like in the database.

**You need:**

- Rust 1.96.1 or newer.
- PostgreSQL. This guide was run against PostgreSQL 16.4.
- A decoder crate with a `postgres` feature. Every decoder Carbon publishes has one, and the [CLI](https://carbon.sevenlabs.org/guides/generate-a-decoder.md) generates it unless you pass `--with-postgres false`.

## Choose a storage model

|            | Typed tables                                                                                                                                     | Generic JSONB tables                                                                          |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| Processor  | `PostgresInstructionProcessor<I, W>`, `PostgresAccountProcessor<A, W>`                                                                           | `PostgresJsonInstructionProcessor<I>`, `PostgresJsonAccountProcessor<A>`                      |
| Tables     | One per instruction (`swap_instruction`) and per account type (`lb_pair_account`), created by the decoder's migrations                           | `instructions` and `accounts`, shared by every program, created by `carbon-core`'s migrations |
| Columns    | Metadata columns plus one column per field: `NUMERIC(20)` for `u64`, `BYTEA` for pubkeys, `JSONB` for nested types                               | Metadata columns plus one `JSONB` column with the whole decoded value                         |
| Query with | Plain SQL on typed columns, indexes per field; the decoder's [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md) module reads these tables | JSON operators (`->`, `->>`, `@>`)                                                            |
| Best for   | One or a few programs you query in depth                                                                                                         | Many programs, exploration, schemas that change                                               |

Both models upsert on a primary key, so writing the same update twice leaves one row.

### Step 1: Start PostgreSQL

Use any PostgreSQL you can reach. With Docker:

```sh
docker run -d --name carbon-pg -e POSTGRES_USER=carbon -e POSTGRES_PASSWORD=carbon -e POSTGRES_DB=carbon -p 5432:5432 postgres:16
export DATABASE_URL=postgres://carbon:carbon@localhost:5432/carbon
```

The program reads the connection string from `DATABASE_URL`. Add `?sslmode=require` for a managed database that requires TLS.

### Step 2: Create the project

```sh
cargo new dlmm-postgres
cd dlmm-postgres
cargo add carbon-core@2.0.0 --features postgres
cargo add carbon-meteora-dlmm-decoder@2.0.0 --features postgres
cargo add carbon-rpc-transaction-crawler-datasource@2.0.0 carbon-log-metrics@2.0.0
cargo add sqlx@0.8.6 --features postgres,runtime-tokio-rustls
cargo add sqlx_migrator@0.17.0 --features postgres
cargo add tokio --features full
cargo add env_logger log
```

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres"] }
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres"] }
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio-rustls"] }
sqlx_migrator = { version = "0.17.0", features = ["postgres"] }
tokio = { version = "1.54.1", features = ["full"] }
```

Patch versions may be newer. Enable `postgres` on both `carbon-core` and the decoder. Keep `sqlx` on 0.8 and `sqlx_migrator` on 0.17, the versions Carbon 2.0 is built with; another major version gives you a second, incompatible `PgPool` type.

### Step 3: Write the pipeline

The program connects, runs the decoder's instruction migration, and writes every decoded DLMM instruction to its table.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::{CarbonResult, Error},
        pipeline::Pipeline,
        postgres::processors::PostgresInstructionProcessor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::{
            postgres::{MeteoraDlmmInstructionWithMetadata, MeteoraDlmmInstructionsMigration},
            MeteoraDlmmInstruction,
        },
        MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    sqlx::postgres::PgPoolOptions,
    sqlx_migrator::{Info, Migrate, Migrator, Plan},
    std::sync::Arc,
};

fn db_error(e: impl std::fmt::Display) -> Error {
    Error::Custom(format!("postgres: {e}"))
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let database_url = std::env::var("DATABASE_URL")
        .map_err(|_| Error::Custom("DATABASE_URL must be set".to_string()))?;
    let pool = PgPoolOptions::new()
        .max_connections(5)
        .connect(&database_url)
        .await
        .map_err(db_error)?;

    // Create one table per instruction (swap_instruction, swap2_instruction, ...).
    let mut migrator = Migrator::default();
    migrator
        .add_migration(Box::new(MeteoraDlmmInstructionsMigration))
        .map_err(db_error)?;
    let mut conn = pool.acquire().await.map_err(db_error)?;
    migrator
        .run(&mut *conn, &Plan::apply_all())
        .await
        .map_err(db_error)?;
    drop(conn);

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(
            MeteoraDlmmDecoder,
            PostgresInstructionProcessor::<MeteoraDlmmInstruction, MeteoraDlmmInstructionWithMetadata>::new(pool),
        )
        .build()?
        .run()
        .await
}
```

The two type parameters of `PostgresInstructionProcessor` are the decoder's instruction enum and its generated `MeteoraDlmmInstructionWithMetadata` wrapper, which knows which table and row type each variant goes to.

### Step 4: Run it

```sh
RUST_LOG=info cargo run
```

Output:

```text
[… INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
…
[… INFO  carbon_log_metrics] 00:01:55 (+4.999493119s) | 42 processed (100%), 42 successful, 0 failed (0%), 0 in queue
…
[… INFO  carbon_log_metrics]   postgres_instructions_upsert_upserted: 12
[… INFO  carbon_log_metrics]   postgres_instructions_upsert_failed: 0
```

The first run creates 76 `*_instruction` tables. The public RPC endpoint is rate-limited, so rows arrive slowly; that's the datasource, not Postgres. Stop with ctrl-C.

### Step 5: Query the tables

```sh
psql "$DATABASE_URL" -c "SELECT left(__signature, 12) AS signature, __instruction_index, __stack_height, __slot, amount_in, min_amount_out FROM swap2_instruction ORDER BY __slot DESC LIMIT 3"
```

Output:

```text
  signature   | __instruction_index | __stack_height |  __slot   | amount_in | min_amount_out 
--------------+---------------------+----------------+-----------+-----------+----------------
 57gWxR71yzrh |                   2 |              2 | 467111933 |  24356122 |              0
 Ez2XgG9LDAvj |                   2 |              2 | 467111933 | 124337400 |              0
 UMQXvmmhKaHw |                   0 |              2 | 467111932 |  26993569 |              0
(3 rows)
```

All three are CPIs (`__stack_height` 2) from aggregators. The schema of one table:

```sh
psql "$DATABASE_URL" -c '\d swap2_instruction'
```

Output:

```text
                     Table "public.swap2_instruction"
         Column          |     Type      | Collation | Nullable | Default 
-------------------------+---------------+-----------+----------+---------
 amount_in               | numeric(20,0) |           | not null | 
 min_amount_out          | numeric(20,0) |           | not null | 
 remaining_accounts_info | jsonb         |           | not null | 
 __signature             | text          |           | not null | 
 __instruction_index     | bigint        |           | not null | 
 __stack_height          | bigint        |           | not null | 
 __slot                  | numeric(20,0) |           |          | 
 __accounts              | jsonb         |           | not null | 
Indexes:
    "swap2_instruction_pkey" PRIMARY KEY, btree (__signature, __instruction_index, __stack_height)
```

Columns starting with `__` are metadata. `__accounts` holds the instruction's accounts as JSON, with each pubkey as an array of 32 bytes: `{"pubkey": [115, 168, …], "is_signer": false, "is_writable": true}`.

## Store everything in one JSONB table

Swap the processor and the migration to write every instruction of every program to one `instructions` table. The decoder's instruction enum must implement `serde::Serialize` and `Deserialize`; its `postgres` feature turns on `serde`.

`src/bin/jsonb.rs`:

```rust
use {
    carbon_core::{
        error::{CarbonResult, Error},
        pipeline::Pipeline,
        postgres::{processors::PostgresJsonInstructionProcessor, rows::GenericInstructionMigration},
    },
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID},
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    sqlx::postgres::PgPoolOptions,
    sqlx_migrator::{Info, Migrate, Migrator, Plan},
};

fn db_error(e: impl std::fmt::Display) -> Error {
    Error::Custom(format!("postgres: {e}"))
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let database_url = std::env::var("DATABASE_URL")
        .map_err(|_| Error::Custom("DATABASE_URL must be set".to_string()))?;
    let pool = PgPoolOptions::new()
        .max_connections(5)
        .connect(&database_url)
        .await
        .map_err(db_error)?;

    // One generic `instructions` table with a JSONB column.
    let mut migrator = Migrator::default();
    migrator
        .add_migration(Box::new(GenericInstructionMigration))
        .map_err(db_error)?;
    let mut conn = pool.acquire().await.map_err(db_error)?;
    migrator
        .run(&mut *conn, &Plan::apply_all())
        .await
        .map_err(db_error)?;
    drop(conn);

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(
            MeteoraDlmmDecoder,
            PostgresJsonInstructionProcessor::<MeteoraDlmmInstruction>::new(pool),
        )
        .build()?
        .run()
        .await
}
```

```sh
cargo run --bin jsonb
psql "$DATABASE_URL" -c "SELECT left(__signature, 12) AS signature, __instruction_index, __stack_height, decoded_instruction->>'type' AS type FROM instructions ORDER BY __slot DESC LIMIT 4"
```

Output:

```text
  signature   | __instruction_index | __stack_height |   type   
--------------+---------------------+----------------+----------
 SjGeh3aLxKbB |                   3 |              2 | Swap2
 SjGeh3aLxKbB |                   3 |              3 | CpiEvent
 4AaBRNtD8ZV2 |                   0 |              2 | Swap
 4AaBRNtD8ZV2 |                   0 |              3 | CpiEvent
(4 rows)
```

`decoded_instruction` is the enum serialized with serde as `{"type": "<Variant>", "data": {"program_id": …, "data": {…}, "accounts": {…}}}`, so the arguments are under `data.data`:

```sh
psql "$DATABASE_URL" -c "SELECT left(__signature, 12) AS signature, (decoded_instruction->'data'->'data'->>'amount_in')::numeric AS amount_in FROM instructions WHERE decoded_instruction->>'type' IN ('Swap', 'Swap2') ORDER BY __slot DESC LIMIT 3"
```

Output:

```text
  signature   | amount_in 
--------------+-----------
 SjGeh3aLxKbB | 169262461
 4AaBRNtD8ZV2 |  13025584
 2ya9GepZqk9Y |  99050000
(3 rows)
```

The table has no index on the JSON. Add one for the fields you filter on, for example `CREATE INDEX ON instructions ((decoded_instruction->>'type'));`.

## Store accounts

Accounts work the same way with the account processors and migrations. Register them on a pipeline whose datasource emits account updates, such as [`RpcProgramSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md), [`RpcGpa`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md) or [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md); see [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md). The transaction crawler above never emits them.

```rust
use carbon_meteora_dlmm_decoder::accounts::{
    postgres::{MeteoraDlmmAccountWithMetadata, MeteoraDlmmAccountsMigration},
    MeteoraDlmmAccount,
};

migrator
    .add_migration(Box::new(MeteoraDlmmAccountsMigration))
    .map_err(db_error)?;

// on the pipeline builder:
.account(
    MeteoraDlmmDecoder,
    PostgresAccountProcessor::<MeteoraDlmmAccount, MeteoraDlmmAccountWithMetadata>::new(pool.clone()),
)
```

`PostgresAccountProcessor` comes from `carbon_core::postgres::processors`. Account tables (`lb_pair_account`, …) are keyed by `__pubkey`, so each account keeps its latest state, with `__slot` set to the slot of the last update written. For the generic model, use `PostgresJsonAccountProcessor::<MeteoraDlmmAccount>` with `carbon_core::postgres::rows::GenericAccountsMigration` (table `accounts`).

Account processors don't handle closures. To delete closed accounts, add `.account_deletions(..)` with a processor that deletes the row, for example with the generated row type's `Delete` implementation (`LbPairRow::delete(pubkey.into(), &pool)` deletes from `lb_pair_account`).

## What the decoder's `postgres` module provides

With the feature on, a generated decoder adds, under `accounts::postgres` and `instructions::postgres`:

- `<Program>AccountWithMetadata` and `<Program>InstructionWithMetadata`: the `W` type for the typed processors. Their `Upsert` and `Insert` match on the variant and write the matching row.
- `<Program>AccountsMigration` and `<Program>InstructionsMigration`: `sqlx_migrator` migrations that create every table (`CREATE TABLE IF NOT EXISTS`).
- One `<Name>Row` per account and instruction, with `from_parts`, `TryFrom<Row>` back to the decoded type, and `Insert`, `Upsert`, `Delete` and `Lookup` from `carbon_core::postgres::operations`.

Use the row types to read data back in your own code:

```rust
use carbon_core::postgres::operations::Lookup;
use carbon_meteora_dlmm_decoder::accounts::postgres::LbPairRow;

let row = LbPairRow::lookup(pubkey.into(), &pool).await?; // Option<LbPairRow>
```

The full list, column types and the generic row types are in the [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md#generated-decoder-types).

## Migrations

`sqlx_migrator` records applied migrations in `_sqlx_migrator_migrations` and skips them on the next run:

```sh
psql "$DATABASE_URL" -c "SELECT app, name FROM _sqlx_migrator_migrations"
```

Output:

```text
     app      |           name            
--------------+---------------------------
 meteora-dlmm | meteora_dlmm_instructions
 meteora-dlmm | meteora_dlmm_accounts
 carbon_core  | generic_instructions
```

Things to know before you rely on them:

- **They only create tables.** A newer decoder version with a changed layout doesn't alter existing tables, and its migration has the same name, so it's skipped. Write your own migration (a type implementing `sqlx_migrator::Migration`) with the `ALTER TABLE` statements, or create the new tables in a fresh schema.
- **No indexes beyond the primary key.** Add the ones your queries need in your own migration.
- **Run them once at startup**, before the pipeline, on one connection, as in `main.rs`. Several replicas starting at once can race; run migrations from one replica or a separate job.

## Connection settings

- **Pool size.** The pipeline processes one update at a time, so a processor uses one connection at a time; `max_connections(5)` leaves room for other code. A [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md) server on the same pool needs more.
- **Latency.** Each instruction is a separate `INSERT … ON CONFLICT` round trip, awaited before the next update. At 1 ms per write, a pipeline tops out near 1,000 instructions per second. Keep the database close to the indexer, watch `postgres_instructions_upsert_duration_milliseconds`, and batch writes in your own processor if you need more.
- **Errors.** A failed write returns `Err`, which the pipeline logs (`error processing update …`), counts in `carbon_updates_failed_total` and `postgres_*_upsert_failed`, and then moves on. The update is not retried. Alert on these metrics; see [Monitor with Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md).

## Common problems

### Two swaps in one transaction, but only one row

Instruction tables are keyed by `(__signature, __instruction_index, __stack_height)`, and `__instruction_index` is the index of the **top-level** instruction. Two CPIs of the same program at the same depth under the same top-level instruction get the same key, and the second upsert overwrites the first. This happens with every multi-hop aggregator route: replaying the Jupiter transaction `SZE5CFK6…` (two DLMM `Swap2` CPIs under top-level instruction 4) leaves one row:

Output:

```text
  signature   | __instruction_index | __stack_height | amount_in 
--------------+---------------------+----------------+-----------
 SZE5CFK6gkG9 |                   4 |              2 | 405807562
(1 row)
```

The first swap (`amount_in` 91444688) is lost. The same happens to the program's two `CpiEvent`s per swap. If you need every CPI, write your own processor and key rows on `InstructionMetadata::absolute_path` (for example `[4, 0]` and `[4, 1]`). See [Instruction primary key](https://carbon.sevenlabs.org/reference/postgres.md#instruction-primary-key).

### `relation "lb_pair_account" does not exist`

The account migration wasn't run. Add `MeteoraDlmmAccountsMigration` (or `GenericAccountsMigration` for the generic model) to the migrator.

### `Error: Custom("postgres: pool timed out while waiting for an open connection")`

The program couldn't connect within sqlx's default 30-second acquire timeout: PostgreSQL isn't running, or `DATABASE_URL` has the wrong host or port. Test it with `psql "$DATABASE_URL" -c 'select 1'`. During a run, the same error means every pooled connection is busy; raise `max_connections` if other code shares the pool.

### `Error: Custom("postgres: error returned from database: role \"…\" does not exist")`

The user, password or database name in `DATABASE_URL` is wrong (`database "…" does not exist` and `password authentication failed` have the same cause). Create the role and database, or fix the URL.

### Pubkeys in `__accounts` or JSONB columns are arrays of numbers

That's how the decoders serialize `Pubkey` with serde. Typed pubkey columns are `BYTEA`. To filter, compare with the bytes: in SQL, `decode('<hex>', 'hex')` for `BYTEA` columns; in Rust, bind `sqlx::types::Json(pubkey.to_bytes())` against a JSONB element, as the [GraphQL guide](https://carbon.sevenlabs.org/guides/graphql.md#add-your-own-queries) does.

### Values above 2^63 fail to read as `i64`

`u64` fields and `__slot` are `NUMERIC(20)`. Read them as `carbon_core::postgres::primitives::U64` or `sqlx::types::BigDecimal`, or cast in SQL.

## Next steps

- [Serve it over GraphQL](https://carbon.sevenlabs.org/guides/graphql.md): Expose these tables with the decoder's generated GraphQL schema.
- [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md): Processors, rows, operations, primitives and generated types.
- [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md): Idempotent writes, backpressure and shutdown.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/postgres · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Serve indexed data over GraphQL (Carbon 2.0)

> Expose the Postgres tables a Carbon pipeline writes as a GraphQL API with the graphql feature, a decoder's generated QueryRoot, and Juniper on Axum.

Carbon's `graphql` feature serves the tables written by the [Postgres processors](https://carbon.sevenlabs.org/guides/postgres.md) as a read-only GraphQL API. A decoder built with its `graphql` feature contains a complete query schema (`QueryRoot`) over its typed tables, and `carbon_core::graphql::server` turns any Juniper schema into an Axum router with `/graphql` and a GraphiQL playground at `/graphiql`.

This guide runs a Meteora DLMM pipeline that writes to Postgres and serves it on port 8080, then adds a custom query next to the generated ones.

**You need:**

- Rust 1.96.1 or newer.
- PostgreSQL and a `DATABASE_URL`, as in [Store decoded data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md). This guide was run against PostgreSQL 16.4.
- A decoder with both `postgres` and `graphql` features. The generated GraphQL module reads the typed Postgres tables, so `graphql` doesn't work without `postgres`.

## What you get

| Piece                                  | Where                                                              | What it does                                                                                                                                                                         |
| -------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `build_schema(query)`                  | `carbon_core::graphql::server`                                     | Wraps a Juniper query root in a `RootNode` with no mutations or subscriptions.                                                                                                       |
| `graphql_router(schema, context)`      | `carbon_core::graphql::server`                                     | Axum `Router` with `GET`/`POST /graphql` and `GET /graphiql`.                                                                                                                        |
| `QueryRoot`                            | `<decoder>::graphql`                                               | One field per account type (`lbPair(pubkey)`) and per instruction (`swap2(signature, instructionIndex)`), plus a `list…(limit, offset)` field for each. 177 fields for Meteora DLMM. |
| `GraphQLContext { pool: Arc<PgPool> }` | `<decoder>::graphql::context`                                      | The Juniper context `QueryRoot` expects.                                                                                                                                             |
| `<Name>GraphQL` types                  | `<decoder>::accounts::graphql`, `<decoder>::instructions::graphql` | GraphQL objects converted from the Postgres row types.                                                                                                                               |
| Scalars                                | `carbon_core::graphql::primitives`                                 | `Pubkey` (base58 string), `U64`, `U128`, `I64`, `I128` (decimal strings, so JavaScript clients keep full precision), `U8`, `U32`, `Json`.                                            |

See the [GraphQL reference](https://carbon.sevenlabs.org/reference/graphql.md) for the signatures.

### Step 1: Create the project

```sh
cargo new dlmm-graphql
cd dlmm-graphql
cargo add carbon-core@2.0.0 --features postgres,graphql
cargo add carbon-meteora-dlmm-decoder@2.0.0 --features postgres,graphql
cargo add carbon-rpc-transaction-crawler-datasource@2.0.0
cargo add sqlx@0.8.6 --features postgres,runtime-tokio-rustls
cargo add sqlx_migrator@0.17.0 --features postgres
cargo add axum@0.8 juniper@0.16 solana-pubkey@4.2.1
cargo add tokio --features full
cargo add env_logger log
```

`Cargo.toml`:

```toml
[dependencies]
axum = "0.8"
carbon-core = { version = "2.0.0", features = ["postgres", "graphql"] }
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres", "graphql"] }
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.11"
juniper = "0.16"
log = "0.4.35"
solana-pubkey = "4.2.1"
sqlx = { version = "0.8.6", features = ["postgres", "runtime-tokio-rustls"] }
sqlx_migrator = { version = "0.17.0", features = ["postgres"] }
tokio = { version = "1.54.1", features = ["full"] }
```

Patch versions of `env_logger`, `log` and `tokio` may be newer. `axum` must be 0.8 and `juniper` 0.16, the versions Carbon's router is built on. `juniper` and `solana-pubkey` are only needed for the custom query later in this guide.

### Step 2: Run the pipeline and the server together

The server reads the same pool the processor writes to. It runs in a spawned task, so it is up while the pipeline is still fetching its first transactions.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::{CarbonResult, Error},
        graphql::server::{build_schema, graphql_router},
        pipeline::Pipeline,
        postgres::processors::PostgresInstructionProcessor,
    },
    carbon_meteora_dlmm_decoder::{
        accounts::postgres::MeteoraDlmmAccountsMigration,
        graphql::{context::GraphQLContext, QueryRoot},
        instructions::{
            postgres::{MeteoraDlmmInstructionWithMetadata, MeteoraDlmmInstructionsMigration},
            MeteoraDlmmInstruction,
        },
        MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    sqlx::postgres::PgPoolOptions,
    sqlx_migrator::{Info, Migrate, Migrator, Plan},
    std::sync::Arc,
};

fn custom(e: impl std::fmt::Display) -> Error {
    Error::Custom(e.to_string())
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let database_url = std::env::var("DATABASE_URL").map_err(custom)?;
    let pool = PgPoolOptions::new()
        .max_connections(10)
        .connect(&database_url)
        .await
        .map_err(custom)?;

    // QueryRoot reads both account and instruction tables, so create both.
    let mut migrator = Migrator::default();
    migrator
        .add_migration(Box::new(MeteoraDlmmAccountsMigration))
        .map_err(custom)?;
    migrator
        .add_migration(Box::new(MeteoraDlmmInstructionsMigration))
        .map_err(custom)?;
    let mut conn = pool.acquire().await.map_err(custom)?;
    migrator
        .run(&mut *conn, &Plan::apply_all())
        .await
        .map_err(custom)?;
    drop(conn);

    // GraphQL API over the tables the pipeline writes.
    let schema = build_schema(QueryRoot);
    let context = GraphQLContext { pool: Arc::new(pool.clone()) };
    let app = graphql_router(schema, context);
    let listener = tokio::net::TcpListener::bind("0.0.0.0:8080")
        .await
        .map_err(custom)?;
    log::info!("GraphiQL at http://localhost:8080/graphiql");
    tokio::spawn(async move {
        if let Err(e) = axum::serve(listener, app).await {
            log::error!("GraphQL server stopped: {e}");
        }
    });

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .instruction(
            MeteoraDlmmDecoder,
            PostgresInstructionProcessor::<MeteoraDlmmInstruction, MeteoraDlmmInstructionWithMetadata>::new(pool),
        )
        .build()?
        .run()
        .await
}
```

```sh
export DATABASE_URL=postgres://carbon:carbon@localhost:5432/carbon
RUST_LOG=info cargo run
```

Output:

```text
[… INFO  dlmm_graphql] GraphiQL at http://localhost:8080/graphiql
[… INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 0, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
```

### Step 3: Query it

Open `http://localhost:8080/graphiql` for the playground, or send a request. Field and argument names are camelCase versions of the Rust names, and the generated `list…` fields require both `limit` and `offset`:

```sh
curl -s localhost:8080/graphql -H 'content-type: application/json' \
  -d '{"query":"{ listSwap2(limit: 1, offset: 0) { instructionMetadata { signature instructionIndex stackHeight slot } amountIn minAmountOut } }"}'
```

Output:

```json
{"data":{"listSwap2":[{"instructionMetadata":{"signature":"Ez2XgG9LDAvjnQnkmTNZM2rLTcjDAdLcSAv9jPCZJ7qSVo9gcyN8riocyJcwkY5FEvBn7mhWn49SvQ8AEagyUHH","instructionIndex":"2","stackHeight":"2","slot":"467111933"},"amountIn":"124337400","minAmountOut":"0"}]}}
```

`U32` and `U64` values are strings. Each instruction type also has a lookup by transaction, which returns every matching instruction of that type under one top-level instruction, ordered by stack height:

```graphql
{
  swap2(signature: "SZE5CFK6gkG91yUWfovgDWd7XYFT9DFaHphCnQq2TbghcBY1c1uSzdksAxsayxgh27aBPhrvbBAahUbUNiNB757", instructionIndex: 4) {
    instructionMetadata { stackHeight }
    amountIn
  }
}
```

Account fields take a base58 pubkey and return `null` when the account isn't stored, for example `{ lbPair(pubkey: "Bnoev1Gp1o247d7zTwSbznRwqjkRmbrx3R6Qt4P16YbQ") { activeId binStep } }`. This pipeline stores instructions only, so account queries return `null` until you add an account pipe ([Store accounts](https://carbon.sevenlabs.org/guides/postgres.md#store-accounts)).

## Add your own queries

The generated fields are lookups and paginated lists. For anything else (filters, joins, aggregates) write your own query root with Juniper, reuse the decoder's row and GraphQL types, and expose the generated root as one field so you keep both. This server runs on its own (port 8081) against the same database, which is how you deploy API replicas separately from the indexer:

`src/bin/api.rs`:

```rust
//! A read-only API with your own queries, next to the generated ones.
use {
    carbon_core::graphql::server::{build_schema, graphql_router},
    carbon_meteora_dlmm_decoder::{
        graphql::{context::GraphQLContext, QueryRoot},
        instructions::{graphql::Swap2GraphQL, postgres::Swap2Row},
    },
    juniper::{graphql_object, FieldError, FieldResult, Value},
    sqlx::postgres::PgPoolOptions,
    solana_pubkey::Pubkey,
    std::{str::FromStr, sync::Arc},
};

pub struct Query;

#[graphql_object(context = GraphQLContext)]
impl Query {
    /// Swap2 instructions in one pool, newest first.
    async fn pool_swaps(
        context: &GraphQLContext,
        lb_pair: String,
        limit: Option<i32>,
    ) -> FieldResult<Vec<Swap2GraphQL>> {
        let lb_pair = Pubkey::from_str(&lb_pair)
            .map_err(|e| FieldError::new(e.to_string(), Value::null()))?;
        // `__accounts` stores each pubkey as a JSON array of 32 bytes; the pool is account 0.
        let rows: Vec<Swap2Row> = sqlx::query_as(
            "SELECT * FROM swap2_instruction \
             WHERE __accounts -> 0 -> 'pubkey' = $1 \
             ORDER BY __slot DESC LIMIT $2",
        )
        .bind(sqlx::types::Json(lb_pair.to_bytes()))
        .bind(i64::from(limit.unwrap_or(20).clamp(1, 100)))
        .fetch_all(&*context.pool)
        .await
        .map_err(|e| FieldError::new(e.to_string(), Value::null()))?;

        Ok(rows.into_iter().filter_map(|row| row.try_into().ok()).collect())
    }

    /// Every query the decoder generates, under `generated { ... }`.
    fn generated() -> QueryRoot {
        QueryRoot
    }
}

#[tokio::main]
async fn main() {
    let pool = PgPoolOptions::new()
        .connect(&std::env::var("DATABASE_URL").expect("DATABASE_URL"))
        .await
        .expect("connect to Postgres");

    let app = graphql_router(
        build_schema(Query),
        GraphQLContext { pool: Arc::new(pool) },
    )
    .route("/healthz", axum::routing::get(|| async { "ok" }));
    let listener = tokio::net::TcpListener::bind("0.0.0.0:8081").await.unwrap();
    println!("GraphiQL at http://localhost:8081/graphiql");
    axum::serve(listener, app).await.unwrap();
}
```

```sh
cargo run --bin api
```

```sh
curl -s localhost:8081/graphql -H 'content-type: application/json' \
  -d '{"query":"{ poolSwaps(lbPair: \"EZyszDEx1LZDt7TsSFV8xdPi49sDKC3mdfv2MVMEQLtU\", limit: 5) { instructionMetadata { signature slot } amountIn } }"}'
```

Output:

```json
{"data":{"poolSwaps":[{"instructionMetadata":{"signature":"SZE5CFK6gkG91yUWfovgDWd7XYFT9DFaHphCnQq2TbghcBY1c1uSzdksAxsayxgh27aBPhrvbBAahUbUNiNB757","slot":"467107994"},"amountIn":"405807562"}]}}
```

The generated fields are now under `generated`: `{ generated { listSwap2(limit: 2, offset: 0) { amountIn } } }`. `graphql_router` returns a plain `axum::Router`, so `.route(..)`, `.merge(..)` and Tower layers work as usual; `/healthz` above answers `ok`.

The SQL compares `__accounts -> 0 -> 'pubkey'` with the pool's 32 bytes as JSON because the generated tables store instruction accounts as JSON with each pubkey as a byte array. For frequent filters, add an index on that expression in your own migration.

## Before you expose it

- **Authentication.** The router has none. Put it behind a reverse proxy or add an Axum middleware layer.
- **Cost limits.** Juniper doesn't limit query depth or result size here. The generated `list…` fields pass `limit` to SQL as given; cap it in your own fields, as `poolSwaps` does with `clamp(1, 100)`.
- **Indexes.** The generated lists sort by `__slot` and the generated tables only index their primary key. Add indexes for the queries you serve.
- **Pool.** API requests and the pipeline share connections when they share a pool; size `max_connections` for both, or give the API its own pool.
- **Errors.** Database errors come back in the GraphQL `errors` array with the database's message, including table names.

## Common problems

### `relation "lb_pair_account" does not exist`

An account field was queried but the account migration never ran. `QueryRoot` covers accounts and instructions; run both `MeteoraDlmmAccountsMigration` and `MeteoraDlmmInstructionsMigration`, as `main.rs` does.

### `Field "listSwap2" argument "offset" of type "Int!" is required but not provided`

Generated list fields take both `limit` and `offset`. Pass `offset: 0`.

### `Unknown field "listSwap2" on type "Query"`

You're querying a custom root that nests the generated fields. Use `{ generated { listSwap2(..) } }`, or the field names of your own root.

### ``cannot find `graphql` in `carbon_meteora_dlmm_decoder` ``

The decoder's `graphql` feature is off. Enable `postgres` and `graphql` on the decoder and on `carbon-core`.

### Mismatched `Router` or `RootNode` types

Your `axum` or `juniper` major version differs from Carbon's. Use `axum` 0.8 and `juniper` 0.16.

### `Address already in use (os error 98)`

Another process listens on 8080. Change the bind address.

## Next steps

- [Store decoded data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md): The tables this API reads.
- [GraphQL reference](https://carbon.sevenlabs.org/reference/graphql.md): build_schema, graphql_router and the scalar types.
- [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md): Run the indexer and the API reliably.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/graphql · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Monitor an indexer with Prometheus (Carbon 2.0)

> Serve Carbon's pipeline and datasource metrics on a Prometheus endpoint, scrape them, and alert on stalls, failures, queue depth and slow updates.

`carbon-prometheus-metrics` serves every metric in Carbon's registry (pipeline counters, datasource metrics, and your own) as Prometheus text on `http://<host>:9464/metrics`. This guide adds it to a pipeline, shows real output from Solana mainnet, and gives a scrape config, queries and alert rules you can copy.

**You need:** Rust 1.96.1 or newer and a Carbon pipeline. The example uses the public RPC endpoint, so no account is needed. To run the scrape and alert steps you also need [Prometheus](https://prometheus.io/download/) (any 2.x or 3.x release; tested with 3.5.0).

For what each metric means, see [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md) and the [metrics reference](https://carbon.sevenlabs.org/reference/metrics.md#built-in-metrics).

### Step 1: Add the crate

In an existing Carbon project, or a new one from the [Quickstart](https://carbon.sevenlabs.org/quickstart.md):

```sh
cargo add carbon-prometheus-metrics@2.0.0
```

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-prometheus-metrics = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.8"
log = "0.4.28"
tokio = { version = "1.48.0", features = ["full"] }
```

The crate's default `http-server` feature provides the HTTP endpoint. Keep it on.

### Step 2: Register the exporter with a server

Use `PrometheusMetrics::with_server`. `PrometheusMetrics::new()` creates an exporter that **starts no server**, so nothing listens on port 9464.

This program indexes Meteora DLMM and also defines its own counter, `dlmm_swaps_decoded_total`. Any `Counter`, `Gauge` or `Histogram` you register in `MetricsRegistry::global()` is served on the same endpoint.

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult,
        instruction::InstructionProcessorInputType,
        metrics::{Counter, MetricsRegistry},
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_prometheus_metrics::{PrometheusMetrics, PrometheusServerConfig},
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::sync::Arc,
};

// Your own metric, served next to Carbon's on the same endpoint.
static SWAPS_DECODED: Counter = Counter::new(
    "dlmm_swaps_decoded_total",
    "Meteora DLMM swap instructions decoded",
);

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    MetricsRegistry::global().register_counter(&SWAPS_DECODED);

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    // Serves http://0.0.0.0:9464/metrics while the pipeline runs.
    let prometheus = PrometheusMetrics::with_server(PrometheusServerConfig::default());

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(prometheus))
        .instruction(MeteoraDlmmDecoder, SwapCounter)
        .build()?
        .run()
        .await
}

struct SwapCounter;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapCounter {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmInstruction::Swap { .. } | MeteoraDlmmInstruction::Swap2 { .. } =
            input.decoded_instruction
        {
            SWAPS_DECODED.inc();
        }
        Ok(())
    }
}
```

`PrometheusServerConfig::default()` listens on `0.0.0.0:9464`, allows any CORS origin and times requests out after 30 seconds. Change them with the builder methods:

```rust
let config = PrometheusServerConfig::new()
    .listen_addr("127.0.0.1:9100".parse().unwrap())
    .enable_cors(false)
    .request_timeout_secs(10);
```

### Step 3: Run the pipeline

```sh
RUST_LOG=info cargo run
```

Output:

```text
[… INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
[… INFO  carbon_prometheus_metrics] Prometheus metrics server starting on http://0.0.0.0:9464/metrics
```

The server starts when `run()` begins and stops when the process receives ctrl-C.

### Step 4: Read the endpoint

In another terminal:

```sh
curl -s localhost:9464/metrics
```

Output after about two minutes on the public RPC (trimmed):

Output:

```text
# HELP dlmm_swaps_decoded_total Meteora DLMM swap instructions decoded
# TYPE dlmm_swaps_decoded_total counter
dlmm_swaps_decoded_total 6
# HELP carbon_updates_received_total Total updates pulled from datasources
# TYPE carbon_updates_received_total counter
carbon_updates_received_total 41
# HELP carbon_updates_processed_total Total updates processed by the pipeline
# TYPE carbon_updates_processed_total counter
carbon_updates_processed_total 41
# HELP carbon_updates_successful_total Updates processed without error
# TYPE carbon_updates_successful_total counter
carbon_updates_successful_total 41
# HELP carbon_updates_failed_total Updates that errored during processing
# TYPE carbon_updates_failed_total counter
carbon_updates_failed_total 0
…
# HELP transaction_crawler_transactions_fetched_total Transactions fetched by transaction crawler
# TYPE transaction_crawler_transactions_fetched_total counter
transaction_crawler_transactions_fetched_total 62
# HELP carbon_updates_queued Current number of updates waiting in queue
# TYPE carbon_updates_queued gauge
carbon_updates_queued 0
# HELP carbon_updates_process_time_nanoseconds Time taken to process updates in nanoseconds
# TYPE carbon_updates_process_time_nanoseconds histogram
carbon_updates_process_time_nanoseconds_bucket{le="1000"} 0
carbon_updates_process_time_nanoseconds_bucket{le="10000"} 0
carbon_updates_process_time_nanoseconds_bucket{le="100000"} 12
carbon_updates_process_time_nanoseconds_bucket{le="1000000"} 39
carbon_updates_process_time_nanoseconds_bucket{le="10000000"} 41
…
carbon_updates_process_time_nanoseconds_bucket{le="+Inf"} 41
carbon_updates_process_time_nanoseconds_count 41
carbon_updates_process_time_nanoseconds_sum 9060388
…
transaction_crawler_transaction_fetch_times_milliseconds_bucket{le="5000"} 39
transaction_crawler_transaction_fetch_times_milliseconds_bucket{le="+Inf"} 58
transaction_crawler_transaction_fetch_times_milliseconds_count 58
transaction_crawler_transaction_fetch_times_milliseconds_sum 371211
```

Datasource metrics (here `transaction_crawler_*`) appear once the datasource starts. Metrics carry no labels; Prometheus adds `job` and `instance` when it scrapes, and you can add your own in the scrape config. The 19 transaction fetches above 5 seconds are the public endpoint rate-limiting requests.

## Scrape it with Prometheus

`prometheus.yml`:

```yaml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:
  - carbon-alerts.yml

scrape_configs:
  - job_name: carbon
    static_configs:
      - targets: ["localhost:9464"]
        labels:
          indexer: dlmm-swaps
```

Give each indexer its own `indexer` label (or its own job) so the queries below can tell them apart. Check the file and start Prometheus:

```sh
promtool check config prometheus.yml
prometheus --config.file=prometheus.yml
```

Output:

```text
Checking prometheus.yml
  SUCCESS: 1 rule files found
 SUCCESS: prometheus.yml is valid prometheus config file syntax
```

Open `http://localhost:9090/targets`: the `carbon` target should be `UP`. `promtool check metrics` on the endpoint's output warns that the `_milliseconds` and `_nanoseconds` histograms don't use seconds as their base unit. That is a naming lint; Prometheus ingests them normally.

## Useful queries

| Question                            | PromQL                                                                                                                  |
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| Is the endpoint up?                 | `up{job="carbon"}`                                                                                                      |
| Updates per second (throughput)     | `rate(carbon_updates_processed_total[5m])`                                                                              |
| Transactions or accounts per second | `rate(carbon_transaction_updates_processed_total[5m])`, `rate(carbon_account_updates_processed_total[5m])`              |
| Failure ratio                       | `rate(carbon_updates_failed_total[5m]) / clamp_min(rate(carbon_updates_processed_total[5m]), 1e-9)`                     |
| Queue depth                         | `carbon_updates_queued`                                                                                                 |
| p99 time per update, ms             | `histogram_quantile(0.99, rate(carbon_updates_process_time_milliseconds_bucket[5m]))`                                   |
| p50 time per update, µs             | `histogram_quantile(0.5, rate(carbon_updates_process_time_nanoseconds_bucket[5m])) / 1e3`                               |
| Mean time per update, ms            | `rate(carbon_updates_process_time_nanoseconds_sum[5m]) / rate(carbon_updates_process_time_nanoseconds_count[5m]) / 1e6` |
| Your own counter                    | `increase(dlmm_swaps_decoded_total[1h])`                                                                                |
| Postgres write failures             | `rate(postgres_instructions_upsert_failed[5m])` (these counters have no `_total` suffix)                                |

These are real results from Prometheus 3.5.0 scraping the program above every 5 seconds, after about three minutes:

Output:

```text
up{job="carbon"}                                                         1
rate(carbon_updates_processed_total[1m])                                 0.509
histogram_quantile(0.99, rate(carbon_updates_process_time_milliseconds_bucket[2m]))   0.99
carbon_updates_queued                                                    0
rate(carbon_updates_failed_total[1m]) / clamp_min(rate(carbon_updates_processed_total[1m]), 1e-9)   0
increase(dlmm_swaps_decoded_total[2m])                                   15.65
```

The millisecond histogram records whole milliseconds, truncated, so every update faster than 1 ms lands in the `le="1"` bucket and quantiles below 1 ms are interpolated. Use the nanosecond histogram for fast processors and the millisecond one for thresholds of 5 ms and above. Both measure the time to run one update through **all** pipes, including your processors' database writes.

## Alert rules

`carbon-alerts.yml`:

```yaml
groups:
  - name: carbon
    rules:
      - alert: CarbonIndexerDown
        expr: up{job="carbon"} == 0
        for: 2m
        annotations:
          summary: "{{ $labels.instance }} metrics endpoint is unreachable"

      - alert: CarbonNoUpdates
        expr: rate(carbon_updates_received_total[5m]) == 0
        for: 5m
        annotations:
          summary: "{{ $labels.indexer }} received no updates for 10 minutes"

      - alert: CarbonProcessingErrors
        expr: |
          rate(carbon_updates_failed_total[5m])
            / clamp_min(rate(carbon_updates_processed_total[5m]), 1e-9) > 0.01
        for: 10m
        annotations:
          summary: "More than 1% of updates fail in {{ $labels.indexer }}"

      - alert: CarbonQueueFilling
        # 1000 is the default channel_buffer_size; use your own value.
        expr: carbon_updates_queued > 0.8 * 1000
        for: 5m
        annotations:
          summary: "{{ $labels.indexer }} is falling behind; streaming datasources drop updates when the queue is full"

      - alert: CarbonSlowUpdates
        expr: |
          histogram_quantile(0.99, rate(carbon_updates_process_time_milliseconds_bucket[5m])) > 250
        for: 10m
        annotations:
          summary: "p99 time per update in {{ $labels.indexer }} is above 250 ms"
```

```sh
promtool check rules carbon-alerts.yml
```

Output:

```text
Checking carbon-alerts.yml
  SUCCESS: 5 rules found
```

What each alert catches:

- **`CarbonIndexerDown`**: the process exited or the port isn't reachable. Carbon's pipeline also returns when every datasource stops, so a crashed datasource ends in this alert.
- **`CarbonNoUpdates`**: the endpoint is up but nothing arrives: a dead upstream, a filter that matches nothing, or a datasource that gave up reconnecting. Tune the window to your program's normal traffic; a quiet program can legitimately have minutes without transactions.
- **`CarbonProcessingErrors`**: processors or decoders return `Err`. The pipeline logs each error and continues, so this is often the only signal. See [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md).
- **`CarbonQueueFilling`**: the processors are slower than the datasource. Most streaming datasources (Yellowstone, LaserStream, RPC program and block subscribe) use a non-blocking send and **drop** updates when the channel is full. Set the threshold from your [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size).
- **`CarbonSlowUpdates`**: per-update latency, usually your database.

For gaps after reconnects, count missed slots in your own counter from `DatasourceDisconnection` events; [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md#detect-gaps-after-reconnects) has the code.

## Common problems

### `curl: (7) Failed to connect to localhost port 9464`

The exporter was created with `PrometheusMetrics::new()`, which starts no server. Use `PrometheusMetrics::with_server(PrometheusServerConfig::default())`. Projects generated with `carbon scaffold -m prometheus` use `new()`; change that line.

### `Prometheus metrics server error: Address already in use (os error 98)`

Another process (often a previous run of the indexer) holds port 9464. The pipeline keeps running without an endpoint. Stop the other process or set a different `listen_addr`.

### Datasource metrics are missing

Datasources register their metrics when they start consuming, so they appear a moment after `run()` begins. A datasource that failed to start never registers them.

### `text format parsing error … second HELP line for metric name`

A metric was registered more than once. Each `register_*` call adds it again and the endpoint prints it twice, which the Prometheus text parser rejects. Register your own metrics once at startup, not inside a processor or a datasource's `consume`. Calling `build()` on more than one pipeline in a process registers Carbon's pipeline metrics once per call, so run one pipeline per process.

### The target is `DOWN` from another host

`listen_addr` is `127.0.0.1`, or a firewall blocks the port. Bind `0.0.0.0` (the default) and open TCP 9464 to the Prometheus server only. The endpoint has no authentication.

## Next steps

- [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md): How the registry and exporters work.
- [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md): Every built-in metric, per datasource.
- [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md): Backpressure, reconnects, shutdown and sizing.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/prometheus · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Write a custom datasource (Carbon 2.0)

> Implement the Datasource trait to feed Carbon from any source, emit transactions and accounts, honour cancellation, or adapt a channel with StreamMessageClient.

A datasource is anything that implements `carbon_core::datasource::Datasource`: it runs in its own task, turns data from somewhere into [`Update`](https://carbon.sevenlabs.org/concepts/updates.md)s and sends them into the pipeline. Write one when no [built-in datasource](https://carbon.sevenlabs.org/datasources.md) reads from your source, for example a provider API, a queue, files on disk, or an RPC method Carbon doesn't wrap.

This guide builds three, all runnable against Solana's public RPC endpoint:

- [`SignaturePoller`](#write-a-transaction-datasource), which polls a program's signatures and emits each transaction.
- [`AccountPoller`](#emit-account-updates), which watches a list of accounts and emits account updates and deletions.
- An [adapter over a channel](#adapt-a-channel-with-streammessageclient) with `carbon-stream-message-datasource`, when you already have a producer and only need to hand its output to Carbon.

**You need:** Rust 1.96.1 or newer. Read [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md) for how the pipeline runs them.

## The trait

```rust
#[async_trait]
pub trait Datasource: Send + Sync {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: tokio::sync::mpsc::Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()>;

    fn update_types(&self) -> Vec<UpdateType>;
}
```

`Datasource` uses `#[async_trait]` (the pipeline stores datasources as `dyn Datasource`), so annotate your `impl` with `#[async_trait]` from the `async-trait` crate. `Processor`, by contrast, uses a plain `async fn`.

The contract:

| Part                 | What you do                                                                                                                                                                                                                                             |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `consume`            | Runs once, in its own Tokio task, when `Pipeline::run` starts. Produce updates and send `(update, id.clone())` on `sender`.                                                                                                                             |
| `id`                 | The datasource's identity. Send it back unchanged with every update; [`DatasourceFilter`](https://carbon.sevenlabs.org/concepts/filters.md) uses it.                                                                                                    |
| `sender`             | Bounded channel into the pipeline ([`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size), 1,000 by default). `send(..).await` waits while it's full. An `Err` means the pipeline has stopped: return. |
| `cancellation_token` | Cancelled on ctrl-C or when your own token passed to `datasource_cancellation_token` is cancelled. Return promptly when it fires, including while you wait on the network or a timer.                                                                   |
| Return value         | `Ok(())` when the source is exhausted. An `Err` is logged as `error consuming datasource: …`. Either way the task ends, and once every datasource task has ended and dropped its sender, the pipeline drains the channel and `run()` returns.           |
| `update_types`       | The `UpdateType`s you emit. The 2.0.0 pipeline doesn't check it; keep it accurate as documentation.                                                                                                                                                     |

## Write a transaction datasource

`SignaturePoller` calls `getSignaturesForAddress` for one address, fetches each new transaction with `getTransaction`, and sends it as an `Update::Transaction`. It's a smaller version of the repository's [`custom-datasource` example](https://github.com/sevenlabs-hq/carbon/tree/v2.0.0/examples/custom-datasource) and of what [`RpcTransactionCrawler`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) does with more concurrency and retries.

### Step 1: Create the project

```sh
cargo new poller
cd poller
cargo add carbon-core@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-token-program-decoder@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-client@=4.2.2 solana-transaction-status@=4.2.2 solana-commitment-config@3.1.1 solana-pubkey@4.2.1 solana-signature@3.4.1
cargo add async-trait tokio-util env_logger log
cargo add tokio --features full
```

`Cargo.toml`:

```toml
[dependencies]
async-trait = "0.1.92"
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-stream-message-datasource = "2.0.0"
carbon-token-program-decoder = "2.0.0"
env_logger = "0.11.11"
log = "0.4.35"
solana-client = "=4.2.2"
solana-commitment-config = "3.1.1"
solana-pubkey = "4.2.1"
solana-signature = "3.4.1"
solana-transaction-status = "=4.2.2"
tokio = { version = "1.54.1", features = ["full"] }
tokio-util = "0.7.20"
```

Patch versions of the unpinned crates may be newer. Use the Solana crate versions Carbon 2.0 depends on (the Agave 4.2 cohort), so each type resolves to one version. A second `solana-pubkey` or `solana-transaction-status` in the graph produces "expected `Pubkey`, found `Pubkey`" errors.

### Step 2: Write the datasource

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{Datasource, DatasourceId, TransactionUpdate, Update, UpdateType},
        error::CarbonResult,
        instruction::InstructionProcessorInputType,
        pipeline::Pipeline,
        processor::Processor,
        transformers::transaction_metadata_from_original_meta,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    solana_client::{
        nonblocking::rpc_client::RpcClient, rpc_client::GetConfirmedSignaturesForAddress2Config,
        rpc_config::RpcTransactionConfig,
    },
    solana_commitment_config::CommitmentConfig,
    solana_pubkey::Pubkey,
    solana_signature::Signature,
    solana_transaction_status::UiTransactionEncoding,
    std::{str::FromStr, time::Duration},
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Polls `getSignaturesForAddress` for one program and emits each new transaction.
pub struct SignaturePoller {
    pub rpc_url: String,
    pub address: Pubkey,
    pub poll_interval: Duration,
    /// Maximum signatures fetched per poll. Anything older than the newest
    /// `max_per_poll` signatures since the last poll is skipped.
    pub max_per_poll: usize,
}

#[async_trait]
impl Datasource for SignaturePoller {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        let rpc = RpcClient::new_with_commitment(self.rpc_url.clone(), CommitmentConfig::confirmed());
        let mut newest_seen: Option<Signature> = None;

        loop {
            let config = GetConfirmedSignaturesForAddress2Config {
                before: None,
                until: newest_seen,
                limit: Some(self.max_per_poll),
                commitment: Some(CommitmentConfig::confirmed()),
            };

            // Stop waiting on the RPC as soon as the pipeline shuts down.
            let result = tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                result = rpc.get_signatures_for_address_with_config(&self.address, config) => result,
            };

            match result {
                Ok(signatures) => {
                    if let Some(first) = signatures.first() {
                        newest_seen = Signature::from_str(&first.signature).ok();
                    }

                    // The RPC returns newest first; emit in chain order.
                    for info in signatures.iter().rev() {
                        if cancellation_token.is_cancelled() {
                            return Ok(());
                        }
                        if info.err.is_some() {
                            continue; // skip failed transactions
                        }
                        let Ok(signature) = Signature::from_str(&info.signature) else {
                            continue;
                        };
                        let index = info.transaction_index.map(u64::from);
                        let Some(update) = fetch_transaction(&rpc, signature, index).await else {
                            continue;
                        };
                        // `send` waits when the pipeline is busy instead of dropping the update.
                        if sender.send((update, id.clone())).await.is_err() {
                            return Ok(()); // the pipeline has stopped
                        }
                    }
                }
                Err(error) => log::warn!("getSignaturesForAddress failed: {error}"),
            }

            tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                _ = tokio::time::sleep(self.poll_interval) => {}
            }
        }
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::Transaction]
    }
}

async fn fetch_transaction(
    rpc: &RpcClient,
    signature: Signature,
    index: Option<u64>,
) -> Option<Update> {
    let config = RpcTransactionConfig {
        encoding: Some(UiTransactionEncoding::Base64), // binary encoding is required
        commitment: Some(CommitmentConfig::confirmed()),
        max_supported_transaction_version: Some(1),     // accept legacy, V0 and V1
    };
    let fetched = match rpc.get_transaction_with_config(&signature, config).await {
        Ok(fetched) => fetched,
        Err(error) => {
            log::warn!("getTransaction {signature} failed: {error}");
            return None;
        }
    };

    let transaction = fetched.transaction.transaction.decode()?;
    let meta = transaction_metadata_from_original_meta(fetched.transaction.meta?).ok()?;

    Some(Update::Transaction(Box::new(TransactionUpdate {
        signature,
        transaction,
        meta,
        is_vote: false,
        slot: fetched.slot,
        index, // position in the block, from getSignaturesForAddress
        block_time: fetched.block_time,
        block_hash: None,
    })))
}

struct InstructionLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for InstructionLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let debug = format!("{:?}", input.decoded_instruction);
        let name = debug.split([' ', '{', '(']).next().unwrap_or_default();
        let metadata = &input.metadata.transaction_metadata;
        println!("slot={} {name} {}", metadata.slot, metadata.signature);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let datasource = SignaturePoller {
        rpc_url: "https://api.mainnet-beta.solana.com".to_string(),
        address: PROGRAM_ID,
        poll_interval: Duration::from_secs(5),
        max_per_poll: 10,
    };

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, InstructionLogger)
        .build()?
        .run()
        .await
}
```

What each part does:

- **Cancellation.** Every wait (the signatures request and the sleep) sits in a `tokio::select!` with `cancellation_token.cancelled()`, and the inner loop checks `is_cancelled()` between transactions. Without that, ctrl-C waits for the current RPC call and the full poll interval.
- **Building the update.** `getTransaction` must use a binary encoding (`Base64`) and `max_supported_transaction_version: Some(1)`. `decode()` rebuilds the `VersionedTransaction`, and `transaction_metadata_from_original_meta` converts the RPC's `UiTransactionStatusMeta` into the `TransactionStatusMeta` Carbon expects. Carbon can't rebuild a transaction from `Json` or `JsonParsed` responses.
- **Metadata.** `index` is the transaction's position in its block, which `getSignaturesForAddress` returns on current RPC nodes; [`SlotRangeFilter`](https://carbon.sevenlabs.org/reference/filters.md) uses it. `block_hash` stays `None` because `getTransaction` doesn't return it.
- **Backpressure.** `sender.send(..).await` waits for room in the channel, so a slow processor slows the poller down instead of losing updates. Several built-in streaming datasources use `try_send` and drop updates instead; see [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md#backpressure-and-the-channel).
- **Order.** The RPC lists signatures newest first; the loop reverses them so processors see chain order.

### Step 3: Run it

```sh
cargo run
```

Output:

```text
slot=467109371 Swap2 2EWz1tSnoreYujahprX6jibXvRXCcmXJCKeoVWzGgSSf8iFVGmh5yYZPQXd3zbX7d2usEPpWyhYv9Z5aoV8TVsxf
slot=467109371 CpiEvent 2EWz1tSnoreYujahprX6jibXvRXCcmXJCKeoVWzGgSSf8iFVGmh5yYZPQXd3zbX7d2usEPpWyhYv9Z5aoV8TVsxf
slot=467109371 CpiEvent 2EWz1tSnoreYujahprX6jibXvRXCcmXJCKeoVWzGgSSf8iFVGmh5yYZPQXd3zbX7d2usEPpWyhYv9Z5aoV8TVsxf
slot=467109396 RemoveLiquidityByRange2 5mZ9yjJxJHBKee7fHZC51ejcq2JgEDd1nZCVskHFQnM6Tm6evV3cacEhaJRpDkHs6CEXzSDmgHXEaeYGSTLZ9efL
slot=467109396 ClaimFee2 5mZ9yjJxJHBKee7fHZC51ejcq2JgEDd1nZCVskHFQnM6Tm6evV3cacEhaJRpDkHs6CEXzSDmgHXEaeYGSTLZ9efL
slot=467109396 ClosePositionIfEmpty 5mZ9yjJxJHBKee7fHZC51ejcq2JgEDd1nZCVskHFQnM6Tm6evV3cacEhaJRpDkHs6CEXzSDmgHXEaeYGSTLZ9efL
slot=467109396 Swap 4sCsDTY5X3RyFBSiAihqutGYwF1N2pyxbnAe9wg5ZitCnde8RZqyXb2HsAgfeMA2yjyXrw1akUaa4cRaLwwdiZPi
…
```

`CpiEvent` lines are the program's self-CPI events, decoded as instructions. Press ctrl-C to stop:

Output:

```text
[… INFO  carbon_core::pipeline] shutting down the pipeline after processing pending updates.
[… INFO  carbon_core::pipeline] pipeline shutdown complete.
```

The public endpoint rate-limits; with `RUST_LOG=solana_rpc_client=debug` you see `Too many requests: server responded with … status: 429` while the client backs off and retries. Use your own RPC URL for anything beyond a test.

This poller is deliberately simple. It skips a transaction whose `getTransaction` call fails, and it skips older signatures when more than `max_per_poll` arrive between two polls. A production version retries failed fetches and pages backwards with `before` until it reaches `until`.

## Emit account updates

For accounts, build an `AccountUpdate` and send `AccountUpdate::into_update()`, not `Update::Account(..)`. `into_update` returns `Update::AccountDeletion` for an account with zero lamports and `Update::Account` otherwise, the same rule every built-in datasource applies. Account decoders then never see closed accounts, and `.account_deletions(..)` processors receive them with the account's final state.

Add this file next to `main.rs`. It polls `getMultipleAccounts` for the USDC and USDT mints and sends an update whenever an account changes.

`src/bin/accounts.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        account::AccountProcessorInputType,
        datasource::{AccountDeletion, AccountUpdate, Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_token_program_decoder::{accounts::TokenProgramAccount, TokenProgramDecoder},
    solana_client::nonblocking::rpc_client::RpcClient,
    solana_commitment_config::CommitmentConfig,
    solana_pubkey::Pubkey,
    std::{collections::HashMap, time::Duration},
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Polls `getMultipleAccounts` and emits an update whenever an account changes.
pub struct AccountPoller {
    pub rpc_url: String,
    pub accounts: Vec<Pubkey>,
    pub poll_interval: Duration,
}

#[async_trait]
impl Datasource for AccountPoller {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        let rpc = RpcClient::new(self.rpc_url.clone());
        let mut last_seen = HashMap::new();

        loop {
            let result = tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                result = rpc.get_multiple_accounts_with_commitment(
                    &self.accounts,
                    CommitmentConfig::confirmed(),
                ) => result,
            };

            match result {
                Ok(response) => {
                    let slot = response.context.slot;
                    for (pubkey, account) in self.accounts.iter().zip(response.value) {
                        // A missing account is reported with zero lamports, so
                        // `into_update` turns it into an `AccountDeletion`.
                        let account = account.unwrap_or_default();
                        if last_seen.get(pubkey) == Some(&account) {
                            continue; // unchanged since the last poll
                        }
                        last_seen.insert(*pubkey, account.clone());

                        let update = AccountUpdate {
                            pubkey: *pubkey,
                            account,
                            slot,
                            transaction_signature: None,
                        }
                        .into_update();

                        if sender.send((update, id.clone())).await.is_err() {
                            return Ok(());
                        }
                    }
                }
                Err(error) => log::warn!("getMultipleAccounts failed: {error}"),
            }

            tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                _ = tokio::time::sleep(self.poll_interval) => {}
            }
        }
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::AccountUpdate, UpdateType::AccountDeletion]
    }
}

struct MintLogger;

impl Processor<AccountProcessorInputType<'_, TokenProgramAccount>> for MintLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, TokenProgramAccount>,
    ) -> CarbonResult<()> {
        if let TokenProgramAccount::Mint(mint) = &input.decoded_account.data {
            println!(
                "slot={} mint={} supply={} decimals={}",
                input.metadata.slot, input.metadata.pubkey, mint.supply, mint.decimals
            );
        }
        Ok(())
    }
}

struct DeletionLogger;

impl Processor<AccountDeletion> for DeletionLogger {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        println!("slot={} closed {}", deletion.slot, deletion.pubkey);
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let datasource = AccountPoller {
        rpc_url: "https://api.mainnet-beta.solana.com".to_string(),
        accounts: vec![
            Pubkey::from_str_const("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"), // USDC mint
            Pubkey::from_str_const("Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB"), // USDT mint
        ],
        poll_interval: Duration::from_secs(2),
    };

    Pipeline::builder()
        .datasource(datasource)
        .account(TokenProgramDecoder, MintLogger)
        .account_deletions(DeletionLogger)
        .build()?
        .run()
        .await
}
```

```sh
cargo run --bin accounts
```

Output:

```text
slot=467109863 mint=EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v supply=8450898376555615 decimals=6
slot=467109863 mint=Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB supply=3839903059562649 decimals=6
slot=467109958 mint=EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v supply=8450897696579348 decimals=6
…
```

The first poll emits every account; later polls emit only the ones whose data or lamports changed. Set `transaction_signature` when your source knows which transaction wrote the account: [`DeduplicationFilter`](https://carbon.sevenlabs.org/concepts/filters.md) uses it to drop duplicate account updates and accepts every update without one.

## Adapt a channel with `StreamMessageClient`

If you already have code that produces Carbon updates (a Kafka consumer, a websocket client, a replay from files), you don't need to implement `Datasource`. `carbon-stream-message-datasource` wraps a `tokio::sync::mpsc::Receiver<UnifiedMessage>`:

```rust
pub enum UnifiedMessage {
    Account(AccountUpdate),
    Transaction(Box<TransactionUpdate>),
}
```

`StreamMessageClient::new(receiver)` forwards each message to the pipeline, calling `into_update()` on accounts. It waits for room in the pipeline's channel, and it ends when the sender side is dropped, which stops the pipeline.

`src/bin/stream.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType,
        datasource::AccountUpdate,
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    carbon_token_program_decoder::{accounts::TokenProgramAccount, TokenProgramDecoder},
    solana_client::nonblocking::rpc_client::RpcClient,
    solana_pubkey::Pubkey,
};

const USDC_MINT: Pubkey = Pubkey::from_str_const("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");

struct MintLogger;

impl Processor<AccountProcessorInputType<'_, TokenProgramAccount>> for MintLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, TokenProgramAccount>,
    ) -> CarbonResult<()> {
        if let TokenProgramAccount::Mint(mint) = &input.decoded_account.data {
            println!("slot={} supply={}", input.metadata.slot, mint.supply);
        }
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let (tx, rx) = tokio::sync::mpsc::channel::<UnifiedMessage>(1_000);

    // Your producer: a Kafka consumer, a websocket, a file replay... Here, three RPC reads.
    tokio::spawn(async move {
        let rpc = RpcClient::new("https://api.mainnet-beta.solana.com".to_string());
        for _ in 0..3 {
            if let Ok(response) = rpc
                .get_account_with_commitment(&USDC_MINT, rpc.commitment())
                .await
            {
                if let Some(account) = response.value {
                    let update = AccountUpdate {
                        pubkey: USDC_MINT,
                        account,
                        slot: response.context.slot,
                        transaction_signature: None,
                    };
                    if tx.send(UnifiedMessage::Account(update)).await.is_err() {
                        break;
                    }
                }
            }
            tokio::time::sleep(std::time::Duration::from_secs(2)).await;
        }
        // Dropping `tx` closes the stream; the pipeline then finishes.
    });

    Pipeline::builder()
        .datasource(StreamMessageClient::new(rx))
        .account(TokenProgramDecoder, MintLogger)
        .build()?
        .run()
        .await
}
```

```sh
cargo run --bin stream
```

Output:

```text
slot=467109975 supply=8450897696579348
slot=467109985 supply=8450897592422230
slot=467109993 supply=8450897592422230
```

The program exits by itself after the third message. A `StreamMessageClient` can be consumed once: its receiver moves into the pipeline on the first `consume` call. It doesn't carry `BlockDetails`; implement `Datasource` if you need them.

## Add metrics to your datasource

Define `Counter`s or `Histogram`s as statics and register them in `MetricsRegistry::global()` at the start of `consume`, as the built-in datasources do. They appear in [`LogMetrics`](https://carbon.sevenlabs.org/concepts/metrics.md) output and on a [Prometheus endpoint](https://carbon.sevenlabs.org/guides/prometheus.md):

```rust
use carbon_core::metrics::{Counter, MetricsRegistry};

static FETCH_FAILURES: Counter = Counter::new(
    "poller_fetch_failures_total",
    "getTransaction calls that failed",
);

// first line of `consume`:
MetricsRegistry::global().register_counter(&FETCH_FAILURES);

// where a fetch fails:
FETCH_FAILURES.inc();
```

## Common problems

### ``lifetime parameters or bounds on method `consume` do not match the trait declaration``

The `impl Datasource` block is missing `#[async_trait]`. Add `async-trait` to `Cargo.toml` and put `#[async_trait]` on the `impl`.

### ctrl-C doesn't stop the program until a request finishes

`consume` awaits something without also watching the token. Wrap every long wait in `tokio::select!` with `cancellation_token.cancelled()`.

### The pipeline exits right after starting

`consume` returned. That's correct for a finite source, but a stream must loop until it's cancelled. Also check the log for `error consuming datasource:`: an `Err` ends the task the same way. If `consume` spawns a task and returns `Ok(())`, move the `sender` into that task; the pipeline keeps running while a sender is alive.

### Processors never see a closed account

You sent `Update::Account` for a zero-lamport account. Use `AccountUpdate { .. }.into_update()` and register `.account_deletions(..)`.

### `error processing update (…): MissingFeePayer`

The `VersionedTransaction` you sent has no account keys, usually because it was built by hand. Every transaction needs its message's account keys; the first is the fee payer.

### Every transaction is skipped

`fetched.transaction.transaction.decode()` returns `None` for transactions requested as `Json` or `JsonParsed`, so `fetch_transaction` returns early. Request `UiTransactionEncoding::Base64`.

### Top-level instructions decode but CPIs never do

The update's `meta.inner_instructions` is `None` or empty. Carbon builds the CPI tree from it, so a source that strips inner instructions only ever yields top-level instructions. Keep the full metadata when you build `TransactionUpdate`.

## Next steps

- [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md): How the pipeline runs datasources.
- [Updates](https://carbon.sevenlabs.org/concepts/updates.md): Every field of TransactionUpdate, AccountUpdate and the other updates.
- [Test decoders and processors](https://carbon.sevenlabs.org/guides/testing.md): Feed saved transactions through a pipeline without a network.
- [Run Carbon in production](https://carbon.sevenlabs.org/guides/production.md): Backpressure, reconnects and shutdown.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/custom-datasource · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Write a decoder by hand (Carbon 2.0)

> Decode a program that has no IDL by implementing CarbonDeserialize, ArrangeAccounts, InstructionDecoder and AccountDecoder yourself, then unit-test it.

Most decoders are generated from an IDL with the [CLI](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Write one by hand when the program has no Anchor or Codama IDL: a native Rust program, a program whose IDL is wrong or missing, or when you only need two instructions out of fifty.

This guide writes a decoder for **Counter**, a native program with a borsh-encoded, one-byte-tagged instruction set, and tests it with unit tests. The same pieces apply to any layout.

**You need:** Rust 1.96.1 or newer, and the program's binary layout: the bytes that identify each instruction and account (the discriminator), the field order and encoding of each, and the order of each instruction's accounts. Get it from the program's source or its client SDK. [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md) explains how decoders fit into the pipeline.

## The layout

| Item                                        | Discriminator | Payload (borsh)                     | Accounts, in order                                           |
| ------------------------------------------- | ------------- | ----------------------------------- | ------------------------------------------------------------ |
| `Initialize` instruction                    | `[0]`         | `initial_count: u64`                | `counter` (writable), `authority` (signer), `system_program` |
| `Increment` instruction                     | `[1]`         | `amount: u64`                       | `counter` (writable), `authority` (signer)                   |
| `Counter` account                           | `[1]`         | `authority: Pubkey`, `count: u64`   |                                                              |
| `CounterIncremented` event (`sol_log_data`) | `b"cinc"`     | `counter: Pubkey`, `new_count: u64` |                                                              |

Anchor programs use 8-byte discriminators (`sha256("global:<name>")[..8]` for instructions, `sha256("account:<Name>")[..8]` for accounts). Native programs often use a one-byte enum tag, and some accounts have none at all; then the account's owner and exact data length identify it.

## The pieces

| Piece                                 | Role                                                                                                                                                                                          |
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CarbonDeserialize`                   | `DISCRIMINATOR` constant plus `deserialize(&[u8]) -> Option<Self>`: check the prefix, decode the rest. Requires `BorshDeserialize`. `InstructionMetadata::decode_log_events::<T>()` needs it. |
| `Type::decode(&[u8]) -> Option<Self>` | An inherent function. `try_decode_instructions!` calls it by that name, so every instruction type needs one. Generated decoders have the same function.                                       |
| `ArrangeAccounts`                     | Turns the instruction's positional `&[AccountMeta]` into a named struct. Return `None` when required accounts are missing.                                                                    |
| `try_decode_instructions!`            | Tries each `Enum::Variant => Type` in order and returns `Some(Enum::Variant { program_id, data, accounts })` for the first type whose `decode` and `arrange_accounts` both succeed.           |
| `InstructionDecoder`                  | What `.instruction(decoder, processor)` takes. Return `None` for instructions that aren't yours.                                                                                              |
| `AccountDecoder`                      | What `.account(decoder, processor)` takes. Return `None` for accounts that aren't yours.                                                                                                      |

### Step 1: Create a library crate

```sh
cargo new --lib counter-decoder
cd counter-decoder
cargo add carbon-core@2.0.0 solana-account@4.3.2 solana-instruction@3.4.1
cargo add borsh --features derive
cargo add solana-pubkey@4.2.1 --features borsh
cargo add --dev base64 solana-transaction-status@=4.2.2
```

`Cargo.toml`:

```toml
[package]
name = "counter-decoder"
version = "0.1.0"
edition = "2021"

[dependencies]
borsh = { version = "1.8.1", features = ["derive"] }
carbon-core = "2.0.0"
solana-account = "4.3.2"
solana-instruction = "3.4.1"
solana-pubkey = { version = "4.2.1", features = ["borsh"] }

[dev-dependencies]
base64 = "0.23.1"
solana-transaction-status = "=4.2.2"
```

Newer patch versions of `borsh` and `base64` work too. `carbon-core`'s default `macros` feature provides `try_decode_instructions!`. The `borsh` feature of `solana-pubkey` lets `Pubkey` fields derive `BorshDeserialize` (`carbon-core` already turns it on; declaring it keeps your crate correct on its own). Use the Solana crate versions Carbon 2.0 depends on so `Pubkey`, `Instruction` and `Account` are the same types Carbon uses.

### Step 2: Write the decoder

`src/lib.rs`:

```rust
//! Hand-written Carbon decoder for the Counter program.
use {
    borsh::BorshDeserialize,
    carbon_core::{
        account::{AccountDecoder, DecodedAccount},
        account_utils::next_account,
        deserialize::{ArrangeAccounts, CarbonDeserialize},
        instruction::InstructionDecoder,
    },
    solana_instruction::{AccountMeta, Instruction},
    solana_pubkey::Pubkey,
};

pub const PROGRAM_ID: Pubkey = Pubkey::from_str_const("Counter111111111111111111111111111111111111");

/// The decoder you register with the pipeline.
pub struct CounterDecoder;

/// Borsh-decode `data` after checking and removing a discriminator prefix.
/// `try_from_slice` rejects trailing bytes, so a longer layout never decodes as a shorter one.
fn decode_prefixed<T: BorshDeserialize>(discriminator: &[u8], data: &[u8]) -> Option<T> {
    let payload = data.strip_prefix(discriminator)?;
    T::try_from_slice(payload).ok()
}

// ---------- Instructions ----------

/// Instruction 0: create a counter. Data: `[0] ++ initial_count: u64`.
#[derive(Debug, Clone, PartialEq, BorshDeserialize)]
pub struct Initialize {
    pub initial_count: u64,
}

/// Instruction 1: add to a counter. Data: `[1] ++ amount: u64`.
#[derive(Debug, Clone, PartialEq, BorshDeserialize)]
pub struct Increment {
    pub amount: u64,
}

impl CarbonDeserialize for Initialize {
    const DISCRIMINATOR: &'static [u8] = &[0];

    fn deserialize(data: &[u8]) -> Option<Self> {
        decode_prefixed(Self::DISCRIMINATOR, data)
    }
}

impl CarbonDeserialize for Increment {
    const DISCRIMINATOR: &'static [u8] = &[1];

    fn deserialize(data: &[u8]) -> Option<Self> {
        decode_prefixed(Self::DISCRIMINATOR, data)
    }
}

// `try_decode_instructions!` calls `Type::decode(&[u8])`.
impl Initialize {
    pub fn decode(data: &[u8]) -> Option<Self> {
        <Self as CarbonDeserialize>::deserialize(data)
    }
}

impl Increment {
    pub fn decode(data: &[u8]) -> Option<Self> {
        <Self as CarbonDeserialize>::deserialize(data)
    }
}

#[derive(Debug, Clone, PartialEq)]
pub struct InitializeAccounts {
    pub counter: Pubkey,
    pub authority: Pubkey,
    pub system_program: Pubkey,
}

#[derive(Debug, Clone, PartialEq)]
pub struct IncrementAccounts {
    pub counter: Pubkey,
    pub authority: Pubkey,
    /// Accounts past the ones the program declares.
    pub remaining: Vec<AccountMeta>,
}

impl ArrangeAccounts for Initialize {
    type ArrangedAccounts = InitializeAccounts;

    fn arrange_accounts(accounts: &[AccountMeta]) -> Option<Self::ArrangedAccounts> {
        let mut iter = accounts.iter();
        Some(InitializeAccounts {
            counter: next_account(&mut iter)?,
            authority: next_account(&mut iter)?,
            system_program: next_account(&mut iter)?,
        })
    }
}

impl ArrangeAccounts for Increment {
    type ArrangedAccounts = IncrementAccounts;

    fn arrange_accounts(accounts: &[AccountMeta]) -> Option<Self::ArrangedAccounts> {
        let mut iter = accounts.iter();
        let counter = next_account(&mut iter)?;
        let authority = next_account(&mut iter)?;
        Some(IncrementAccounts {
            counter,
            authority,
            remaining: iter.as_slice().to_vec(),
        })
    }
}

/// One variant per instruction, in the shape `try_decode_instructions!` builds.
#[derive(Debug, Clone, PartialEq)]
pub enum CounterInstruction {
    Initialize {
        program_id: Pubkey,
        data: Initialize,
        accounts: InitializeAccounts,
    },
    Increment {
        program_id: Pubkey,
        data: Increment,
        accounts: IncrementAccounts,
    },
}

impl InstructionDecoder<'_> for CounterDecoder {
    type InstructionType = CounterInstruction;

    fn decode_instruction(&self, instruction: &Instruction) -> Option<Self::InstructionType> {
        if instruction.program_id != PROGRAM_ID {
            return None;
        }

        carbon_core::try_decode_instructions!(
            instruction,
            PROGRAM_ID,
            CounterInstruction::Initialize => Initialize,
            CounterInstruction::Increment => Increment,
        )
    }
}

// ---------- Accounts ----------

/// Counter state. Layout: `[1] ++ authority: Pubkey ++ count: u64` (41 bytes).
#[derive(Debug, Clone, PartialEq, BorshDeserialize)]
pub struct Counter {
    pub authority: Pubkey,
    pub count: u64,
}

impl CarbonDeserialize for Counter {
    const DISCRIMINATOR: &'static [u8] = &[1];

    fn deserialize(data: &[u8]) -> Option<Self> {
        decode_prefixed(Self::DISCRIMINATOR, data)
    }
}

#[derive(Debug, Clone, PartialEq)]
pub enum CounterAccount {
    Counter(Counter),
}

impl<'a> AccountDecoder<'a> for CounterDecoder {
    type AccountType = CounterAccount;

    fn decode_account(
        &self,
        account: &'a solana_account::Account,
    ) -> Option<DecodedAccount<Self::AccountType>> {
        if account.owner != PROGRAM_ID {
            return None;
        }

        let data = <Counter as CarbonDeserialize>::deserialize(&account.data)?;
        Some(DecodedAccount {
            lamports: account.lamports,
            data: CounterAccount::Counter(data),
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

// ---------- Events ----------

/// Logged with `sol_log_data` by `Increment`. Data: `b"cinc" ++ counter ++ new_count: u64`.
#[derive(Debug, Clone, PartialEq, BorshDeserialize)]
pub struct CounterIncremented {
    pub counter: Pubkey,
    pub new_count: u64,
}

impl CarbonDeserialize for CounterIncremented {
    const DISCRIMINATOR: &'static [u8] = b"cinc";

    fn deserialize(data: &[u8]) -> Option<Self> {
        decode_prefixed(Self::DISCRIMINATOR, data)
    }
}
```

Notes on the choices:

- **`try_from_slice`, not `deserialize(&mut slice)`.** Generated decoders read with `BorshDeserialize::deserialize(&mut slice)`, which stops after the last field and ignores extra bytes. `try_from_slice` fails on extra bytes, so a payload from a longer, newer layout is rejected instead of decoded wrongly. Use the lenient form if the program appends optional trailing data.
- **Fully qualified calls.** `CarbonDeserialize::deserialize` and `BorshDeserialize::deserialize` have the same name. With both traits in scope, call `<Counter as CarbonDeserialize>::deserialize(..)`.
- **Program check first.** `decode_instruction` compares `program_id` before decoding and `decode_account` compares the owner. Discriminators are short and collide across programs; a one-byte tag matches many unrelated instructions.
- **`remaining`.** Keeping accounts past the declared ones, as generated decoders do, preserves extra accounts that some instructions take, such as remaining accounts for routing.
- **Order in `try_decode_instructions!`.** The first match wins. With distinct discriminators the order doesn't matter; list the longer layout first if two types share one.

### Step 3: Test it

Unit tests build instruction and account bytes by hand and check every rejection path, so a layout mistake fails here rather than silently in production. The event test builds the `InstructionMetadata` the pipeline would pass and checks that `decode_log_events` finds the event in the logs of instruction `[0]`.

`tests/decoder.rs`:

```rust
use {
    base64::{engine::general_purpose::STANDARD, Engine},
    carbon_core::{
        account::AccountDecoder,
        instruction::{InstructionDecoder, InstructionMetadata},
        transaction::TransactionMetadata,
    },
    counter_decoder::{
        Counter, CounterAccount, CounterDecoder, CounterIncremented, CounterInstruction, Increment,
        PROGRAM_ID,
    },
    solana_instruction::{AccountMeta, Instruction},
    solana_pubkey::Pubkey,
    solana_transaction_status::TransactionStatusMeta,
    std::sync::Arc,
};

fn increment_ix(data: Vec<u8>) -> (Instruction, Pubkey, Pubkey) {
    let counter = Pubkey::new_unique();
    let authority = Pubkey::new_unique();
    let instruction = Instruction {
        program_id: PROGRAM_ID,
        accounts: vec![
            AccountMeta::new(counter, false),
            AccountMeta::new_readonly(authority, true),
        ],
        data,
    };
    (instruction, counter, authority)
}

#[test]
fn decodes_increment() {
    let mut data = vec![1];
    data.extend_from_slice(&42u64.to_le_bytes());
    let (instruction, counter, authority) = increment_ix(data);

    let decoded = CounterDecoder.decode_instruction(&instruction).expect("decodes");

    let CounterInstruction::Increment { data, accounts, .. } = decoded else {
        panic!("expected Increment, got {decoded:?}");
    };
    assert_eq!(data, Increment { amount: 42 });
    assert_eq!(accounts.counter, counter);
    assert_eq!(accounts.authority, authority);
    assert!(accounts.remaining.is_empty());
}

#[test]
fn rejects_other_programs_and_bad_data() {
    let mut data = vec![1];
    data.extend_from_slice(&42u64.to_le_bytes());

    let (mut other_program, _, _) = increment_ix(data.clone());
    other_program.program_id = Pubkey::new_unique();
    assert!(CounterDecoder.decode_instruction(&other_program).is_none());

    let (truncated, _, _) = increment_ix(data[..5].to_vec());
    assert!(CounterDecoder.decode_instruction(&truncated).is_none());

    let (unknown, _, _) = increment_ix(vec![9, 0, 0, 0, 0, 0, 0, 0, 0]);
    assert!(CounterDecoder.decode_instruction(&unknown).is_none());

    let (mut missing_accounts, _, _) = increment_ix(data);
    missing_accounts.accounts.truncate(1);
    assert!(CounterDecoder.decode_instruction(&missing_accounts).is_none());
}

#[test]
fn decodes_counter_account() {
    let authority = Pubkey::new_unique();
    let mut data = vec![1];
    data.extend_from_slice(authority.as_ref());
    data.extend_from_slice(&7u64.to_le_bytes());

    let account = solana_account::Account {
        lamports: 1_183_200,
        data,
        owner: PROGRAM_ID,
        executable: false,
        rent_epoch: u64::MAX,
    };

    let decoded = CounterDecoder.decode_account(&account).expect("decodes");
    assert_eq!(decoded.data, CounterAccount::Counter(Counter { authority, count: 7 }));
}

#[test]
fn decodes_event_from_logs() {
    let counter = Pubkey::new_unique();
    let mut event = b"cinc".to_vec();
    event.extend_from_slice(counter.as_ref());
    event.extend_from_slice(&43u64.to_le_bytes());

    let logs = vec![
        format!("Program {PROGRAM_ID} invoke [1]"),
        format!("Program data: {}", STANDARD.encode(&event)),
        format!("Program {PROGRAM_ID} success"),
    ];
    let metadata = InstructionMetadata {
        transaction_metadata: Arc::new(TransactionMetadata {
            meta: TransactionStatusMeta {
                log_messages: Some(logs),
                ..Default::default()
            },
            ..Default::default()
        }),
        stack_height: 1,
        index: 0,
        absolute_path: vec![0],
    };

    let events = metadata.decode_log_events::<CounterIncremented>();
    assert_eq!(events, vec![CounterIncremented { counter, new_count: 43 }]);
}
```

```sh
cargo test
```

Output:

```text
     Running tests/decoder.rs (target/debug/deps/decoder-3c61c1f19ffcfb96)

running 4 tests
test decodes_counter_account ... ok
test rejects_other_programs_and_bad_data ... ok
test decodes_increment ... ok
test decodes_event_from_logs ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
```

Add a test with a real instruction and account from mainnet once the program is deployed; [Test decoders and processors](https://carbon.sevenlabs.org/guides/testing.md) shows how to capture them as fixtures.

## Use it in a pipeline

The decoder plugs in like a generated one. A processor receives `CounterInstruction` and can decode the events its instruction logged:

```rust
use {
    carbon_core::{error::CarbonResult, instruction::InstructionProcessorInputType, processor::Processor},
    counter_decoder::{CounterIncremented, CounterInstruction},
};

struct CounterProcessor;

impl Processor<InstructionProcessorInputType<'_, CounterInstruction>> for CounterProcessor {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, CounterInstruction>,
    ) -> CarbonResult<()> {
        if let CounterInstruction::Increment { data, accounts, .. } = input.decoded_instruction {
            // Events this instruction logged with `sol_log_data`.
            for event in input.metadata.decode_log_events::<CounterIncremented>() {
                println!("{} +{} -> {}", accounts.counter, data.amount, event.new_count);
            }
        }
        Ok(())
    }
}
```

Register it with `.instruction(counter_decoder::CounterDecoder, CounterProcessor)`, and the account side with `.account(counter_decoder::CounterDecoder, processor)`. `InstructionProcessorInputType` and `AccountProcessorInputType` are described in [Processor inputs](https://carbon.sevenlabs.org/reference/processor-inputs.md).

`decode_log_events` reads `Program data: <base64>` log lines that the instruction at exactly this position in the call tree emitted, skips the ones whose bytes `T::deserialize` rejects, and returns the rest in order. Events logged by an inner instruction belong to that inner instruction, not its parent. Events emitted as a self-CPI (Anchor's `emit_cpi!`) are instructions, not logs: decode them with your `InstructionDecoder` as a separate variant.

## Use it in a transaction pipe

`.transaction(..)` pipes need an [`InstructionDecoderCollection`](https://carbon.sevenlabs.org/concepts/transactions.md), whose enum must be `Eq + Hash + Serialize`. Add `#[derive(Eq, Hash, serde::Serialize)]` to `CounterInstruction` and to every type it contains (with `serde` and the `serde` feature of `solana-pubkey`), then list the decoder in `instruction_decoder_collection!`.

## Common problems

### ``no associated function or constant named `decode` found for struct `Increment` ``

`try_decode_instructions!` calls `Type::decode`. Add `impl Type { pub fn decode(data: &[u8]) -> Option<Self> { .. } }` for each instruction type, delegating to `CarbonDeserialize`.

### `multiple applicable items in scope` on `deserialize`

Both `CarbonDeserialize` and `BorshDeserialize` are in scope. Call `<Type as CarbonDeserialize>::deserialize(data)`.

### Every instruction decodes as the wrong variant

Two types accept the same bytes: a shared or empty discriminator, or lenient decoding that ignores trailing bytes. Give each type its own discriminator check, use `try_from_slice`, and put the more specific type first in `try_decode_instructions!`.

### Instructions decode in tests but not in the pipeline

The program ID constant doesn't match the deployed program, or `arrange_accounts` requires more accounts than real transactions pass. Test with a captured mainnet instruction.

## Next steps

- [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md): The decoder traits and how decoding works.
- [Test decoders and processors](https://carbon.sevenlabs.org/guides/testing.md): Fixtures from real transactions and accounts.
- [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md): Generate the same pieces from an Anchor or Codama IDL.
- [Macros reference](https://carbon.sevenlabs.org/reference/macros.md): try_decode_instructions!, instruction_decoder_collection! and InstructionType.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/custom-decoder · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Test decoders and processors (Carbon 2.0)

> Unit-test Carbon decoders with carbon-test-utils fixtures captured from mainnet, and run processors through a real pipeline in tests without a network.

A Carbon indexer has two things worth testing: that the **decoder** turns real bytes into the right types, and that your **processor** does the right thing with them. This guide tests both offline, from fixtures captured once from mainnet:

- Decoder tests with [`carbon-test-utils`](https://carbon.sevenlabs.org/reference/test-utils.md): instruction data from an explorer, an instruction fixture, and an account fixture.
- Pipeline tests that replay a saved transaction through `Pipeline::run` with a fixture datasource and assert on what the processor recorded.

The code under test is `SwapStats`, a processor that sums Meteora DLMM swap volume per pool. The fixture is a real Jupiter route that swaps through two DLMM pools in one transaction.

**You need:** Rust 1.96.1 or newer, plus `curl` and `jq` to capture fixtures. Tests themselves need no network.

## How the Carbon repository tests

- `carbon-core` keeps real transaction metadata under `crates/core/tests/fixtures/*.json` (with the Solscan links in a README) and loads them with `carbon_test_utils::read_transaction_meta` in unit tests of its transformers.
- Decoders get unit tests next to the code: `carbon-token-program-decoder` packs SPL accounts with `solana_program_pack::Pack` and decodes the bytes; `carbon-token-2022-decoder` builds mints with extensions in `tests/`.
- Generated decoders add `carbon-test-utils` as a dev-dependency so you can add fixture tests the same way.

`carbon-test-utils` provides:

| Item                               | Use                                                                                                                                                           |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `read_instruction(path)`           | Load an `Instruction` from JSON: `program_id`, `accounts` (`pubkey`, `is_signer`, `is_writable`) and hex `data`.                                              |
| `read_account(path)`               | Load an `Account` from JSON: base64 `data`, `executable`, `lamports`, `owner`, `rent_epoch`.                                                                  |
| `read_transaction_meta(path)`      | Load a `UiTransactionStatusMeta` (the `meta` object of a `getTransaction` response). Convert it with `transformers::transaction_metadata_from_original_meta`. |
| `base58_deserialize::ix_data(str)` | Decode base58 instruction data, as explorers display it, into bytes.                                                                                          |

All three readers return `anyhow::Result`. The full formats are in the [test-utils reference](https://carbon.sevenlabs.org/reference/test-utils.md).

### Step 1: Create the crate

```sh
cargo new --lib swap-stats
cd swap-stats
mkdir -p tests/fixtures src/bin
```

`Cargo.toml`:

```toml
[package]
name = "swap-stats"
version = "0.1.0"
edition = "2021"

[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
hex = "0.4.3"
serde_json = "1.0.145"
solana-pubkey = "4.2.1"
solana-transaction-status = "=4.2.2"

[dev-dependencies]
async-trait = "0.1.89"
carbon-test-utils = "2.0.0"
solana-transaction-error = "3.3.2"
tokio = { version = "1.48.0", features = ["full"] }
tokio-util = "0.7.17"
```

`hex` and `serde_json` are used by the fixture tool below; the rest of the library needs only Carbon and the two Solana crates.

### Step 2: Write the code under test

`SwapStats` keeps its totals behind an `Arc<Mutex<..>>`, so a test can keep a clone and read them after the pipeline has finished. `load_transaction` turns a saved `getTransaction` response into the `TransactionUpdate` a datasource would send; it's what the RPC datasources do with a live response.

`src/lib.rs`:

```rust
use {
    carbon_core::{
        datasource::TransactionUpdate,
        error::{CarbonResult, Error},
        instruction::InstructionProcessorInputType,
        processor::Processor,
        transformers::transaction_metadata_from_original_meta,
    },
    carbon_meteora_dlmm_decoder::instructions::MeteoraDlmmInstruction,
    solana_pubkey::Pubkey,
    solana_transaction_status::EncodedConfirmedTransactionWithStatusMeta,
    std::{
        collections::HashMap,
        path::Path,
        sync::{Arc, Mutex},
    },
};

#[derive(Debug, Default, Clone, PartialEq)]
pub struct PairTotals {
    pub swaps: u64,
    pub amount_in: u128,
}

/// The pool and input amount of a swap instruction, if it is one.
pub fn swap_amount(instruction: &MeteoraDlmmInstruction) -> Option<(Pubkey, u64)> {
    match instruction {
        MeteoraDlmmInstruction::Swap { data, accounts, .. } => Some((accounts.lb_pair, data.amount_in)),
        MeteoraDlmmInstruction::Swap2 { data, accounts, .. } => {
            Some((accounts.lb_pair, data.amount_in))
        }
        _ => None,
    }
}

/// Sums swap volume per DLMM pool. Clone it to keep a handle on the totals.
#[derive(Clone, Default)]
pub struct SwapStats {
    pub totals: Arc<Mutex<HashMap<Pubkey, PairTotals>>>,
}

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapStats {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        if input.metadata.transaction_metadata.meta.status.is_err() {
            return Ok(()); // failed transactions moved no funds
        }
        if let Some((pair, amount_in)) = swap_amount(input.decoded_instruction) {
            let mut totals = self.totals.lock().map_err(|e| Error::Custom(e.to_string()))?;
            let entry = totals.entry(pair).or_default();
            entry.swaps += 1;
            entry.amount_in += u128::from(amount_in);
        }
        Ok(())
    }
}

/// Loads a `getTransaction` result saved as JSON (Base64 encoding) into a `TransactionUpdate`.
pub fn load_transaction(path: impl AsRef<Path>) -> TransactionUpdate {
    let path = path.as_ref();
    let json = std::fs::read(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()));
    let fetched: EncodedConfirmedTransactionWithStatusMeta =
        serde_json::from_slice(&json).expect("a getTransaction result");

    let transaction = fetched
        .transaction
        .transaction
        .decode()
        .expect("Base64-encoded transaction");
    let meta = transaction_metadata_from_original_meta(fetched.transaction.meta.expect("meta"))
        .expect("convertible meta");

    TransactionUpdate {
        signature: transaction.signatures[0],
        transaction,
        meta,
        is_vote: false,
        slot: fetched.slot,
        index: None,
        block_time: fetched.block_time,
        block_hash: None,
    }
}
```

### Step 3: Capture a transaction

Save the `getTransaction` result for one signature. Request `base64` encoding: Carbon can't rebuild a transaction from `json` or `jsonParsed`.

```sh
SIG=SZE5CFK6gkG91yUWfovgDWd7XYFT9DFaHphCnQq2TbghcBY1c1uSzdksAxsayxgh27aBPhrvbBAahUbUNiNB757
curl -s https://api.mainnet-beta.solana.com -H 'content-type: application/json' \
  -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"getTransaction\",\"params\":[\"$SIG\",{\"encoding\":\"base64\",\"maxSupportedTransactionVersion\":1}]}" \
  | jq '.result' > tests/fixtures/route.json
jq '.slot, (.meta.logMessages | map(select(contains("Instruction: Swap"))))' tests/fixtures/route.json
```

Output:

```text
467107994
[
  "Program log: Instruction: Swap2",
  "Program log: Instruction: Swap2",
  "Program log: Instruction: SwapV2"
]
```

Two `Swap2` instructions are Meteora DLMM CPIs from Jupiter; `SwapV2` belongs to another program. If `route.json` contains `null`, the RPC rejected the request (rate limit or unknown signature): run `curl` without `jq` to see the error.

### Step 4: Extract instruction fixtures

`read_instruction` expects resolved accounts with signer and writable flags, which a raw transaction doesn't contain. This tool extracts them with Carbon's own instruction extraction and writes one fixture per DLMM instruction:

`src/bin/extract-instructions.rs`:

```rust
//! Writes every Meteora DLMM instruction of a saved transaction as a
//! `carbon-test-utils` instruction fixture.
use {
    carbon_core::{transaction::TransactionMetadata, transformers::extract_instructions_with_metadata},
    carbon_meteora_dlmm_decoder::PROGRAM_ID,
    serde_json::json,
    std::sync::Arc,
};

fn main() {
    let path = std::env::args().nth(1).expect("usage: extract-instructions <tx.json>");
    let update = swap_stats::load_transaction(&path);
    let metadata = Arc::new(TransactionMetadata::try_from(update.clone()).expect("metadata"));

    let instructions = extract_instructions_with_metadata(&metadata, &update).expect("instructions");
    for (n, (meta, ix)) in instructions.iter().enumerate() {
        if ix.program_id != PROGRAM_ID {
            continue;
        }
        let fixture = json!({
            "program_id": ix.program_id.to_string(),
            "accounts": ix.accounts.iter().map(|a| json!({
                "pubkey": a.pubkey.to_string(),
                "is_signer": a.is_signer,
                "is_writable": a.is_writable,
            })).collect::<Vec<_>>(),
            "data": hex::encode(&ix.data),
        });
        let out = format!("{}.ix{n}.json", path.trim_end_matches(".json"));
        std::fs::write(&out, serde_json::to_string_pretty(&fixture).unwrap()).unwrap();
        println!("{out}: path={:?} stack_height={}", meta.absolute_path, meta.stack_height);
    }
}
```

```sh
cargo run --bin extract-instructions -- tests/fixtures/route.json
```

Output:

```text
tests/fixtures/route.ix9.json: path=[4, 0] stack_height=2
tests/fixtures/route.ix10.json: path=[4, 0, 0] stack_height=3
tests/fixtures/route.ix13.json: path=[4, 0, 3] stack_height=3
tests/fixtures/route.ix14.json: path=[4, 1] stack_height=2
tests/fixtures/route.ix15.json: path=[4, 1, 0] stack_height=3
tests/fixtures/route.ix18.json: path=[4, 1, 3] stack_height=3
```

The two stack-height-2 entries are the swaps; the others are the DLMM program's self-CPI events. Keep the swaps under readable names and delete the rest:

```sh
mv tests/fixtures/route.ix9.json tests/fixtures/swap2_first.json
mv tests/fixtures/route.ix14.json tests/fixtures/swap2_second.json
rm tests/fixtures/route.ix*.json
```

### Step 5: Capture an account

`read_account` expects the account fields with `data` as a base64 string. `getAccountInfo` with `base64` encoding returns `data` as `[string, "base64"]`, so reshape it with `jq`:

```sh
curl -s https://api.mainnet-beta.solana.com -H 'content-type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"getAccountInfo","params":["Bnoev1Gp1o247d7zTwSbznRwqjkRmbrx3R6Qt4P16YbQ",{"encoding":"base64"}]}' \
  | jq '.result.value | {data: .data[0], executable, lamports, owner, rent_epoch: .rentEpoch}' \
  > tests/fixtures/lb_pair.json
jq '{executable, lamports, owner, rent_epoch}' tests/fixtures/lb_pair.json
```

Output:

```text
{
  "executable": false,
  "lamports": 6535656,
  "owner": "LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo",
  "rent_epoch": 18446744073709551615
}
```

`rent_epoch` is `u64::MAX` for rent-exempt accounts. Use jq 1.7 or newer, which keeps large integers exact; older versions round it to a float and `read_account` fails to parse it.

### Step 6: Write decoder tests

The first test decodes instruction data copied from an explorer, without any fixture file. The others load the fixtures and go through `MeteoraDlmmDecoder`, so they also check account arrangement and account discriminators.

`tests/decoder.rs`:

```rust
use {
    carbon_core::{account::AccountDecoder, instruction::InstructionDecoder},
    carbon_meteora_dlmm_decoder::{
        accounts::MeteoraDlmmAccount,
        instructions::{MeteoraDlmmInstruction, Swap2},
        MeteoraDlmmDecoder,
    },
    carbon_test_utils::{base58_deserialize, read_account, read_instruction},
    solana_pubkey::Pubkey,
};

const POOL: Pubkey = Pubkey::from_str_const("Bnoev1Gp1o247d7zTwSbznRwqjkRmbrx3R6Qt4P16YbQ");
const WSOL: Pubkey = Pubkey::from_str_const("So11111111111111111111111111111111111111112");

#[test]
fn decodes_swap2_data_from_base58() {
    // Instruction data as an explorer shows it (base58).
    let data = base58_deserialize::ix_data("fx9RHbGFfZA1Se5ZmKqQdyyUnuKrrydfLZcDLj");

    let swap = Swap2::decode(&data).expect("Swap2 data");
    assert_eq!(swap.amount_in, 91_444_688);
    assert_eq!(swap.min_amount_out, 0);
}

#[test]
fn decodes_swap2_instruction_fixture() {
    let instruction = read_instruction("tests/fixtures/swap2_first.json").expect("fixture");

    let decoded = MeteoraDlmmDecoder
        .decode_instruction(&instruction)
        .expect("a DLMM instruction");

    let MeteoraDlmmInstruction::Swap2 { data, accounts, .. } = decoded else {
        panic!("expected Swap2, got {decoded:?}");
    };
    assert_eq!(data.amount_in, 91_444_688);
    assert_eq!(accounts.lb_pair, POOL);
    assert_eq!(accounts.token_y_mint, WSOL);
}

#[test]
fn decodes_lb_pair_account_fixture() {
    let account = read_account("tests/fixtures/lb_pair.json").expect("fixture");

    let decoded = MeteoraDlmmDecoder.decode_account(&account).expect("a DLMM account");

    let MeteoraDlmmAccount::LbPair(pair) = decoded.data else {
        panic!("expected LbPair");
    };
    assert_eq!(pair.token_y_mint, WSOL);
}
```

### Step 7: Write pipeline tests

`FixtureDatasource` sends saved updates and returns. Once it returns, the pipeline processes what's queued and `run()` returns, so the test can assert right after. The second test edits the saved transaction to check that failed transactions are ignored.

`tests/pipeline.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{Datasource, DatasourceId, Update, UpdateType},
        error::{CarbonResult, Error},
        pipeline::Pipeline,
    },
    carbon_meteora_dlmm_decoder::MeteoraDlmmDecoder,
    solana_pubkey::Pubkey,
    swap_stats::{load_transaction, PairTotals, SwapStats},
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Sends saved updates once, then returns. The pipeline stops when it has processed them.
struct FixtureDatasource(Vec<Update>);

#[async_trait]
impl Datasource for FixtureDatasource {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        _cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        for update in &self.0 {
            sender
                .send((update.clone(), id.clone()))
                .await
                .map_err(|e| Error::Custom(e.to_string()))?;
        }
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::Transaction]
    }
}

#[tokio::test]
async fn sums_both_swaps_of_a_routed_transaction() {
    let transaction = load_transaction("tests/fixtures/route.json");
    let stats = SwapStats::default();

    Pipeline::builder()
        .datasource(FixtureDatasource(vec![Update::Transaction(Box::new(transaction))]))
        .instruction(MeteoraDlmmDecoder, stats.clone())
        .build()
        .unwrap()
        .run()
        .await
        .unwrap();

    let totals = stats.totals.lock().unwrap();
    assert_eq!(totals.len(), 2);
    assert_eq!(
        totals[&Pubkey::from_str_const("Bnoev1Gp1o247d7zTwSbznRwqjkRmbrx3R6Qt4P16YbQ")],
        PairTotals { swaps: 1, amount_in: 91_444_688 }
    );
    assert_eq!(
        totals[&Pubkey::from_str_const("EZyszDEx1LZDt7TsSFV8xdPi49sDKC3mdfv2MVMEQLtU")],
        PairTotals { swaps: 1, amount_in: 405_807_562 }
    );
}

#[tokio::test]
async fn ignores_failed_transactions() {
    let mut transaction = load_transaction("tests/fixtures/route.json");
    transaction.meta.status = Err(solana_transaction_error::TransactionError::AccountInUse);
    let stats = SwapStats::default();

    Pipeline::builder()
        .datasource(FixtureDatasource(vec![Update::Transaction(Box::new(transaction))]))
        .instruction(MeteoraDlmmDecoder, stats.clone())
        .build()
        .unwrap()
        .run()
        .await
        .unwrap();

    assert!(stats.totals.lock().unwrap().is_empty());
}
```

### Step 8: Run the tests

```sh
cargo test
```

Output:

```text
     Running tests/decoder.rs (target/debug/deps/decoder-e08750c32c07deb0)

running 3 tests
test decodes_swap2_instruction_fixture ... ok
test decodes_swap2_data_from_base58 ... ok
test decodes_lb_pair_account_fixture ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s

     Running tests/pipeline.rs (target/debug/deps/pipeline-f12b833c4b961edf)

running 2 tests
test sums_both_swaps_of_a_routed_transaction ... ok
test ignores_failed_transactions ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
…
```

## What to test

- **Every variant you handle.** One fixture per instruction and account type your processors match on. A decoder regenerated from a newer IDL that changes a layout then fails a test instead of returning `None` in production.
- **CPIs, not only top-level calls.** Most DEX volume arrives through aggregators. The route fixture above has both swaps at stack height 2; a processor that only looked at top-level instructions would count zero.
- **Rejections.** Failed transactions, other programs' instructions, truncated data. See the rejection test in [Write a decoder by hand](https://carbon.sevenlabs.org/guides/custom-decoder.md#test-it).
- **Your write path.** For processors that write to a database, run the pipeline test against a disposable database and query it afterwards. With the [`postgres` feature](https://carbon.sevenlabs.org/guides/postgres.md), check that every row you expect is there: sibling CPIs can [share a primary key](https://carbon.sevenlabs.org/reference/postgres.md#instruction-primary-key).

## Test a processor without a pipeline

`process` is an ordinary async method, but its input borrows metadata, the decoded value, the nested instructions and the raw instruction. Building that by hand is more work than running the pipeline, so prefer the fixture datasource above. Logic that doesn't need the input, like `swap_amount` here, is easiest to test as a plain function.

## Common problems

### `Couldn't read fixture: No such file or directory`

Paths in tests are relative to the crate root (`CARGO_MANIFEST_DIR`), where `cargo test` runs them. Use `tests/fixtures/…`, or build the path with `concat!(env!("CARGO_MANIFEST_DIR"), "/tests/fixtures/route.json")` for tests that may run from elsewhere.

### `Couldn't deserialize fixture: invalid type: floating point …, expected u64`

`rent_epoch` was rounded by an old `jq`. Upgrade to jq 1.7 or newer, or edit the value back to `18446744073709551615`.

### `a getTransaction result: Error("invalid type: null …")`

The saved file is `null`: the RPC returned an error instead of the transaction. Capture it again and check the raw response.

### `Base64-encoded transaction` panic in `load_transaction`

The transaction was captured with `json` or `jsonParsed` encoding. Capture it with `"encoding":"base64"`.

### The pipeline test hangs

The datasource never returns, or keeps a clone of `sender` alive in a spawned task. The pipeline only stops when every sender is dropped or the cancellation token is cancelled.

### Decoding the account fixture returns `None`

The account was captured from a different program, or after it was closed and reopened with another layout. Check `owner` in the fixture and the first 8 bytes of `data` against the decoder's discriminator.

## Next steps

- [test-utils reference](https://carbon.sevenlabs.org/reference/test-utils.md): Every fixture type and reader.
- [Write a decoder by hand](https://carbon.sevenlabs.org/guides/custom-decoder.md): Decoders for programs without an IDL, with unit tests.
- [Write a custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md): The same Datasource trait the fixture datasource implements.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/testing · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Run Carbon in production (Carbon 2.0)

> Choose datasources, handle reconnects and gaps, size the channel, stop cleanly on SIGTERM, write idempotently, and monitor a Carbon 2.0 indexer in production.

This page collects what matters when a Carbon indexer runs unattended: where data comes from, what happens when a connection drops or the processors fall behind, how the process stops, and what to monitor. Every behaviour described here is that of Carbon 2.0.0; links point to the code.

## Checklist

| Area          | Do this                                                                                               | Why                                                                     |
| ------------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| Datasource    | Stream with Yellowstone or LaserStream; backfill with an RPC crawler                                  | Streams are fast but can drop data; crawlers are complete but slow      |
| Redundancy    | Two streams from different providers plus `DeduplicationFilter` on every instruction and account pipe | A single upstream is a single point of failure                          |
| Gaps          | Pass a `disconnect_notifier` and backfill the reported slots                                          | Reconnects don't replay missed slots unless you enable replay           |
| Backpressure  | Set `channel_buffer_size` from your peak rate; alert on `carbon_updates_queued`                       | Streaming datasources drop updates when the channel is full             |
| Shutdown      | Handle SIGINT and SIGTERM yourself and cancel the token                                               | The pipeline only handles SIGINT, and can miss it while busy            |
| Exit          | Exit non-zero when `run()` returns without a shutdown request                                         | `run()` returns `Ok` when all datasources have stopped                  |
| Commitment    | `Confirmed` or `Finalized`                                                                            | Carbon 2.0 has no rollback events for `Processed` data                  |
| Writes        | Upsert on a natural key; skip failed transactions                                                     | Reconnects, backfills and redundant sources deliver the same data twice |
| TLS           | Install a rustls `CryptoProvider` first thing in `main`                                               | gRPC datasources panic on their first TLS connection otherwise          |
| Build         | `cargo build --release`                                                                               | Debug builds are many times slower                                      |
| Observability | `RUST_LOG=info`, Prometheus endpoint, alerts                                                          | Processor errors are logged and skipped, not fatal                      |

## A production-shaped indexer

The rest of the page explains each part of this program. It streams Meteora DLMM transactions from one or more Yellowstone endpoints, deduplicates them, reports gaps, serves Prometheus metrics, stops on SIGTERM and exits non-zero if its datasources die.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-prometheus-metrics = "2.0.0"
carbon-yellowstone-grpc-datasource = "2.0.0"
env_logger = "0.11.8"
log = "0.4.28"
rustls = { version = "0.23.35", default-features = false, features = ["std", "aws_lc_rs"] }
tokio = { version = "1.48.0", features = ["full"] }
tokio-util = "0.7.17"
yellowstone-grpc-proto = { version = "=12.7.0", default-features = false }

[profile.release]
codegen-units = 1
lto = "thin"
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        datasource::{DatasourceDisconnection, DatasourceId},
        error::{CarbonResult, Error},
        filter::DeduplicationFilter,
        instruction::InstructionProcessorInputType,
        metrics::{Counter, MetricsRegistry},
        pipeline::{Pipeline, ShutdownStrategy},
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_prometheus_metrics::{PrometheusMetrics, PrometheusServerConfig},
    carbon_yellowstone_grpc_datasource::{
        BlockFilters, YellowstoneGrpcClientConfig, YellowstoneGrpcGeyserClient,
    },
    std::{collections::HashMap, env, sync::Arc, time::Duration},
    tokio::signal::unix::{signal, SignalKind},
    tokio_util::sync::CancellationToken,
    yellowstone_grpc_proto::geyser::{CommitmentLevel, SubscribeRequestFilterTransactions},
};

static MISSED_SLOTS: Counter = Counter::new(
    "indexer_missed_slots_total",
    "Slots skipped while a datasource was disconnected",
);

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();
    rustls::crypto::aws_lc_rs::default_provider()
        .install_default()
        .map_err(|_| Error::Custom("install rustls crypto provider".into()))?;
    MetricsRegistry::global().register_counter(&MISSED_SLOTS);

    // One or more Yellowstone endpoints, ideally from different providers.
    let endpoints = env::var("GEYSER_URLS")
        .map_err(|_| Error::Custom("GEYSER_URLS not set".into()))?;
    let x_token = env::var("X_TOKEN").ok();

    // Gap reports: one message per reconnect, with the slots that were missed.
    let (disconnect_tx, mut disconnect_rx) =
        tokio::sync::mpsc::channel::<DatasourceDisconnection>(64);
    tokio::spawn(async move {
        while let Some(gap) = disconnect_rx.recv().await {
            MISSED_SLOTS.inc_by(gap.missed_slots);
            log::error!(
                "{} reconnected: slots {}..{} missed ({} slots); backfill them",
                gap.source,
                gap.last_slot_before_disconnect,
                gap.first_slot_after_reconnect,
                gap.missed_slots,
            );
        }
    });

    let mut builder = Pipeline::builder();
    for (n, endpoint) in endpoints.split(',').enumerate() {
        let transaction_filters = HashMap::from([(
            "dlmm".to_string(),
            SubscribeRequestFilterTransactions {
                vote: Some(false),
                failed: Some(false),
                account_include: vec![PROGRAM_ID.to_string()],
                ..Default::default()
            },
        )]);
        let datasource = YellowstoneGrpcGeyserClient::new(
            endpoint.trim().to_string(),
            x_token.clone(),
            Some(CommitmentLevel::Confirmed),
            HashMap::new(),
            transaction_filters,
            BlockFilters::default(),
            YellowstoneGrpcClientConfig::default(),
            Some(disconnect_tx.clone()),
            Some(Duration::from_secs(30)),
        );
        builder = builder.datasource_with_id(datasource, DatasourceId::new_named(&format!("geyser-{n}")));
    }
    drop(disconnect_tx);

    // Stop on SIGTERM (systemd, Docker, Kubernetes) and ctrl-C. Listening in a task of its
    // own means a signal that arrives while a processor is running isn't missed.
    let shutdown = CancellationToken::new();
    let on_signal = shutdown.clone();
    let mut sigterm = signal(SignalKind::terminate()).map_err(|e| Error::Custom(e.to_string()))?;
    tokio::spawn(async move {
        tokio::select! {
            _ = sigterm.recv() => log::info!("SIGTERM received, stopping"),
            _ = tokio::signal::ctrl_c() => log::info!("ctrl-C received, stopping"),
        }
        on_signal.cancel();
    });

    builder
        .channel_buffer_size(10_000)
        .metrics(Arc::new(PrometheusMetrics::with_server(
            PrometheusServerConfig::default(),
        )))
        .instruction_with_filters(
            MeteoraDlmmDecoder,
            SwapWriter,
            vec![Box::new(DeduplicationFilter::new(Duration::from_secs(120)))],
        )
        .datasource_cancellation_token(shutdown.clone())
        .shutdown_strategy(ShutdownStrategy::ProcessPending)
        .build()?
        .run()
        .await?;

    if shutdown.is_cancelled() {
        Ok(())
    } else {
        // Every datasource ended on its own: exit non-zero so the supervisor restarts us.
        Err(Error::Custom("all datasources stopped".into()))
    }
}

struct SwapWriter;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapWriter {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmInstruction::Swap2 { data, accounts, .. } = input.decoded_instruction {
            let tx = &input.metadata.transaction_metadata;
            if tx.meta.status.is_err() {
                return Ok(()); // the stream filter drops failed transactions; backfills may not
            }
            // Idempotent key: the same instruction always maps to the same row.
            let key = (tx.signature, input.metadata.absolute_path.clone());
            log::info!("{key:?} pool={} amount_in={}", accounts.lb_pair, data.amount_in);
        }
        Ok(())
    }
}
```

You need a Yellowstone gRPC endpoint (from a provider such as Triton, Helius or QuickNode) to run it; see [Stream with Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md). Without a reachable endpoint the datasource fails at startup and the program exits with an error, which is what a supervisor needs to see:

```sh
GEYSER_URLS=http://127.0.0.1:1 RUST_LOG=info ./target/release/prod-indexer
```

Output:

```text
[… INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
[… INFO  carbon_prometheus_metrics] Prometheus metrics server starting on http://0.0.0.0:9464/metrics
[… ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("gRPC transport error: transport error")
[… INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[… INFO  carbon_core::pipeline] pipeline shutdown complete.
Error: Custom("all datasources stopped")
```

The exit status is 1.

## Choose datasources for latency and completeness

| Datasource                                                                                                                                                                                                                                                         | Latency                                           | Completeness                                                                                                     | Use for                               |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md), [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)                                                                                             | Lowest: pushed as the validator processes them    | Misses whatever arrives while disconnected; LaserStream replays from the last slot by default                    | Live indexing                         |
| [RPC program subscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md), [RPC block subscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md), [Helius Atlas WS](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md) | Low, over websockets                              | Websocket drops lose data; block subscribe isn't available on many RPC nodes, including Solana's public endpoint | Live indexing without a gRPC provider |
| [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                                                                                             | Polling interval plus one request per transaction | Complete for one address, from `getSignaturesForAddress` history                                                 | Backfills, low-volume programs        |
| [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)                                                                                                                                                                                 | One `getBlock` per slot                           | Complete for a slot range                                                                                        | Filling known gaps                    |
| [RPC GPA](https://carbon.sevenlabs.org/datasources/rpc-gpa.md), [Helius GPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md), [validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md)                                   | One-off                                           | Current state of every account                                                                                   | Bootstrapping account state           |

A common setup runs a stream for live data and a crawler (or block crawler) for history and gaps, writing to the same idempotent tables. See [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md) and [Use multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md).

## Run redundant streams and deduplicate

Give each datasource a stable ID with `datasource_with_id(.., DatasourceId::new_named(..))` and put a `DeduplicationFilter` on every pipe:

- For instructions it keys on `(signature, absolute_path)`, for accounts on `(transaction_signature, pubkey)`, and keeps keys for the TTL you pass (entries are purged about once a minute).
- Account updates without a `transaction_signature` are always accepted. Yellowstone and LaserStream set it; program subscribe and GPA datasources don't, so their duplicates pass through.
- It doesn't implement `filter_transaction`, so `.transaction(..)` pipes see every copy. Deduplicate those in your processor or with your own [filter](https://carbon.sevenlabs.org/concepts/filters.md).
- Each pipe owns its own filter instance and its own key set. That's what you want: each pipe should see each instruction once.
- Pick a TTL longer than the time between two sources delivering the same transaction: 60 seconds covers two healthy live streams; raise it to a few minutes if a source can lag further, for example while it reconnects. Memory grows with the number of distinct keys in the window.

## Detect gaps after reconnects

What each streaming datasource does when its connection drops:

| Datasource                                         | Reconnect                                                                                                                      | Reports the gap                                                                                   | Gives up                                                                  |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| Yellowstone gRPC                                   | Resubscribes with backoff from 100 ms to 3 s; a stream silent for `stream_timeout` (30 s default) counts as dropped            | `DatasourceDisconnection` on `disconnect_notifier` with the last slot before and first slot after | Never, once connected. A failed **first** connection ends the datasource. |
| Yellowstone with `with_reconnect(ReconnectConfig)` | The gRPC client reconnects and replays missed data inside the stream, within its slot retention                                | No                                                                                                | Per the client's backoff                                                  |
| Helius LaserStream                                 | Resubscribes every 3 s; with `replay_enabled` (default) it resumes from the last slot it saw (31 slots earlier at `Processed`) | No                                                                                                | After 10 consecutive failures                                             |
| RPC block subscribe                                | Reconnects every 3 s                                                                                                           | `DatasourceDisconnection`                                                                         | After 10 consecutive failures                                             |
| RPC program subscribe                              | Reconnects every 3 s, also when the pipeline channel is full                                                                   | No                                                                                                | After 10 consecutive failures                                             |

`DatasourceDisconnection` has `source`, `disconnect_time`, `last_slot_before_disconnect`, `first_slot_after_reconnect` and `missed_slots` (their difference). The program above counts missed slots in a Prometheus counter and logs them. To close the gap, crawl the range with [`RpcBlockCrawler`](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md) from `last_slot_before_disconnect` (inclusive, because the stream may have dropped mid-slot) to `first_slot_after_reconnect`, through the same processors. Because writes are idempotent, the overlap is harmless.

To use client-side replay instead, add `yellowstone-grpc-client = "=13.5.0"` and:

```rust
use yellowstone_grpc_client::ReconnectConfig;

let config = YellowstoneGrpcClientConfig::default().with_reconnect(ReconnectConfig::default());
```

Replay depends on your provider keeping enough history.

## Backpressure and the channel

Datasources and the processing loop are connected by one bounded channel, [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size) long (1,000 by default). The loop takes one update at a time and runs it through every matching pipe before taking the next, so throughput is about one update per average processing time.

When processors fall behind, the channel fills, and what happens next depends on the datasource:

- **Yellowstone, LaserStream, RPC block subscribe, RPC block crawler, RPC transaction crawler (default), Helius Atlas WS:** `try_send` fails and the update is **dropped**, with an error such as `Failed to send transaction update with signature … at slot …: …` (Yellowstone).
- **RPC program subscribe:** the send failure (`Error sending account update`) ends the subscription, which reconnects after 3 s.
- **RPC transaction crawler with `blocking_send: true`, RPC GPA, Helius GPA v2, Helius GTFA, stream message, validator snapshot, and datasources you write with `send(..).await`:** the datasource waits. Nothing is lost, the source slows down.

So size the buffer to absorb bursts (`.channel_buffer_size(10_000)` above), alert when `carbon_updates_queued` stays high, and keep processors fast: batch database writes, or hand work to a task of your own through your own channel. Memory used by the buffer grows with its length times the size of an update; a full transaction with metadata is typically several kilobytes, so measure your process's resident memory at peak rather than guessing.

## Stop cleanly

What the 2.0.0 pipeline does on shutdown, measured with 2,000 queued updates and a 2 ms processor:

- **SIGTERM** isn't handled. Docker, Kubernetes and systemd stop processes with SIGTERM, which kills the process immediately unless you handle it.
- **SIGINT (ctrl-C)** is only noticed while the loop is waiting for the next update: the loop creates a fresh `tokio::signal::ctrl_c()` listener on each iteration, and a signal that arrives while no listener exists (while an update is being processed) is lost. Tokio has already replaced the default handler, so the process doesn't die either: in the test, a SIGINT sent while the backlog was being processed was ignored, and the pipeline went on to process all 2,000 updates before a second SIGINT stopped it.
- **Cancelling your own token** (`datasource_cancellation_token`) is reliable: the pipeline stops after the update in progress. Updates still in the channel are dropped; in the test, 128 of 2,000 were processed.
- **`ShutdownStrategy::ProcessPending`**, the default, is meant to drain the channel after SIGINT, but the loop exits as soon as it sees the cancelled token, so it doesn't drain. See [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).

So: install your own handlers for SIGINT and SIGTERM, cancel the token from them in a task of their own, as the program above does, and design for at-least-once delivery. Updates queued at shutdown are lost, so on restart resume from a checkpoint a bit before the last slot you wrote (a stream's `from_slot`, a crawler's `until` signature or start slot) and let idempotent writes absorb the overlap.

To finish queued updates before exiting, stop the **source** instead of the pipeline. A datasource you write can watch its own token (not the one passed to `datasource_cancellation_token`), stop sending and return. Once every datasource has returned and dropped its sender, the pipeline processes what's left in the channel and `run()` returns. The built-in datasources only watch the pipeline's token, so this works only for your own.

`run()` also returns `Ok(())` when every datasource has stopped on its own, for example after LaserStream gives up reconnecting. The program above checks whether shutdown was requested and otherwise returns an error, so the process exits with status 1 and your supervisor restarts it (`Restart=on-failure` in systemd, `restartPolicy: Always` in Kubernetes).

## Commitment, forks and finality

Streams let you choose a commitment level (`CommitmentLevel` for Yellowstone and LaserStream); the RPC datasources default to `confirmed`.

- **`Processed`** is fastest, but the block can still be skipped. Carbon 2.0.0 emits no event when a slot is dropped, so data from an abandoned fork stays in your database unless you reconcile it.
- **`Confirmed`** has been voted on by a supermajority. Rollbacks are very rare. This is the usual choice.
- **`Finalized`** can't be rolled back but arrives roughly 32 slots later.

If you must index at `Processed`, store the slot with every row and reconcile later, for example by re-checking recent slots against a `Finalized` source and deleting rows whose transactions aren't there.

## Write idempotently

The same data arrives more than once after reconnects, from redundant sources, from replay and from backfills. Make every write an upsert on a natural key:

- **Instructions:** `(signature, absolute_path)`. `absolute_path` (for example `[4, 1]`) identifies the instruction's position in the CPI tree. Don't key on `InstructionMetadata::index` and `stack_height` alone: inner instructions share their top-level instruction's `index`, so two CPIs under one top-level instruction collide. The generated Postgres tables have this problem; see [Store decoded data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md#two-swaps-in-one-transaction-but-only-one-row).
- **Accounts:** `pubkey`, and only overwrite when the incoming slot is not older: `ON CONFLICT (pubkey) DO UPDATE SET … WHERE excluded.slot >= accounts.slot`. Updates from two sources can arrive out of order. Carbon's generated account upserts overwrite unconditionally.
- **Failed transactions:** the pipeline decodes instructions of failed transactions too. Check `metadata.transaction_metadata.meta.status.is_err()` and skip them unless you want them; some datasource filters (`failed: Some(false)` for Yellowstone) drop them upstream.

## Install a TLS crypto provider

The Yellowstone and LaserStream crates use `rustls` through `tonic`, and Carbon's dependency graph doesn't select a rustls crypto backend by itself. Without one, the first TLS connection panics inside the datasource task:

Output:

```text
thread 'tokio-rt-worker' panicked at …/rustls-0.23.46/src/crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
Call CryptoProvider::install_default() before this point to select a provider manually, or make sure exactly one of the 'aws-lc-rs' and 'ring' features is enabled.
```

The panic kills only that task, so the pipeline logs `update_receiver closed, shutting down.` and `run()` returns `Ok(())`: without the exit check below, the process exits with status 0 and looks like a clean stop. Add `rustls` with the `aws_lc_rs` feature and install the provider before building any datasource, as `main` above does:

```rust
rustls::crypto::aws_lc_rs::default_provider()
    .install_default()
    .map_err(|_| Error::Custom("install rustls crypto provider".into()))?;
```

`install_default` fails only if a provider is already installed, which you can ignore if another library installs the same one.

## Build for release

```sh
cargo build --release
```

Always deploy release builds. With the profile above, building this program from scratch took under a minute on an 8-core machine with a peak of 680 MB of memory and produced a 12.6 MB binary. `lto = "thin"` and `codegen-units = 1` make the binary a little faster at the cost of build time; they're optional. Pin your dependencies with a committed `Cargo.lock`, and build on the same Rust version you test with (1.96.1 or newer).

## Logs

Carbon logs through the `log` crate, so the binary decides the format and level; with `env_logger`, set `RUST_LOG`:

```sh
RUST_LOG=info,carbon_log_metrics=warn ./target/release/prod-indexer
```

`info` shows startup, reconnects and shutdown. `LogMetrics` prints every metric every 5 seconds at `info`; silence it with its target as above if you also export to Prometheus, or use `LogMetrics::new_with_flush_interval(60)`. Messages worth alerting on in your log platform:

| Message                                                                        | Meaning                                                                                                                                                                           |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `error consuming datasource: …`                                                | A datasource stopped with an error                                                                                                                                                |
| `error processing update (…): …`                                               | A decoder conversion or processor returned `Err`; the update was skipped for the remaining pipes. The message includes the full update with `Debug` formatting, so it can be long |
| `Failed to send transaction update …`, `Failed to send account event …`        | Yellowstone dropped an update because the channel was full                                                                                                                        |
| `Stream timeout - no messages for …`, `Reconnected. Slots: … -> … (missed: …)` | Yellowstone reconnect                                                                                                                                                             |
| `Max reconnection attempts reached: 10`                                        | LaserStream gave up                                                                                                                                                               |
| `Prometheus metrics server error: …`                                           | The metrics endpoint didn't start                                                                                                                                                 |

Log as JSON for your log platform by swapping `env_logger` for a structured logger; Carbon's messages are plain strings, so the fields above are in the message text.

## Metrics and alerts

Serve metrics with [`carbon-prometheus-metrics`](https://carbon.sevenlabs.org/guides/prometheus.md) and alert on at least: the endpoint being down, no updates received for longer than your program's quiet periods, a failure ratio above zero, `carbon_updates_queued` near `channel_buffer_size`, p99 processing time, and any increase of your own missed-slots counter. [Monitor with Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md#alert-rules) has rules for all of these.

## Size resources

- **CPU.** Processing runs on one task, so one fast core does the decoding and processor work; datasource tasks (gRPC decoding, RPC calls) run on other Tokio worker threads. More cores don't speed up the processing loop. If one core isn't enough, run several indexers, each on a subset of programs.
- **Memory.** Dominated by the channel buffer, `DeduplicationFilter` key sets and your own state. Measure resident memory at peak load and leave headroom for bursts and reconnect replays.
- **Network.** A Yellowstone subscription delivers only what its filters match, but a busy program's full transactions still add up to a lot of bandwidth. Filter as narrowly as you can upstream (`account_include`, `failed: Some(false)`, `vote: Some(false)`).
- **Database.** Each processed instruction is one round trip with Carbon's Postgres processors, so database latency caps throughput. Keep the database in the same region, or batch.

## Next steps

- [Monitor with Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md): The endpoint, queries and alert rules.
- [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md): Cancellation tokens and shutdown strategies in detail.
- [Use multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md): Combine streams and crawlers in one pipeline.
- [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md): Symptoms and fixes for common failures.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/production · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Migrate from Carbon 1.x to 2.0 (Carbon 2.0)

> Upgrade from Carbon 1.x to 2.0: a checklist with before/after code for the toolchain, Solana crates, transaction configs, account deletions and the CLI.

Carbon 2.0 is the Agave 4 compatibility release: it moves to the Agave 4.2 Solana crates, supports Transaction V1 end to end, and delivers account closures the same way from every datasource. The pipeline, processor and decoder APIs are unchanged, so most indexers only need dependency bumps and a few constructor changes. This page lists every breaking change from 1.x, from the [2.0.0 changelog](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/CHANGELOG.md) and the crates themselves.

## Checklist

- [ ] [Rust 1.96.1 or newer](#toolchain), Node.js 20.18.0 or newer for the CLI.
- [ ] [Every Carbon crate at `2.0.0`, Solana crates on the Agave 4.2 cohort](#dependencies).
- [ ] [Remove Jetstreamer and Jito Shredstream](#datasources-not-in-20) if you use them; they aren't on the 2.0 line.
- [ ] [Regenerate decoders](#regenerate-decoders-with-the-20-cli) with `@sevenlabs-hq/carbon-cli@2.0.0`.
- [ ] [Request Base64 and transaction version 1](#transaction-source-configuration) in RPC block and Helius Atlas configs you build yourself.
- [ ] [Remove `account_deletions_tracked`](#account-deletions) from `YellowstoneGrpcGeyserClient`, `LaserStreamGeyserClient`, `HeliusWebsocket` and `StreamMessageClient`.
- [ ] [Handle zero-lamport accounts as deletions](#zero-lamport-accounts-are-now-always-deletions); give `AccountDeletion` its new `account` field.
- [ ] [Use `into_update()` in custom datasources](#custom-datasources).
- [ ] [Add `from_slot` to `LaserStreamClientConfig::new`](#laserstream-config).
- [ ] [Opt into Yellowstone reconnect](#yellowstone-client-reconnection-is-opt-in) if you want client-side replay.
- [ ] [Replace `Mint::decode`, `Token::decode` and `Multisig::decode`](#spl-token-account-helpers).
- [ ] [Match `VersionedMessage::V1`](#transaction-v1) wherever you match on message versions.
- [ ] [Update dashboards for sanitized Prometheus names](#other-behaviour-changes) if you used custom metric names with `.` or `-`.

## Toolchain

Carbon 2.0 crates declare `rust-version = "1.96.1"`; 1.x built with Rust 1.88.

```sh
rustup update stable
rustc --version
```

If your project pins a toolchain, update it:

`rust-toolchain.toml`:

```diff
 [toolchain]
-channel = "1.88.0"
+channel = "1.96.1"
```

The CLI and its renderer need Node.js 20.18.0 or newer (the 1.x-era CLI, 0.12.0, accepted Node 18).

## Dependencies

Bump every Carbon crate you declare to `2.0.0`, and every Solana crate you declare directly to the version Carbon 2.0 uses. Carbon 1.x used the Solana 3.x crates (`solana-pubkey` 3, `solana-client` 3, `yellowstone-grpc-proto` 10).

`Cargo.toml`:

```diff
 [dependencies]
-carbon-core = "1.0.0"
-carbon-yellowstone-grpc-datasource = "1.0.0"
-carbon-pump-swap-decoder = "1.0.0"
-solana-pubkey = "3.0.0"
-solana-client = "3.0.0"
-yellowstone-grpc-proto = "10.0.0"
+carbon-core = "2.0.0"
+carbon-yellowstone-grpc-datasource = "2.0.0"
+carbon-pump-swap-decoder = "2.0.0"
+solana-pubkey = "4.2.1"
+solana-client = "=4.2.2"
+yellowstone-grpc-proto = { version = "=12.7.0", default-features = false }
```

The versions Carbon 2.0 pins, for the crates you're most likely to declare:

| Crate                                                                  | Version            |
| ---------------------------------------------------------------------- | ------------------ |
| `solana-pubkey`                                                        | `4.2.1`            |
| `solana-account`                                                       | `4.3.2`            |
| `solana-signature`                                                     | `3.4.1`            |
| `solana-instruction`                                                   | `3.4.1`            |
| `solana-message`                                                       | `4.4.1`            |
| `solana-hash`                                                          | `4.5.0`            |
| `solana-transaction`                                                   | `4.1.6`            |
| `solana-commitment-config`                                             | `3.1.1`            |
| `solana-client`, `solana-transaction-status`, `solana-account-decoder` | `=4.2.2`           |
| `yellowstone-grpc-proto`                                               | `=12.7.0`          |
| `yellowstone-grpc-client`                                              | `=13.5.0`          |
| `helius`                                                               | `2.0.0`            |
| `sqlx` / `sqlx_migrator` (with `postgres`)                             | `0.8.6` / `0.17.0` |
| `juniper` / `axum` (with `graphql`)                                    | `0.16` / `0.8`     |

Then update and list the versions your own crate depends on directly:

```sh
cargo update
cargo tree -e normal --depth 1 | grep -E "solana|yellowstone|carbon"
```

Output:

```text
my-indexer v0.1.0 (/home/you/my-indexer)
├── carbon-core v2.0.0
├── carbon-meteora-dlmm-decoder v2.0.0
├── carbon-prometheus-metrics v2.0.0
├── carbon-yellowstone-grpc-datasource v2.0.0
├── solana-hash v4.5.0
├── solana-message v4.4.1
├── solana-pubkey v4.2.1
├── solana-signature v3.4.1
├── solana-transaction v4.1.6
├── solana-transaction-status v4.2.2
├── yellowstone-grpc-client v13.5.0
└── yellowstone-grpc-proto v12.7.0
```

Every Carbon crate must be `2.0.0`, and every `solana-*` crate you list must match the table. Don't use `cargo tree -d` as the check: Agave 4.2's own crates already pull a second copy of some type crates (`solana-account` 3.4.0 and `solana-pubkey` 3.0.0, through `solana-config-interface`), which is expected and harmless.

A Solana type crate on the wrong major version fails where a Carbon type meets yours:

Output:

```text
error[E0308]: mismatched types
   |  expected `solana_account::Account`, found a different `solana_account::Account`
note: there are multiple different versions of crate `solana_account` in the dependency graph
```

Don't mix Carbon 1.x and 2.0 crates in one graph either.

### Datasources not in 2.0

The 2.0 release publishes twelve datasource crates. `carbon-jito-shredstream-grpc-datasource` (no Transaction V1 support yet) and `carbon-jetstreamer-datasource` (not yet on the Solana v4 stack) stay on 1.x and can't be used in a Carbon 2.0 dependency graph. Keep indexers that depend on them on 1.x for now, or move them to [another datasource](https://carbon.sevenlabs.org/datasources.md).

## Regenerate decoders with the 2.0 CLI

Decoders generated by earlier CLI versions depend on Carbon 1.x crates. Regenerate them from the same IDL with the 2.0 CLI. The CLI's version on npm is `2.0.0` (the changelog calls this release of the CLI, renderer and version-registry packages 0.13; npm has no 0.13).

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./idl.json --out-dir ./decoders/my-program --name my-program
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./idl.json --out-dir ./decoders/my-program --name my-program
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./idl.json --out-dir ./decoders/my-program --name my-program
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./idl.json --out-dir ./decoders/my-program --name my-program
```

`parse` deletes `--out-dir` first, so commit or move any hand edits before you run it. Other CLI changes:

- `scaffold` now rejects `--with-graphql true` with `--with-postgres false`: the generated GraphQL layer reads the Postgres tables.
- Codama IDLs: native Codama events are rendered, and event-CPI discriminators are exported (`--event-cpi-discriminator` overrides the default Anchor tag).
- With `--with-base58 true`, pubkeys in instruction metadata serialize as base58 too.

See [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md) and the [CLI reference](https://carbon.sevenlabs.org/reference/cli.md).

## Transaction source configuration

Carbon 2.0 reconstructs transactions from binary encodings only and needs `max_supported_transaction_version: Some(1)` to receive V1 transactions. Configs you pass yourself must say so. `RpcTransactionCrawler` and the Helius GTFA datasource set both internally.

### `RpcBlockSubscribe`

```rust
// Carbon 1.x
let filters = Filters::new(
    RpcBlockSubscribeFilter::MentionsAccountOrProgram(program_id.to_string()),
    Some(RpcBlockSubscribeConfig {
        encoding: Some(UiTransactionEncoding::Base64),
        transaction_details: Some(TransactionDetails::Full),
        max_supported_transaction_version: Some(0),
        ..RpcBlockSubscribeConfig::default()
    }),
);
```

```rust
// Carbon 2.0
let filters = Filters::new(
    RpcBlockSubscribeFilter::MentionsAccountOrProgram(program_id.to_string()),
    Some(RpcBlockSubscribeConfig {
        encoding: Some(UiTransactionEncoding::Base64),
        transaction_details: Some(TransactionDetails::Full),
        max_supported_transaction_version: Some(1),
        ..RpcBlockSubscribeConfig::default()
    }),
);
let datasource = RpcBlockSubscribe::new(rpc_ws_url, filters);
```

If you requested `Json` or `JsonParsed`, switch to `Base64`: Carbon can't rebuild the original transaction from them.

### `RpcBlockCrawler`

```rust
// Carbon 2.0
let datasource = RpcBlockCrawler::new(
    rpc_url,
    467_000_000,       // start_slot
    Some(467_000_100), // end_slot
    None,              // block_interval
    RpcBlockConfig {
        encoding: Some(UiTransactionEncoding::Base64),
        transaction_details: Some(TransactionDetails::Full),
        max_supported_transaction_version: Some(1),
        ..RpcBlockConfig::default()
    },
    None, // max_concurrent_requests
    None, // channel_buffer_size
);
```

The 1.x code is the same with `max_supported_transaction_version: Some(0)`.

### `HeliusWebsocket` transaction subscriptions

The Helius SDK's `TransactionSubscribeOptions::default()` asks for `jsonParsed` and version 0. Override both:

```rust
// Carbon 2.0
use helius::types::{
    Cluster, RpcTransactionsConfig, TransactionSubscribeFilter, TransactionSubscribeOptions,
    UiEnhancedTransactionEncoding,
};

let filters = Filters::new(
    vec![],
    Some(RpcTransactionsConfig {
        filter: TransactionSubscribeFilter {
            account_include: Some(vec![program_id.to_string()]),
            ..TransactionSubscribeFilter::default()
        },
        options: TransactionSubscribeOptions {
            encoding: Some(UiEnhancedTransactionEncoding::Base64),
            max_supported_transaction_version: Some(1),
            ..TransactionSubscribeOptions::default()
        },
    }),
)?;
let datasource = HeliusWebsocket::new(api_key, filters, Cluster::MainnetBeta);
```

`Filters` here is `carbon_helius_atlas_ws_datasource::Filters`; add `helius = "2.0.0"` to use the SDK types.

## Account deletions

### The `account_deletions_tracked` argument is gone

In 1.x these four constructors took an `Arc<RwLock<HashSet<Pubkey>>>` of accounts to report closures for. Remove the argument.

**`YellowstoneGrpcGeyserClient::new`**

```rust
// Carbon 1.x
let datasource = YellowstoneGrpcGeyserClient::new(
    endpoint,
    x_token,
    Some(CommitmentLevel::Confirmed),
    account_filters,
    transaction_filters,
    BlockFilters::default(),
    Arc::new(RwLock::new(HashSet::new())), // account_deletions_tracked
    YellowstoneGrpcClientConfig::default(),
    None,
    None,
);
```

```rust
// Carbon 2.0
let datasource = YellowstoneGrpcGeyserClient::new(
    endpoint,
    x_token,
    Some(CommitmentLevel::Confirmed),
    account_filters,
    transaction_filters,
    BlockFilters::default(),
    YellowstoneGrpcClientConfig::default(),
    None, // disconnect_notifier
    None, // stream_timeout
);
```

**`LaserStreamGeyserClient::new`**

```rust
// Carbon 1.x
let datasource = LaserStreamGeyserClient::new(
    endpoint,
    api_key,
    Some(CommitmentLevel::Confirmed),
    account_filters,
    transaction_filters,
    BlockFilters::default(),
    Arc::new(RwLock::new(HashSet::new())), // account_deletions_tracked
    config,
);
```

```rust
// Carbon 2.0
let datasource = LaserStreamGeyserClient::new(
    endpoint,
    api_key,
    Some(CommitmentLevel::Confirmed),
    account_filters,
    transaction_filters,
    BlockFilters::default(),
    config,
);
```

**`HeliusWebsocket::new`**

```rust
// Carbon 1.x
let datasource = HeliusWebsocket::new(api_key, filters, Arc::new(RwLock::new(HashSet::new())), Cluster::MainnetBeta);
```

```rust
// Carbon 2.0
let datasource = HeliusWebsocket::new(api_key, filters, Cluster::MainnetBeta);
```

**`StreamMessageClient::new`**

```rust
// Carbon 1.x
let datasource = StreamMessageClient::new(Arc::new(RwLock::new(HashSet::new())), rx);
```

```rust
// Carbon 2.0
let datasource = StreamMessageClient::new(rx);
```

The matching public `account_deletions_tracked` fields were removed too.

### Zero-lamport accounts are now always deletions

|                                                                                   | Carbon 1.x                | Carbon 2.0                |
| --------------------------------------------------------------------------------- | ------------------------- | ------------------------- |
| Zero-lamport update for a tracked pubkey, empty data, owned by the System Program | `Update::AccountDeletion` | `Update::AccountDeletion` |
| Same, pubkey not tracked                                                          | Dropped                   | `Update::AccountDeletion` |
| Zero lamports, other owner or non-empty data                                      | `Update::Account`         | `Update::AccountDeletion` |
| Non-zero lamports                                                                 | `Update::Account`         | `Update::Account`         |

Consequences:

- Account pipes (`.account(..)`) no longer receive zero-lamport accounts. If a processor handled closures by checking `lamports == 0`, move that logic to an `.account_deletions(..)` processor.
- `.account_deletions(..)` processors now receive a deletion for every closed account the datasource sees, not only tracked ones. Filter by owner or pubkey in the processor if you only care about some.
- `AccountDeletion` carries the final account state in a new `account: Account` field. Its `owner` and `data` are whatever the source reported for the closing update, so check them before you rely on them.

If you construct `AccountDeletion` yourself, add the field:

```rust
// Carbon 1.x
let deletion = AccountDeletion { pubkey, slot, transaction_signature: None };
```

```rust
// Carbon 2.0
let deletion = AccountDeletion { pubkey, account, slot, transaction_signature: None };
```

### Custom datasources

Build an `AccountUpdate` and call `into_update()` instead of choosing between `Update::Account` and `Update::AccountDeletion` yourself, so your datasource applies the same rule as the built-in ones:

```rust
// Carbon 1.x
let update = Update::Account(AccountUpdate { pubkey, account, slot, transaction_signature: None });
```

```rust
// Carbon 2.0
let update = AccountUpdate { pubkey, account, slot, transaction_signature: None }.into_update();
```

See [Write a custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md#emit-account-updates).

## Datasource configuration changes

### LaserStream config

`LaserStreamClientConfig::new` takes a new last argument, `from_slot: Option<u64>`. Pass `None` to keep the previous behaviour, or a slot to start the stream from it.

```rust
// Carbon 1.x
let config = LaserStreamClientConfig::new(
    None,                              // compression
    Some(Duration::from_secs(15)),     // connect_timeout
    Some(Duration::from_secs(15)),     // timeout
    None,                              // max_decoding_message_size
    None,                              // tls_config
    None,                              // tcp_nodelay
    true,                              // replay_enabled
);
```

```rust
// Carbon 2.0
let config = LaserStreamClientConfig::new(
    None,                              // compression
    Some(Duration::from_secs(15)),     // connect_timeout
    Some(Duration::from_secs(15)),     // timeout
    None,                              // max_decoding_message_size
    None,                              // tls_config
    None,                              // tcp_nodelay
    true,                              // replay_enabled
    None,                              // from_slot
);
```

`LaserStreamClientConfig::default()` also works and sets `from_slot: None`.

### Yellowstone client reconnection is opt-in

`YellowstoneGrpcClientConfig` has a new `reconnect` field, off by default, so nothing changes unless you opt in. With it, the gRPC client reconnects and replays missed data inside the stream, and Carbon no longer sees (or reports) the disconnect. Use it only if your provider supports replay for the range you need:

```rust
// Carbon 2.0, optional; add yellowstone-grpc-client = "=13.5.0"
use yellowstone_grpc_client::ReconnectConfig;

let config = YellowstoneGrpcClientConfig::default().with_reconnect(ReconnectConfig::default());
```

If you build `YellowstoneGrpcClientConfig` with a struct literal, add `reconnect: None`. `YellowstoneGrpcClientConfig::new(..)` keeps its six arguments.

## SPL Token account helpers

`carbon-token-program-decoder` no longer has `Mint::decode`, `Token::decode` and `Multisig::decode`. Decoding through `TokenProgramDecoder` in a pipeline needs no change. If you called them directly, unpack with the SPL layout and convert:

```rust
// Carbon 1.x
let token = carbon_token_program_decoder::accounts::token::Token::decode(data);
```

```rust
// Carbon 2.0; add solana-program-pack = "3.1.0" and spl-token-interface = "3.0.0"
use solana_program_pack::Pack;

let token = spl_token_interface::state::Account::unpack(data)
    .ok()
    .map(carbon_token_program_decoder::accounts::token::Token::from);
let mint = spl_token_interface::state::Mint::unpack(data)
    .ok()
    .map(carbon_token_program_decoder::accounts::mint::Mint::from);
let multisig = spl_token_interface::state::Multisig::unpack(data)
    .ok()
    .map(carbon_token_program_decoder::accounts::multisig::Multisig::from);
```

`unpack` also checks that the account is initialized, which the 1.x `decode` (a length check plus borsh) didn't.

## Transaction V1

Agave 4.2 adds V1 transactions, and Carbon 2.0 decodes their instructions, inner instructions and account metadata like legacy and V0. Your code only changes if it matches on the message version: `solana_message::VersionedMessage` has a third variant.

```rust
// Carbon 2.0
match &input.metadata.transaction_metadata.message {
    VersionedMessage::Legacy(_) => "legacy",
    VersionedMessage::V0(_) => "v0",
    VersionedMessage::V1(_) => "v1",
}
```

## Other behaviour changes

These don't need code changes but can change what you see:

- **Prometheus metric names are sanitized.** Characters other than letters, digits, `_` and `:` become `_` (`rpc.block-crawler.fetch.failed` is served as `rpc_block_crawler_fetch_failed`). Built-in names were already valid; update dashboards that query custom metrics with `.` or `-` in their names.
- **Short log events decode.** `decode_log_events` no longer skips `Program data:` payloads shorter than 8 bytes, so events with short discriminators are returned.
- **Invalid CPI stack heights are skipped.** Inner instructions with a missing or out-of-range `stackHeight` stop that inner-instruction group with a warning instead of producing a wrong tree.
- **Yellowstone** backs off between failed subscribe attempts (100 ms doubling to 3 s) and keeps its filters on ping resubscribes.
- **Token Program accounts** decode with the SPL `Pack` layout.

## Verify the migration

1. `cargo build` with no duplicate Solana crates (`cargo tree -d`).
2. Run against a known slot range and compare decoded counts with your 1.x indexer. Expect more `AccountDeletion`s and fewer zero-lamport account updates.
3. If you index V1 transactions, check that some arrive: `VersionedMessage::V1` in `transaction_metadata.message`.

## Common problems

### ``expected `solana_account::Account`, found a different `solana_account::Account` ``

A Solana type crate you depend on is still on its 1.x-era major version (here `solana-account` 3), or a Carbon 1.x crate is left in the graph. Bump it to the version in the [table](#dependencies) and run `cargo update`. `solana-pubkey` 3 and 4 share one `Pubkey` type, so pubkeys rarely show this error; accounts, transactions and messages do.

### `this function takes 9 arguments but 10 arguments were supplied`

A 1.x `YellowstoneGrpcGeyserClient::new` call still passes `account_deletions_tracked`. Remove it. The same error with other counts comes from `LaserStreamGeyserClient::new` (7), `HeliusWebsocket::new` (3) and `StreamMessageClient::new` (1).

### `this function takes 8 arguments but 7 arguments were supplied` on `LaserStreamClientConfig::new`

Add the trailing `from_slot` argument (`None`).

### ``no associated function or constant named `decode` found for struct `Token` ``

Use `spl_token_interface::state::Account::unpack` and `Token::from`, as shown above.

### `Failed to decode transaction` in the logs of an RPC block datasource

Your config still requests `Json`/`JsonParsed` or version 0. Set `UiTransactionEncoding::Base64` and `max_supported_transaction_version: Some(1)`.

## Next steps

- [Changelog](https://carbon.sevenlabs.org/changelog.md): Everything in 2.0.0.
- [Installation](https://carbon.sevenlabs.org/installation.md): Toolchain, crates and features for Carbon 2.0.
- [Datasources](https://carbon.sevenlabs.org/datasources.md): The twelve datasources in Carbon 2.0.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/guides/migrate-to-v2 · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Pipeline builder (Carbon 2.0)

> Every PipelineBuilder method, Pipeline::run, ShutdownStrategy and the pipeline defaults in Carbon 2.0.0.

`Pipeline` connects datasources to processors. You create one with `Pipeline::builder()`, register datasources, pipes and metrics exporters on the returned `PipelineBuilder`, call `build()`, and then `run().await`. This page lists every method in `carbon_core::pipeline` for Carbon 2.0.0. For the model behind it, see [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md).

Source: [`crates/core/src/pipeline.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/pipeline.rs).

## Example

This complete program crawls recent Meteora DLMM transactions on the public RPC endpoint. It uses a named datasource, an instruction pipe with two filters, a metrics exporter, a cancellation token, an explicit shutdown strategy and a larger channel. The fragments in the entries below come from this program.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11"
tokio = { version = "1", features = ["full"] }
tokio-util = "0.7"
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        datasource::DatasourceId,
        error::CarbonResult,
        filter::{DatasourceFilter, DeduplicationFilter, Filter},
        instruction::InstructionProcessorInputType,
        pipeline::{Pipeline, ShutdownStrategy},
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::{sync::Arc, time::Duration},
    tokio_util::sync::CancellationToken,
};

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmInstruction::Swap { data, .. } = input.decoded_instruction {
            println!(
                "swap {} slot={} amount_in={}",
                input.metadata.transaction_metadata.signature,
                input.metadata.transaction_metadata.slot,
                data.amount_in
            );
        }
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let crawler_id = DatasourceId::new_named("dlmm-crawler");
    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    // Stop on Ctrl-C, or after 60 seconds.
    let token = CancellationToken::new();
    let stop = token.clone();
    tokio::spawn(async move {
        tokio::select! {
            _ = tokio::signal::ctrl_c() => {}
            _ = tokio::time::sleep(Duration::from_secs(60)) => {}
        }
        stop.cancel();
    });

    let filters: Vec<Box<dyn Filter>> = vec![
        Box::new(DatasourceFilter::new(crawler_id.clone())),
        Box::new(DeduplicationFilter::new(Duration::from_secs(300))),
    ];

    Pipeline::builder()
        .datasource_with_id(datasource, crawler_id)
        .instruction_with_filters(MeteoraDlmmDecoder, SwapLogger, filters)
        .metrics(Arc::new(LogMetrics::new()))
        .datasource_cancellation_token(token)
        .shutdown_strategy(ShutdownStrategy::Immediate)
        .channel_buffer_size(10_000)
        .build()?
        .run()
        .await
}
```

```sh
RUST_LOG=info cargo run
```

The public endpoint rate-limits the crawler, so swaps arrive slowly. After 60 seconds the token fires and the pipeline exports metrics one last time and exits with status 0:

Output:

```text
[2026-11-09T00:39:21Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
swap 21qMkWAt3sGx1cN3E1gfyLqMPmXtd2Npnou5rREVBxnDfdW1wS7NSgyZx3Y4Y9uEfzXdyh83HSybpNBosV7BWaeR slot=467119379 amount_in=4313584534
[2026-11-09T00:40:21Z INFO  carbon_log_metrics] 00:01:00 (+376.772µs) | 34 processed (100%), 34 successful, 0 failed (0%), 0 in queue
[2026-11-09T00:40:21Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:40:21Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

## Defaults

| Setting                                                    | Default                                                                                              | Change with                                                       |
| ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| Channel capacity between datasources and the pipeline loop | `1_000` updates (`DEFAULT_CHANNEL_BUFFER_SIZE`)                                                      | [`channel_buffer_size`](#channel_buffer_size)                     |
| Shutdown strategy                                          | `ShutdownStrategy::ProcessPending`                                                                   | [`shutdown_strategy`](#shutdown_strategy)                         |
| Cancellation token                                         | None. `run` creates a private token, so only Ctrl-C or the end of all datasources stops the pipeline | [`datasource_cancellation_token`](#datasource_cancellation_token) |
| Datasource ID                                              | A random UUID v4 per datasource (`DatasourceId::new_unique()`)                                       | [`datasource_with_id`](#datasource_with_id)                       |
| Filters                                                    | None. Every update reaches every pipe of the matching kind                                           | the `*_with_filters` methods                                      |
| Metrics exporters                                          | None                                                                                                 | [`metrics`](#metrics)                                             |

## `Pipeline::builder`

Returns an empty `PipelineBuilder` with the [defaults](#defaults).

```rust
pub fn builder() -> PipelineBuilder
```

**Details.** Equivalent to `PipelineBuilder::new()` and `PipelineBuilder::default()`. A pipeline with no datasources builds and runs, but `run` returns as soon as it starts because the update channel closes immediately.

**Example.**

```rust
let builder = Pipeline::builder();
```

**See also.** [`build`](#build), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/struct.Pipeline.html#method.builder).

## `PipelineBuilder::new`

Same as [`Pipeline::builder`](#pipelinebuilder).

```rust
pub fn new() -> Self
```

**Details.** `PipelineBuilder` also implements `Default`. All of its fields are public (`datasources`, `account_pipes`, `account_deletion_pipes`, `block_details_pipes`, `instruction_pipes`, `transaction_pipes`, `exporters`, `datasource_cancellation_token`, `shutdown_strategy`, `channel_buffer_size`), but the pipe fields hold internal `*Pipe` types. Use the methods instead.

**Example.**

```rust
let builder = PipelineBuilder::new();
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/struct.PipelineBuilder.html).

## `datasource`

Adds a datasource with a random ID.

```rust
pub fn datasource(self, datasource: impl Datasource + 'static) -> Self
```

**Details.** The ID is `DatasourceId::new_unique()`, a UUID v4 string you cannot read back. Use [`datasource_with_id`](#datasource_with_id) when a filter needs to match it. `run` spawns one Tokio task per datasource. You can add any number of datasources; their updates share one channel and are processed in arrival order.

**Example.**

```rust
Pipeline::builder().datasource(datasource)
```

**See also.** [`datasource_with_id`](#datasource_with_id), [`Datasource`](https://carbon.sevenlabs.org/reference/traits.md#datasource), [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/struct.PipelineBuilder.html#method.datasource).

## `datasource_with_id`

Adds a datasource with an ID you choose.

```rust
pub fn datasource_with_id(self, datasource: impl Datasource + 'static, id: DatasourceId) -> Self
```

**Details.** Create the ID with `DatasourceId::new_named("name")`. Every update from this datasource carries the ID, and filters read it from `FilterContext::datasource_id`. The pipeline does not check that IDs are unique: two datasources with the same ID are indistinguishable to filters.

**Example.**

```rust
let crawler_id = DatasourceId::new_named("dlmm-crawler");
Pipeline::builder().datasource_with_id(datasource, crawler_id.clone())
```

**See also.** [`DatasourceId`](https://carbon.sevenlabs.org/reference/updates.md#datasourceid), [`DatasourceFilter`](https://carbon.sevenlabs.org/reference/filters.md#datasourcefilter), [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md).

## `account`

Adds an account pipe: a decoder and the processor that receives what it decodes.

```rust
pub fn account<T, P>(
    self,
    decoder: impl for<'a> AccountDecoder<'a, AccountType = T> + Send + Sync + 'static,
    processor: P,
) -> Self
where
    T: Send + Sync + 'static,
    P: for<'a> Processor<AccountProcessorInputType<'a, T>> + Send + Sync + 'static,
```

**Details.** For every `Update::Account`, the pipe calls `decoder.decode_account(&account)`. When it returns `Some`, the processor receives an [`AccountProcessorInputType<T>`](https://carbon.sevenlabs.org/reference/processor-inputs.md#accountprocessorinputtype). When it returns `None` (for example, the owner is a different program), the pipe skips the update. Zero-lamport accounts arrive as `Update::AccountDeletion` instead and never reach account pipes; register [`account_deletions`](#account_deletions) for them.

**Example.**

```rust
Pipeline::builder()
    .datasource(GpaDatasource::new(rpc_url, PROGRAM_ID))
    .account(MeteoraDlmmDecoder, PoolLogger)
```

**See also.** [`account_with_filters`](#account_with_filters), [`AccountDecoder`](https://carbon.sevenlabs.org/reference/traits.md#accountdecoder), [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md).

## `account_with_filters`

Adds an account pipe that only runs for updates every filter accepts.

```rust
pub fn account_with_filters<T, P>(
    self,
    decoder: impl for<'a> AccountDecoder<'a, AccountType = T> + Send + Sync + 'static,
    processor: P,
    filters: Vec<Box<dyn Filter + 'static>>,
) -> Self
```

**Details.** Same bounds as [`account`](#account). The pipeline calls `Filter::filter_account` on each filter in order and stops at the first `Reject`, so put stateful filters such as `DeduplicationFilter` last. Filters run before the decoder.

**Example.**

```rust
Pipeline::builder().account_with_filters(
    MeteoraDlmmDecoder,
    PoolLogger,
    vec![Box::new(SlotRangeFilter::from(400_000_000, None))],
)
```

**See also.** [Filters reference](https://carbon.sevenlabs.org/reference/filters.md), [Filters](https://carbon.sevenlabs.org/concepts/filters.md).

## `account_deletions`

Adds a processor for account closures.

```rust
pub fn account_deletions<P>(self, processor: P) -> Self
where
    P: Processor<AccountDeletion> + Send + Sync + 'static,
```

**Details.** The processor receives `&AccountDeletion` for every `Update::AccountDeletion`, with no decoding step. In 2.0.0 the datasources that emit deletions are Yellowstone gRPC, Helius LaserStream, Helius Atlas WebSocket, RPC program subscribe and stream message: any account update with `lamports == 0` becomes a deletion. The deletion carries the account's final state in `account`.

**Example.**

```rust
Pipeline::builder().account_deletions(DeletionLogger)
```

**See also.** [`AccountDeletion`](https://carbon.sevenlabs.org/reference/updates.md#accountdeletion), [`account_deletions_with_filters`](#account_deletions_with_filters).

## `account_deletions_with_filters`

Adds a deletion processor behind filters.

```rust
pub fn account_deletions_with_filters<P>(
    self,
    processor: P,
    filters: Vec<Box<dyn Filter + 'static>>,
) -> Self
where
    P: Processor<AccountDeletion> + Send + Sync + 'static,
```

**Details.** Calls `Filter::filter_account_deletion`. `DeduplicationFilter` does not implement this hook, so it accepts every deletion.

**Example.**

```rust
Pipeline::builder().account_deletions_with_filters(
    DeletionLogger,
    vec![Box::new(DatasourceFilter::new(DatasourceId::new_named("geyser")))],
)
```

**See also.** [Filters reference](https://carbon.sevenlabs.org/reference/filters.md#filter).

## `block_details`

Adds a processor for block-level metadata.

```rust
pub fn block_details<P>(self, processor: P) -> Self
where
    P: Processor<BlockDetails> + Send + Sync + 'static,
```

**Details.** The processor receives `&BlockDetails` for every `Update::BlockDetails`. In 2.0.0 only the RPC block subscribe datasource emits block details; with any other datasource this processor never runs.

**Example.**

```rust
Pipeline::builder().block_details(BlockLogger)
```

**See also.** [`BlockDetails`](https://carbon.sevenlabs.org/reference/updates.md#blockdetails), [`block_details_with_filters`](#block_details_with_filters).

## `block_details_with_filters`

Adds a block-details processor behind filters.

```rust
pub fn block_details_with_filters<P>(
    self,
    processor: P,
    filters: Vec<Box<dyn Filter + 'static>>,
) -> Self
where
    P: Processor<BlockDetails> + Send + Sync + 'static,
```

**Details.** Calls `Filter::filter_block_details`.

**Example.**

```rust
Pipeline::builder().block_details_with_filters(
    BlockLogger,
    vec![Box::new(SlotRangeFilter::between(400_000_000, None, 400_100_000, None))],
)
```

**See also.** [Filters reference](https://carbon.sevenlabs.org/reference/filters.md#slotrangefilter).

## `instruction`

Adds an instruction pipe: a decoder and the processor that receives each instruction it decodes.

```rust
pub fn instruction<T, P>(
    self,
    decoder: impl for<'a> InstructionDecoder<'a, InstructionType = T> + Send + Sync + 'static,
    processor: P,
) -> Self
where
    T: Send + Sync + 'static,
    P: for<'a> Processor<InstructionProcessorInputType<'a, T>> + Send + Sync + 'static,
```

**Details.** For every `Update::Transaction`, the pipeline rebuilds the instruction tree and flattens it depth-first: each top-level instruction, then its CPIs. The pipe tries the decoder on **every** instruction, including inner ones, so a program reached by CPI is processed like a top-level call. When the decoder returns `Some`, the processor receives an [`InstructionProcessorInputType<T>`](https://carbon.sevenlabs.org/reference/processor-inputs.md#instructionprocessorinputtype), including the instruction's own children in `nested_instructions`.

**Example.**

```rust
Pipeline::builder()
    .datasource(datasource)
    .instruction(MeteoraDlmmDecoder, SwapLogger)
```

**See also.** [`instruction_with_filters`](#instruction_with_filters), [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md), [`InstructionDecoder`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecoder).

## `instruction_with_filters`

Adds an instruction pipe behind filters.

```rust
pub fn instruction_with_filters<T, P>(
    self,
    decoder: impl for<'a> InstructionDecoder<'a, InstructionType = T> + Send + Sync + 'static,
    processor: P,
    filters: Vec<Box<dyn Filter + 'static>>,
) -> Self
```

**Details.** Same bounds as [`instruction`](#instruction). Calls `Filter::filter_instruction` once per instruction, before decoding, so filters also see instructions of other programs.

**Example.**

```rust
let filters: Vec<Box<dyn Filter>> = vec![
    Box::new(DatasourceFilter::new(crawler_id.clone())),
    Box::new(DeduplicationFilter::new(Duration::from_secs(300))),
];
Pipeline::builder().instruction_with_filters(MeteoraDlmmDecoder, SwapLogger, filters)
```

**See also.** [`DeduplicationFilter`](https://carbon.sevenlabs.org/reference/filters.md#deduplicationfilter).

## `transaction`

Adds a transaction pipe: one processor call per transaction with every instruction the collection `T` recognizes.

```rust
pub fn transaction<T, P>(self, processor: P) -> Self
where
    T: InstructionDecoderCollection + 'static,
    P: for<'a> Processor<TransactionProcessorInputType<'a, T>> + Send + Sync + 'static,
```

**Details.** There is no decoder argument: `T::parse_instruction` runs on each instruction of the flattened list (top-level and inner), and the ones it parses are passed in order in `instructions`. The processor runs for **every** transaction, even when `instructions` is empty, so return early when it is. You usually have to name `T` with a turbofish or let the processor's type infer it.

**Example.**

```rust
Pipeline::builder()
    .datasource(datasource)
    .transaction::<AllInstructions, _>(TxLogger)
```

**See also.** [`transaction_with_filters`](#transaction_with_filters), [`InstructionDecoderCollection`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecodercollection), [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md).

## `transaction_with_filters`

Adds a transaction pipe behind filters.

```rust
pub fn transaction_with_filters<T, P>(
    self,
    processor: P,
    filters: Vec<Box<dyn Filter + 'static>>,
) -> Self
where
    T: InstructionDecoderCollection + 'static,
    P: for<'a> Processor<TransactionProcessorInputType<'a, T>> + Send + Sync + 'static,
```

**Details.** Calls `Filter::filter_transaction`. `DeduplicationFilter` does not implement this hook, so it does not deduplicate transaction pipes.

**Example.**

```rust
Pipeline::builder().transaction_with_filters::<AllInstructions, _>(
    TxLogger,
    vec![Box::new(SlotRangeFilter::from(400_000_000, Some(0)))],
)
```

**See also.** [Filters reference](https://carbon.sevenlabs.org/reference/filters.md).

## `metrics`

Adds a metrics exporter.

```rust
pub fn metrics(self, exporter: Arc<dyn MetricsExporter>) -> Self
```

**Details.** You can add several exporters. `run` calls each exporter's `initialize` once at startup, then `export` and `shutdown` once when the pipeline stops. It does not call `export` while running: exporters that report periodically, such as `LogMetrics`, start their own timer in `initialize`. An error from any of these calls makes `run` return that error.

**Example.**

```rust
Pipeline::builder().metrics(Arc::new(LogMetrics::new()))
```

**See also.** [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md), [`MetricsExporter`](https://carbon.sevenlabs.org/reference/traits.md#metricsexporter).

## `datasource_cancellation_token`

Gives the pipeline a `CancellationToken` you control.

```rust
pub fn datasource_cancellation_token(self, cancellation_token: CancellationToken) -> Self
```

**Details.** `run` passes a clone of the token to every datasource's `consume`. When you cancel it, the pipeline finishes the update it is processing, exports metrics, shuts down exporters and returns `Ok(())`. It does **not** drain updates still in the channel, whatever the [`shutdown_strategy`](#shutdown_strategy). Without this method, `run` creates a private token that only Ctrl-C cancels. Use this method in production: see [Ctrl-C is ignored while processing](#ctrl-c-is-ignored-while-an-update-is-processing).

**Example.**

```rust
let token = CancellationToken::new();
let stop = token.clone();
tokio::spawn(async move {
    tokio::signal::ctrl_c().await.ok();
    stop.cancel();
});
Pipeline::builder().datasource_cancellation_token(token)
```

**See also.** [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md), [`run`](#run).

## `shutdown_strategy`

Sets what happens when the pipeline receives Ctrl-C.

```rust
pub fn shutdown_strategy(self, shutdown_strategy: ShutdownStrategy) -> Self
```

**Details.** Applies only to Ctrl-C handled by `run` itself. Cancelling your own token always behaves like `Immediate`. See [`ShutdownStrategy`](#shutdownstrategy) for the exact behaviour of each variant.

**Example.**

```rust
Pipeline::builder().shutdown_strategy(ShutdownStrategy::Immediate)
```

**See also.** [`ShutdownStrategy`](#shutdownstrategy), [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).

## `channel_buffer_size`

Sets the capacity of the channel between datasources and the pipeline loop.

```rust
pub fn channel_buffer_size(self, size: usize) -> Self
```

**Details.** Default `1_000` updates. All datasources share the one channel. When it is full, datasources that use `send().await` wait (backpressure), and datasources that use `try_send` drop the update and log an error. Most streaming datasources in 2.0.0 use `try_send`, and so does the RPC transaction crawler unless `ConnectionConfig::blocking_send` is `true`. Raise the size when processors are slower than the source. A size of `0` panics in `run` (Tokio channels need a capacity of at least 1). The `carbon_updates_queued` gauge shows the queue length.

**Example.**

```rust
Pipeline::builder().channel_buffer_size(10_000)
```

**See also.** [`DEFAULT_CHANNEL_BUFFER_SIZE`](#default_channel_buffer_size), [Production](https://carbon.sevenlabs.org/guides/production.md).

## `build`

Finishes the builder and returns the `Pipeline`.

```rust
pub fn build(self) -> CarbonResult<Pipeline>
```

**Details.** Registers the [pipeline metrics](https://carbon.sevenlabs.org/reference/metrics.md) in the global `MetricsRegistry` (and the Postgres metrics when carbon-core's `postgres` feature is on), then moves every field into the `Pipeline`. It never returns `Err` in 2.0.0, and it does not validate anything: no check for zero datasources, duplicate IDs, or datasources whose `update_types()` don't match the registered pipes. Registration is not deduplicated, so building two pipelines in one process lists each pipeline metric twice in exported snapshots.

**Example.**

```rust
let mut pipeline = Pipeline::builder()
    .datasource(datasource)
    .instruction(MeteoraDlmmDecoder, SwapLogger)
    .build()?;
```

**See also.** [`run`](#run), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/struct.PipelineBuilder.html#method.build).

## `run`

Runs the pipeline until it stops.

```rust
pub async fn run(&mut self) -> CarbonResult<()>
```

**Details.** In order, `run`:

1. Logs `starting pipeline. num_datasources: …` at `info` level.
2. Calls `initialize` on each exporter.
3. Creates the update channel (capacity `channel_buffer_size`) and spawns one Tokio task per datasource, each calling `consume(id, sender, token)`.
4. Processes updates one at a time, in arrival order. For each update, it runs every matching pipe in registration order: account pipes for `Account`, deletion pipes for `AccountDeletion`, block pipes for `BlockDetails`. For `Transaction`, it runs all instruction pipes over the flattened instruction list, then all transaction pipes.
5. Stops when one of the [stop conditions](#stop-conditions) happens, then calls `export` with a final snapshot and `shutdown` on each exporter.

Processing is sequential: a slow processor delays every other pipe and fills the channel. Use `tokio::spawn` inside a processor to move slow work off the loop.

**Errors.** When a processor or decoder step returns `Err`, the pipeline logs `error processing update (…): …` at `error` level, increments `carbon_updates_failed_total` and moves on to the next update. The remaining pipes for that same update are skipped. When a datasource's `consume` returns `Err`, the pipeline logs `error consuming datasource: …` and keeps running with the other datasources. `run` itself returns `Err` only when an exporter's `initialize`, `export` or `shutdown` fails. A pipeline whose only datasource failed still returns `Ok(())`, so your process exits with status 0. See [Errors](https://carbon.sevenlabs.org/reference/errors.md).

**Example.**

```rust
Pipeline::builder()
    .datasource(datasource)
    .instruction(MeteoraDlmmDecoder, SwapLogger)
    .build()?
    .run()
    .await?;
```

**See also.** [Architecture](https://carbon.sevenlabs.org/concepts/architecture.md), [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md).

### Stop conditions

| Event                                                                           | What `run` does                                                                                                                                                     | Updates still in the channel |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- |
| Every datasource task returns (finite sources such as GPA or a bounded crawler) | Processes everything left in the channel, logs `update_receiver closed, shutting down.`, exports, returns `Ok(())`                                                  | Processed                    |
| Your cancellation token is cancelled                                            | Finishes the current update, exports, returns `Ok(())`                                                                                                              | Dropped                      |
| Ctrl-C with `ShutdownStrategy::Immediate`                                       | Cancels the token, logs `shutting down the pipeline immediately.`, exports, returns `Ok(())`                                                                        | Dropped                      |
| Ctrl-C with `ShutdownStrategy::ProcessPending`                                  | Cancels the token, logs `shutting down the pipeline after processing pending updates.`, then stops at the next loop iteration where the cancelled token is selected | Mostly dropped (see below)   |

### Ctrl-C is ignored while an update is processing

`run` listens for Ctrl-C with a fresh `tokio::signal::ctrl_c()` future on each loop iteration, and it installs Tokio's signal handler, which replaces the default "terminate on SIGINT" behaviour. A signal that arrives while a processor is running is lost. So a pipeline that is always busy can ignore Ctrl-C entirely, until you send SIGTERM or SIGKILL. Listen for Ctrl-C yourself and cancel a token, as in the [example](#example), so the signal is never missed.

## `ShutdownStrategy`

What `run` does when it receives Ctrl-C.

```rust
#[derive(Default, PartialEq, Debug)]
pub enum ShutdownStrategy {
    Immediate,
    #[default]
    ProcessPending,
}
```

**Details.**

| Variant                    | Behaviour in 2.0.0                                                                                                                                                                                                                                                            |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Immediate`                | Cancels the datasources' token, exports metrics, shuts down exporters and returns. Updates already in the channel are dropped.                                                                                                                                                |
| `ProcessPending` (default) | Cancels the token and keeps looping. On the next iteration the cancelled token and the channel are both ready, and `tokio::select!` picks between them at random, so the pipeline usually exits after zero or a few more updates. It does **not** reliably drain the channel. |

Neither variant applies when you cancel your own token: that path always stops without draining. To process everything that was fetched, let the datasources finish on their own (the channel then drains completely), or stop your datasource from producing and wait for the `carbon_updates_queued` gauge to reach 0 before you cancel.

**Example.**

```rust
Pipeline::builder().shutdown_strategy(ShutdownStrategy::Immediate)
```

**See also.** [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/enum.ShutdownStrategy.html).

## `DEFAULT_CHANNEL_BUFFER_SIZE`

The default channel capacity.

```rust
pub const DEFAULT_CHANNEL_BUFFER_SIZE: usize = 1_000;
```

**Details.** Used by `PipelineBuilder::default()`. Override it with [`channel_buffer_size`](#channel_buffer_size).

**Example.**

```rust
let size = DEFAULT_CHANNEL_BUFFER_SIZE * 10;
Pipeline::builder().channel_buffer_size(size)
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/constant.DEFAULT_CHANNEL_BUFFER_SIZE.html).

## `Pipeline` fields

The built pipeline. Its fields are public.

```rust
pub struct Pipeline {
    pub datasources: Vec<(DatasourceId, Arc<dyn Datasource>)>,
    pub account_pipes: Vec<Box<dyn AccountPipes>>,
    pub account_deletion_pipes: Vec<Box<dyn AccountDeletionPipes>>,
    pub block_details_pipes: Vec<Box<dyn BlockDetailsPipes>>,
    pub instruction_pipes: Vec<Box<dyn for<'a> InstructionPipes<'a>>>,
    pub transaction_pipes: Vec<Box<dyn for<'a> TransactionPipes<'a>>>,
    pub exporters: Vec<Arc<dyn MetricsExporter>>,
    pub datasource_cancellation_token: Option<CancellationToken>,
    pub shutdown_strategy: ShutdownStrategy,
    pub channel_buffer_size: usize,
}
```

**Details.** The fields are the builder's values, unchanged. You can read them (for example `pipeline.datasources.len()`) or adjust `channel_buffer_size` and `shutdown_strategy` before calling `run`. The `*Pipes` traits are internal plumbing implemented by `AccountPipe`, `InstructionPipe` and the others; you don't need to implement them.

**Example.**

```rust
let mut pipeline = Pipeline::builder().datasource(datasource).build()?;
println!("{} datasource(s)", pipeline.datasources.len());
pipeline.run().await?;
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/pipeline/struct.Pipeline.html).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/pipeline-builder · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Updates (Carbon 2.0)

> The Update enum and every type a datasource emits in Carbon 2.0.0, with all fields.

Datasources send `(Update, DatasourceId)` pairs into the pipeline. This page lists every type in `carbon_core::datasource` that makes up an update, field by field. You need it when you write a [custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md) or a processor for account deletions or block details. For how updates move through the pipeline, see [Updates](https://carbon.sevenlabs.org/concepts/updates.md).

Source: [`crates/core/src/datasource.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/datasource.rs).

## `Update`

The payload a datasource sends into the pipeline.

```rust
#[derive(Debug, Clone)]
pub enum Update {
    Account(AccountUpdate),
    Transaction(Box<TransactionUpdate>),
    AccountDeletion(AccountDeletion),
    BlockDetails(BlockDetails),
}
```

**Details.**

| Variant           | Routed to                                                                                                            | Emitted in 2.0.0 by                                                                                                                                        |
| ----------------- | -------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Account`         | account pipes ([`account`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account))                      | RPC GPA, Helius GPA V2, validator snapshot, and every streaming source for funded accounts                                                                 |
| `Transaction`     | instruction pipes, then transaction pipes                                                                            | RPC transaction crawler, RPC block crawler, RPC block subscribe, Yellowstone gRPC, Helius LaserStream, Helius Atlas WebSocket, Helius GTFA, stream message |
| `AccountDeletion` | deletion pipes ([`account_deletions`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account_deletions)) | Yellowstone gRPC, Helius LaserStream, Helius Atlas WebSocket, RPC program subscribe, stream message                                                        |
| `BlockDetails`    | block-details pipes ([`block_details`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#block_details))    | RPC block subscribe only                                                                                                                                   |

`Transaction` is boxed because `TransactionUpdate` is large. `update_type()` returns the matching [`UpdateType`](#updatetype).

**Example.**

```rust
match &update {
    Update::Account(account) => println!("account {}", account.pubkey),
    Update::Transaction(tx) => println!("tx {}", tx.signature),
    Update::AccountDeletion(deletion) => println!("closed {}", deletion.pubkey),
    Update::BlockDetails(block) => println!("block {}", block.slot),
}
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/enum.Update.html), [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md).

## `Update::update_type`

Returns the variant as an `UpdateType`.

```rust
pub fn update_type(&self) -> UpdateType
```

**Details.** `Account` → `UpdateType::AccountUpdate`, `Transaction` → `UpdateType::Transaction`, `AccountDeletion` → `UpdateType::AccountDeletion`, `BlockDetails` → `UpdateType::BlockDetails`.

**Example.**

```rust
assert_eq!(update.update_type(), UpdateType::AccountUpdate);
```

**See also.** [`UpdateType`](#updatetype).

## `UpdateType`

The kinds of update a datasource declares it can emit.

```rust
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum UpdateType {
    AccountUpdate,
    Transaction,
    AccountDeletion,
    BlockDetails,
}
```

**Details.** A datasource returns these from `Datasource::update_types()`. In 2.0.0 the value is informational: the pipeline never calls `update_types()` and never checks it against the pipes you registered or the updates the datasource sends. Several built-in datasources under-report (for example, RPC block subscribe declares only `Transaction` but also emits `BlockDetails`, and Yellowstone gRPC doesn't list `AccountDeletion`). Don't rely on it to decide what a datasource emits; use the table under [`Update`](#update).

**Example.**

```rust
fn update_types(&self) -> Vec<UpdateType> {
    vec![UpdateType::AccountUpdate, UpdateType::AccountDeletion]
}
```

**See also.** [`Datasource`](https://carbon.sevenlabs.org/reference/traits.md#datasource), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/enum.UpdateType.html).

## `AccountUpdate`

The state of one account at a slot.

```rust
#[derive(Debug, Clone)]
pub struct AccountUpdate {
    pub pubkey: Pubkey,
    pub account: Account,
    pub slot: u64,
    pub transaction_signature: Option<Signature>,
}
```

**Details.**

| Field                   | Type                                  | Meaning                                                                                                                                                                                                                             |
| ----------------------- | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pubkey`                | `solana_pubkey::Pubkey`               | Address of the account.                                                                                                                                                                                                             |
| `account`               | `solana_account::Account`             | Full account: `lamports`, `data`, `owner`, `executable`, `rent_epoch`.                                                                                                                                                              |
| `slot`                  | `u64`                                 | Slot the state was observed at. For GPA and snapshot sources, the slot of the RPC context or the snapshot.                                                                                                                          |
| `transaction_signature` | `Option<solana_signature::Signature>` | Transaction that wrote this state. Yellowstone and LaserStream fill it from the update's `txn_signature` when present; every other built-in datasource sends `None`. `DeduplicationFilter` only deduplicates accounts that have it. |

The pipeline turns `AccountUpdate` into an [`AccountMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md#accountmetadata) plus the raw `Account` before calling your account decoder.

**Example.**

```rust
let update = AccountUpdate {
    pubkey,
    account: Account { lamports: 1_000_000, data, owner, executable: false, rent_epoch: 0 },
    slot,
    transaction_signature: None,
}
.into_update();
```

**See also.** [`into_update`](#accountupdateinto_update), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.AccountUpdate.html).

## `AccountUpdate::into_update`

Wraps the account update in an `Update`, turning zero-lamport accounts into deletions.

```rust
pub fn into_update(self) -> Update
```

**Details.** Returns `Update::AccountDeletion` with the same four fields when `account.lamports == 0`, and `Update::Account` otherwise. Custom datasources should always build updates this way so closures reach deletion processors, as the built-in streaming datasources do. The GPA and snapshot datasources build `Update::Account` directly, which is safe because they only return live accounts.

**Example.**

```rust
let update = AccountUpdate { pubkey, account, slot, transaction_signature: None }.into_update();
sender
    .send((update, id.clone()))
    .await
    .map_err(|e| Error::FailedToConsumeDatasource(e.to_string()))?;
```

**See also.** [`AccountDeletion`](#accountdeletion), [Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md).

## `TransactionUpdate`

A confirmed transaction with its status metadata.

```rust
#[derive(Debug, Clone)]
pub struct TransactionUpdate {
    pub signature: Signature,
    pub transaction: VersionedTransaction,
    pub meta: TransactionStatusMeta,
    pub is_vote: bool,
    pub slot: u64,
    pub index: Option<u64>,
    pub block_time: Option<i64>,
    pub block_hash: Option<Hash>,
}
```

**Details.**

| Field         | Type                                                  | Meaning                                                                                                                                                                                                                                                                                            |
| ------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `signature`   | `solana_signature::Signature`                         | First signature of the transaction.                                                                                                                                                                                                                                                                |
| `transaction` | `solana_transaction::versioned::VersionedTransaction` | Signatures and message (legacy, V0 or V1). The pipeline reads the fee payer and instructions from `transaction.message`.                                                                                                                                                                           |
| `meta`        | `solana_transaction_status::TransactionStatusMeta`    | Execution result: `status`, `fee`, balances, `inner_instructions`, `log_messages`, token balances, `rewards`, `loaded_addresses`, `return_data`, `compute_units_consumed`, `cost_units`. Inner instructions and loaded addresses must be present for CPIs and V0 lookup-table accounts to resolve. |
| `is_vote`     | `bool`                                                | Vote transaction flag. Only Yellowstone and LaserStream set it from the source; the RPC and Helius GTFA datasources always set `false`.                                                                                                                                                            |
| `slot`        | `u64`                                                 | Slot the transaction landed in.                                                                                                                                                                                                                                                                    |
| `index`       | `Option<u64>`                                         | Position of the transaction in its block. `Some` for RPC block crawler, RPC block subscribe, Yellowstone and LaserStream; GTFA passes the provider's value; `None` for the RPC transaction crawler and Helius Atlas. `SlotRangeFilter` uses it for transaction-index bounds.                       |
| `block_time`  | `Option<i64>`                                         | Unix timestamp in seconds, when the source provides it.                                                                                                                                                                                                                                            |
| `block_hash`  | `Option<solana_hash::Hash>`                           | Hash of the block. Only the RPC block crawler and RPC block subscribe set it; every other built-in datasource sends `None`.                                                                                                                                                                        |

The pipeline converts the update into a [`TransactionMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md#transactionmetadata). If the message has no account keys, the conversion fails with [`MissingFeePayer`](https://carbon.sevenlabs.org/reference/errors.md#missingfeepayer).

**Example.**

```rust
let update = Update::Transaction(Box::new(TransactionUpdate {
    signature,
    transaction,
    meta,
    is_vote: false,
    slot,
    index: Some(tx_index),
    block_time: Some(block_time),
    block_hash: None,
}));
```

**See also.** [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.TransactionUpdate.html).

## `AccountDeletion`

An account that was closed, with its final state.

```rust
#[derive(Debug, Clone)]
pub struct AccountDeletion {
    pub pubkey: Pubkey,
    pub account: Account,
    pub slot: u64,
    pub transaction_signature: Option<Signature>,
}
```

**Details.**

| Field                   | Type                                  | Meaning                                                                                                                                                                                                                                    |
| ----------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `pubkey`                | `solana_pubkey::Pubkey`               | Address of the closed account.                                                                                                                                                                                                             |
| `account`               | `solana_account::Account`             | Final state as the source reported it, unchanged: `lamports` is `0`. Programs often clear the data and reassign the account to the System Program when closing it, so `owner` and `data` may no longer identify the program. New in 2.0.0. |
| `slot`                  | `u64`                                 | Slot of the closure.                                                                                                                                                                                                                       |
| `transaction_signature` | `Option<solana_signature::Signature>` | Closing transaction, when the source reports it.                                                                                                                                                                                           |

Deletion processors receive `&AccountDeletion` directly, with no decoder. To know which program the account belonged to, check `account.owner` or keep your own index of addresses.

**Example.**

```rust
impl Processor<AccountDeletion> for DeletionLogger {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        println!("closed {} at slot {}", deletion.pubkey, deletion.slot);
        Ok(())
    }
}
```

**See also.** [`account_deletions`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account_deletions), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.AccountDeletion.html).

## `BlockDetails`

Block-level metadata.

```rust
#[derive(Debug, Clone)]
pub struct BlockDetails {
    pub slot: u64,
    pub block_hash: Option<Hash>,
    pub previous_block_hash: Option<Hash>,
    pub rewards: Option<Rewards>,
    pub num_reward_partitions: Option<u64>,
    pub block_time: Option<i64>,
    pub block_height: Option<u64>,
}
```

**Details.**

| Field                   | Type                                         | Meaning                                                             |
| ----------------------- | -------------------------------------------- | ------------------------------------------------------------------- |
| `slot`                  | `u64`                                        | Slot of the block.                                                  |
| `block_hash`            | `Option<solana_hash::Hash>`                  | Hash of this block.                                                 |
| `previous_block_hash`   | `Option<solana_hash::Hash>`                  | Hash of the parent block.                                           |
| `rewards`               | `Option<solana_transaction_status::Rewards>` | Block rewards (`Vec<Reward>`), when the subscription requests them. |
| `num_reward_partitions` | `Option<u64>`                                | Number of partitions for partitioned epoch rewards, when reported.  |
| `block_time`            | `Option<i64>`                                | Unix timestamp in seconds.                                          |
| `block_height`          | `Option<u64>`                                | Block height.                                                       |

In 2.0.0 only `RpcBlockSubscribe` emits block details, one per block, before that block's transactions. It copies every field from the RPC `UiConfirmedBlock`.

**Example.**

```rust
impl Processor<BlockDetails> for BlockLogger {
    async fn process(&mut self, block: &BlockDetails) -> CarbonResult<()> {
        println!("block {} height={:?}", block.slot, block.block_height);
        Ok(())
    }
}
```

**See also.** [`block_details`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#block_details), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.BlockDetails.html).

## `DatasourceId`

Identifies the datasource an update came from.

```rust
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct DatasourceId(String);

impl DatasourceId {
    pub fn new_unique() -> Self;
    pub fn new_named(name: &str) -> Self;
}
```

**Details.**

| Constructor       | Value                                                             |
| ----------------- | ----------------------------------------------------------------- |
| `new_unique()`    | A random UUID v4 string. `PipelineBuilder::datasource` uses this. |
| `new_named(name)` | The string you pass. Two IDs with the same name are equal.        |

The inner `String` is private and there is no accessor or `Display` impl; compare IDs with `==` or print them with `{:?}` (which shows `DatasourceId("name")`). Filters receive the ID in [`FilterContext`](https://carbon.sevenlabs.org/reference/filters.md#filtercontext), and `Datasource::consume` receives it so the datasource can tag what it sends.

**Example.**

```rust
let geyser = DatasourceId::new_named("geyser");
assert_eq!(geyser, DatasourceId::new_named("geyser"));
println!("{geyser:?}");
```

**See also.** [`datasource_with_id`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#datasource_with_id), [`DatasourceFilter`](https://carbon.sevenlabs.org/reference/filters.md#datasourcefilter), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.DatasourceId.html).

## `DatasourceDisconnection`

Describes a gap in a streaming datasource after it reconnects.

```rust
#[derive(Debug, Clone)]
pub struct DatasourceDisconnection {
    pub source: String,
    pub disconnect_time: DateTime<Utc>,
    pub last_slot_before_disconnect: Slot,
    pub first_slot_after_reconnect: Slot,
    pub missed_slots: u64,
}
```

**Details.** This is not an `Update` variant and never goes through the pipeline. Two datasources in 2.0.0 can report gaps: `RpcBlockSubscribe` (through `RpcBlockSubscribe::with_disconnect_notifier`) and `YellowstoneGrpcGeyserClient` (through the `disconnect_notifier` argument of `new`). Each takes a `tokio::sync::mpsc::Sender<DatasourceDisconnection>` and sends one event with the first message after a reconnect. You read the receiver yourself, for example to backfill the missed range with a crawler. Yellowstone sends with `try_send`, so the event is dropped if your channel is full.

| Field                         | Type                         | Meaning                                                                                                                                                                                    |
| ----------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `source`                      | `String`                     | `"rpc-websocket"` for RPC block subscribe, `"yellowstone-grpc"` for Yellowstone.                                                                                                           |
| `disconnect_time`             | `chrono::DateTime<Utc>`      | When the stream was detected as closed or timed out.                                                                                                                                       |
| `last_slot_before_disconnect` | `u64` (`solana_clock::Slot`) | Last slot processed before the disconnect.                                                                                                                                                 |
| `first_slot_after_reconnect`  | `u64`                        | Slot of the first message received after reconnecting.                                                                                                                                     |
| `missed_slots`                | `u64`                        | `first_slot_after_reconnect - last_slot_before_disconnect` (saturating). The slots strictly between the two are the ones you may have missed, so the true gap is one less than this value. |

**Example.**

```rust
let (tx, mut rx) = tokio::sync::mpsc::channel::<DatasourceDisconnection>(16);
tokio::spawn(async move {
    while let Some(gap) = rx.recv().await {
        eprintln!(
            "{} missed {} slots ({}..{})",
            gap.source, gap.missed_slots, gap.last_slot_before_disconnect, gap.first_slot_after_reconnect
        );
    }
});
let datasource = RpcBlockSubscribe::with_disconnect_notifier(ws_url, filters, tx);
```

**See also.** [Production](https://carbon.sevenlabs.org/guides/production.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/struct.DatasourceDisconnection.html).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/updates · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Processor inputs (Carbon 2.0)

> Every type a Carbon 2.0.0 processor receives, field by field, for account, instruction, transaction, deletion and block-details pipes.

A processor implements `Processor<T>` for the input type of the pipe it is registered on. This page lists each input type and every field in it, so you know what data your `process` method can read. For how to write a processor, see [Processors](https://carbon.sevenlabs.org/concepts/processors.md).

| Builder method                                                                                      | Processor implements                              | Defined in                                                                                              |
| --------------------------------------------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [`account`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account)                     | `Processor<AccountProcessorInputType<'_, T>>`     | `carbon_core::account`                                                                                  |
| [`instruction`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#instruction)             | `Processor<InstructionProcessorInputType<'_, T>>` | `carbon_core::instruction`                                                                              |
| [`transaction`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#transaction)             | `Processor<TransactionProcessorInputType<'_, T>>` | `carbon_core::transaction`                                                                              |
| [`account_deletions`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account_deletions) | `Processor<AccountDeletion>`                      | `carbon_core::datasource` ([fields](https://carbon.sevenlabs.org/reference/updates.md#accountdeletion)) |
| [`block_details`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#block_details)         | `Processor<BlockDetails>`                         | `carbon_core::datasource` ([fields](https://carbon.sevenlabs.org/reference/updates.md#blockdetails))    |

Every input is passed by reference and borrows data the pipeline owns for the duration of one `process` call. Clone what you need to keep.

## Example

This program prints the instruction input for each Meteora DLMM `Swap2` the transaction crawler finds on the public RPC endpoint.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = "2.0.0"
carbon-rpc-transaction-crawler-datasource = "2.0.0"
tokio = { version = "1", features = ["full"] }
tokio-util = "0.7"
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::time::Duration,
    tokio_util::sync::CancellationToken,
};

struct InputPrinter;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for InputPrinter {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        let MeteoraDlmmInstruction::Swap2 { data, accounts, .. } = input.decoded_instruction else {
            return Ok(());
        };
        let meta = input.metadata;
        let tx = &meta.transaction_metadata;
        println!("signature:      {}", tx.signature);
        println!("slot:           {} (block_time {:?})", tx.slot, tx.block_time);
        println!("fee payer:      {}", tx.fee_payer);
        println!("stack height:   {}", meta.stack_height);
        println!("index:          {}", meta.index);
        println!("absolute path:  {:?}", meta.absolute_path);
        println!("inner ixs:      {}", input.nested_instructions.len());
        println!("raw accounts:   {}", input.raw_instruction.accounts.len());
        println!("lb_pair:        {}", accounts.lb_pair);
        println!("amount_in:      {}", data.amount_in);
        println!();
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    let token = CancellationToken::new();
    let stop = token.clone();
    tokio::spawn(async move {
        tokio::time::sleep(Duration::from_secs(90)).await;
        stop.cancel();
    });

    Pipeline::builder()
        .datasource(datasource)
        .instruction(MeteoraDlmmDecoder, InputPrinter)
        .datasource_cancellation_token(token)
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

The first swap is a top-level instruction; the second is a CPI from another program, so its `stack_height` is 2 and its `index` is the index of the top-level instruction that called it:

Output:

```text
signature:      43eGCzcyUFYtJsuoWwmFsirBBu51tm6BRs7WiCWYRfR7txoz1DATD5iSQNFfS1LXuGJvYG2AXTA3t38N1DG9eYJp
slot:           467120934 (block_time Some(1794185094))
fee payer:      6TWMHcW3gGXRnzRoe6vTgENjJhCbTFApcA5yLEJJaBDh
stack height:   1
index:          2
absolute path:  [2]
inner ixs:      4
raw accounts:   17
lb_pair:        GeUkx21Vc6yg63YZ1BdXY95ZATm9LeBYCgSN1uJ3o18S
amount_in:      404287484

signature:      2W5fitdRrbLo2pNF4ChwDPPcmRNC3EfR1u7viGGyPXdnHZM8HBu2dFnJbVxktupvJKbXcxheZaadphndEMkCfNTi
slot:           467120933 (block_time Some(1794185094))
fee payer:      AgmLJBMDCqWynYnQiPCuj9ewsNNsBJXyzoUhD9LJzN51
stack height:   2
index:          1
absolute path:  [1, 1]
inner ixs:      4
raw accounts:   19
lb_pair:        CcG3fyDZn6uXTV5p2TV9cceCA2YhE3bUMLeXGn6vZSNr
amount_in:      680804450
```

## `AccountProcessorInputType`

What an account processor receives for each account its decoder accepts.

```rust
#[derive(Debug)]
pub struct AccountProcessorInputType<'a, T> {
    pub metadata: &'a AccountMetadata,
    pub decoded_account: &'a DecodedAccount<T>,
    pub raw_account: &'a solana_account::Account,
}
```

**Details.**

| Field             | Type                       | Meaning                                                                               |
| ----------------- | -------------------------- | ------------------------------------------------------------------------------------- |
| `metadata`        | `&AccountMetadata`         | Slot, address and writing transaction. See [`AccountMetadata`](#accountmetadata).     |
| `decoded_account` | `&DecodedAccount<T>`       | Decoder output. `T` is the decoder's `AccountType`, for example `MeteoraDlmmAccount`. |
| `raw_account`     | `&solana_account::Account` | The account exactly as the datasource sent it, including the undecoded `data` bytes.  |

**Example.**

```rust
impl Processor<AccountProcessorInputType<'_, MeteoraDlmmAccount>> for PoolLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, MeteoraDlmmAccount>,
    ) -> CarbonResult<()> {
        if let MeteoraDlmmAccount::LbPair(pool) = &input.decoded_account.data {
            println!(
                "{} active_id={} ({} bytes, slot {})",
                input.metadata.pubkey,
                pool.active_id,
                input.raw_account.data.len(),
                input.metadata.slot
            );
        }
        Ok(())
    }
}
```

**See also.** [`account`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/account/struct.AccountProcessorInputType.html).

## `AccountMetadata`

Where and when an account update was observed.

```rust
#[derive(Debug, Clone)]
pub struct AccountMetadata {
    pub slot: u64,
    pub pubkey: Pubkey,
    pub transaction_signature: Option<Signature>,
}
```

**Details.** Copied from the [`AccountUpdate`](https://carbon.sevenlabs.org/reference/updates.md#accountupdate).

| Field                   | Type                                  | Meaning                                                                                                         |
| ----------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `slot`                  | `u64`                                 | Slot of the update.                                                                                             |
| `pubkey`                | `solana_pubkey::Pubkey`               | Address of the account.                                                                                         |
| `transaction_signature` | `Option<solana_signature::Signature>` | Transaction that wrote this state. Only Yellowstone gRPC and Helius LaserStream set it; `None` everywhere else. |

**Example.**

```rust
let key = (input.metadata.pubkey, input.metadata.slot);
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/account/struct.AccountMetadata.html).

## `DecodedAccount`

An account decoder's output: the typed data plus the account header.

```rust
#[derive(Debug, Clone)]
pub struct DecodedAccount<T> {
    pub lamports: u64,
    pub data: T,
    pub owner: Pubkey,
    pub executable: bool,
    pub rent_epoch: u64,
}
```

**Details.**

| Field        | Type                    | Meaning                                                                                                                                             |
| ------------ | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `lamports`   | `u64`                   | Balance. Always above 0 here: zero-lamport accounts arrive as deletions.                                                                            |
| `data`       | `T`                     | The decoded account. Generated decoders use an enum with one boxed variant per account type, for example `MeteoraDlmmAccount::LbPair(Box<LbPair>)`. |
| `owner`      | `solana_pubkey::Pubkey` | Owning program.                                                                                                                                     |
| `executable` | `bool`                  | Whether the account is a program.                                                                                                                   |
| `rent_epoch` | `u64`                   | Rent epoch from the account.                                                                                                                        |

Generated decoders copy the four header fields from the raw account. A hand-written [`AccountDecoder`](https://carbon.sevenlabs.org/reference/traits.md#accountdecoder) builds this struct itself.

**Example.**

```rust
let decoded = &input.decoded_account;
println!("owner={} lamports={}", decoded.owner, decoded.lamports);
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/account/struct.DecodedAccount.html).

## `InstructionProcessorInputType`

What an instruction processor receives for each instruction its decoder accepts.

```rust
#[derive(Debug)]
pub struct InstructionProcessorInputType<'a, T> {
    pub metadata: &'a InstructionMetadata,
    pub decoded_instruction: &'a T,
    pub nested_instructions: &'a NestedInstructions,
    pub raw_instruction: &'a solana_instruction::Instruction,
}
```

**Details.**

| Field                 | Type                               | Meaning                                                                                                                                                                                                               |
| --------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `metadata`            | `&InstructionMetadata`             | Position in the transaction and the shared transaction data. See [`InstructionMetadata`](#instructionmetadata).                                                                                                       |
| `decoded_instruction` | `&T`                               | Decoder output. For generated decoders, an enum with struct variants: `MeteoraDlmmInstruction::Swap { program_id, data, accounts }`.                                                                                  |
| `nested_instructions` | `&NestedInstructions`              | This instruction's own CPIs (its subtree), not the whole transaction. Empty when it made no CPIs.                                                                                                                     |
| `raw_instruction`     | `&solana_instruction::Instruction` | The instruction with `program_id`, resolved `accounts` (`AccountMeta` with `is_signer` and `is_writable`) and raw `data`. Lookup-table accounts of V0 transactions are already resolved from `meta.loaded_addresses`. |

The pipe runs once per matching instruction, top-level or inner. If a program calls your program through CPI, you receive that inner instruction with `stack_height > 1`.

**Example.**

```rust
for child in input.nested_instructions.iter() {
    println!("CPI to {}", child.instruction.program_id);
}
```

**See also.** [`instruction`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#instruction), [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/struct.InstructionProcessorInputType.html).

## `InstructionMetadata`

Where an instruction sits in its transaction.

```rust
#[derive(Debug, Clone)]
pub struct InstructionMetadata {
    pub transaction_metadata: Arc<TransactionMetadata>,
    pub stack_height: u32,
    pub index: u32,
    pub absolute_path: Vec<u8>,
}
```

**Details.**

| Field                  | Type                       | Meaning                                                                                                                                                                                                                                                                                                      |
| ---------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `transaction_metadata` | `Arc<TransactionMetadata>` | The transaction this instruction belongs to, shared by all its instructions. See [`TransactionMetadata`](#transactionmetadata).                                                                                                                                                                              |
| `stack_height`         | `u32`                      | `1` for a top-level instruction, `2` for a CPI made by it, up to `5` (`MAX_INSTRUCTION_STACK_DEPTH`).                                                                                                                                                                                                        |
| `index`                | `u32`                      | Index of the **top-level** instruction in the message. Inner instructions carry the index of the top-level instruction they descend from, not their position among siblings.                                                                                                                                 |
| `absolute_path`        | `Vec<u8>`                  | Path from the root of the instruction tree: `[top_level_index, child_position, grandchild_position, …]`, one entry per stack level, with positions counted from 0. `[2]` is the third top-level instruction; `[1, 1]` is the second CPI under the second top-level instruction. Unique within a transaction. |

Use `(transaction_metadata.signature, absolute_path)` as a unique key for an instruction; `(signature, index, stack_height)` is not unique when one instruction makes several CPIs at the same depth.

Carbon reads inner instructions from `meta.inner_instructions` and their depth from each inner instruction's `stack_height`. When the source reports a missing or out-of-range `stack_height` (outside 2 to 5), Carbon logs `invalid inner instruction stack height (…), dropping the remaining inner instructions of this group` at `warn` level and skips the rest of that group.

**Example.**

```rust
let key = (
    input.metadata.transaction_metadata.signature,
    input.metadata.absolute_path.clone(),
);
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/struct.InstructionMetadata.html), [source](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/transformers.rs).

## `InstructionMetadata::decode_log_events`

Decodes the program log events emitted by this instruction.

```rust
pub fn decode_log_events<T: CarbonDeserialize>(&self) -> Vec<T>
```

**Details.** Walks `transaction_metadata.meta.log_messages`, tracks `Program … invoke [n]` and `Program … success` / `failed` lines to follow the call stack, and keeps the log lines emitted while this instruction (matched by `absolute_path`) was executing. For each such line, it base64-decodes the last whitespace-separated token and calls `T::deserialize`; lines that fail to decode or deserialize are skipped. This targets Anchor `emit!` events, logged as `Program data: <base64>`. Logs emitted by this instruction's own CPIs are not included.

Precompile instructions (Ed25519, Secp256k1, Secp256r1) log nothing, so Carbon subtracts the number of precompiles before this instruction from the top-level index before matching. Returns an empty `Vec` when the transaction has no log messages. If the runtime truncated the logs (`Log truncated`), events after the cut are missing.

Generated decoders in 2.0.0 do not implement `CarbonDeserialize` for their event types; they expose an inherent `decode` method instead, and Anchor events emitted through self-CPI (`emit_cpi!`) arrive as a decoded `CpiEvent` instruction. Implement `CarbonDeserialize` yourself to use this method; see [`CarbonDeserialize`](https://carbon.sevenlabs.org/reference/traits.md#carbondeserialize).

**Example.**

```rust
for event in input.metadata.decode_log_events::<MyEvent>() {
    println!("{} moved {}", event.pool, event.amount);
}
```

**See also.** [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/struct.InstructionMetadata.html#method.decode_log_events).

## `NestedInstruction`

One node in a transaction's instruction tree.

```rust
#[derive(Debug, Clone)]
pub struct NestedInstruction {
    pub metadata: InstructionMetadata,
    pub instruction: solana_instruction::Instruction,
    pub inner_instructions: NestedInstructions,
}
```

**Details.**

| Field                | Type                              | Meaning                                                                                          |
| -------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------ |
| `metadata`           | `InstructionMetadata`             | Position of this instruction.                                                                    |
| `instruction`        | `solana_instruction::Instruction` | The raw instruction. It is not decoded: run a decoder on it yourself if you need its typed form. |
| `inner_instructions` | `NestedInstructions`              | The CPIs this instruction made, recursively.                                                     |

**Example.**

```rust
fn count(instructions: &NestedInstructions) -> usize {
    instructions
        .iter()
        .map(|ix| 1 + count(&ix.inner_instructions))
        .sum()
}
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/struct.NestedInstruction.html).

## `NestedInstructions`

An ordered list of `NestedInstruction`s.

```rust
#[derive(Debug, Default)]
pub struct NestedInstructions(pub Vec<NestedInstruction>);

impl NestedInstructions {
    pub fn len(&self) -> usize;
    pub fn is_empty(&self) -> bool;
    pub fn push(&mut self, nested_instruction: NestedInstruction);
}
```

**Details.** Derefs to `[NestedInstruction]`, so slice methods such as `iter()`, `first()` and indexing work. It also implements `Clone`, owned `IntoIterator`, and `From<InstructionsWithMetadata>`, which builds the tree from a flat `Vec<(InstructionMetadata, Instruction)>` ordered top-level-first as the runtime reports it. Filters receive the whole transaction's tree in `filter_transaction`; instruction processors receive one subtree in `nested_instructions`.

`From<InstructionsWithMetadata>` asserts that every `stack_height` is between 1 and `MAX_INSTRUCTION_STACK_DEPTH` (5) and panics otherwise. The pipeline only builds trees from validated data, but if you call it yourself, check heights first.

**Example.**

```rust
if let Some(first) = input.nested_instructions.first() {
    println!("first CPI: {:?}", first.metadata.absolute_path);
}
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/struct.NestedInstructions.html).

## `TransactionProcessorInputType`

What a transaction processor receives, once per transaction.

```rust
#[derive(Debug)]
pub struct TransactionProcessorInputType<'a, T> {
    pub metadata: &'a Arc<TransactionMetadata>,
    pub instructions: &'a [(InstructionMetadata, T)],
}
```

**Details.**

| Field          | Type                          | Meaning                                                                                                                                                                                                                                                                                                                  |
| -------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `metadata`     | `&Arc<TransactionMetadata>`   | The transaction. See [`TransactionMetadata`](#transactionmetadata).                                                                                                                                                                                                                                                      |
| `instructions` | `&[(InstructionMetadata, T)]` | Every instruction that `T::parse_instruction` recognized, top-level and inner, in execution order (each top-level instruction followed by its CPIs). Unrecognized instructions are omitted. `T` is your [`InstructionDecoderCollection`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecodercollection). |

The processor runs for every transaction the datasource sends, including transactions where `instructions` is empty. There is no nested tree here; use `InstructionMetadata::stack_height` and `absolute_path` to rebuild parent-child relations.

**Example.**

```rust
impl Processor<TransactionProcessorInputType<'_, AllInstructions>> for TxLogger {
    async fn process(
        &mut self,
        input: &TransactionProcessorInputType<'_, AllInstructions>,
    ) -> CarbonResult<()> {
        if input.instructions.is_empty() {
            return Ok(());
        }
        println!(
            "{}: {} known instructions",
            input.metadata.signature,
            input.instructions.len()
        );
        Ok(())
    }
}
```

**See also.** [`transaction`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#transaction), [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/transaction/struct.TransactionProcessorInputType.html).

## `TransactionMetadata`

Data shared by every instruction of one transaction.

```rust
#[derive(Debug, Clone, Default)]
pub struct TransactionMetadata {
    pub slot: u64,
    pub signature: Signature,
    pub fee_payer: Pubkey,
    pub meta: solana_transaction_status::TransactionStatusMeta,
    pub message: solana_message::VersionedMessage,
    pub index: Option<u64>,
    pub block_time: Option<i64>,
    pub block_hash: Option<Hash>,
}
```

**Details.** Built from the [`TransactionUpdate`](https://carbon.sevenlabs.org/reference/updates.md#transactionupdate) with `TryFrom`, which fails with [`MissingFeePayer`](https://carbon.sevenlabs.org/reference/errors.md#missingfeepayer) when the message has no account keys.

| Field        | Type                               | Meaning                                                                                                                                                                                                                                                                                                                                   |
| ------------ | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `slot`       | `u64`                              | Slot the transaction landed in.                                                                                                                                                                                                                                                                                                           |
| `signature`  | `solana_signature::Signature`      | Transaction signature.                                                                                                                                                                                                                                                                                                                    |
| `fee_payer`  | `solana_pubkey::Pubkey`            | First static account key of the message.                                                                                                                                                                                                                                                                                                  |
| `meta`       | `TransactionStatusMeta`            | Status, fee, balances, inner instructions, logs, token balances, rewards, loaded addresses, return data, compute units. For RPC datasources Carbon converts it from the JSON-RPC form: token balances without an `owner` or `programId` are dropped, and `return_data.data` holds the base64 text as bytes rather than the decoded bytes. |
| `message`    | `solana_message::VersionedMessage` | The full message (legacy, V0 or V1), including `recent_blockhash`, static account keys and compiled instructions.                                                                                                                                                                                                                         |
| `index`      | `Option<u64>`                      | Position in the block, when the datasource knows it. `None` for the RPC transaction crawler and Helius Atlas.                                                                                                                                                                                                                             |
| `block_time` | `Option<i64>`                      | Unix timestamp in seconds, when provided.                                                                                                                                                                                                                                                                                                 |
| `block_hash` | `Option<solana_hash::Hash>`        | Only set by the RPC block crawler and RPC block subscribe.                                                                                                                                                                                                                                                                                |

Check `meta.status.is_err()` to detect failed transactions. Whether you receive failed transactions depends on the datasource: the RPC transaction crawler, RPC block crawler and RPC block subscribe drop them before they reach the pipeline; Yellowstone and LaserStream follow your subscription's `failed` filter; Helius GTFA follows its `status` filter.

**Example.**

```rust
let tx = &input.metadata.transaction_metadata;
if tx.meta.status.is_err() {
    return Ok(());
}
println!("{} paid {} lamports", tx.fee_payer, tx.meta.fee);
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/transaction/struct.TransactionMetadata.html).

## `MAX_INSTRUCTION_STACK_DEPTH`

The deepest instruction nesting Carbon accepts.

```rust
pub const MAX_INSTRUCTION_STACK_DEPTH: usize = 5;
```

**Details.** Matches the Agave runtime's limit of a top-level instruction plus four levels of CPI. Inner instructions reported deeper than this are dropped with a warning.

**Example.**

```rust
assert!(input.metadata.stack_height as usize <= MAX_INSTRUCTION_STACK_DEPTH);
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/constant.MAX_INSTRUCTION_STACK_DEPTH.html).

## Deletions and block details

Account-deletion and block-details processors receive the update itself, with no metadata wrapper and no decoder:

```rust
impl Processor<AccountDeletion> for DeletionLogger {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        println!("closed {} (owner {})", deletion.pubkey, deletion.account.owner);
        Ok(())
    }
}

impl Processor<BlockDetails> for BlockLogger {
    async fn process(&mut self, block: &BlockDetails) -> CarbonResult<()> {
        println!("block {} at {:?}", block.slot, block.block_time);
        Ok(())
    }
}
```

See [`AccountDeletion`](https://carbon.sevenlabs.org/reference/updates.md#accountdeletion) and [`BlockDetails`](https://carbon.sevenlabs.org/reference/updates.md#blockdetails) for their fields.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/processor-inputs · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Traits (Carbon 2.0)

> Every trait you implement or call in Carbon 2.0.0, with required methods, associated types and a compiled implementation example.

Carbon is built from a few traits: datasources produce updates, decoders turn raw bytes into types, processors handle the result, filters gate pipes, and exporters publish metrics. This page lists each trait in `carbon-core` 2.0.0 with its exact definition and an implementation that compiles against the published crates.

Examples on this page assume these dependencies:

`Cargo.toml`:

```toml
[dependencies]
async-trait = "0.1"
borsh = { version = "1.5.7", features = ["derive"] }
carbon-core = "2.0.0"
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["serde"] }
carbon-token-program-decoder = { version = "2.0.0", features = ["serde"] }
serde = { version = "1", features = ["derive"] }
solana-account = "4.3.2"
solana-instruction = "3.4.1"
solana-pubkey = "4.2.1"
tokio = { version = "1", features = ["full"] }
tokio-util = "0.7"
```

| Trait                                                           | Module                     | You implement it to                                          |
| --------------------------------------------------------------- | -------------------------- | ------------------------------------------------------------ |
| [`Datasource`](#datasource)                                     | `carbon_core::datasource`  | Feed updates from a new source                               |
| [`Processor`](#processor)                                       | `carbon_core::processor`   | Handle decoded data                                          |
| [`AccountDecoder`](#accountdecoder)                             | `carbon_core::account`     | Decode accounts of a program without a generated decoder     |
| [`InstructionDecoder`](#instructiondecoder)                     | `carbon_core::instruction` | Decode instructions of a program without a generated decoder |
| [`CarbonDeserialize`](#carbondeserialize)                       | `carbon_core::deserialize` | Decode log events with `decode_log_events`                   |
| [`ArrangeAccounts`](#arrangeaccounts)                           | `carbon_core::deserialize` | Map an instruction's account list to named fields            |
| [`InstructionDecoderCollection`](#instructiondecodercollection) | `carbon_core::collection`  | Use transaction pipes over one or more programs              |
| [`Filter`](#filter)                                             | `carbon_core::filter`      | Gate which updates reach a pipe                              |
| [`MetricsExporter`](#metricsexporter)                           | `carbon_core::metrics`     | Send metrics to a backend                                    |
| [`Metric`](#metric)                                             | `carbon_core::metrics`     | Read a metric's name and help text                           |

## `Datasource`

Produces updates and sends them into the pipeline.

```rust
#[async_trait]
pub trait Datasource: Send + Sync {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: tokio::sync::mpsc::Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()>;

    fn update_types(&self) -> Vec<UpdateType>;
}
```

**Details.** `Datasource` uses `#[async_trait]` because the pipeline stores datasources as `Arc<dyn Datasource>`; put `#[async_trait::async_trait]` on your `impl` too.

| Method         | Required | Description                                                                                                                                                                                                                                                                                                                                                                                   |
| -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `consume`      | Yes      | Called once by `Pipeline::run`, in its own Tokio task. Send each update as `(update, id.clone())` on `sender`. Return when `cancellation_token` is cancelled. Returning (with `Ok` or `Err`) ends this datasource; the pipeline stops when all datasources have returned and the channel is empty. An `Err` is logged as `error consuming datasource: …` and does not stop other datasources. |
| `update_types` | Yes      | The [`UpdateType`](https://carbon.sevenlabs.org/reference/updates.md#updatetype)s this datasource emits. Not used by the pipeline in 2.0.0, but return an accurate list for readers and future versions.                                                                                                                                                                                      |

Use `sender.send(...).await` to wait when the pipeline is behind (backpressure), or `try_send` to drop updates instead. Build account updates with [`AccountUpdate::into_update`](https://carbon.sevenlabs.org/reference/updates.md#accountupdateinto_update) so zero-lamport accounts become deletions. `&self` is shared, so keep mutable state inside `consume` or behind a lock.

**Example.**

`src/ticker.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{AccountUpdate, Datasource, DatasourceId, Update, UpdateType},
        error::{CarbonResult, Error},
    },
    solana_account::Account,
    solana_pubkey::Pubkey,
    std::time::Duration,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Emits one synthetic account update per second until cancelled.
pub struct Ticker {
    pub owner: Pubkey,
}

#[async_trait]
impl Datasource for Ticker {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        let mut slot: u64 = 0;
        loop {
            tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                _ = tokio::time::sleep(Duration::from_secs(1)) => {}
            }
            slot += 1;

            let mut data = b"counter!".to_vec();
            data.extend_from_slice(&slot.to_le_bytes());

            let update = AccountUpdate {
                pubkey: Pubkey::new_unique(),
                account: Account {
                    lamports: 1_000_000,
                    data,
                    owner: self.owner,
                    executable: false,
                    rent_epoch: 0,
                },
                slot,
                transaction_signature: None,
            }
            .into_update();

            sender
                .send((update, id.clone()))
                .await
                .map_err(|e| Error::FailedToConsumeDatasource(e.to_string()))?;
        }
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::AccountUpdate]
    }
}
```

**See also.** [Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md), [Updates](https://carbon.sevenlabs.org/reference/updates.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/datasource/trait.Datasource.html).

## `Processor`

Handles one input value at a time.

```rust
pub trait Processor<T>
where
    T: Sync,
{
    fn process(&mut self, data: &T) -> impl Future<Output = CarbonResult<()>> + Send;
}
```

**Details.** Implement it with a native `async fn process(&mut self, data: &T) -> CarbonResult<()>`; no `#[async_trait]` is needed. `T` is the pipe's input type, listed in [Processor inputs](https://carbon.sevenlabs.org/reference/processor-inputs.md). For input types with a lifetime, write `impl Processor<InstructionProcessorInputType<'_, MyInstruction>> for MyProcessor`; the builder requires the impl for every lifetime, which `'_` provides.

The pipeline owns the processor and calls `process` sequentially, so `&mut self` state needs no locking. The returned future must be `Send`: don't hold a `std::sync::MutexGuard` or `Rc` across an `.await`. Returning `Err` logs the error, increments `carbon_updates_failed_total`, skips the remaining pipes for that update and continues with the next update.

**Example.**

`src/deletions.rs`:

```rust
use carbon_core::{datasource::AccountDeletion, error::CarbonResult, processor::Processor};

#[derive(Default)]
pub struct DeletionCounter {
    pub seen: u64,
}

impl Processor<AccountDeletion> for DeletionCounter {
    async fn process(&mut self, deletion: &AccountDeletion) -> CarbonResult<()> {
        self.seen += 1;
        println!("closed {} at slot {} ({} so far)", deletion.pubkey, deletion.slot, self.seen);
        Ok(())
    }
}
```

**See also.** [Processors](https://carbon.sevenlabs.org/concepts/processors.md), [Processor inputs](https://carbon.sevenlabs.org/reference/processor-inputs.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/processor/trait.Processor.html).

## `AccountDecoder`

Turns a raw account into a typed value.

```rust
pub trait AccountDecoder<'a> {
    type AccountType;

    fn decode_account(
        &self,
        account: &'a solana_account::Account,
    ) -> Option<DecodedAccount<Self::AccountType>>;
}
```

**Details.**

| Item               | Description                                                                                               |
| ------------------ | --------------------------------------------------------------------------------------------------------- |
| `type AccountType` | What your processor receives in `decoded_account.data`. Must be `Send + Sync + 'static` to register.      |
| `decode_account`   | Return `None` for accounts you don't handle; the pipe then skips the update. Check `account.owner` first. |

Generated decoders implement this trait on their decoder struct (`MeteoraDlmmDecoder`, `TokenProgramDecoder`, …): they check the owner, then try each account type's discriminator in turn. Implement it by hand for programs without an IDL. Register the decoder with [`account`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account), which requires the impl for every lifetime (`impl<'a> AccountDecoder<'a> for …`).

**Example.**

`src/counter.rs`:

```rust
use {
    carbon_core::account::{AccountDecoder, DecodedAccount},
    solana_account::Account,
    solana_pubkey::Pubkey,
};

pub const COUNTER_PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("Counter111111111111111111111111111111111111");

#[derive(Debug, Clone)]
pub struct CounterAccount {
    pub count: u64,
}

pub struct CounterDecoder;

impl<'a> AccountDecoder<'a> for CounterDecoder {
    type AccountType = CounterAccount;

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<CounterAccount>> {
        if account.owner != COUNTER_PROGRAM_ID {
            return None;
        }
        let payload = account.data.strip_prefix(b"counter!")?;
        let count = u64::from_le_bytes(payload.get(..8)?.try_into().ok()?);

        Some(DecodedAccount {
            lamports: account.lamports,
            data: CounterAccount { count },
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}
```

**See also.** [`DecodedAccount`](https://carbon.sevenlabs.org/reference/processor-inputs.md#decodedaccount), [Custom decoder](https://carbon.sevenlabs.org/guides/custom-decoder.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/account/trait.AccountDecoder.html).

## `InstructionDecoder`

Turns a raw instruction into a typed value.

```rust
pub trait InstructionDecoder<'a> {
    type InstructionType;

    fn decode_instruction(
        &self,
        instruction: &'a solana_instruction::Instruction,
    ) -> Option<Self::InstructionType>;
}
```

**Details.**

| Item                   | Description                                                                                                                                                  |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `type InstructionType` | What your processor receives in `decoded_instruction`. Must be `Send + Sync + 'static` to register.                                                          |
| `decode_instruction`   | Called for every instruction of every transaction, top-level and inner. Return `None` quickly for other programs by checking `instruction.program_id` first. |

The `instruction` has resolved `AccountMeta`s, so you can read `is_signer` and `is_writable`. Generated decoders return an enum with one struct variant per instruction (`{ program_id, data, accounts }`) built with [`try_decode_instructions!`](https://carbon.sevenlabs.org/reference/macros.md#try_decode_instructions).

**Example.**

`src/memo.rs`:

```rust
use {
    carbon_core::instruction::InstructionDecoder,
    solana_instruction::Instruction,
    solana_pubkey::Pubkey,
};

pub const MEMO_PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("MemoSq4gqABAXKb96qnH8TysNcWxMyWCqXgDLGmfcHr");

#[derive(Debug, Clone)]
pub struct Memo {
    pub text: String,
    pub signers: Vec<Pubkey>,
}

pub struct MemoDecoder;

impl<'a> InstructionDecoder<'a> for MemoDecoder {
    type InstructionType = Memo;

    fn decode_instruction(&self, instruction: &'a Instruction) -> Option<Memo> {
        if instruction.program_id != MEMO_PROGRAM_ID {
            return None;
        }
        Some(Memo {
            text: String::from_utf8(instruction.data.clone()).ok()?,
            signers: instruction
                .accounts
                .iter()
                .filter(|meta| meta.is_signer)
                .map(|meta| meta.pubkey)
                .collect(),
        })
    }
}
```

**See also.** [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md), [Custom decoder](https://carbon.sevenlabs.org/guides/custom-decoder.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/instruction/trait.InstructionDecoder.html).

## `CarbonDeserialize`

Discriminator-prefixed Borsh deserialization.

```rust
pub trait CarbonDeserialize
where
    Self: Sized + borsh::BorshDeserialize,
{
    const DISCRIMINATOR: &'static [u8];

    fn deserialize(data: &[u8]) -> Option<Self>;
}
```

**Details.**

| Item                  | Description                                                                                                           |
| --------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `const DISCRIMINATOR` | The prefix that identifies this type, usually 8 bytes for Anchor. Not read by Carbon itself; use it in `deserialize`. |
| `deserialize`         | Check and strip the discriminator, then Borsh-decode the rest. Return `None` on mismatch.                             |

In 2.0.0 the only consumer is [`InstructionMetadata::decode_log_events`](https://carbon.sevenlabs.org/reference/processor-inputs.md#instructionmetadatadecode_log_events). Generated decoders do **not** implement this trait: their account, instruction and event types have an inherent `decode(&[u8]) -> Option<Self>` method instead. To decode a generated event type from logs, wrap it or implement the trait on your own type. `carbon_core::deserialize::extract_discriminator(length, data)` splits `data` into `(discriminator, rest)` and returns `None` when `data` is shorter than `length`.

**Example.**

`src/events.rs`:

```rust
use {
    borsh::BorshDeserialize,
    carbon_core::{
        deserialize::CarbonDeserialize, error::CarbonResult,
        instruction::InstructionProcessorInputType, processor::Processor,
    },
    carbon_meteora_dlmm_decoder::instructions::MeteoraDlmmInstruction,
    solana_pubkey::Pubkey,
};

/// An Anchor `emit!` event: 8-byte discriminator, then the Borsh body.
#[derive(Debug, BorshDeserialize)]
pub struct MyEvent {
    pub pool: Pubkey,
    pub amount: u64,
}

impl CarbonDeserialize for MyEvent {
    const DISCRIMINATOR: &'static [u8] = &[0x1d, 0x2c, 0x3b, 0x4a, 0x59, 0x68, 0x77, 0x86];

    fn deserialize(data: &[u8]) -> Option<Self> {
        let body = data.strip_prefix(Self::DISCRIMINATOR)?;
        borsh::from_slice(body).ok()
    }
}

pub struct EventLogger;

impl Processor<InstructionProcessorInputType<'_, MeteoraDlmmInstruction>> for EventLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, MeteoraDlmmInstruction>,
    ) -> CarbonResult<()> {
        for event in input.metadata.decode_log_events::<MyEvent>() {
            println!("{} moved {}", event.pool, event.amount);
        }
        Ok(())
    }
}
```

**See also.** [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/deserialize/trait.CarbonDeserialize.html).

## `ArrangeAccounts`

Maps an instruction's positional accounts to a struct.

```rust
pub trait ArrangeAccounts {
    type ArrangedAccounts: Clone + Send + Sync + std::fmt::Debug;

    fn arrange_accounts(
        accounts: &[solana_instruction::AccountMeta],
    ) -> Option<Self::ArrangedAccounts>;
}
```

**Details.** An associated function (no `self`), implemented on the instruction's data type. Return `None` when required accounts are missing; [`try_decode_instructions!`](https://carbon.sevenlabs.org/reference/macros.md#try_decode_instructions) then rejects the instruction. Generated decoders implement it for every instruction, treat optional accounts as `Option<Pubkey>`, and collect extra accounts into `remaining: Vec<AccountMeta>`. `carbon_core::account_utils::next_account(&mut iter)` returns the next account's `Pubkey`, or `None` at the end.

**Example.**

`src/transfer.rs`:

```rust
use {
    carbon_core::{account_utils::next_account, deserialize::ArrangeAccounts},
    solana_instruction::AccountMeta,
    solana_pubkey::Pubkey,
};

pub struct Transfer;

#[derive(Debug, Clone)]
pub struct TransferAccounts {
    pub source: Pubkey,
    pub destination: Pubkey,
    pub authority: Pubkey,
    pub remaining: Vec<AccountMeta>,
}

impl ArrangeAccounts for Transfer {
    type ArrangedAccounts = TransferAccounts;

    fn arrange_accounts(accounts: &[AccountMeta]) -> Option<TransferAccounts> {
        let mut iter = accounts.iter();
        let source = next_account(&mut iter)?;
        let destination = next_account(&mut iter)?;
        let authority = next_account(&mut iter)?;

        Some(TransferAccounts {
            source,
            destination,
            authority,
            remaining: iter.as_slice().to_vec(),
        })
    }
}
```

**See also.** [`try_decode_instructions!`](https://carbon.sevenlabs.org/reference/macros.md#try_decode_instructions), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/deserialize/trait.ArrangeAccounts.html).

## `InstructionDecoderCollection`

A single enum over the instructions of several programs, used by transaction pipes.

```rust
pub trait InstructionDecoderCollection:
    Clone + std::fmt::Debug + Send + Sync + Eq + std::hash::Hash + Serialize + 'static
{
    type InstructionType: Clone + std::fmt::Debug + PartialEq + Eq + Send + Sync + 'static;

    fn parse_instruction(instruction: &solana_instruction::Instruction) -> Option<Self>;

    fn get_type(&self) -> Self::InstructionType;
}
```

**Details.**

| Item                   | Description                                                                                                                                               |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `type InstructionType` | A tag enum identifying the variant. Carbon does not read it in 2.0.0.                                                                                     |
| `parse_instruction`    | Associated function (no decoder instance). Return the variant for the instruction's program, or `None`. Called on every instruction of every transaction. |
| `get_type`             | Return the tag for `self`.                                                                                                                                |

The supertraits include `Eq`, `Hash` and `serde::Serialize`. Generated decoder enums derive only `Debug`, `Clone` and `PartialEq` (plus `Serialize`/`Deserialize` with the `serde` feature), so wrap them in your own enum and implement `Eq` and `Hash` by hand, as below. The [`instruction_decoder_collection!`](https://carbon.sevenlabs.org/reference/macros.md#instruction_decoder_collection) macro generates this trait, but only for instruction enums that derive `Hash`, `Eq` and `InstructionType`, which generated decoders don't.

**Example.**

`src/collection.rs`:

```rust
use {
    carbon_core::{collection::InstructionDecoderCollection, instruction::InstructionDecoder},
    carbon_meteora_dlmm_decoder::{instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder},
    carbon_token_program_decoder::{instructions::TokenProgramInstruction, TokenProgramDecoder},
    solana_instruction::Instruction,
    std::hash::{Hash, Hasher},
};

#[derive(Debug, Clone, PartialEq, serde::Serialize)]
pub enum AllInstructions {
    MeteoraDlmm(MeteoraDlmmInstruction),
    Token(TokenProgramInstruction),
}

// Generated instruction enums derive `PartialEq` only, so provide `Eq` and
// `Hash` by hand. Hashing the variant alone is consistent with `PartialEq`.
impl Eq for AllInstructions {}

impl Hash for AllInstructions {
    fn hash<H: Hasher>(&self, state: &mut H) {
        std::mem::discriminant(self).hash(state);
    }
}

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AllInstructionTypes {
    MeteoraDlmm,
    Token,
}

impl InstructionDecoderCollection for AllInstructions {
    type InstructionType = AllInstructionTypes;

    fn parse_instruction(instruction: &Instruction) -> Option<Self> {
        if let Some(ix) = MeteoraDlmmDecoder.decode_instruction(instruction) {
            return Some(Self::MeteoraDlmm(ix));
        }
        TokenProgramDecoder
            .decode_instruction(instruction)
            .map(Self::Token)
    }

    fn get_type(&self) -> AllInstructionTypes {
        match self {
            Self::MeteoraDlmm(_) => AllInstructionTypes::MeteoraDlmm,
            Self::Token(_) => AllInstructionTypes::Token,
        }
    }
}
```

Register it with `Pipeline::builder().transaction::<AllInstructions, _>(processor)`.

**See also.** [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md), [`TransactionProcessorInputType`](https://carbon.sevenlabs.org/reference/processor-inputs.md#transactionprocessorinputtype), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/collection/trait.InstructionDecoderCollection.html).

## `Filter`

Decides whether an update reaches a pipe.

```rust
pub trait Filter: Send + Sync {
    fn filter_account(&self, _context: &FilterContext, _account_metadata: &AccountMetadata, _account: &solana_account::Account) -> FilterResult { FilterResult::Accept }
    fn filter_instruction(&self, _context: &FilterContext, _nested_instruction: &NestedInstruction) -> FilterResult { FilterResult::Accept }
    fn filter_transaction(&self, _context: &FilterContext, _transaction_metadata: &TransactionMetadata, _nested_instructions: &NestedInstructions) -> FilterResult { FilterResult::Accept }
    fn filter_account_deletion(&self, _context: &FilterContext, _account_deletion: &AccountDeletion) -> FilterResult { FilterResult::Accept }
    fn filter_block_details(&self, _context: &FilterContext, _block_details: &BlockDetails) -> FilterResult { FilterResult::Accept }
}
```

**Details.** Every method has a default that returns `Accept`, so implement only the hooks for the pipe kinds you filter. Hooks take `&self`: use atomics or a lock for state, as `DeduplicationFilter` does. See the [filters reference](https://carbon.sevenlabs.org/reference/filters.md#filter) for when each hook runs.

**Example.**

`src/filters.rs`:

```rust
use {
    carbon_core::{
        filter::{Filter, FilterContext, FilterResult},
        instruction::{NestedInstruction, NestedInstructions},
        transaction::TransactionMetadata,
    },
};

/// Drops failed transactions and every instruction inside them.
pub struct SuccessfulOnly;

impl Filter for SuccessfulOnly {
    fn filter_instruction(
        &self,
        _context: &FilterContext,
        nested_instruction: &NestedInstruction,
    ) -> FilterResult {
        if nested_instruction.metadata.transaction_metadata.meta.status.is_ok() {
            FilterResult::Accept
        } else {
            FilterResult::Reject
        }
    }

    fn filter_transaction(
        &self,
        _context: &FilterContext,
        transaction_metadata: &TransactionMetadata,
        _nested_instructions: &NestedInstructions,
    ) -> FilterResult {
        if transaction_metadata.meta.status.is_ok() {
            FilterResult::Accept
        } else {
            FilterResult::Reject
        }
    }
}
```

**See also.** [Filters reference](https://carbon.sevenlabs.org/reference/filters.md), [Filters](https://carbon.sevenlabs.org/concepts/filters.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/trait.Filter.html).

## `MetricsExporter`

Publishes metric snapshots to a backend.

```rust
pub trait MetricsExporter: Send + Sync {
    fn initialize(self: Arc<Self>) -> CarbonResult<()> {
        let _ = self;
        Ok(())
    }

    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()>;

    fn shutdown(&self) -> CarbonResult<()> {
        Ok(())
    }
}
```

**Details.**

| Method       | Required | When the pipeline calls it                                                                                                                                                                |
| ------------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `initialize` | No       | Once, at the start of `run`, with an `Arc` of the exporter. Start periodic work here (for example `tokio::spawn` a loop that calls `self.export(&MetricsRegistry::global().snapshot())`). |
| `export`     | Yes      | Once, when the pipeline stops, with a final snapshot. Not called while the pipeline runs.                                                                                                 |
| `shutdown`   | No       | Once, after the final `export`.                                                                                                                                                           |

An `Err` from any of these makes `Pipeline::run` return that error. Register with [`PipelineBuilder::metrics`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#metrics).

**Example.**

`src/exporter.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult,
        metrics::{MetricsExporter, MetricsRegistry, MetricsSnapshot},
    },
    std::{sync::Arc, time::Duration},
};

/// Prints every counter every 10 seconds, and once more at shutdown.
pub struct StdoutExporter;

impl MetricsExporter for StdoutExporter {
    fn initialize(self: Arc<Self>) -> CarbonResult<()> {
        tokio::spawn(async move {
            let mut interval = tokio::time::interval(Duration::from_secs(10));
            loop {
                interval.tick().await;
                let _ = self.export(&MetricsRegistry::global().snapshot());
            }
        });
        Ok(())
    }

    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()> {
        for (name, _help, value) in &snapshot.counters {
            println!("{name} {value}");
        }
        Ok(())
    }
}
```

**See also.** [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/trait.MetricsExporter.html).

## `Metric`

Name and help text shared by `Counter`, `Gauge` and `Histogram`.

```rust
pub trait Metric: Send + Sync {
    fn name(&self) -> &'static str;
    fn help(&self) -> &'static str {
        ""
    }
}
```

**Details.** Implemented by the three metric types in `carbon_core::metrics`. `MetricsRegistry` only stores `&'static Counter`, `&'static Gauge` and `&'static Histogram`, so implementing `Metric` on your own type does not make it exportable. To add a metric, declare a `static` `Counter`, `Gauge` or `Histogram` and register it.

**Example.**

`src/metrics.rs`:

```rust
use carbon_core::metrics::{Counter, Metric, MetricsRegistry};

static SWAPS_SEEN: Counter = Counter::new("my_indexer_swaps_seen_total", "Swaps decoded");

pub fn register() {
    MetricsRegistry::global().register_counter(&SWAPS_SEEN);
    assert_eq!(SWAPS_SEEN.name(), "my_indexer_swaps_seen_total");
}
```

**See also.** [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/trait.Metric.html).

## Complete example

This program implements a datasource and an account decoder, runs them for 3.5 seconds and needs no network.

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        account::{AccountDecoder, AccountProcessorInputType, DecodedAccount},
        datasource::{AccountUpdate, Datasource, DatasourceId, Update, UpdateType},
        error::{CarbonResult, Error},
        pipeline::Pipeline,
        processor::Processor,
    },
    solana_account::Account,
    solana_pubkey::Pubkey,
    std::time::Duration,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

const COUNTER_PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("Counter111111111111111111111111111111111111");

/// Emits one synthetic account update per second until cancelled.
struct Ticker;

#[async_trait]
impl Datasource for Ticker {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        let mut slot: u64 = 0;
        loop {
            tokio::select! {
                _ = cancellation_token.cancelled() => return Ok(()),
                _ = tokio::time::sleep(Duration::from_secs(1)) => {}
            }
            slot += 1;

            let mut data = b"counter!".to_vec();
            data.extend_from_slice(&slot.to_le_bytes());

            let update = AccountUpdate {
                pubkey: Pubkey::new_unique(),
                account: Account {
                    lamports: 1_000_000,
                    data,
                    owner: COUNTER_PROGRAM_ID,
                    executable: false,
                    rent_epoch: 0,
                },
                slot,
                transaction_signature: None,
            }
            .into_update();

            sender
                .send((update, id.clone()))
                .await
                .map_err(|e| Error::FailedToConsumeDatasource(e.to_string()))?;
        }
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::AccountUpdate]
    }
}

#[derive(Debug, Clone)]
struct CounterAccount {
    count: u64,
}

struct CounterDecoder;

impl<'a> AccountDecoder<'a> for CounterDecoder {
    type AccountType = CounterAccount;

    fn decode_account(&self, account: &'a Account) -> Option<DecodedAccount<CounterAccount>> {
        if account.owner != COUNTER_PROGRAM_ID {
            return None;
        }
        let payload = account.data.strip_prefix(b"counter!")?;
        let count = u64::from_le_bytes(payload.get(..8)?.try_into().ok()?);

        Some(DecodedAccount {
            lamports: account.lamports,
            data: CounterAccount { count },
            owner: account.owner,
            executable: account.executable,
            rent_epoch: account.rent_epoch,
        })
    }
}

struct CounterLogger;

impl Processor<AccountProcessorInputType<'_, CounterAccount>> for CounterLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, CounterAccount>,
    ) -> CarbonResult<()> {
        println!(
            "slot {}: {} count={}",
            input.metadata.slot, input.metadata.pubkey, input.decoded_account.data.count
        );
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    let token = CancellationToken::new();
    let stop = token.clone();
    tokio::spawn(async move {
        tokio::time::sleep(Duration::from_millis(3_500)).await;
        stop.cancel();
    });

    Pipeline::builder()
        .datasource(Ticker)
        .account(CounterDecoder, CounterLogger)
        .datasource_cancellation_token(token)
        .build()?
        .run()
        .await
}
```

```sh
cargo run
```

Output:

```text
slot 1: 11157t3sqMV725NVRLrVQbAu98Jjfk1uCKehJnXXQs count=1
slot 2: 1117mWrzzrZr312ebPDHu8tbfMwFNvCvMbr6WepCNG count=2
slot 3: 1119DWteoLSdjvrT6g6L8C2PfDD2faiTQUpsjY2RiF count=3
```

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/traits · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Filters reference (Carbon 2.0)

> Filter, FilterContext, FilterResult and the built-in DeduplicationFilter, DatasourceFilter and SlotRangeFilter in Carbon 2.0.0, with exact semantics.

Filters decide, per pipe, which updates reach the processor. This page documents the `carbon_core::filter` module in 2.0.0: the trait, its context and result types, and the three built-in filters, with the exact rules each one applies. For when to use filters, see [Filters](https://carbon.sevenlabs.org/concepts/filters.md).

Source: [`crates/core/src/filter.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/filter.rs).

## How filters are evaluated

- You attach filters to one pipe with the `*_with_filters` builder methods. Each pipe owns its filters; two pipes never share filter state, even if you create them with the same arguments.
- For each update and pipe, the pipeline calls the hook that matches the pipe kind on each filter in order, and stops at the first `Reject`: the filters are combined with AND. A rejected update is skipped for that pipe only.
- Hooks run before decoding. Instruction filters see every instruction of every transaction, including instructions of other programs.
- Because evaluation stops at the first `Reject`, a stateful filter later in the list only sees updates the earlier filters accepted. Put `DeduplicationFilter` last.

| Pipe registered with             | Hook called               | Called once per                   |
| -------------------------------- | ------------------------- | --------------------------------- |
| `account_with_filters`           | `filter_account`          | account update                    |
| `instruction_with_filters`       | `filter_instruction`      | instruction (top-level and inner) |
| `transaction_with_filters`       | `filter_transaction`      | transaction                       |
| `account_deletions_with_filters` | `filter_account_deletion` | account deletion                  |
| `block_details_with_filters`     | `filter_block_details`    | block                             |

## Built-in filters at a glance

| Filter                | Account                               | Instruction                                  | Transaction                | Deletion        | Block details   |
| --------------------- | ------------------------------------- | -------------------------------------------- | -------------------------- | --------------- | --------------- |
| `DeduplicationFilter` | Deduplicated by `(signature, pubkey)` | Deduplicated by `(signature, absolute_path)` | Always accepted            | Always accepted | Always accepted |
| `DatasourceFilter`    | Checked                               | Checked                                      | Checked                    | Checked         | Checked         |
| `SlotRangeFilter`     | Slot only                             | Slot and transaction index                   | Slot and transaction index | Slot only       | Slot only       |

## `Filter`

The trait every filter implements.

```rust
pub trait Filter: Send + Sync {
    fn filter_account(
        &self,
        _context: &FilterContext,
        _account_metadata: &AccountMetadata,
        _account: &solana_account::Account,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_instruction(
        &self,
        _context: &FilterContext,
        _nested_instruction: &NestedInstruction,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_transaction(
        &self,
        _context: &FilterContext,
        _transaction_metadata: &TransactionMetadata,
        _nested_instructions: &NestedInstructions,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_account_deletion(
        &self,
        _context: &FilterContext,
        _account_deletion: &AccountDeletion,
    ) -> FilterResult {
        FilterResult::Accept
    }

    fn filter_block_details(
        &self,
        _context: &FilterContext,
        _block_details: &BlockDetails,
    ) -> FilterResult {
        FilterResult::Accept
    }
}
```

**Details.**

| Hook                      | Receives                                                                                                                                                                                                                                       |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `filter_account`          | [`AccountMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md#accountmetadata) and the raw `Account`                                                                                                                          |
| `filter_instruction`      | The [`NestedInstruction`](https://carbon.sevenlabs.org/reference/processor-inputs.md#nestedinstruction): metadata, raw instruction, and its CPIs                                                                                               |
| `filter_transaction`      | [`TransactionMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md#transactionmetadata) and the whole transaction's [`NestedInstructions`](https://carbon.sevenlabs.org/reference/processor-inputs.md#nestedinstructions) tree |
| `filter_account_deletion` | The [`AccountDeletion`](https://carbon.sevenlabs.org/reference/updates.md#accountdeletion)                                                                                                                                                     |
| `filter_block_details`    | The [`BlockDetails`](https://carbon.sevenlabs.org/reference/updates.md#blockdetails)                                                                                                                                                           |

All hooks default to `Accept`. Hooks take `&self`, so keep state behind atomics or locks. Pipes store filters as `Box<dyn Filter + 'static>`, so build the list as `Vec<Box<dyn Filter>>`.

**Example.**

```rust
use {
    carbon_core::{
        account::AccountMetadata,
        filter::{Filter, FilterContext, FilterResult},
    },
    solana_account::Account,
};

/// Accepts accounts whose data is at least `min_len` bytes.
pub struct MinDataLen(pub usize);

impl Filter for MinDataLen {
    fn filter_account(
        &self,
        _context: &FilterContext,
        _account_metadata: &AccountMetadata,
        account: &Account,
    ) -> FilterResult {
        if account.data.len() >= self.0 {
            FilterResult::Accept
        } else {
            FilterResult::Reject
        }
    }
}
```

**See also.** [`Filter` in traits](https://carbon.sevenlabs.org/reference/traits.md#filter), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/trait.Filter.html).

## `FilterContext`

Read-only context passed to every hook.

```rust
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct FilterContext<'a> {
    pub datasource_id: &'a DatasourceId,
}
```

**Details.**

| Field           | Type            | Meaning                                                                                                                   |
| --------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `datasource_id` | `&DatasourceId` | ID of the datasource that sent the update: the one you passed to `datasource_with_id`, or a random UUID for `datasource`. |

**Example.**

```rust
fn filter_block_details(&self, context: &FilterContext, _block: &BlockDetails) -> FilterResult {
    if *context.datasource_id == DatasourceId::new_named("primary") {
        FilterResult::Accept
    } else {
        FilterResult::Reject
    }
}
```

**See also.** [`DatasourceId`](https://carbon.sevenlabs.org/reference/updates.md#datasourceid), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/struct.FilterContext.html).

## `FilterResult`

A hook's decision.

```rust
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum FilterResult {
    Accept,
    Reject,
}
```

**Details.** `Accept` passes the update to the next filter, or to the pipe if this was the last one. `Reject` skips the update for this pipe; other pipes are unaffected.

**Example.**

```rust
let result = if slot >= 400_000_000 { FilterResult::Accept } else { FilterResult::Reject };
```

**See also.** [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/enum.FilterResult.html).

## `DeduplicationFilter`

Drops instructions and accounts already seen within a time window.

```rust
pub struct DeduplicationFilter { /* private fields */ }

impl DeduplicationFilter {
    pub fn new(ttl: Duration) -> Self;
    pub fn cleanup_expired(&self);
}
```

**Details.** Use it when two datasources deliver the same data, for example two Yellowstone endpoints for redundancy, or a crawler that overlaps a stream.

| Update                               | Key                                      | Behaviour                                                                                                                                                                |
| ------------------------------------ | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Instruction                          | `(transaction signature, absolute_path)` | First occurrence is accepted and recorded; any later occurrence of the same key is rejected while the entry is stored.                                                   |
| Account                              | `(transaction_signature, pubkey)`        | Same, but only when `AccountMetadata::transaction_signature` is `Some`. Accounts without a signature (RPC GPA, program subscribe, snapshots, Atlas) are always accepted. |
| Transaction, deletion, block details | none                                     | Always accepted. Transaction pipes are not deduplicated.                                                                                                                 |

The key does not include the datasource or the slot, so the same instruction from two different sources is a duplicate, which is the point. Entries record when they were first seen; a duplicate does not refresh the timestamp.

**TTL and memory.** Expired entries are removed lazily: on each hook call, if at least 60 seconds have passed since the last cleanup, the filter removes every entry older than `ttl`. An entry therefore lives between `ttl` and `ttl + 60 s`, or longer if no updates arrive. Memory grows with the number of distinct keys seen in that window: every instruction the pipe sees is recorded, including instructions of other programs (the filter runs before the decoder). A duplicate arriving after its entry was removed is accepted again. `cleanup_expired` runs the cleanup immediately.

If the internal lock is poisoned, the filter accepts everything instead of panicking.

**Example.**

```rust
let filters: Vec<Box<dyn Filter>> = vec![
    Box::new(DeduplicationFilter::new(Duration::from_secs(300))),
];
Pipeline::builder()
    .datasource_with_id(primary, DatasourceId::new_named("primary"))
    .datasource_with_id(backup, DatasourceId::new_named("backup"))
    .instruction_with_filters(MeteoraDlmmDecoder, SwapLogger, filters)
```

**See also.** [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/struct.DeduplicationFilter.html).

## `DatasourceFilter`

Accepts only updates from the listed datasources.

```rust
pub struct DatasourceFilter {
    pub allowed_datasources: Vec<DatasourceId>,
}

impl DatasourceFilter {
    pub fn new(datasource_id: DatasourceId) -> Self;
    pub fn new_many(datasource_ids: Vec<DatasourceId>) -> Self;
}
```

**Details.** Implements all five hooks: an update is accepted when `context.datasource_id` is in `allowed_datasources` (compared with `==`), rejected otherwise. `new` allows one ID, `new_many` several. An empty list rejects everything. Register the datasources with [`datasource_with_id`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#datasource_with_id); IDs from `datasource` are random and can't be matched.

**Example.**

```rust
let geyser = DatasourceId::new_named("geyser");
let snapshot = DatasourceId::new_named("snapshot");

Pipeline::builder()
    .datasource_with_id(geyser_client, geyser.clone())
    .datasource_with_id(snapshot_loader, snapshot.clone())
    .account_with_filters(
        MeteoraDlmmDecoder,
        PoolLogger,
        vec![Box::new(DatasourceFilter::new_many(vec![geyser, snapshot]))],
    )
```

**See also.** [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/struct.DatasourceFilter.html).

## `SlotRangeFilter`

Accepts only updates inside a slot range, optionally refined by transaction index.

```rust
#[derive(Debug, Clone)]
pub struct SlotRangeFilter { /* private fields */ }

impl SlotRangeFilter {
    pub fn from(slot: u64, transaction_index: Option<u64>) -> Self;
    pub fn to(slot: u64, transaction_index: Option<u64>) -> Self;
    pub fn between(
        from_slot: u64,
        from_transaction_index: Option<u64>,
        to_slot: u64,
        to_transaction_index: Option<u64>,
    ) -> Self;
    pub fn contains(&self, slot: u64, index: Option<u64>) -> bool;
}
```

**Details.** `from` sets only a lower bound, `to` only an upper bound, `between` both. Every hook calls `contains(slot, index)`:

| Update                           | `slot`                      | `index`                      |
| -------------------------------- | --------------------------- | ---------------------------- |
| Instruction                      | `transaction_metadata.slot` | `transaction_metadata.index` |
| Transaction                      | `TransactionMetadata::slot` | `TransactionMetadata::index` |
| Account, deletion, block details | the update's `slot`         | `None`                       |

`contains` applies these rules:

| Bound                                         | Rejects when                                                                                                |
| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| Lower (`from_slot`, `from_transaction_index`) | `slot < from_slot`, or `slot == from_slot` and both indexes are `Some` and `index < from_transaction_index` |
| Upper (`to_slot`, `to_transaction_index`)     | `slot > to_slot`, or `slot == to_slot` and both indexes are `Some` and `index >= to_transaction_index`      |

So the lower bound is inclusive. The upper slot is **inclusive** unless you give a `to_transaction_index` and the update has an index, in which case the range stops just before that transaction. When the update has no index (accounts, or transactions from the RPC transaction crawler and Helius Atlas), the index bounds are ignored and only slots are compared. The source code comment describes the range as half-open `[from, to)`; that is only true at the transaction-index level.

**Example.** This program prints how `contains` treats boundary cases:

`src/main.rs`:

```rust
use carbon_core::filter::SlotRangeFilter;

fn main() {
    let to = SlotRangeFilter::to(100, None);
    let to_idx = SlotRangeFilter::to(100, Some(5));
    let from_idx = SlotRangeFilter::from(100, Some(5));
    let between = SlotRangeFilter::between(100, None, 200, None);

    println!("to(100, None)        slot 100      -> {}", to.contains(100, None));
    println!("to(100, None)        slot 101      -> {}", to.contains(101, None));
    println!("to(100, Some(5))     slot 100 idx 4 -> {}", to_idx.contains(100, Some(4)));
    println!("to(100, Some(5))     slot 100 idx 5 -> {}", to_idx.contains(100, Some(5)));
    println!("to(100, Some(5))     slot 100 None -> {}", to_idx.contains(100, None));
    println!("from(100, Some(5))   slot 100 idx 4 -> {}", from_idx.contains(100, Some(4)));
    println!("from(100, Some(5))   slot 100 None -> {}", from_idx.contains(100, None));
    println!("between(100, 200)    slot 200      -> {}", between.contains(200, None));
}
```

```sh
cargo run
```

Output:

```text
to(100, None)        slot 100      -> true
to(100, None)        slot 101      -> false
to(100, Some(5))     slot 100 idx 4 -> true
to(100, Some(5))     slot 100 idx 5 -> false
to(100, Some(5))     slot 100 None -> true
from(100, Some(5))   slot 100 idx 4 -> false
from(100, Some(5))   slot 100 None -> true
between(100, 200)    slot 200      -> true
```

To register it on a pipe:

```rust
Pipeline::builder().instruction_with_filters(
    MeteoraDlmmDecoder,
    SwapLogger,
    vec![Box::new(SlotRangeFilter::between(400_000_000, None, 400_100_000, None))],
)
```

**See also.** [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/filter/struct.SlotRangeFilter.html).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/filters · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Metrics (Carbon 2.0)

> Every metric Carbon core and each datasource emit, the metrics primitives, and the LogMetrics and PrometheusMetrics exporters.

This page lists every built-in metric in Carbon 2.0.0, the primitives in [`carbon_core::metrics`](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/index.html), and the two exporter crates. For how metrics fit into a pipeline, see [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md); to scrape them, see [Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md).

## How metrics are collected

- Every metric is a `static` [`Counter`](#counter), [`Gauge`](#gauge) or [`Histogram`](#histogram) registered in one process-wide [`MetricsRegistry`](#metricsregistry).
- Metrics have no labels. Each crate prefixes its own names instead (`carbon_`, `yellowstone_grpc_`, `transaction_crawler_` and so on).
- Pipeline metrics are registered by [`PipelineBuilder::build`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#build). With the carbon-core `postgres` feature enabled, `build` also registers the Postgres metrics, whether or not you use a Postgres processor.
- Datasource metrics are registered when the datasource's `consume` starts, so they appear in snapshots only after [`Pipeline::run`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#run) has started that datasource.
- `Pipeline::run` itself exports a snapshot only once, at shutdown. Periodic output comes from the exporter: [`LogMetrics`](#logmetrics) runs its own timer, and [`PrometheusMetrics`](#prometheusmetrics) renders the registry on each scrape.

> **Warning: Registration is not deduplicated.** `MetricsRegistry::register_*` appends without checking for duplicates. Calling `build()` on two pipelines in one process, or running two instances of the same datasource crate, registers the same metrics twice. Snapshots then contain each entry twice, and the Prometheus output repeats the `# TYPE` line for those names, which Prometheus rejects as an invalid scrape. Run one pipeline per process, with at most one instance of each datasource crate, if you scrape with Prometheus.

## Built-in metrics

Histogram buckets are listed as their upper bounds. Each histogram also has an implicit overflow bucket (`+Inf`).

### carbon-core pipeline

Defined in [`pipeline.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/pipeline.rs). Updated by `Pipeline::run` once per update it receives.

| Name                                         | Type      | Unit    | Meaning                                                                                                                                                                                             |
| -------------------------------------------- | --------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `carbon_updates_received_total`              | counter   | updates | Updates taken off the datasource channel.                                                                                                                                                           |
| `carbon_updates_processed_total`             | counter   | updates | Updates the pipeline finished handling, successfully or not.                                                                                                                                        |
| `carbon_updates_successful_total`            | counter   | updates | Updates for which every matching pipe returned `Ok`.                                                                                                                                                |
| `carbon_updates_failed_total`                | counter   | updates | Updates for which a pipe, decoder conversion or processor returned `Err`. The error is logged and the pipeline continues. Pipes after the failing one do not run for that update.                   |
| `carbon_updates_queued`                      | gauge     | updates | Channel length after the last processed update: how many updates are waiting. Compare with [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size). |
| `carbon_account_updates_processed_total`     | counter   | updates | `Update::Account` updates that went through all account pipes without error.                                                                                                                        |
| `carbon_transaction_updates_processed_total` | counter   | updates | `Update::Transaction` updates that went through all instruction and transaction pipes without error.                                                                                                |
| `carbon_account_deletions_processed_total`   | counter   | updates | `Update::AccountDeletion` updates that went through all deletion pipes without error.                                                                                                               |
| `carbon_block_details_processed_total`       | counter   | updates | `Update::BlockDetails` updates that went through all block-details pipes without error.                                                                                                             |
| `carbon_updates_process_time_nanoseconds`    | histogram | ns      | Time to run one update through every pipe. Buckets: 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9.                                                                                                              |
| `carbon_updates_process_time_milliseconds`   | histogram | ms      | Same measurement in whole milliseconds (truncated, so sub-millisecond updates record `0`). Buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000.                                                          |

### carbon-core `postgres` feature

Defined in [`postgres/processors.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/processors.rs). Updated by `PostgresAccountProcessor`, `PostgresJsonAccountProcessor`, `PostgresInstructionProcessor` and `PostgresJsonInstructionProcessor` (see the [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md)). The counter names do not end in `_total`.

| Name                                                 | Type      | Unit | Meaning                                                                                     |
| ---------------------------------------------------- | --------- | ---- | ------------------------------------------------------------------------------------------- |
| `postgres_accounts_upsert_upserted`                  | counter   | rows | Account upserts that succeeded.                                                             |
| `postgres_accounts_upsert_failed`                    | counter   | rows | Account upserts that returned an error.                                                     |
| `postgres_accounts_upsert_duration_milliseconds`     | histogram | ms   | Duration of successful account upserts. Buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000.     |
| `postgres_instructions_upsert_upserted`              | counter   | rows | Instruction upserts that succeeded.                                                         |
| `postgres_instructions_upsert_failed`                | counter   | rows | Instruction upserts that returned an error.                                                 |
| `postgres_instructions_upsert_duration_milliseconds` | histogram | ms   | Duration of successful instruction upserts. Buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000. |

### Datasource histogram buckets

Most datasource histograms use one of three bucket sets:

| Set      | Buckets                                              |
| -------- | ---------------------------------------------------- |
| ns       | 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9 (nanoseconds)      |
| ms-short | 1, 5, 10, 25, 50, 100, 250, 500, 1000 (milliseconds) |
| ms-long  | 1, 10, 50, 100, 500, 1000, 5000 (milliseconds)       |

### carbon-yellowstone-grpc-datasource

See [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md). Updates are queued with a non-blocking `try_send`. When the pipeline channel is full, the update is dropped and an error is logged.

| Name                                                         | Type      | Unit        | Meaning                                                                                                               |
| ------------------------------------------------------------ | --------- | ----------- | --------------------------------------------------------------------------------------------------------------------- |
| `yellowstone_grpc_account_updates_received_total`            | counter   | updates     | Account notifications converted to `Update::Account`. Counted even when `try_send` failed and the update was dropped. |
| `yellowstone_grpc_account_deletions_received_total`          | counter   | updates     | Zero-lamport account notifications converted to `Update::AccountDeletion`. Counted even when the send failed.         |
| `yellowstone_grpc_transaction_updates_received_total`        | counter   | updates     | Transactions queued to the pipeline. Not counted when the send failed.                                                |
| `yellowstone_grpc_account_process_time_nanoseconds`          | histogram | ns (ns set) | Time to convert one account notification and attempt the send.                                                        |
| `yellowstone_grpc_account_deletion_process_time_nanoseconds` | histogram | ns (ns set) | Same, for deletions.                                                                                                  |
| `yellowstone_grpc_transaction_process_time_nanoseconds`      | histogram | ns (ns set) | Time to convert one protobuf transaction and queue it.                                                                |

### carbon-helius-laserstream-datasource

See [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md). Same behaviour as Yellowstone, including `try_send`.

| Name                                                    | Type      | Unit        | Meaning                                                                                               |
| ------------------------------------------------------- | --------- | ----------- | ----------------------------------------------------------------------------------------------------- |
| `laserstream_account_updates_received_total`            | counter   | updates     | Account notifications converted to `Update::Account`, counted even when the send failed.              |
| `laserstream_account_deletions_received_total`          | counter   | updates     | Zero-lamport notifications converted to `Update::AccountDeletion`, counted even when the send failed. |
| `laserstream_transaction_updates_received_total`        | counter   | updates     | Transactions queued to the pipeline.                                                                  |
| `laserstream_account_process_time_nanoseconds`          | histogram | ns (ns set) | Time to convert one account notification and attempt the send.                                        |
| `laserstream_account_deletion_process_time_nanoseconds` | histogram | ns (ns set) | Same, for deletions.                                                                                  |
| `laserstream_transaction_process_time_nanoseconds`      | histogram | ns (ns set) | Time to convert one transaction and queue it.                                                         |

### carbon-stream-message-datasource

See [Stream message](https://carbon.sevenlabs.org/datasources/stream-message.md). Names use the `agave_grpc_` prefix. Sends wait for channel capacity, so updates are not dropped when the pipeline is slow, and the process-time histograms include that wait.

| Name                                                   | Type      | Unit        | Meaning                                                                       |
| ------------------------------------------------------ | --------- | ----------- | ----------------------------------------------------------------------------- |
| `agave_grpc_account_updates_received_total`            | counter   | updates     | Account messages forwarded as `Update::Account`.                              |
| `agave_grpc_account_deletions_received_total`          | counter   | updates     | Zero-lamport account messages forwarded as `Update::AccountDeletion`.         |
| `agave_grpc_transaction_updates_received_total`        | counter   | updates     | Transactions forwarded to the pipeline.                                       |
| `agave_grpc_account_process_time_nanoseconds`          | histogram | ns (ns set) | Time to forward one account message, including the wait for channel capacity. |
| `agave_grpc_account_deletion_process_time_nanoseconds` | histogram | ns (ns set) | Same, for deletions.                                                          |
| `agave_grpc_transaction_process_time_nanoseconds`      | histogram | ns (ns set) | Same, for transactions.                                                       |

### carbon-helius-atlas-ws-datasource

See [Helius Atlas WebSocket](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md). Counters are incremented before the non-blocking `try_send`, so they include dropped updates.

| Name                                                        | Type      | Unit        | Meaning                                                                                                                                              |
| ----------------------------------------------------------- | --------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `helius_atlas_ws_account_updates_received_total`            | counter   | updates     | Account notifications converted to `Update::Account`.                                                                                                |
| `helius_atlas_ws_account_deletions_received_total`          | counter   | updates     | Zero-lamport notifications converted to `Update::AccountDeletion`.                                                                                   |
| `helius_atlas_ws_transaction_updates_received_total`        | counter   | updates     | Transaction notifications converted to `Update::Transaction`.                                                                                        |
| `helius_atlas_ws_account_process_time_nanoseconds`          | histogram | ns (ns set) | Time to convert one account notification.                                                                                                            |
| `helius_atlas_ws_account_deletion_process_time_nanoseconds` | histogram | ns (ns set) | Same, for deletions.                                                                                                                                 |
| `helius_atlas_ws_transaction_process_time_nanoseconds`      | histogram | ns (ns set) | Time to convert one transaction notification.                                                                                                        |
| `helius_atlas_ws_clock_process_time_nanoseconds`            | histogram | ns (ns set) | Time to deserialize one Clock sysvar notification. The datasource watches the Clock sysvar to detect stalls and slot gaps and to trigger reconnects. |

### carbon-rpc-block-subscribe-datasource

See [RPC block subscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md). Failed transactions (non-`Ok` status) are skipped and not counted. If `try_send` fails, the remaining transactions of that block are dropped.

| Name                                                   | Type      | Unit         | Meaning                                                                   |
| ------------------------------------------------------ | --------- | ------------ | ------------------------------------------------------------------------- |
| `block_subscribe_transactions_processed_total`         | counter   | transactions | Transactions converted to `Update::Transaction`, counted before the send. |
| `block_subscribe_blocks_received_total`                | counter   | blocks       | Block notifications fully handled.                                        |
| `block_subscribe_transaction_process_time_nanoseconds` | histogram | ns (ns set)  | Time to decode and convert one transaction.                               |
| `block_subscribe_block_process_time_nanoseconds`       | histogram | ns (ns set)  | Time to handle one block notification, all transactions included.         |

### carbon-rpc-block-crawler-datasource

See [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md). Failed transactions are skipped and not counted. If `try_send` fails, the remaining transactions of that block are dropped.

| Name                                                 | Type      | Unit             | Meaning                                                                                                                                                 |
| ---------------------------------------------------- | --------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `block_crawler_blocks_fetched_total`                 | counter   | blocks           | Successful `getBlock` calls.                                                                                                                            |
| `block_crawler_blocks_skipped_total`                 | counter   | blocks           | `getBlock` calls that failed with RPC error `-32004`, `-32007` or `-32009` (skipped slot or block not available). Other errors are logged, not counted. |
| `block_crawler_blocks_received_total`                | counter   | blocks           | Fetched blocks picked up by the processing task.                                                                                                        |
| `block_crawler_blocks_processed_total`               | counter   | blocks           | Blocks whose transactions were all handled.                                                                                                             |
| `block_crawler_transactions_processed_total`         | counter   | transactions     | Transactions converted to `Update::Transaction`, counted before the send.                                                                               |
| `block_crawler_blocks_fetch_times_milliseconds`      | histogram | ms (ms-long set) | Duration of each successful `getBlock` call.                                                                                                            |
| `block_crawler_transaction_process_time_nanoseconds` | histogram | ns (ns set)      | Time to decode and convert one transaction.                                                                                                             |
| `block_crawler_block_process_time_nanoseconds`       | histogram | ns (ns set)      | Time to handle one block, all transactions included.                                                                                                    |

### carbon-rpc-transaction-crawler-datasource

See [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md).

| Name                                                        | Type      | Unit             | Meaning                                                                                                                  |
| ----------------------------------------------------------- | --------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `transaction_crawler_signatures_fetched_total`              | counter   | signatures       | Signatures returned by `getSignaturesForAddress`, summed over pages.                                                     |
| `transaction_crawler_transactions_fetched_total`            | counter   | transactions     | `getTransaction` requests that completed, including those that still failed after all retries.                           |
| `transaction_crawler_signatures_fetch_times_milliseconds`   | histogram | ms (ms-long set) | Time per `getSignaturesForAddress` page, including handing the signatures to the fetch stage.                            |
| `transaction_crawler_transaction_fetch_times_milliseconds`  | histogram | ms (ms-long set) | Time per successful `getTransaction`, measured from the first attempt, so retries and backoff are included.              |
| `transaction_crawler_transaction_process_time_milliseconds` | histogram | ms (ms-long set) | Time to convert one fetched transaction into an `Update::Transaction`. Failed transactions are skipped and not recorded. |

### carbon-rpc-gpa-datasource

See [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md).

| Name                                  | Type      | Unit              | Meaning                                           |
| ------------------------------------- | --------- | ----------------- | ------------------------------------------------- |
| `rpc_gpa_fetch_duration_milliseconds` | histogram | ms (ms-short set) | Duration of the single `getProgramAccounts` call. |
| `rpc_gpa_accounts_processed_total`    | counter   | accounts          | Accounts successfully sent to the pipeline.       |

### carbon-rpc-program-subscribe-datasource

See [RPC program subscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md). If `try_send` fails, the subscription loop exits.

| Name                                                 | Type      | Unit        | Meaning                                                                                               |
| ---------------------------------------------------- | --------- | ----------- | ----------------------------------------------------------------------------------------------------- |
| `program_subscribe_accounts_processed_total`         | counter   | accounts    | `programSubscribe` notifications converted to an account update or deletion, counted before the send. |
| `program_subscribe_account_process_time_nanoseconds` | histogram | ns (ns set) | Time to decode and convert one notification.                                                          |

### carbon-helius-gpa-v2-datasource

See [Helius getProgramAccountsV2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md).

| Name                                        | Type      | Unit              | Meaning                                     |
| ------------------------------------------- | --------- | ----------------- | ------------------------------------------- |
| `helius_gpa_v2_pages_fetched_total`         | counter   | pages             | Pages fetched from the paginated API.       |
| `helius_gpa_v2_accounts_processed_total`    | counter   | accounts          | Accounts successfully sent to the pipeline. |
| `helius_gpa_v2_fetch_duration_milliseconds` | histogram | ms (ms-short set) | Time to fetch one page.                     |

### carbon-helius-gtfa-datasource

See [Helius getTransactionsForAddress](https://carbon.sevenlabs.org/datasources/helius-gtfa.md).

| Name                                       | Type      | Unit              | Meaning                                         |
| ------------------------------------------ | --------- | ----------------- | ----------------------------------------------- |
| `helius_gtfa_pages_fetched_total`          | counter   | pages             | Pages fetched from the API.                     |
| `helius_gtfa_transactions_processed_total` | counter   | transactions      | Transactions successfully sent to the pipeline. |
| `helius_gtfa_fetch_duration_milliseconds`  | histogram | ms (ms-short set) | Time to fetch one page.                         |

### carbon-validator-snapshot-datasource

See [Validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md). Histogram buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000, 5000, 10000 ms. A snapshot load usually takes longer than 10 s, so expect it in the `+Inf` bucket.

| Name                                            | Type      | Unit     | Meaning                                                     |
| ----------------------------------------------- | --------- | -------- | ----------------------------------------------------------- |
| `validator_snapshot_load_duration_milliseconds` | histogram | ms       | Time to load the snapshot from a local path or download it. |
| `validator_snapshot_scan_duration_milliseconds` | histogram | ms       | Time to scan every account after loading.                   |
| `validator_snapshot_accounts_processed_total`   | counter   | accounts | Accounts successfully sent to the pipeline.                 |

### carbon-jetstreamer-datasource (Carbon 1.x)

`carbon-jetstreamer-datasource` is published at 1.0.0 on Carbon 1.x and is not compatible with carbon-core 2.0.0. Listed for completeness.

| Name                                          | Type    | Unit         | Meaning                                                   |
| --------------------------------------------- | ------- | ------------ | --------------------------------------------------------- |
| `jetstreamer_blocks_sent_total`               | counter | blocks       | Block details sent to the pipeline.                       |
| `jetstreamer_transactions_sent_total`         | counter | transactions | Transactions sent to the pipeline.                        |
| `jetstreamer_transactions_filtered_in_total`  | counter | transactions | Transactions that matched the datasource filters.         |
| `jetstreamer_transactions_filtered_out_total` | counter | transactions | Transactions that did not match the filters.              |
| `jetstreamer_internal_slots_processed`        | gauge   | slots        | Slots processed by the firehose, from its internal stats. |
| `jetstreamer_internal_blocks_processed`       | gauge   | blocks       | Blocks processed by the firehose.                         |
| `jetstreamer_internal_transactions_processed` | gauge   | transactions | Transactions processed by the firehose.                   |

### carbon-jito-shredstream-grpc-datasource (Carbon 1.x)

Also published at 1.0.0 on Carbon 1.x only.

| Name                                                   | Type      | Unit        | Meaning                               |
| ------------------------------------------------------ | --------- | ----------- | ------------------------------------- |
| `jito_shredstream_grpc_entry_updates_received_total`   | counter   | entries     | Entries received, summed per message. |
| `jito_shredstream_grpc_duplicate_entries_total`        | counter   | entries     | Duplicate entries skipped.            |
| `jito_shredstream_grpc_entry_process_time_nanoseconds` | histogram | ns (ns set) | Time to process one entries message.  |

## `Metric`

Name and help text shared by `Counter`, `Gauge` and `Histogram`.

```rust
pub trait Metric: Send + Sync {
    fn name(&self) -> &'static str;
    fn help(&self) -> &'static str { "" }
}
```

**Details.** Implemented by the three primitives. The registry stores the concrete types, not `dyn Metric`, so implementing `Metric` on your own type does not make it exportable.

**Example.**

```rust
use carbon_core::metrics::{Counter, Metric};

static SEEN: Counter = Counter::new("my_seen_total", "Things seen");
assert_eq!(SEEN.name(), "my_seen_total");
```

**See also.** [`Counter`](#counter), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/trait.Metric.html).

## `Counter`

A monotonically increasing `u64`.

```rust
pub struct Counter { /* private */ }

impl Counter {
    pub const fn new(name: &'static str, help: &'static str) -> Self;
    pub fn inc(&self);
    pub fn inc_by(&self, v: u64);
    pub fn get(&self) -> u64;
}
```

**Details.** `new` is `const`, so you can declare a counter as a plain `static`. `inc` and `inc_by` are a single relaxed atomic add, cheap enough for the hot path. A counter is exported only after you pass it to [`MetricsRegistry::register_counter`](#metricsregistry).

| Method            | Description                                                                                        |
| ----------------- | -------------------------------------------------------------------------------------------------- |
| `new(name, help)` | Creates a counter at 0. Use Prometheus-style `snake_case` names, ending in `_total` by convention. |
| `inc()`           | Adds 1.                                                                                            |
| `inc_by(v)`       | Adds `v`.                                                                                          |
| `get()`           | Returns the current value.                                                                         |

**Example.** See [Register custom metrics](#register-custom-metrics).

**See also.** [`Gauge`](#gauge), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.Counter.html).

## `Gauge`

An `f64` value that can go up and down.

```rust
pub struct Gauge { /* private */ }

impl Gauge {
    pub const fn new(name: &'static str, help: &'static str) -> Self;
    pub fn set(&self, value: f64);
    pub fn add(&self, delta: f64);
    pub fn get(&self) -> f64;
}
```

**Details.** Stored as the bit pattern of an `f64` in an `AtomicU64`. Starts at `0.0`. `add` accepts negative deltas.

| Method            | Description                                             |
| ----------------- | ------------------------------------------------------- |
| `new(name, help)` | Creates a gauge at 0. `const`, so usable in a `static`. |
| `set(value)`      | Replaces the value.                                     |
| `add(delta)`      | Adds `delta` atomically (compare-and-swap loop).        |
| `get()`           | Returns the current value.                              |

**Example.** See [Register custom metrics](#register-custom-metrics).

**See also.** [`Counter`](#counter), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.Gauge.html).

## `Histogram`

A bucketed distribution of `f64` observations with fixed upper bounds.

```rust
pub struct Histogram { /* private */ }

impl Histogram {
    pub fn new(name: &'static str, help: &'static str, boundaries: Vec<f64>) -> Self;
    pub fn record(&self, value: f64);
    pub fn get(&self) -> HistogramSnapshot;
}
```

**Details.** `boundaries` are inclusive upper bounds and must be sorted ascending. The histogram keeps `boundaries.len() + 1` buckets: a value goes into the first bucket whose bound is `>=` the value, and values above the last bound go into the final overflow bucket. `record` also adds the value to a running sum. Bucket counts are per bucket, not cumulative; `PrometheusMetrics` makes them cumulative when it renders them.

`new` allocates, so it is not `const`. Wrap a static histogram in `std::sync::LazyLock`.

| Method                        | Description                                          |
| ----------------------------- | ---------------------------------------------------- |
| `new(name, help, boundaries)` | Creates an empty histogram.                          |
| `record(value)`               | Counts `value` in its bucket and adds it to the sum. |
| `get()`                       | Returns a [`HistogramSnapshot`](#histogramsnapshot). |

**Example.** See [Register custom metrics](#register-custom-metrics).

**See also.** [`HistogramSnapshot`](#histogramsnapshot), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.Histogram.html).

## `HistogramSnapshot`

Point-in-time copy of a histogram.

```rust
pub struct HistogramSnapshot {
    pub counts: Vec<u64>,
    pub boundaries: Vec<f64>,
    pub sum: f64,
}
```

**Details.**

| Field        | Type       | Meaning                                                                                                       |
| ------------ | ---------- | ------------------------------------------------------------------------------------------------------------- |
| `counts`     | `Vec<u64>` | Observations per bucket, not cumulative. Has `boundaries.len() + 1` entries; the last is the overflow bucket. |
| `boundaries` | `Vec<f64>` | Bucket upper bounds, as passed to `Histogram::new`.                                                           |
| `sum`        | `f64`      | Sum of all recorded values.                                                                                   |

The total count is `counts.iter().sum::<u64>()`.

**Example.**

```rust
let h = SWAP_SIZE.get();
let total: u64 = h.counts.iter().sum();
println!("{} observations, mean {}", total, h.sum / total.max(1) as f64);
```

**See also.** [`MetricsSnapshot`](#metricssnapshot), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.HistogramSnapshot.html).

## `MetricsSnapshot`

Point-in-time values of every registered metric, passed to [`MetricsExporter::export`](#metricsexporter).

```rust
pub struct MetricsSnapshot {
    pub counters: Vec<(&'static str, &'static str, u64)>,
    pub gauges: Vec<(&'static str, &'static str, f64)>,
    pub histograms: Vec<(&'static str, &'static str, HistogramSnapshot)>,
}
```

**Details.** Each tuple is `(name, help, value)`. Entries are in registration order. A metric registered twice appears twice.

**Example.**

```rust
let snapshot = carbon_core::metrics::MetricsRegistry::global().snapshot();
let received = snapshot
    .counters
    .iter()
    .find(|(name, _, _)| *name == "carbon_updates_received_total")
    .map(|(_, _, v)| *v)
    .unwrap_or(0);
```

**See also.** [`MetricsRegistry`](#metricsregistry), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.MetricsSnapshot.html).

## `MetricsRegistry`

The process-wide registry that exporters read from.

```rust
pub struct MetricsRegistry { /* private */ }

impl MetricsRegistry {
    pub fn global() -> &'static MetricsRegistry;
    pub fn register_counter(&self, counter: &'static Counter);
    pub fn register_gauge(&self, gauge: &'static Gauge);
    pub fn register_histogram(&self, histogram: &'static Histogram);
    pub fn snapshot(&self) -> MetricsSnapshot;
}
```

**Details.** There is one registry per process, returned by `global()`; you cannot create another. It holds `&'static` references, so metrics must live in `static`s. Registration appends without deduplication (see the [warning above](#how-metrics-are-collected)). There is no way to unregister a metric. The internal locks panic with `metrics counters lock poisoned` (or `gauges`, `histograms`) if a thread panicked while holding them.

**Example.** See [Register custom metrics](#register-custom-metrics).

**See also.** [`MetricsSnapshot`](#metricssnapshot), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/struct.MetricsRegistry.html).

### Register custom metrics

Declare metrics as statics, register them once at startup (before `Pipeline::run`), and update them from your processors. Every exporter picks them up with the built-in metrics.

`src/main.rs`:

```rust
use {
    carbon_core::metrics::{Counter, Gauge, Histogram, MetricsRegistry},
    std::sync::LazyLock,
};

static SWAPS_SEEN: Counter = Counter::new("my_swaps_seen_total", "Swaps decoded by my processor");
static LAST_SLOT: Gauge = Gauge::new("my_last_slot", "Slot of the last processed swap");
static SWAP_SIZE: LazyLock<Histogram> = LazyLock::new(|| {
    Histogram::new(
        "my_swap_amount_in",
        "Swap input amount in lamports",
        vec![1_000.0, 1_000_000.0, 1_000_000_000.0],
    )
});

fn main() {
    let registry = MetricsRegistry::global();
    registry.register_counter(&SWAPS_SEEN);
    registry.register_gauge(&LAST_SLOT);
    registry.register_histogram(&SWAP_SIZE);

    SWAPS_SEEN.inc();
    SWAPS_SEEN.inc_by(2);
    LAST_SLOT.set(312_000_000.0);
    SWAP_SIZE.record(500.0);
    SWAP_SIZE.record(2_500_000.0);

    let snapshot = registry.snapshot();
    for (name, _help, value) in &snapshot.counters {
        println!("counter {name} = {value}");
    }
    for (name, _help, value) in &snapshot.gauges {
        println!("gauge {name} = {value}");
    }
    for (name, _help, h) in &snapshot.histograms {
        println!("histogram {name}: counts={:?} boundaries={:?} sum={}", h.counts, h.boundaries, h.sum);
    }
}
```

Output:

```text
counter my_swaps_seen_total = 3
gauge my_last_slot = 312000000
histogram my_swap_amount_in: counts=[1, 0, 1, 0] boundaries=[1000.0, 1000000.0, 1000000000.0] sum=2500500
```

This program needs only `carbon-core = "2.0.0"`.

## `MetricsExporter`

A sink for metrics snapshots, registered with [`PipelineBuilder::metrics`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#metrics).

```rust
pub trait MetricsExporter: Send + Sync {
    fn initialize(self: Arc<Self>) -> CarbonResult<()> { Ok(()) }
    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()>;
    fn shutdown(&self) -> CarbonResult<()> { Ok(()) }
}
```

**Details.** `Pipeline::run` calls these methods at fixed points:

| Method       | When the pipeline calls it                                                                        | Notes                                                                                                                                                   |
| ------------ | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `initialize` | Once per exporter, at the start of `run`, inside the Tokio runtime, before any datasource starts. | Spawn periodic work here. Returning `Err` makes `run` return that error.                                                                                |
| `export`     | Once per exporter, at shutdown, with a final snapshot.                                            | The pipeline does not call `export` per update or on a timer. The `carbon-core` doc comment saying it runs "after each processed update" is inaccurate. |
| `shutdown`   | Once per exporter, after the final `export`.                                                      | Stop background tasks here.                                                                                                                             |

The final `export` and `shutdown` run on every shutdown path: Ctrl-C, cancellation through [`datasource_cancellation_token`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#datasource_cancellation_token), and all datasources finishing. An `Err` from either makes `run` return early with that error.

**Example.** An exporter that prints counters every 10 seconds and once more at shutdown:

`src/exporter.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult,
        metrics::{MetricsExporter, MetricsRegistry, MetricsSnapshot},
    },
    std::{sync::Arc, time::Duration},
};

/// Prints every counter to stdout every 10 seconds and once more at shutdown.
struct StdoutExporter;

impl MetricsExporter for StdoutExporter {
    fn initialize(self: Arc<Self>) -> CarbonResult<()> {
        // The pipeline only calls `export` at shutdown, so run your own loop
        // for periodic exports.
        tokio::spawn(async move {
            let mut interval = tokio::time::interval(Duration::from_secs(10));
            loop {
                interval.tick().await;
                let _ = self.export(&MetricsRegistry::global().snapshot());
            }
        });
        Ok(())
    }

    fn export(&self, snapshot: &MetricsSnapshot) -> CarbonResult<()> {
        for (name, _help, value) in &snapshot.counters {
            println!("{name} {value}");
        }
        Ok(())
    }
}
```

Register it with `.metrics(Arc::new(StdoutExporter))`.

**See also.** [`LogMetrics`](#logmetrics), [`PrometheusMetrics`](#prometheusmetrics), [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/metrics/trait.MetricsExporter.html).

## `LogMetrics`

An exporter that writes a summary and every metric to the [`log`](https://docs.rs/log) crate at `info` level. Crate: `carbon-log-metrics = "2.0.0"`.

```rust
pub struct LogMetrics { /* private */ }

impl LogMetrics {
    pub fn new() -> Self;
    pub fn new_with_flush_interval(interval_secs: u64) -> Self;
}

impl Default for LogMetrics { /* same as new() */ }
impl MetricsExporter for LogMetrics { /* … */ }
```

**Details.**

| Constructor                                          | Flush interval                                                                                                           |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `LogMetrics::new()` / `LogMetrics::default()`        | 5 seconds                                                                                                                |
| `LogMetrics::new_with_flush_interval(interval_secs)` | `interval_secs` seconds. Must be at least 1: a value of `0` makes the background task panic, and periodic logging stops. |

- `initialize` spawns a Tokio task that logs a snapshot every interval. The first tick fires immediately, so you see one block at startup.
- The pipeline's shutdown `export` logs one last block, then `shutdown` cancels the task.
- Output goes through `log::info!` with target `carbon_log_metrics`. Nothing is printed unless you install a logger (for example `env_logger`) with `info` enabled for that target.
- Each block starts with a summary line: `HH:MM:SS (+time since last flush) | N processed (P%), N successful, N failed (F%), Q in queue`. The processed percentage is `carbon_updates_processed_total` divided by `carbon_updates_received_total` plus `carbon_updates_queued`; the failed percentage is relative to processed.
- Then one line per counter and gauge (`  name: value`) and one per histogram (`  name: count=N`; buckets and sum are not logged).

**Example.** This program runs a pipeline with a datasource that emits three empty blocks and then returns, which closes the channel and ends the pipeline.

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{BlockDetails, Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    std::sync::Arc,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Emits three empty blocks, then returns, which ends the pipeline.
struct ThreeBlocks;

#[async_trait]
impl Datasource for ThreeBlocks {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        _cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        for slot in 1..=3 {
            let block = BlockDetails {
                slot,
                block_hash: None,
                previous_block_hash: None,
                rewards: None,
                num_reward_partitions: None,
                block_time: None,
                block_height: None,
            };
            let _ = sender.send((Update::BlockDetails(block), id.clone())).await;
        }
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::BlockDetails]
    }
}

struct Noop;

impl Processor<BlockDetails> for Noop {
    async fn process(&mut self, _data: &BlockDetails) -> CarbonResult<()> {
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

    Pipeline::builder()
        .datasource(ThreeBlocks)
        .block_details(Noop)
        .metrics(Arc::new(LogMetrics::new()))
        .build()?
        .run()
        .await
}
```

`Cargo.toml`:

```toml
[dependencies]
async-trait = "0.1.89"
carbon-core = "2.0.0"
carbon-log-metrics = "2.0.0"
env_logger = "0.11.8"
tokio = { version = "1.48.0", features = ["full"] }
tokio-util = "0.7.17"
```

```sh
cargo run
```

Output:

```text
[2026-11-08T23:53:11Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 0, num_transaction_pipes: 0
[2026-11-08T23:53:11Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-08T23:53:11Z INFO  carbon_log_metrics] 00:00:00 (+4.145248ms) | 3 processed (100%), 3 successful, 0 failed (0%), 0 in queue
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_received_total: 3
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_processed_total: 3
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_successful_total: 3
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_failed_total: 0
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_account_updates_processed_total: 0
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_transaction_updates_processed_total: 0
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_account_deletions_processed_total: 0
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_block_details_processed_total: 3
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_queued: 0
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_process_time_nanoseconds: count=3
[2026-11-08T23:53:11Z INFO  carbon_log_metrics]   carbon_updates_process_time_milliseconds: count=3
[2026-11-08T23:53:11Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

The pipeline ended before the first periodic tick ran, so only the shutdown export appears. With the carbon-core `postgres` feature enabled, the six `postgres_*` metrics are listed too.

**See also.** [`MetricsExporter`](#metricsexporter), [docs.rs](https://docs.rs/carbon-log-metrics/2.0.0/carbon_log_metrics/struct.LogMetrics.html).

## `PrometheusMetrics`

An exporter that serves the registry in the Prometheus text format over HTTP. Crate: `carbon-prometheus-metrics = "2.0.0"`.

```rust
pub struct PrometheusMetrics { /* private */ }

impl PrometheusMetrics {
    pub fn new() -> Self;
    pub fn with_server(config: PrometheusServerConfig) -> Self;              // feature "http-server"
    pub fn start_server(
        config: PrometheusServerConfig,
    ) -> tokio::task::JoinHandle<Result<(), Box<dyn std::error::Error + Send + Sync>>>; // feature "http-server"
    pub fn prometheus_text() -> String;
    pub fn format_snapshot(snapshot: &MetricsSnapshot) -> String;
}

impl Default for PrometheusMetrics { /* same as new() */ }
impl MetricsExporter for PrometheusMetrics { /* … */ }
```

**Details.**

| Item                        | Behaviour                                                                                                                                                                        |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `new()` / `default()`       | An exporter that does nothing: no server is started and `export` is a no-op. Use it only if you serve `prometheus_text()` yourself or call `start_server`.                       |
| `with_server(config)`       | When the pipeline calls `initialize`, spawns an HTTP server on `config.listen_addr`. Bind errors are logged (`Prometheus metrics server error: …`) and do not stop the pipeline. |
| `start_server(config)`      | Spawns the same server immediately on the current Tokio runtime and returns its `JoinHandle`. Independent of any pipeline.                                                       |
| `prometheus_text()`         | Renders a fresh snapshot of the global registry.                                                                                                                                 |
| `format_snapshot(snapshot)` | Renders a given snapshot.                                                                                                                                                        |
| `export`                    | No-op. Values are read live on each scrape, so the shutdown export changes nothing.                                                                                              |
| `shutdown`                  | Default no-op. The server stops on its own when the process receives Ctrl-C; it is not stopped by a cancellation token.                                                          |

The server:

- Serves `GET /metrics` only, with `content-type: text/plain; charset=utf-8; version=0.0.4` and `cache-control: no-cache, no-store, must-revalidate`.
- Applies a request timeout (`408 Request Timeout` after `request_timeout_secs`) and, when `enable_cors` is set, a permissive CORS layer.
- Logs `Prometheus metrics server starting on http://<addr>/metrics` at startup.

Rendering:

- Characters other than ASCII letters, digits, `_` and `:` in names are replaced with `_`. All built-in names are already valid.
- `# HELP` is written when the help text is non-empty, then `# TYPE` (`counter`, `gauge` or `histogram`).
- Histograms are written as cumulative `<name>_bucket{le="…"}` lines, a `le="+Inf"` bucket, `<name>_count` and `<name>_sum`. There are no other labels.

**Example.** Serve metrics on `127.0.0.1:9464` for the lifetime of the pipeline. The datasource emits three blocks and then waits for Ctrl-C.

`src/main.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{BlockDetails, Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
        pipeline::Pipeline,
        processor::Processor,
    },
    carbon_prometheus_metrics::{PrometheusMetrics, PrometheusServerConfig},
    std::sync::Arc,
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

/// Emits three empty blocks, then waits for shutdown.
struct ThreeBlocks;

#[async_trait]
impl Datasource for ThreeBlocks {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        for slot in 1..=3 {
            let block = BlockDetails {
                slot,
                block_hash: None,
                previous_block_hash: None,
                rewards: None,
                num_reward_partitions: None,
                block_time: None,
                block_height: None,
            };
            let _ = sender.send((Update::BlockDetails(block), id.clone())).await;
        }
        cancellation_token.cancelled().await;
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::BlockDetails]
    }
}

struct Noop;

impl Processor<BlockDetails> for Noop {
    async fn process(&mut self, _data: &BlockDetails) -> CarbonResult<()> {
        Ok(())
    }
}

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

    let config = PrometheusServerConfig::new().listen_addr("127.0.0.1:9464".parse().unwrap());

    Pipeline::builder()
        .datasource(ThreeBlocks)
        .block_details(Noop)
        .metrics(Arc::new(PrometheusMetrics::with_server(config)))
        .build()?
        .run()
        .await
}
```

Add `carbon-prometheus-metrics = "2.0.0"` to the dependencies from the [`LogMetrics` example](#logmetrics). Start it with `cargo run`, then scrape it from another terminal:

```sh
curl -s http://127.0.0.1:9464/metrics
```

Output:

```text
# HELP carbon_updates_received_total Total updates pulled from datasources
# TYPE carbon_updates_received_total counter
carbon_updates_received_total 3
# HELP carbon_updates_processed_total Total updates processed by the pipeline
# TYPE carbon_updates_processed_total counter
carbon_updates_processed_total 3
…
# HELP carbon_updates_queued Current number of updates waiting in queue
# TYPE carbon_updates_queued gauge
carbon_updates_queued 0
# HELP carbon_updates_process_time_nanoseconds Time taken to process updates in nanoseconds
# TYPE carbon_updates_process_time_nanoseconds histogram
carbon_updates_process_time_nanoseconds_bucket{le="1000"} 2
carbon_updates_process_time_nanoseconds_bucket{le="10000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="100000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="1000000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="10000000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="100000000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="1000000000"} 3
carbon_updates_process_time_nanoseconds_bucket{le="+Inf"} 3
carbon_updates_process_time_nanoseconds_count 3
carbon_updates_process_time_nanoseconds_sum 7030
…
```

Press Ctrl-C to stop both the pipeline and the server.

**See also.** [`PrometheusServerConfig`](#prometheusserverconfig), [Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md), [docs.rs](https://docs.rs/carbon-prometheus-metrics/2.0.0/carbon_prometheus_metrics/struct.PrometheusMetrics.html).

## `PrometheusServerConfig`

Configuration for the `/metrics` HTTP server. Requires the `http-server` feature (on by default).

```rust
#[derive(Debug, Clone)]
pub struct PrometheusServerConfig {
    pub listen_addr: std::net::SocketAddr,
    pub enable_cors: bool,
    pub request_timeout_secs: u64,
}

impl PrometheusServerConfig {
    pub fn new() -> Self;
    pub fn listen_addr(self, addr: std::net::SocketAddr) -> Self;
    pub fn enable_cors(self, enable: bool) -> Self;
    pub fn request_timeout_secs(self, timeout: u64) -> Self;
}
```

**Details.** `new()` and `Default` give these values; each builder method replaces one.

| Field                  | Type         | Default        | Description                                                      |
| ---------------------- | ------------ | -------------- | ---------------------------------------------------------------- |
| `listen_addr`          | `SocketAddr` | `0.0.0.0:9464` | Address and port to bind. The default listens on all interfaces. |
| `enable_cors`          | `bool`       | `true`         | Adds a permissive CORS layer (`access-control-allow-origin: *`). |
| `request_timeout_secs` | `u64`        | `30`           | Requests running longer get `408 Request Timeout`.               |

**Example.** Serve metrics yourself, without a pipeline exporter:

`src/main.rs`:

```rust
use carbon_prometheus_metrics::{PrometheusMetrics, PrometheusServerConfig};

#[tokio::main]
async fn main() {
    // Serve /metrics yourself instead of through the exporter.
    let server = PrometheusMetrics::start_server(
        PrometheusServerConfig::new()
            .listen_addr("0.0.0.0:9100".parse().unwrap())
            .enable_cors(false)
            .request_timeout_secs(10),
    );

    // Or render the exposition text directly, e.g. from your own HTTP handler.
    let text: String = PrometheusMetrics::prometheus_text();
    println!("{text}");

    server.abort();
}
```

**See also.** [`run_metrics_server`](#run_metrics_server), [docs.rs](https://docs.rs/carbon-prometheus-metrics/2.0.0/carbon_prometheus_metrics/struct.PrometheusServerConfig.html).

## `run_metrics_server`

Free functions that run the `/metrics` server on the current task until Ctrl-C. Require the `http-server` feature.

```rust
pub async fn run_metrics_server(
    listen_addr: std::net::SocketAddr,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;

pub async fn run_default_metrics_server() -> Result<(), Box<dyn std::error::Error + Send + Sync>>;

pub async fn run_metrics_server_with_config(
    config: PrometheusServerConfig,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
```

**Details.** `run_metrics_server` uses `listen_addr` with the other defaults; `run_default_metrics_server` uses `PrometheusServerConfig::default()` (`0.0.0.0:9464`). They return `Err` only if binding fails. A server error after startup is logged and the function returns `Ok(())`. Each function also returns `Ok(())` after Ctrl-C. `PrometheusMetrics::with_server` and `start_server` call `run_metrics_server_with_config` inside `tokio::spawn`.

**Example.**

```rust
tokio::spawn(carbon_prometheus_metrics::run_metrics_server(
    "127.0.0.1:9464".parse().unwrap(),
));
```

**See also.** [`PrometheusMetrics`](#prometheusmetrics), [docs.rs](https://docs.rs/carbon-prometheus-metrics/2.0.0/carbon_prometheus_metrics/fn.run_metrics_server.html).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/metrics · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Errors (Carbon 2.0)

> Every carbon_core::error::Error variant in Carbon 2.0.0 with its message, cause and fix, plus common errors from RPC, rustls and Cargo.

This page lists every variant of `carbon_core::error::Error` in Carbon 2.0.0 as **error → cause → fix**, followed by errors that come from Carbon's dependencies but show up in Carbon pipelines. For how errors flow through a pipeline, see [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md).

Source: [`crates/core/src/error.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/error.rs).

## How Carbon reports errors

```rust
#[derive(Error, Debug)]
pub enum Error {
    #[error("Missing update type in datasource")]
    MissingUpdateTypeInDatasource(UpdateType),
    #[error("Failed to receive updates({0})")]
    FailedToReceiveUpdates(String),
    #[error("Transaction missing fee payer")]
    MissingFeePayer,
    #[error("Missing inner instructions")]
    MissingInnerInstructions,
    #[error("Missing account in transaction")]
    MissingAccountInTransaction,
    #[error("Missing instruction data")]
    MissingInstructionData,
    #[error("Failed to consume datasource ({0})")]
    FailedToConsumeDatasource(String),
    #[error("Custom error: {0}")]
    Custom(String),
}

pub type CarbonResult<T> = Result<T, Error>;
```

The pipeline logs errors with `{:?}`, so logs show the **Debug** form (`FailedToConsumeDatasource("Failed to fetch program accounts")`), not the `Display` message. Both forms are listed below. Where an error surfaces decides what happens next:

| Where it is returned                   | Log line (`error` level, target `carbon_core::pipeline`) | Effect                                                                                                                          |
| -------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `Datasource::consume`                  | `error consuming datasource: <Debug>`                    | That datasource stops. The others keep running. When none are left, the pipeline drains the channel and `run` returns `Ok(())`. |
| A decoder step or `Processor::process` | `error processing update (<update Debug>): <Debug>`      | The update is counted in `carbon_updates_failed_total`; the remaining pipes for that update are skipped; processing continues.  |
| A metrics exporter                     | none                                                     | `Pipeline::run` returns the error.                                                                                              |

> **Warning: A failed datasource does not fail the process.** `Pipeline::run` returns `Ok(())` after its last datasource returns an error, so a binary that ends with `.run().await` exits with status 0. Check the logs, or alert on `carbon_updates_received_total` staying flat.

Enable logs with a logger such as `env_logger` and `RUST_LOG=info` (Carbon's own messages are at `info`, `warn` and `error`).

## `FailedToConsumeDatasource`

**Error.** `Failed to consume datasource (<message>)`. Logged as `error consuming datasource: FailedToConsumeDatasource("<message>")`.

**Cause.** A datasource's `consume` could not start or continue. Built-in producers and their messages:

| Datasource                           | Messages                                                                                                                                                                                                                                                                                                                             |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| RPC GPA                              | `Failed to fetch program accounts` (any RPC or deserialization failure of `getProgramAccounts`, including rate limits and timeouts); `Failed to parse pubkey: …`; `Failed to decode account`                                                                                                                                         |
| Helius GPA V2                        | `HTTP request failed: …`, `Failed to parse response: …`, `RPC error: <message> (code: <code>)`, `Response missing context or value field`                                                                                                                                                                                            |
| Helius GTFA                          | `Failed to serialize request config: …`, `HTTP request failed: …`, `HTTP error <status>: <body>`, `Failed to parse response: …`, `RPC error: <message> (code: <code>)`, `Response missing result field`                                                                                                                              |
| Yellowstone gRPC, Helius LaserStream | The `tonic` or client error text when building the endpoint, setting `x_token`, or connecting (invalid URL, DNS, TLS, refused connection, bad token)                                                                                                                                                                                 |
| Validator snapshot                   | `Ledger directory does not exist: …`, `Genesis archive not found: …. Expected genesis.tar.bz2`, `No loadable full snapshot archive found in …`, `Failed to load bank forks from snapshot: …`, `Invalid snapshot: loaded bank has slot 0, …`, download errors such as `Failed to download <url> after 3 retries: …`, `Scan failed: …` |

**Fix.** Read the message after the variant name; it names the failing step.

1. For connection errors, check the endpoint URL and scheme (`https://` for TLS gRPC), the token, and network access.
2. For `Failed to fetch program accounts` on the public endpoint, add `filters` (such as a `dataSize` or `memcmp` on the discriminator) to `RpcProgramAccountsConfig`: unfiltered `getProgramAccounts` on large programs times out or is rejected. Use a paid RPC for large programs.
3. For snapshot errors, check the paths and that the archive matches the cluster.
4. Restart the pipeline, or wrap the datasource in your own retry loop: datasources don't restart themselves after `consume` returns.

**See also.** [Datasources](https://carbon.sevenlabs.org/datasources.md), [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md), [Load a snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md).

## `Custom`

**Error.** `Custom error: <message>`. Logged as `Custom("<message>")`.

**Cause.** A catch-all used by:

| Producer                                            | Message                                                                                                                                                                          |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| RPC block subscribe                                 | `Failed to create RPC subscribe client after 10 attempts: …`, `Failed to subscribe after 10 attempts: …`                                                                         |
| RPC program subscribe                               | `Failed to create RPC subscribe client after 10 attempts: …`, `Failed to subscribe after 10 attempts: …`                                                                         |
| Helius Atlas WebSocket                              | `Error creating Filters for the Helius WebSocket: accounts and transactions can't be both empty`, `Failed to create Helius client after … attempts: …`                           |
| `carbon_core::postgres` and generated Postgres code | The `sqlx` error text, or `out of range`, `invalid numeric string`, `value exceeds maximum for target type`, `invalid scale for i128`, `invalid sign/scale for unsigned integer` |
| Your code                                           | Anything you wrap, for example `Error::Custom(e.to_string())`                                                                                                                    |

The subscribe datasources wait 3 seconds between attempts.

**Fix.** For subscribe failures, read the wrapped error: `Method not found (-32601)` means the endpoint doesn't support the method (see [`blockSubscribe` is not supported](#method-not-found-for-blocksubscribe)); connection errors mean a wrong `wss://` URL or a blocked network. For Atlas, pass at least one account or transaction filter. For Postgres errors, see the [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md). In your own code, prefer `Custom` with a descriptive message over `unwrap`, so the pipeline logs the failure and continues.

**See also.** [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md), [Postgres](https://carbon.sevenlabs.org/guides/postgres.md).

## `MissingFeePayer`

**Error.** `Transaction missing fee payer`. Logged as `error processing update (Transaction(…)): MissingFeePayer`.

**Cause.** The pipeline converts each `TransactionUpdate` into `TransactionMetadata` and takes the fee payer from the first static account key of the message. The message has no account keys. Built-in datasources never send such transactions; it happens when a custom datasource builds a `VersionedTransaction` with an empty or default message.

**Fix.** Build `TransactionUpdate::transaction` from the full decoded transaction (for RPC, `EncodedTransaction::decode()` on a Base64 payload), not from `VersionedTransaction::default()`. The update is skipped and the pipeline continues.

**See also.** [`TransactionUpdate`](https://carbon.sevenlabs.org/reference/updates.md#transactionupdate), [Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md).

## `MissingAccountInTransaction`

**Error.** `Missing account in transaction`. Logged as `MissingAccountInTransaction`.

**Cause.** Returned only by the public helper `carbon_core::transformers::extract_account_metas(compiled_instruction, message)` when an account index points past the message's **static** account keys. The pipeline does not call this helper. It happens when you call it on a V0 transaction whose instruction uses accounts from an address lookup table, because the helper ignores `meta.loaded_addresses`.

**Fix.** Use the instruction the pipeline already built (`raw_instruction` in instruction processors, or `NestedInstruction::instruction`), whose accounts include lookup-table addresses. If you need the helper, append `meta.loaded_addresses.writable` and `readonly` to the keys yourself.

**See also.** [`InstructionProcessorInputType`](https://carbon.sevenlabs.org/reference/processor-inputs.md#instructionprocessorinputtype).

## `MissingUpdateTypeInDatasource`

**Error.** `Missing update type in datasource`. Logged as `MissingUpdateTypeInDatasource(<UpdateType>)`.

**Cause.** Nothing in Carbon 2.0.0 returns this variant. It is reserved for checking a datasource's `update_types()` against registered pipes, which the 2.0.0 pipeline does not do.

**Fix.** If you see it, it comes from your own code or a third-party crate; read that code. Carbon itself does not validate `update_types()`.

**See also.** [`UpdateType`](https://carbon.sevenlabs.org/reference/updates.md#updatetype).

## `FailedToReceiveUpdates`

**Error.** `Failed to receive updates(<message>)` (no space before the parenthesis). Logged as `FailedToReceiveUpdates("<message>")`.

**Cause.** Not returned anywhere in Carbon 2.0.0. Available for custom datasources that lose their upstream stream.

**Fix.** Read the wrapped message from the datasource that returned it. If you write a datasource, either variant is fine; `FailedToConsumeDatasource` is what the built-in ones use.

**See also.** [`Datasource`](https://carbon.sevenlabs.org/reference/traits.md#datasource).

## `MissingInnerInstructions`

**Error.** `Missing inner instructions`. Logged as `MissingInnerInstructions`.

**Cause.** Not returned anywhere in Carbon 2.0.0. When `meta.inner_instructions` is `None`, the pipeline treats the transaction as having no CPIs instead of failing.

**Fix.** If CPIs are missing from your processors, the source did not send inner instructions: for custom RPC datasources, request `transaction_details: Full`; for Yellowstone, use a full transaction subscription. See also [invalid inner instruction stack height](#invalid-inner-instruction-stack-height).

**See also.** [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md).

## `MissingInstructionData`

**Error.** `Missing instruction data`. Logged as `MissingInstructionData`.

**Cause.** Not returned anywhere in Carbon 2.0.0.

**Fix.** Read the code that returned it; decoders signal "not mine" by returning `None`, not this error.

**See also.** [`InstructionDecoder`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecoder).

## `ConversionError`

**Error.** A separate error type, `carbon_core::transformers::yellowstone::ConversionError` (behind carbon-core's `yellowstone` feature). Its variants and `Display` messages:

```text
MissingField(field)                       missing required protobuf field `<field>`
InvalidByteLength { field, expected, actual }  invalid byte length for `<field>`: expected <expected>, got <actual>
IntegerOutOfRange { field, value }        value <value> for `<field>` does not fit in a u8
InvalidEnum { field, value }              invalid enum value <value> for `<field>`
InvalidNumber { field, value }            invalid numeric value `<value>` for `<field>`
InvalidTransactionError(_)                failed to decode the protobuf transaction error with wincode
```

**Cause.** The Yellowstone and LaserStream datasources convert protobuf transactions with `create_tx_versioned` and `create_tx_meta`. A malformed or incompatible message fails the conversion. They log `Failed to create transaction meta: <Debug>` and skip that transaction; a failed `create_tx_versioned` skips the transaction silently.

**Fix.** Usually a server running a Yellowstone protocol version that doesn't match `yellowstone-grpc-proto` 12.7 (Carbon 2.0.0's pin). Check the provider's version. The pipeline continues with the next transaction.

**See also.** [Stream with Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md).

## Related errors

These errors don't come from `carbon_core::error::Error`, but readers hit them while running Carbon pipelines.

### `Method not found` for `blockSubscribe`

**Error.** With `RpcBlockSubscribe` on `wss://api.mainnet-beta.solana.com`:

```text
[2026-11-09T00:15:41Z ERROR carbon_rpc_block_subscribe_datasource] Failed to subscribe to block updates: SubscribeFailed { reason: "Method not found (-32601)", message: "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":-32601,\"message\":\"Method not found\"},\"id\":1}" }
[2026-11-09T00:16:42Z ERROR carbon_core::pipeline] error consuming datasource: Custom("Failed to subscribe after 10 attempts: subscribe failed: Method not found (-32601)")
[2026-11-09T00:16:42Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:16:42Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

**Cause.** `blockSubscribe` is an unstable RPC method that validators only serve with `--rpc-pubsub-enable-block-subscription`. Solana's public endpoints don't enable it. The datasource retries 11 times, 3 seconds apart, then gives up, and the pipeline exits with status 0.

**Fix.** Use an RPC provider that supports `blockSubscribe`, or switch datasource: the [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) works on the public endpoint for one program's history, and [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) or [Helius LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md) stream every block.

**See also.** [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md).

### `429 Too Many Requests`

**Error.** With the RPC transaction crawler (or GPA) on the public endpoint, at `RUST_LOG=debug`:

```text
[2026-11-09T00:18:44Z DEBUG solana_rpc_client::http_sender] Too many requests: server responded with Response { url: "https://api.mainnet-beta.solana.com/", status: 429, headers: {…, "retry-after": "10", …} }
```

and, once the client's own retries are exhausted, at `warn` and `error` level:

```text
[2026-11-09T00:19:39Z WARN  carbon_rpc_transaction_crawler_datasource] Failed to fetch transaction 2RPWo3shvDtwGNSD2tY79uwFJ6Gbx1M7g5knfsJEDpcdahojCL8csGJV6KCEyvzZC2UhBnctSPcFSwSPr4i4jHct (attempt 1/3), retrying in 1000ms: Error { request: Some(GetTransaction), kind: Reqwest(reqwest::Error { kind: Status(429, None), url: "https://api.mainnet-beta.solana.com/" }) }
```

**Cause.** The public endpoint rate-limits per IP. `solana-rpc-client` retries 429s internally (logged at `debug`), then the crawler retries with backoff (`RetryConfig::default()`: 3 retries, 1 s initial, ×2, 10 s max). After the last retry it logs `Failed to fetch transaction <sig> after 3 retries: …` and **skips that transaction**.

**Fix.**

1. Lower `ConnectionConfig::max_concurrent_requests` (default 5) and raise `polling_interval`.
2. Raise `RetryConfig::max_retries` and the backoff so fewer transactions are skipped.
3. For anything beyond a demo, use a paid RPC endpoint.

**See also.** [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md), [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md).

### `Could not automatically determine the process-level CryptoProvider`

**Error.** At startup, with the Yellowstone gRPC or Helius LaserStream datasource and a TLS (`https://`) endpoint:

```text
thread 'tokio-rt-worker' panicked at /home/you/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rustls-0.23.46/src/crypto/mod.rs:249:14:

Could not automatically determine the process-level CryptoProvider from Rustls crate features.
Call CryptoProvider::install_default() before this point to select a provider manually, or make sure exactly one of the 'aws-lc-rs' and 'ring' features is enabled.
See the documentation of the CryptoProvider type for more information.

[2026-11-09T00:51:58Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:51:58Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

**Cause.** The gRPC stack pulls in `rustls` 0.23 with both the `ring` and `aws-lc-rs` backends, so rustls can't pick one. The panic happens inside the datasource's task, so the pipeline logs a clean shutdown and exits with status 0.

**Fix.** Install a provider once at the start of `main`, before building the datasource:

`Cargo.toml`:

```toml
[dependencies]
rustls = { version = "0.23", default-features = false, features = ["aws-lc-rs"] }
```

`src/main.rs`:

```rust
rustls::crypto::aws_lc_rs::default_provider()
    .install_default()
    .expect("failed to install the rustls crypto provider");
```

**See also.** [Stream with Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md), [Stream with LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md).

### `there are multiple different versions of crate carbon_core`

**Error.** At compile time, when a Carbon 1.x crate is mixed with Carbon 2.0.0:

```text
error[E0277]: the trait bound `for<'a> MemoProgramDecoder: InstructionDecoder<'a>` is not satisfied
   --> src/main.rs:19:53
    |
 19 |     let _pipeline = Pipeline::builder().instruction(MemoProgramDecoder, Memos).build()?;
    |                                         ----------- ^^^^^^^^^^^^^^^^^^ the trait `for<'a> InstructionDecoder<'a>` is not implemented for `MemoProgramDecoder`
    |                                         |
    |                                         required by a bound introduced by this call
    |
note: there are multiple different versions of crate `carbon_core` in the dependency graph
   --> /home/you/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/carbon-core-2.0.0/src/instruction.rs:187:1
    |
187 | pub trait InstructionDecoder<'a> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this is the expected trait
    |
   ::: /home/you/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/carbon-core-1.0.0/src/instruction.rs:188:1
    |
188 | pub trait InstructionDecoder<'a> {
    | -------------------------------- this is the found trait
```

**Cause.** Here `carbon-memo-program-decoder = "1.0.0"` sits next to `carbon-core = "2.0.0"`, so two `carbon_core` crates are compiled and the decoder implements the 1.x trait. The same happens with Solana types: Carbon 1 uses the Solana 3 crates (`solana-account` 3, `solana-transaction-status` 3) and Carbon 2 the Agave 4.2 cohort (`solana-account` 4.3, `solana-transaction-status` 4.2), so you get `expected Account, found a different Account` with a "multiple different versions of crate `solana_account`" note. `Pubkey` is the exception: `solana-pubkey` 3 and 4 are the same type, so pubkeys don't trigger it.

**Fix.**

1. Set every Carbon crate in `Cargo.toml` to `2.0.0`.
2. Set direct Solana dependencies to the versions Carbon 2.0.0 pins (for example `solana-account = "4.3.2"`, `solana-instruction = "3.4.1"`, `solana-pubkey = "4.2.1"`, `solana-client = "=4.2.2"`).
3. Find the stray crate with `cargo tree -i carbon-core@1.0.0` (or `-i solana-account@3.0.0`).
4. Regenerate decoders produced by an older CLI with `@sevenlabs-hq/carbon-cli@2.0.0`. `carbon-jetstreamer-datasource` and `carbon-jito-shredstream-grpc-datasource` have no 2.0 release; don't use them with Carbon 2.

**See also.** [Migrate to v2](https://carbon.sevenlabs.org/guides/migrate-to-v2.md), [Installation](https://carbon.sevenlabs.org/installation.md).

### `Failed to decode transaction` with `Json` encoding

**Error.** With `RpcBlockSubscribe` or `RpcBlockCrawler` configured with `UiTransactionEncoding::Json` or `JsonParsed`, every transaction logs:

```text
ERROR carbon_rpc_block_subscribe_datasource] Failed to decode transaction: EncodedTransactionWithStatusMeta { transaction: Json(UiTransaction { … }), … }
```

**Cause.** Carbon rebuilds the binary transaction with `EncodedTransaction::decode()`, which only works for binary encodings. JSON payloads can't be decoded, so each transaction is skipped and no instruction processor runs.

**Fix.** Request a binary encoding and V1 support in the config you pass:

```rust
RpcBlockSubscribeConfig {
    encoding: Some(UiTransactionEncoding::Base64),
    transaction_details: Some(TransactionDetails::Full),
    max_supported_transaction_version: Some(1),
    ..RpcBlockSubscribeConfig::default()
}
```

Use the same fields on `RpcBlockConfig` for the block crawler. The transaction crawler and Helius GTFA set Base64 and V1 themselves.

**See also.** [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md), [Migrate to v2](https://carbon.sevenlabs.org/guides/migrate-to-v2.md).

### `Full(..)` when sending updates

**Error.** One of:

```text
ERROR carbon_yellowstone_grpc_datasource] Failed to send transaction update with signature … at slot …: "Full(..)"
WARN  carbon_rpc_transaction_crawler_datasource] Failed to send update: "Full(..)"
ERROR carbon_rpc_block_crawler_datasource] Error sending transaction update: "Full(..)"
```

**Cause.** The pipeline's update channel is full because processors are slower than the source, and the datasource uses `try_send`, which drops the update instead of waiting. Yellowstone, LaserStream, Helius Atlas, RPC block subscribe, RPC block crawler, RPC program subscribe and (by default) the RPC transaction crawler all drop updates this way. Block subscribe and block crawler also stop sending the rest of that block.

**Fix.**

1. Raise [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size) (default 1,000).
2. Make processors faster, or move slow work (database writes, HTTP calls) to a spawned task or a batch.
3. For the transaction crawler, set `ConnectionConfig::blocking_send = true` to apply backpressure instead of dropping.
4. Watch the `carbon_updates_queued` gauge.

**See also.** [Production](https://carbon.sevenlabs.org/guides/production.md), [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md).

### Invalid inner instruction stack height

**Error.**

```text
WARN  carbon_core::transformers] invalid inner instruction stack height (None) in transaction <signature> at instruction <index>, dropping the remaining inner instructions of this group
```

**Cause.** The source sent inner instructions without a `stack_height` (old transactions, or providers that strip it), or with a height outside 2 to 5. Carbon needs the height to rebuild the CPI tree, so it drops the rest of that instruction's inner instructions. Their instruction processors don't run.

**Fix.** Use a source that reports `stackHeight` (current RPC nodes and Yellowstone do). For historical data from before stack heights were recorded, decode inner instructions yourself from `meta.inner_instructions` in a transaction processor.

**See also.** [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md).

### Ctrl-C does not stop the pipeline

**Error.** The process keeps running after Ctrl-C, usually under load.

**Cause.** `Pipeline::run` only listens for Ctrl-C between updates and installs a handler that replaces the default exit, so a signal that arrives while a processor runs is lost.

**Fix.** Handle Ctrl-C yourself and cancel a token passed to [`datasource_cancellation_token`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#datasource_cancellation_token). See [Ctrl-C is ignored while an update is processing](https://carbon.sevenlabs.org/reference/pipeline-builder.md#ctrl-c-is-ignored-while-an-update-is-processing).

**See also.** [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/errors · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Feature flags (Carbon 2.0)

> Every Cargo feature in carbon-core, the datasource and metrics crates, and generated decoders, with what each one enables.

This page lists every Cargo feature in the Carbon 2.0.0 crates and what turning it on adds. Use it when a module or type from another reference page is missing from your build, or when you want to trim dependencies.

## Summary

| Crate                                                      | Features                                                                                                                    | Default       |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------- |
| `carbon-core`                                              | [`macros`](#macros), [`postgres`](#postgres), [`graphql`](#graphql), [`yellowstone`](#yellowstone)                          | `macros`      |
| `carbon-prometheus-metrics`                                | [`http-server`](#http-server)                                                                                               | `http-server` |
| Generated decoders (`carbon-*-decoder`)                    | [`serde`](#serde), [`postgres`](#postgres-decoder), [`graphql`](#graphql-decoder), [`base58`](#base58) (CLI-generated only) | none          |
| `carbon-macros`, `carbon-proc-macros`, `carbon-test-utils` | none                                                                                                                        |               |
| `carbon-log-metrics`                                       | none                                                                                                                        |               |
| Every datasource crate                                     | none                                                                                                                        |               |

## carbon-core

Defined in [`crates/core/Cargo.toml`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/Cargo.toml).

### `macros`

Re-exports the declarative and procedural macros at the `carbon_core` crate root. On by default.

```toml
carbon-core = "2.0.0" # includes "macros"
```

**Details.** Adds the `carbon-macros` and `carbon-proc-macros` dependencies and re-exports everything they export, so you can write `carbon_core::try_decode_instructions!`, `carbon_core::instruction_decoder_collection!` and `#[derive(carbon_core::InstructionType)]`. It also re-exports `log` (hidden from the docs) for use inside the generated macro code. Every decoder crate enables this feature on `carbon-core` itself, so turning it off in your own `Cargo.toml` has no effect once you depend on a decoder.

**Example.**

```toml
carbon-core = { version = "2.0.0", default-features = false } # no macro re-exports
```

**See also.** [Macros](https://carbon.sevenlabs.org/reference/macros.md).

### `postgres`

Adds the `carbon_core::postgres` module: Postgres row types, processors and SQL primitives built on `sqlx`.

```toml
carbon-core = { version = "2.0.0", features = ["postgres"] }
```

**Details.** Enables the optional dependencies `sqlx` (with `postgres`, `rust_decimal` and `bigdecimal`), `sqlx_migrator`, `num-traits` and `bigdecimal`. It also changes [`PipelineBuilder::build`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#build): with this feature, `build` registers the six `postgres_*` metrics, so they appear in every metrics export even if you use no Postgres processor. You add `sqlx` yourself as well, with a runtime feature such as `runtime-tokio-rustls`, to open a connection pool.

**Example.**

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres"] }
sqlx = { version = "0.8.6", features = ["runtime-tokio-rustls", "postgres"] }
sqlx_migrator = "0.17.0"
```

**See also.** [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md), [Postgres guide](https://carbon.sevenlabs.org/guides/postgres.md), [Metrics](https://carbon.sevenlabs.org/reference/metrics.md#carbon-core-postgres-feature).

### `graphql`

Adds the `carbon_core::graphql` module: Juniper scalars for Solana types and an Axum router helper.

```toml
carbon-core = { version = "2.0.0", features = ["graphql"] }
```

**Details.** Enables the optional dependencies `juniper`, `axum` and `juniper_axum`, and compiles `carbon_core::graphql::primitives` and `carbon_core::graphql::server`. It does not enable `postgres`. The GraphQL code that generated decoders emit queries Postgres, so in practice you enable both.

**Example.**

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres", "graphql"] }
axum = "0.8.7"
```

**See also.** [GraphQL reference](https://carbon.sevenlabs.org/reference/graphql.md), [GraphQL guide](https://carbon.sevenlabs.org/guides/graphql.md).

### `yellowstone`

Adds `carbon_core::transformers::yellowstone`, which converts Yellowstone gRPC protobuf messages into Solana transaction types.

```toml
carbon-core = { version = "2.0.0", features = ["yellowstone"] }
```

**Details.** Enables the optional dependencies `yellowstone-grpc-proto`, `solana-account-decoder-client-types`, `solana-transaction-error` and `wincode`. The module provides `create_tx_versioned`, `create_tx_meta`, `ConversionError` and `ConversionResult`. `carbon-yellowstone-grpc-datasource` and `carbon-helius-laserstream-datasource` turn this feature on for you; enable it yourself only to convert protobuf messages in your own datasource.

**Example.**

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["yellowstone"] }
yellowstone-grpc-proto = { version = "=12.7.0", default-features = false }
```

**See also.** [Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md), [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md).

### Implicit dependency features

`carbon-core` names its optional dependencies without the `dep:` prefix in `macros`, `postgres` and `graphql`. Cargo therefore also exposes `carbon-macros`, `carbon-proc-macros`, `sqlx`, `sqlx_migrator`, `num-traits`, `bigdecimal`, `juniper`, `axum` and `juniper_axum` as feature names. Enabling one of these adds the dependency but does not compile any Carbon module, because the code checks for `macros`, `postgres` and `graphql`. Use the named features above instead.

## carbon-prometheus-metrics

Defined in [`metrics/prometheus-metrics/Cargo.toml`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/metrics/prometheus-metrics/Cargo.toml).

### `http-server`

Adds the built-in `/metrics` HTTP server. On by default.

```toml
carbon-prometheus-metrics = "2.0.0" # includes "http-server"
```

**Details.** Enables the optional dependencies `axum`, `tokio`, `tower` and `tower-http`. It provides `PrometheusServerConfig`, `PrometheusMetrics::with_server`, `PrometheusMetrics::start_server`, `run_metrics_server`, `run_default_metrics_server` and `run_metrics_server_with_config`. Without it, only `PrometheusMetrics::new`, `prometheus_text` and `format_snapshot` remain, and you serve the text from your own HTTP stack.

**Example.**

```toml
carbon-prometheus-metrics = { version = "2.0.0", default-features = false } # text rendering only
```

**See also.** [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md#prometheusmetrics), [Prometheus guide](https://carbon.sevenlabs.org/guides/prometheus.md).

## Datasource crates

No datasource crate defines features. Two of them turn on a `carbon-core` feature for you:

| Crate                                       | Version | Enables on `carbon-core` |
| ------------------------------------------- | ------- | ------------------------ |
| `carbon-yellowstone-grpc-datasource`        | 2.0.0   | `yellowstone`            |
| `carbon-helius-laserstream-datasource`      | 2.0.0   | `yellowstone`            |
| `carbon-helius-atlas-ws-datasource`         | 2.0.0   | none                     |
| `carbon-helius-gpa-v2-datasource`           | 2.0.0   | none                     |
| `carbon-helius-gtfa-datasource`             | 2.0.0   | none                     |
| `carbon-rpc-block-crawler-datasource`       | 2.0.0   | none                     |
| `carbon-rpc-block-subscribe-datasource`     | 2.0.0   | none                     |
| `carbon-rpc-gpa-datasource`                 | 2.0.0   | none                     |
| `carbon-rpc-program-subscribe-datasource`   | 2.0.0   | none                     |
| `carbon-rpc-transaction-crawler-datasource` | 2.0.0   | none                     |
| `carbon-stream-message-datasource`          | 2.0.0   | none                     |
| `carbon-validator-snapshot-datasource`      | 2.0.0   | none                     |

Every datasource depends on `carbon-core` with default features, so `macros` is always on in a pipeline that uses one.

`carbon-jetstreamer-datasource` and `carbon-jito-shredstream-grpc-datasource` are excluded from the 2.0.0 workspace. They remain at 1.0.0 on the Carbon 1.x and Solana 3 stack and do not work with `carbon-core` 2.0.0: mixing them in causes duplicate `solana-pubkey` type errors (see [Errors](https://carbon.sevenlabs.org/reference/errors.md#related-errors)).

## Generated decoders

Every decoder crate in the repository (63 of them at 2.0.0) and every decoder generated by [`carbon-cli parse`](https://carbon.sevenlabs.org/reference/cli.md) has the same feature set. All are off by default. Examples below use [`carbon-meteora-dlmm-decoder`](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md); the same applies to [`carbon-token-program-decoder`](https://carbon.sevenlabs.org/decoders/token-program.md) and the rest.

### `serde`

Derives `serde::Serialize` and `serde::Deserialize` on every account, instruction, event and type in the decoder.

```toml
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["serde"] }
```

**Details.** Enables the optional `serde` dependency, plus `serde-big-array` in decoders that have fixed arrays longer than 32 elements (`meteora-dlmm` does, `token-program` does not), because serde only supports arrays up to 32 elements. The account and instruction enums are serialized with an adjacent tag, for example `{"type": "Swap", "data": {…}}`. A `Pubkey` serializes as a JSON array of 32 numbers, not a base58 string, unless the decoder also has [`base58`](#base58).

**Example.**

```rust
// With features = ["serde"]
let json = serde_json::to_string(&decoded_instruction)?;
```

**See also.** [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md).

### `postgres` (decoder)

Adds `accounts::postgres` and `instructions::postgres`: one row type per account and instruction, plus `sqlx_migrator` migrations that create their tables.

```toml
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres"] }
```

**Details.** Enables `carbon-core/postgres`, `sqlx`, `async-trait`, `sqlx_migrator` and the decoder's `serde` feature. You get types such as `accounts::postgres::LbPairRow` and the migrations `MeteoraDlmmAccountsMigration` and `MeteoraDlmmInstructionsMigration`, named after the program.

**Example.**

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres"] }
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres"] }
```

**See also.** [Postgres guide](https://carbon.sevenlabs.org/guides/postgres.md), [Postgres reference](https://carbon.sevenlabs.org/reference/postgres.md).

### `graphql` (decoder)

Adds the decoder's GraphQL schema types and a `graphql` module with `QueryRoot` and `GraphQLContext`, which read rows from Postgres.

```toml
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres", "graphql"] }
```

**Details.** Enables `carbon-core/graphql`, `juniper` and the decoder's `serde` feature. It does **not** enable the decoder's `postgres` feature, but the generated GraphQL code uses the Postgres row types and `sqlx`. Enabling `graphql` alone fails to compile with errors such as ``cannot find `postgres` in `instructions` `` and ``cannot find module or crate `sqlx` ``. Always enable `postgres` with it.

**Example.**

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres", "graphql"] }
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres", "graphql"] }
```

**See also.** [GraphQL guide](https://carbon.sevenlabs.org/guides/graphql.md), [GraphQL reference](https://carbon.sevenlabs.org/reference/graphql.md).

### `base58`

Serializes `Pubkey` fields as base58 strings in serde output. Only in decoders you generate with `--with-base58 true`.

```toml
my-program-decoder = { path = "../decoder", features = ["base58"] }
```

**Details.** Defined as `base58 = ["serde"]` by the CLI renderer. It adds a private `base58` module with `serialize` and `serialize_option` helpers and `#[serde(serialize_with = …)]` attributes on `Pubkey` and `Option<Pubkey>` fields of instructions, instruction accounts and events. Deserialization is unchanged. None of the decoder crates published to crates.io define this feature.

**Example.**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse --idl ./idl.json --out-dir ./decoder --with-base58 true
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

**See also.** [CLI reference](https://carbon.sevenlabs.org/reference/cli.md), [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md).

## Scaffolded indexer features

`carbon-cli scaffold` writes an indexer crate whose `[features]` table contains `default = []`, plus `postgres = []` and `graphql = []` when you choose those options. These features are empty markers: the scaffold enables the real `postgres`/`graphql` features directly on its `carbon-core` and decoder dependencies. See the [CLI reference](https://carbon.sevenlabs.org/reference/cli.md).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/feature-flags · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Macros (Carbon 2.0)

> The macros Carbon 2.0.0 exports: try_decode_instructions!, instruction_decoder_collection! and #[derive(InstructionType)], with their exact expansion and requirements.

Carbon exports three macros, used mostly by generated decoders and by code that combines several programs in one transaction pipe. This page documents what each one expects and what it generates in 2.0.0.

| Macro                                                                | Kind                     | Defined in           | Use it to                                                                |
| -------------------------------------------------------------------- | ------------------------ | -------------------- | ------------------------------------------------------------------------ |
| [`try_decode_instructions!`](#try_decode_instructions)               | declarative              | `carbon-macros`      | Write an `InstructionDecoder` that tries each instruction type in turn   |
| [`#[derive(InstructionType)]`](#deriveinstructiontype)               | derive                   | `carbon-proc-macros` | Generate a tag enum and `get_instruction_type()` for an instruction enum |
| [`instruction_decoder_collection!`](#instruction_decoder_collection) | function-like procedural | `carbon-proc-macros` | Generate an `InstructionDecoderCollection` over several decoders         |

All three are re-exported from the root of `carbon_core` when its default `macros` feature is on, so import them as `carbon_core::try_decode_instructions`, `carbon_core::InstructionType` and `carbon_core::instruction_decoder_collection`. You don't need to depend on `carbon-macros` or `carbon-proc-macros` directly. With `carbon-core = { version = "2.0.0", default-features = false }` the macros are not available.

Examples on this page use:

`Cargo.toml`:

```toml
[dependencies]
borsh = { version = "1.5.7", features = ["derive"] }
carbon-core = "2.0.0"
serde = { version = "1", features = ["derive"] }
solana-instruction = "3.4.1"
solana-pubkey = "4.2.1"
```

## `try_decode_instructions!`

Tries a list of instruction types against one instruction and returns the first that decodes.

```rust
try_decode_instructions!(
    $instruction:expr,
    $program_id:expr,
    $( $Enum::$Variant => $Type ),* $(,)?
)
```

**Details.** For each `Enum::Variant => Type` pair, in order, the macro expands to:

```rust
if let Some(data) = <Type>::decode(instruction.data.as_slice()) {
    if let Some(accounts) = <Type>::arrange_accounts(&instruction.accounts) {
        return Some(Enum::Variant { program_id: $program_id, data, accounts });
    }
}
```

and ends with `None`. Requirements:

| Requirement                                                                                                  | Why                                                                                                                                                                                                              |
| ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Use it as the tail expression of a function returning `Option<Enum>`                                         | It uses `return Some(…)` on a match.                                                                                                                                                                             |
| Each `Type` has an associated `fn decode(&[u8]) -> Option<Self>`                                             | Called as `<Type>::decode`. An inherent method works; `CarbonDeserialize::deserialize` does **not** (different name). The macro's own doc comment says `CarbonDeserialize` is required, which is wrong in 2.0.0. |
| Each `Type` implements [`ArrangeAccounts`](https://carbon.sevenlabs.org/reference/traits.md#arrangeaccounts) | Called as `<Type>::arrange_accounts`. The macro imports the trait itself.                                                                                                                                        |
| Each `Enum::Variant` is a struct variant with fields `program_id`, `data`, `accounts`                        | Built with those field names.                                                                                                                                                                                    |
| The crate `carbon_core` is reachable by that name                                                            | The expansion uses `carbon_core::deserialize::ArrangeAccounts`.                                                                                                                                                  |

An instruction whose data decodes but whose account list is too short is skipped, and the next pair is tried. Order matters when discriminators overlap. Generated decoders call this macro from `decode_instruction` after checking the program ID.

**Example.**

`src/vault.rs`:

```rust
use {
    carbon_core::{
        account_utils::next_account, deserialize::ArrangeAccounts,
        instruction::InstructionDecoder, try_decode_instructions,
    },
    solana_instruction::{AccountMeta, Instruction},
    solana_pubkey::Pubkey,
};

pub const PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("Vau1t11111111111111111111111111111111111111");

#[derive(Debug, Clone, borsh::BorshDeserialize)]
pub struct Deposit {
    pub amount: u64,
}

#[derive(Debug, Clone)]
pub struct DepositAccounts {
    pub vault: Pubkey,
    pub user: Pubkey,
}

impl Deposit {
    pub fn decode(data: &[u8]) -> Option<Self> {
        let body = data.strip_prefix(&[0u8])?;
        borsh::from_slice(body).ok()
    }
}

impl ArrangeAccounts for Deposit {
    type ArrangedAccounts = DepositAccounts;

    fn arrange_accounts(accounts: &[AccountMeta]) -> Option<DepositAccounts> {
        let mut iter = accounts.iter();
        Some(DepositAccounts {
            vault: next_account(&mut iter)?,
            user: next_account(&mut iter)?,
        })
    }
}

#[derive(Debug, Clone, borsh::BorshDeserialize)]
pub struct Withdraw {
    pub amount: u64,
}

impl Withdraw {
    pub fn decode(data: &[u8]) -> Option<Self> {
        let body = data.strip_prefix(&[1u8])?;
        borsh::from_slice(body).ok()
    }
}

impl ArrangeAccounts for Withdraw {
    type ArrangedAccounts = DepositAccounts;

    fn arrange_accounts(accounts: &[AccountMeta]) -> Option<DepositAccounts> {
        Deposit::arrange_accounts(accounts)
    }
}

#[derive(Debug, Clone)]
pub enum VaultInstruction {
    Deposit { program_id: Pubkey, data: Deposit, accounts: DepositAccounts },
    Withdraw { program_id: Pubkey, data: Withdraw, accounts: DepositAccounts },
}

pub struct VaultDecoder;

impl InstructionDecoder<'_> for VaultDecoder {
    type InstructionType = VaultInstruction;

    fn decode_instruction(&self, instruction: &Instruction) -> Option<VaultInstruction> {
        if instruction.program_id != PROGRAM_ID {
            return None;
        }
        try_decode_instructions!(
            instruction,
            PROGRAM_ID,
            VaultInstruction::Deposit => Deposit,
            VaultInstruction::Withdraw => Withdraw,
        )
    }
}
```

**See also.** [`InstructionDecoder`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecoder), [Custom decoder](https://carbon.sevenlabs.org/guides/custom-decoder.md), [source](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/macros/src/try_decode_ixs.rs), [docs.rs](https://docs.rs/carbon-macros/2.0.0/carbon_macros/macro.try_decode_instructions.html).

## `#[derive(InstructionType)]`

Generates a tag enum with the same variant names and a `get_instruction_type` method.

```rust
#[derive(InstructionType)]
pub enum MyInstruction { /* variants */ }

// generates:
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]
pub enum MyInstructionType { /* same variant names, no fields */ }

impl MyInstruction {
    pub fn get_instruction_type(&self) -> MyInstructionType;
}
```

**Details.** Works with unit, tuple and struct variants; fields are dropped in the tag enum. The tag enum is always `pub` and named `<Enum>Type`, and its derive uses the path `serde::Serialize`, so your crate must depend on `serde`. Generics on the source enum are not supported. Generated decoders in 2.0.0 don't use this derive, so `MeteoraDlmmInstructionType` and similar don't exist.

**Example.**

```rust
use carbon_core::InstructionType;

#[derive(Debug, Clone, InstructionType)]
pub enum CounterInstruction {
    Initialize,
    Increment(u64),
    Reset { to: u64 },
}

pub fn demo() {
    let ix = CounterInstruction::Increment(5);
    assert_eq!(ix.get_instruction_type(), CounterInstructionType::Increment);
}
```

**See also.** [`instruction_decoder_collection!`](#instruction_decoder_collection), [docs.rs](https://docs.rs/carbon-proc-macros/2.0.0/carbon_proc_macros/derive.InstructionType.html).

## `instruction_decoder_collection!`

Generates an enum over several programs' instructions and its `InstructionDecoderCollection` impl.

```rust
instruction_decoder_collection!(
    InstructionsEnum, InstructionTypesEnum, ProgramsEnum,
    // 4-part entry
    Variant => PROGRAM_ID_PATH => decoder_expr => InstructionType,
    // 3-part entry
    Variant => decoder_expr => InstructionType,
    …
);
```

**Details.** The macro generates three `pub` enums and one impl:

| Generated item                                                                    | Definition                                                                                                                                                                                                                                                                                                                                                                |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `InstructionsEnum`                                                                | `#[derive(Debug, Clone, Hash, serde::Serialize, serde::Deserialize, PartialEq, Eq)]`, one tuple variant `Variant(InstructionType)` per entry                                                                                                                                                                                                                              |
| `InstructionTypesEnum`                                                            | `#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]`, one variant `Variant(InstructionTypeType)` per entry, wrapping the tag from `#[derive(InstructionType)]`                                                                                                                                                                                                      |
| `ProgramsEnum`                                                                    | `#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]`, one unit variant per entry. Not used by anything; available for your code.                                                                                                                                                                                                                                    |
| `impl carbon_core::collection::InstructionDecoderCollection for InstructionsEnum` | `type InstructionType = InstructionTypesEnum`. `parse_instruction` matches `instruction.program_id` against every 4-part `PROGRAM_ID_PATH`; on a match it returns that decoder's result (or `None`, without trying others). When no 4-part path matches, it tries the 3-part decoders in declaration order. `get_type` calls `get_instruction_type()` on the inner value. |

Requirements, each of which is a compile error when missing:

1. Each `InstructionType` derives `Debug`, `Clone`, `PartialEq`, `Eq`, `Hash`, `serde::Serialize`, `serde::Deserialize` and [`InstructionType`](#deriveinstructiontype).
2. The `InstructionType` path's last segment is used as a bare name, and so is `<Name>Type`: both must be in scope (import them with `use`).
3. `PROGRAM_ID_PATH` must be a `const` path (it is used as a `match` pattern), not a `static` or an expression.
4. `decoder_expr` is an expression evaluated on every call (`MemoDecoder`, or `MyDecoder::new()`), and must implement `InstructionDecoder`.
5. Your crate depends on `serde` and `solana-instruction`, which the expansion names directly.

Generated decoders don't meet requirement 1: their instruction enums derive neither `Eq`, `Hash` nor `InstructionType`. Passing `MeteoraDlmmInstruction` fails with:

```text
error[E0277]: the trait bound `MeteoraDlmmInstruction: Eq` is not satisfied
error[E0277]: the trait bound `MeteoraDlmmInstruction: Hash` is not satisfied
error[E0425]: cannot find type `MeteoraDlmmInstructionType` in this scope
error[E0599]: no method named `get_instruction_type` found for reference `&MeteoraDlmmInstruction` in the current scope
```

For generated decoders, write the collection by hand as shown in [`InstructionDecoderCollection`](https://carbon.sevenlabs.org/reference/traits.md#instructiondecodercollection).

**Example.** Two hand-written decoders, one 4-part and one 3-part entry:

`src/main.rs`:

```rust
use {
    carbon_core::{instruction::InstructionDecoder, instruction_decoder_collection, InstructionType},
    serde::{Deserialize, Serialize},
    solana_instruction::Instruction,
    solana_pubkey::Pubkey,
};

pub const MEMO_PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("MemoSq4gqABAXKb96qnH8TysNcWxMyWCqXgDLGmfcHr");
pub const NOOP_PROGRAM_ID: Pubkey =
    Pubkey::from_str_const("noopb9bkMVfRPU8AsbpTUg8AQkHtKwMYZiFUjNRtMmV");

#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, InstructionType)]
pub enum MemoInstruction {
    Memo(String),
}

pub struct MemoDecoder;

impl InstructionDecoder<'_> for MemoDecoder {
    type InstructionType = MemoInstruction;

    fn decode_instruction(&self, ix: &Instruction) -> Option<MemoInstruction> {
        (ix.program_id == MEMO_PROGRAM_ID)
            .then(|| String::from_utf8(ix.data.clone()).ok())
            .flatten()
            .map(MemoInstruction::Memo)
    }
}

#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, InstructionType)]
pub enum NoopInstruction {
    Log(Vec<u8>),
}

pub struct NoopDecoder;

impl InstructionDecoder<'_> for NoopDecoder {
    type InstructionType = NoopInstruction;

    fn decode_instruction(&self, ix: &Instruction) -> Option<NoopInstruction> {
        (ix.program_id == NOOP_PROGRAM_ID).then(|| NoopInstruction::Log(ix.data.clone()))
    }
}

instruction_decoder_collection!(
    AllInstructions, AllInstructionTypes, AllPrograms,
    // 4-part: matched on the program ID first.
    Memo => MEMO_PROGRAM_ID => MemoDecoder => MemoInstruction,
    // 3-part: tried in order when no 4-part entry matches.
    Noop => NoopDecoder => NoopInstruction,
);
```

Add a `main` to the same file that parses one instruction for each program:

```rust
use carbon_core::collection::InstructionDecoderCollection;

fn main() {
    let memo = Instruction {
        program_id: MEMO_PROGRAM_ID,
        accounts: vec![],
        data: b"gm".to_vec(),
    };
    let noop = Instruction {
        program_id: NOOP_PROGRAM_ID,
        accounts: vec![],
        data: vec![1, 2, 3],
    };

    for ix in [memo, noop] {
        let parsed = AllInstructions::parse_instruction(&ix).unwrap();
        println!("{parsed:?} -> {:?}", parsed.get_type());
    }
}
```

```sh
cargo run
```

Output:

```text
Memo(Memo("gm")) -> Memo(Memo)
Noop(Log([1, 2, 3])) -> Noop(Log)
```

Register the collection on a pipeline with `Pipeline::builder().transaction::<AllInstructions, _>(processor)`.

**See also.** [Transactions](https://carbon.sevenlabs.org/concepts/transactions.md), [Multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md), [source](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/proc-macros/src/lib.rs), [docs.rs](https://docs.rs/carbon-proc-macros/2.0.0/carbon_proc_macros/macro.instruction_decoder_collection.html).

## Re-exported crates

`carbon_core` also re-exports two crates that generated code relies on.

```rust
pub use borsh;
#[doc(hidden)]
pub use log; // only with the `macros` feature
```

**Details.** `carbon_core::borsh` is borsh 1.x. Generated decoders derive `carbon_core::borsh::BorshDeserialize` with `use carbon_core::borsh;` in scope, so a decoder crate doesn't need its own `borsh` dependency for those derives. `carbon_core::log` is hidden from the docs and not part of the public API; depend on `log` yourself.

**Example.**

```rust
use carbon_core::borsh::{self, BorshDeserialize};

#[derive(BorshDeserialize)]
pub struct Header {
    pub version: u8,
}
```

**See also.** [Feature flags](https://carbon.sevenlabs.org/reference/feature-flags.md), [docs.rs](https://docs.rs/carbon-core/2.0.0/carbon_core/).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/macros · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Postgres API (Carbon 2.0)

> Every public type in carbon_core::postgres — processors, row types, CRUD traits, migrations, metadata columns and sqlx primitives.

`carbon_core::postgres` persists decoded accounts and instructions to PostgreSQL through [`sqlx`](https://docs.rs/sqlx/0.8). Use this page when you write a Postgres-backed processor, query the tables Carbon creates, or implement your own row type. For a walkthrough, see [Store data in Postgres](https://carbon.sevenlabs.org/guides/postgres.md).

The module is behind the `postgres` feature of `carbon-core`:

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres"] }
# A generated decoder with its own typed tables:
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres"] }
sqlx = { version = "0.8.6", features = ["runtime-tokio-rustls", "postgres"] }
sqlx_migrator = { version = "0.17.0", features = ["postgres"] }
async-trait = "0.1.89"
```

> **Note: Not on docs.rs.** docs.rs builds `carbon-core` 2.0.0 with default features only, so the `postgres` module has no docs.rs page. The links below point to the source at `v2.0.0`. Run `cargo doc -p carbon-core --features postgres --open` for local API docs.

The module has five submodules:

| Module                      | Contents                                                                     |
| --------------------------- | ---------------------------------------------------------------------------- |
| [`processors`](#processors) | Four `Processor` implementations that upsert every decoded update.           |
| [`rows`](#rows)             | `AccountRow<T>` and `InstructionRow<T>` (JSONB tables) and their migrations. |
| [`operations`](#operations) | The `Table`, `Insert`, `Upsert`, `Delete` and `Lookup` traits.               |
| [`metadata`](#metadata)     | The `__`-prefixed metadata columns every row carries.                        |
| [`primitives`](#primitives) | sqlx newtypes for `Pubkey` and unsigned and 128-bit integers.                |

## Two storage models

Carbon offers two ways to store data. Pick one per pipeline.

|                    | Typed tables                                                                                                             | Generic JSONB tables                                                                                                                     |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Processors         | [`PostgresAccountProcessor`](#postgresaccountprocessor), [`PostgresInstructionProcessor`](#postgresinstructionprocessor) | [`PostgresJsonAccountProcessor`](#postgresjsonaccountprocessor), [`PostgresJsonInstructionProcessor`](#postgresjsoninstructionprocessor) |
| Tables             | One per account type (`<name>_account`) and instruction type (`<name>_instruction`), with one column per field           | `accounts` and `instructions`, decoded body in a `JSONB` column                                                                          |
| Migrations         | `<Program>AccountsMigration`, `<Program>InstructionsMigration` from the decoder crate                                    | [`GenericAccountsMigration`](#genericaccountsmigration), [`GenericInstructionMigration`](#genericinstructionmigration)                   |
| Requires           | Decoder built with its `postgres` feature                                                                                | Decoded type implements `serde::Serialize` and `Deserialize` (the decoder's `serde` feature)                                             |
| Works with GraphQL | Yes, the decoder's generated `QueryRoot` reads these tables                                                              | No generated schema; write your own queries                                                                                              |

## Processors

`carbon_core::postgres::processors`. Source: [`processors.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/processors.rs).

Every processor:

- calls `upsert` once per decoded update and returns its error, so a failed write is logged by the pipeline and counted in `carbon_updates_failed_total`; the pipeline keeps running;
- counts successes and failures and records the upsert duration in milliseconds. See [Postgres metrics](https://carbon.sevenlabs.org/reference/metrics.md) for the metric names;
- holds a `sqlx::PgPool` (cheap to clone; pass `pool.clone()` to each processor).

### `PostgresAccountProcessor`

Upserts each decoded account through a wrapper type `W` that knows how to write it.

```rust
pub struct PostgresAccountProcessor<T, W> { /* private fields */ }

impl<T, W> PostgresAccountProcessor<T, W> {
    pub fn new(pool: sqlx::PgPool) -> Self
}

impl<'a, T, W> Processor<AccountProcessorInputType<'a, T>> for PostgresAccountProcessor<T, W>
where
    T: Clone + Send + Sync + 'static,
    W: From<(T, AccountMetadata)> + Upsert + Send + 'static,
```

**Details.** For every update the processor clones `decoded_account.data` and the [`AccountMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md), builds `W::from((data, metadata))` and calls `W::upsert(&pool)`. Lamports, owner and the raw account bytes are not passed to `W`.

| Type parameter | Description                                                                                                                        |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `T`            | The decoder's account type, for example `MeteoraDlmmAccount`.                                                                      |
| `W`            | The row wrapper. Generated decoders provide `<Program>AccountWithMetadata`, which dispatches to one typed row per account variant. |

**Example.** Register a generated decoder's typed account tables:

```rust
.account(
    MeteoraDlmmDecoder,
    PostgresAccountProcessor::<MeteoraDlmmAccount, MeteoraDlmmAccountWithMetadata>::new(pool.clone()),
)
```

See [the complete program](#example-typed-tables) and [a hand-written wrapper](#example-custom-row).

**See also.** [`Upsert`](#upsert), [Generated decoder types](#generated-decoder-types), [`PostgresJsonAccountProcessor`](#postgresjsonaccountprocessor).

### `PostgresInstructionProcessor`

Upserts each decoded instruction through a wrapper type `W`.

```rust
pub struct PostgresInstructionProcessor<T, W> { /* private fields */ }

impl<T, W> PostgresInstructionProcessor<T, W> {
    pub fn new(pool: sqlx::PgPool) -> Self
}

impl<'a, T, W> Processor<InstructionProcessorInputType<'a, T>> for PostgresInstructionProcessor<T, W>
where
    T: Clone + Send + Sync + 'static,
    W: From<(T, InstructionMetadata, Vec<AccountMeta>)> + Upsert + Send + 'static,
```

**Details.** Builds `W::from((decoded_instruction, metadata, raw_instruction.accounts))` and calls `upsert`. The third tuple element is the instruction's full `Vec<solana_instruction::AccountMeta>`; generated rows store it in an `__accounts` `JSONB` column. Nested instructions are not passed to `W`; each inner instruction reaches the processor as its own update.

| Type parameter | Description                                                                      |
| -------------- | -------------------------------------------------------------------------------- |
| `T`            | The decoder's instruction enum, for example `MeteoraDlmmInstruction`.            |
| `W`            | The row wrapper, for example the generated `MeteoraDlmmInstructionWithMetadata`. |

**Example.**

```rust
.instruction(
    MeteoraDlmmDecoder,
    PostgresInstructionProcessor::<MeteoraDlmmInstruction, MeteoraDlmmInstructionWithMetadata>::new(pool.clone()),
)
```

**See also.** [Instruction primary key](#instruction-primary-key), [`InstructionRowMetadata`](#instructionrowmetadata).

### `PostgresJsonAccountProcessor`

Upserts each decoded account into the generic `accounts` table as JSONB.

```rust
pub struct PostgresJsonAccountProcessor<T> { /* private fields */ }

impl<T> PostgresJsonAccountProcessor<T> {
    pub fn new(pool: sqlx::PgPool) -> Self
}

impl<'a, T> Processor<AccountProcessorInputType<'a, T>> for PostgresJsonAccountProcessor<T>
where
    T: serde::Serialize + for<'de> serde::Deserialize<'de> + Clone + Send + Sync + Unpin + 'static,
```

**Details.** Builds an [`AccountRow<T>`](#accountrow) and calls its `upsert`, which runs `INSERT INTO accounts (__pubkey, __slot, data) … ON CONFLICT (__pubkey) DO UPDATE`. Each pubkey keeps one row holding its latest state; the update with the higher slot does not win automatically, the last one processed does. Create the table first with [`GenericAccountsMigration`](#genericaccountsmigration). Generated account enums serialize as `{"type": "<Variant>", "data": {…}}`, so filter with `data->>'type' = 'LbPair'`. Pubkey fields inside the JSON are arrays of 32 numbers.

**Example.**

```rust
.account(
    TokenProgramDecoder,
    PostgresJsonAccountProcessor::<TokenProgramAccount>::new(pool.clone()),
)
```

**See also.** [`AccountRow`](#accountrow), [Two storage models](#two-storage-models).

### `PostgresJsonInstructionProcessor`

Upserts each decoded instruction into the generic `instructions` table as JSONB.

```rust
pub struct PostgresJsonInstructionProcessor<T> { /* private fields */ }

impl<T> PostgresJsonInstructionProcessor<T> {
    pub fn new(pool: sqlx::PgPool) -> Self
}

impl<'a, T> Processor<InstructionProcessorInputType<'a, T>> for PostgresJsonInstructionProcessor<T>
where
    T: serde::Serialize + for<'de> serde::Deserialize<'de> + Clone + Send + Sync + Unpin + 'static,
```

**Details.** Builds an [`InstructionRow<T>`](#instructionrow) and upserts it on `(__signature, __instruction_index, __stack_height)`. The instruction's accounts are not stored separately; they are in the JSON only if the decoded type includes them (generated instruction enums do, under `accounts`). Create the table with [`GenericInstructionMigration`](#genericinstructionmigration). See [the complete program](#example-generic-jsonb-tables).

**Example.**

```rust
.instruction(
    MeteoraDlmmDecoder,
    PostgresJsonInstructionProcessor::<MeteoraDlmmInstruction>::new(pool.clone()),
)
```

**See also.** [Instruction primary key](#instruction-primary-key), [`InstructionRow`](#instructionrow).

### `register_postgres_metrics`

Registers the six Postgres metrics with the global [`MetricsRegistry`](https://carbon.sevenlabs.org/reference/metrics.md).

```rust
pub fn register_postgres_metrics()
```

**Details.** [`PipelineBuilder::build`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#build) calls it whenever `carbon-core` is compiled with `postgres`, so you never call it yourself. The metrics therefore appear in every exporter's output when the feature is on, even if no Postgres processor is registered. Calling it again registers the metrics a second time and duplicates them in snapshots.

**Example.**

```rust
// Only needed if you read MetricsRegistry::global() without building a pipeline.
carbon_core::postgres::processors::register_postgres_metrics();
```

**See also.** [Metrics reference](https://carbon.sevenlabs.org/reference/metrics.md).

## Rows

`carbon_core::postgres::rows`. Source: [`rows.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/rows.rs).

The generic tables these rows map to:

```sql
CREATE TABLE IF NOT EXISTS accounts (
    __pubkey BYTEA NOT NULL,
    __slot NUMERIC,
    data JSONB NOT NULL,
    PRIMARY KEY (__pubkey)
);

CREATE TABLE IF NOT EXISTS instructions (
    __signature TEXT NOT NULL,
    __instruction_index BIGINT NOT NULL,
    __stack_height BIGINT NOT NULL,
    __slot NUMERIC,
    decoded_instruction JSONB NOT NULL,
    PRIMARY KEY (__signature, __instruction_index, __stack_height)
);
```

### `AccountRow`

One row of the generic `accounts` table.

```rust
#[derive(sqlx::FromRow, Debug, Clone)]
pub struct AccountRow<T>
where
    T: serde::Serialize + for<'de> serde::Deserialize<'de> + Clone + Send + Sync + Unpin + 'static,
{
    #[sqlx(flatten)]
    pub metadata: AccountRowMetadata,
    pub data: sqlx::types::Json<T>,
}

impl<T> AccountRow<T> {
    pub fn from_parts(source: T, metadata: AccountMetadata) -> Self
}
```

**Details.**

| Field      | Type                                        | Column               | Description                                        |
| ---------- | ------------------------------------------- | -------------------- | -------------------------------------------------- |
| `metadata` | [`AccountRowMetadata`](#accountrowmetadata) | `__pubkey`, `__slot` | Account address and slot.                          |
| `data`     | `sqlx::types::Json<T>`                      | `data`               | The decoded account, serialized with `serde_json`. |

Trait implementations:

| Trait               | SQL                                                             | Key                             |
| ------------------- | --------------------------------------------------------------- | ------------------------------- |
| [`Insert`](#insert) | `INSERT INTO accounts …`; fails on an existing pubkey           | —                               |
| [`Upsert`](#upsert) | `… ON CONFLICT (__pubkey) DO UPDATE SET __slot = $2, data = $3` | —                               |
| [`Delete`](#delete) | `DELETE FROM accounts WHERE __pubkey = $1`                      | [`primitives::Pubkey`](#pubkey) |
| [`Lookup`](#lookup) | `SELECT * FROM accounts WHERE __pubkey = $1`                    | [`primitives::Pubkey`](#pubkey) |

Database errors are returned as [`Error::Custom`](https://carbon.sevenlabs.org/reference/errors.md#custom) with the sqlx message.

**Example.** Read back what `PostgresJsonAccountProcessor` wrote:

```rust
use carbon_core::postgres::{operations::Lookup, primitives::Pubkey, rows::AccountRow};

let row = AccountRow::<MeteoraDlmmAccount>::lookup(Pubkey(pubkey), &pool).await?;
if let Some(row) = row {
    println!("slot={:?} data={:?}", row.metadata.slot, row.data.0);
}
```

**See also.** [`PostgresJsonAccountProcessor`](#postgresjsonaccountprocessor), [`GenericAccountsMigration`](#genericaccountsmigration).

### `InstructionRow`

One row of the generic `instructions` table.

```rust
#[derive(sqlx::FromRow, Debug, Clone)]
pub struct InstructionRow<T>
where
    T: serde::Serialize + for<'de> serde::Deserialize<'de> + Clone + Send + Sync + Unpin + 'static,
{
    #[sqlx(flatten)]
    pub metadata: InstructionRowMetadata,
    pub decoded_instruction: sqlx::types::Json<T>,
}

impl<T> InstructionRow<T> {
    pub fn from_parts(decoded_instruction: T, metadata: InstructionMetadata) -> Self
}
```

**Details.**

| Field                 | Type                                                | Column                                                           | Description                      |
| --------------------- | --------------------------------------------------- | ---------------------------------------------------------------- | -------------------------------- |
| `metadata`            | [`InstructionRowMetadata`](#instructionrowmetadata) | `__signature`, `__instruction_index`, `__stack_height`, `__slot` | Where the instruction sits.      |
| `decoded_instruction` | `sqlx::types::Json<T>`                              | `decoded_instruction`                                            | The decoded instruction as JSON. |

`Insert`, `Upsert` (`ON CONFLICT (__signature, __instruction_index, __stack_height) DO UPDATE SET __slot = $4, decoded_instruction = $5`), `Delete` and `Lookup` are implemented. The `Delete` and `Lookup` key is `(String, U32, U32)`: signature in base58, instruction index, stack height.

**Example.**

```rust
use carbon_core::postgres::{operations::Lookup, rows::InstructionRow};

// Top-level instruction 0 of a transaction.
let key = (signature.to_string(), 0u32.into(), 1u32.into());
let row = InstructionRow::<MeteoraDlmmInstruction>::lookup(key, &pool).await?;
```

**See also.** [Instruction primary key](#instruction-primary-key), [`GenericInstructionMigration`](#genericinstructionmigration).

### Instruction primary key

Instruction tables, generic and generated, use the primary key `(__signature, __instruction_index, __stack_height)`. `__instruction_index` comes from [`InstructionMetadata::index`](https://carbon.sevenlabs.org/reference/processor-inputs.md), which for an inner instruction is the index of its **top-level** instruction, not its position among siblings.

> **Warning: Sibling CPIs overwrite each other.** Two inner instructions of the same program at the same stack height under the same top-level instruction (for example, two `Swap` CPIs from one router call) produce the same key. Because the processors upsert, only the last one processed remains in the table. If you need every CPI, write your own wrapper that stores `InstructionMetadata::absolute_path` and keys on it.

### `GenericAccountsMigration`

`sqlx_migrator` migration that creates the generic `accounts` table.

```rust
pub struct GenericAccountsMigration;

impl sqlx_migrator::Migration<sqlx::Postgres> for GenericAccountsMigration
```

**Details.** App `"carbon_core"`, name `"generic_accounts"`, no parents, one operation: [`AccountRowMigrationOperation`](#accountrowmigrationoperation). Rolling it back drops the table.

**Example.**

```rust
use sqlx_migrator::{Info, Migrate, Plan};

let mut migrator = sqlx_migrator::Migrator::default();
migrator
    .add_migration(Box::new(GenericAccountsMigration))
    .map_err(|e| Error::Custom(e.to_string()))?;
let mut conn = pool.acquire().await.map_err(|e| Error::Custom(e.to_string()))?;
migrator
    .run(&mut *conn, &Plan::apply_all())
    .await
    .map_err(|e| Error::Custom(e.to_string()))?;
```

**See also.** [`GenericInstructionMigration`](#genericinstructionmigration).

### `GenericInstructionMigration`

`sqlx_migrator` migration that creates the generic `instructions` table.

```rust
pub struct GenericInstructionMigration;

impl sqlx_migrator::Migration<sqlx::Postgres> for GenericInstructionMigration
```

**Details.** App `"carbon_core"`, name `"generic_instructions"`, no parents, one operation: [`InstructionRowMigrationOperation`](#instructionrowmigrationoperation).

**Example.**

```rust
migrator
    .add_migration(Box::new(GenericInstructionMigration))
    .map_err(|e| Error::Custom(e.to_string()))?;
```

**See also.** [`GenericAccountsMigration`](#genericaccountsmigration).

### `AccountRowMigrationOperation`

The `sqlx_migrator::Operation` behind `GenericAccountsMigration`.

```rust
pub struct AccountRowMigrationOperation;

impl sqlx_migrator::Operation<sqlx::Postgres> for AccountRowMigrationOperation
```

**Details.** `up` runs `CREATE TABLE IF NOT EXISTS accounts (…)` (see [Rows](#rows)); `down` runs `DROP TABLE IF EXISTS accounts`. Use it to include the table in a migration of your own.

**Example.**

```rust
fn operations(&self) -> Vec<Box<dyn sqlx_migrator::Operation<sqlx::Postgres>>> {
    vec![Box::new(AccountRowMigrationOperation)]
}
```

**See also.** [`InstructionRowMigrationOperation`](#instructionrowmigrationoperation).

### `InstructionRowMigrationOperation`

The `sqlx_migrator::Operation` behind `GenericInstructionMigration`.

```rust
pub struct InstructionRowMigrationOperation;

impl sqlx_migrator::Operation<sqlx::Postgres> for InstructionRowMigrationOperation
```

**Details.** `up` creates the `instructions` table; `down` runs `DROP TABLE IF EXISTS instructions`.

**Example.**

```rust
fn operations(&self) -> Vec<Box<dyn sqlx_migrator::Operation<sqlx::Postgres>>> {
    vec![Box::new(InstructionRowMigrationOperation)]
}
```

**See also.** [`AccountRowMigrationOperation`](#accountrowmigrationoperation).

## Operations

`carbon_core::postgres::operations`. Source: [`operations.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/operations.rs). `Insert`, `Upsert`, `Delete` and `Lookup` are `#[async_trait]` traits: annotate your impls with `#[async_trait::async_trait]`.

### `Table`

Static table name and column list.

```rust
pub trait Table {
    fn table() -> &'static str;
    fn columns() -> Vec<&'static str>;
}
```

**Details.** Nothing in `carbon-core` calls it. Generated decoders implement it on each decoded type (not the row), for example `Swap::table()` returns `"swap_instruction"`. Use it to build queries without hard-coding names.

**Example.**

```rust
use carbon_core::postgres::operations::Table;
use carbon_meteora_dlmm_decoder::instructions::swap::Swap;

let sql = format!("SELECT count(*) FROM {}", Swap::table());
```

**See also.** [Generated decoder types](#generated-decoder-types).

### `Insert`

Inserts a row, failing if the primary key exists.

```rust
#[async_trait::async_trait]
pub trait Insert {
    async fn insert(&self, pool: &sqlx::PgPool) -> CarbonResult<()>;
}
```

**Details.** No built-in processor calls `insert`; they all use [`Upsert`](#upsert). Implemented by `AccountRow`, `InstructionRow` and every generated row and wrapper.

**Example.**

```rust
AccountRow::from_parts(decoded, metadata).insert(&pool).await?;
```

**See also.** [`Upsert`](#upsert).

### `Upsert`

Inserts a row or updates it on primary-key conflict.

```rust
#[async_trait::async_trait]
pub trait Upsert {
    async fn upsert(&self, pool: &sqlx::PgPool) -> CarbonResult<()>;
}
```

**Details.** The bound on `W` in [`PostgresAccountProcessor`](#postgresaccountprocessor) and [`PostgresInstructionProcessor`](#postgresinstructionprocessor). Return `Ok(())` without writing to skip an update.

**Example.** See [a custom row](#example-custom-row).

**See also.** [`Insert`](#insert).

### `Delete`

Deletes the row matching a key.

```rust
#[async_trait::async_trait]
pub trait Delete {
    type Key;

    async fn delete(key: Self::Key, pool: &sqlx::PgPool) -> CarbonResult<()>;
}
```

**Details.** An associated function, not a method: call `AccountRow::<T>::delete(key, &pool)`. No built-in processor deletes rows; account closures arrive as [`AccountDeletion`](https://carbon.sevenlabs.org/reference/updates.md) updates, which you handle with a [`account_deletions`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#account_deletions) processor that calls `delete`.

**Example.**

```rust
use carbon_core::postgres::{operations::Delete, rows::AccountRow};

AccountRow::<MeteoraDlmmAccount>::delete(deletion.pubkey.into(), &pool).await?;
```

**See also.** [`Lookup`](#lookup).

### `Lookup`

Selects the row matching a key, or `None`.

```rust
#[async_trait::async_trait]
pub trait Lookup: Sized {
    type Key;

    async fn lookup(key: Self::Key, pool: &sqlx::PgPool) -> CarbonResult<Option<Self>>;
}
```

**Details.** Generated account rows implement it with `Key = primitives::Pubkey`; the generated GraphQL `QueryRoot` uses it for single-account fields.

**Example.** See [`AccountRow`](#accountrow).

**See also.** [`Delete`](#delete).

## Metadata

`carbon_core::postgres::metadata`. Source: [`metadata.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/metadata.rs). Row types embed these with `#[sqlx(flatten)]`; the `__` prefix keeps metadata columns from colliding with program field names.

### `AccountRowMetadata`

Account metadata columns.

```rust
#[derive(Debug, Clone, PartialEq, sqlx::FromRow, serde::Serialize, serde::Deserialize)]
pub struct AccountRowMetadata {
    #[sqlx(rename = "__pubkey")]
    pub pubkey: Pubkey,
    #[sqlx(rename = "__slot")]
    pub slot: Option<U64>,
}

impl From<AccountMetadata> for AccountRowMetadata
```

**Details.**

| Field    | Type                           | Column     | Postgres type      | Description                                                              |
| -------- | ------------------------------ | ---------- | ------------------ | ------------------------------------------------------------------------ |
| `pubkey` | [`Pubkey`](#pubkey)            | `__pubkey` | `BYTEA` (32 bytes) | Account address.                                                         |
| `slot`   | `Option<`[`U64`](#u64-u128)`>` | `__slot`   | `NUMERIC`          | Slot of the update. Always `Some` when converted from `AccountMetadata`. |

`AccountMetadata::transaction_signature` is not stored.

**Example.**

```rust
let columns: AccountRowMetadata = input.metadata.clone().into();
```

**See also.** [`InstructionRowMetadata`](#instructionrowmetadata).

### `InstructionRowMetadata`

Instruction metadata columns.

```rust
#[derive(Debug, Clone, PartialEq, sqlx::FromRow, serde::Serialize, serde::Deserialize)]
pub struct InstructionRowMetadata {
    #[sqlx(rename = "__signature")]
    pub signature: String,
    #[sqlx(rename = "__instruction_index")]
    pub instruction_index: U32,
    #[sqlx(rename = "__stack_height")]
    pub stack_height: U32,
    #[sqlx(rename = "__slot")]
    pub slot: Option<U64>,
}

impl From<InstructionMetadata> for InstructionRowMetadata
```

**Details.**

| Field               | Type                           | Column                | Postgres type | Description                                                                                   |
| ------------------- | ------------------------------ | --------------------- | ------------- | --------------------------------------------------------------------------------------------- |
| `signature`         | `String`                       | `__signature`         | `TEXT`        | Transaction signature, base58.                                                                |
| `instruction_index` | [`U32`](#u8-u16-u32)           | `__instruction_index` | `BIGINT`      | `InstructionMetadata::index`: the top-level instruction's index, also for inner instructions. |
| `stack_height`      | [`U32`](#u8-u16-u32)           | `__stack_height`      | `BIGINT`      | `1` for top-level instructions, `2`+ for CPIs.                                                |
| `slot`              | `Option<`[`U64`](#u64-u128)`>` | `__slot`              | `NUMERIC`     | Transaction slot. Always `Some` when converted.                                               |

`absolute_path`, block time and the rest of `TransactionMetadata` are not stored.

**Example.**

```rust
let columns: InstructionRowMetadata = input.metadata.clone().into();
```

**See also.** [Instruction primary key](#instruction-primary-key).

## Primitives

`carbon_core::postgres::primitives`. Source: [`primitives.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/postgres/primitives.rs). Postgres has no unsigned integers and no 32-byte key type, so these newtypes implement sqlx `Type`, `Encode`, `Decode` and `PgHasArrayType` with a lossless mapping. Each derefs to its inner value, is a single public tuple field, and serializes transparently with serde.

| Type                 | Wraps                   | Postgres type | Inner value             |
| -------------------- | ----------------------- | ------------- | ----------------------- |
| [`Pubkey`](#pubkey)  | `solana_pubkey::Pubkey` | `BYTEA`       | `solana_pubkey::Pubkey` |
| [`U8`](#u8-u16-u32)  | `u8`                    | `INT2`        | `i16`                   |
| [`U16`](#u8-u16-u32) | `u16`                   | `INT4`        | `i32`                   |
| [`U32`](#u8-u16-u32) | `u32`                   | `INT8`        | `i64`                   |
| [`U64`](#u64-u128)   | `u64`                   | `NUMERIC`     | `u64`                   |
| [`U128`](#u64-u128)  | `u128`                  | `NUMERIC`     | `u128`                  |
| [`I128`](#i128)      | `i128`                  | `NUMERIC`     | `i128`                  |

Signed types up to 64 bits use sqlx's own `i16`/`i32`/`i64` mappings directly.

### `Pubkey`

A Solana public key stored as 32 raw bytes.

```rust
#[derive(Clone, Copy, PartialEq, Eq, Debug, serde::Serialize, serde::Deserialize)]
#[serde(transparent)]
pub struct Pubkey(pub solana_pubkey::Pubkey);

impl From<solana_pubkey::Pubkey> for Pubkey
impl From<Vec<u8>> for Pubkey
impl From<&[u8]> for Pubkey
impl Deref for Pubkey { type Target = solana_pubkey::Pubkey; }
```

**Details.** Encodes as `BYTEA`, arrays as `BYTEA[]`. Decoding fails if the column does not hold exactly 32 bytes. `From<Vec<u8>>` and `From<&[u8]>` return the all-zero key `11111111111111111111111111111111` for input of the wrong length instead of failing. It is a different type from `carbon_core::graphql::primitives::Pubkey`.

**Example.**

```rust
let key: carbon_core::postgres::primitives::Pubkey = solana_pubkey::Pubkey::new_unique().into();
println!("{}", *key); // Deref to solana_pubkey::Pubkey
```

**See also.** [`AccountRowMetadata`](#accountrowmetadata).

### `U8`, `U16`, `U32`

Unsigned integers stored in the next wider signed column.

```rust
pub struct U8(pub i16);   // INT2
pub struct U16(pub i32);  // INT4
pub struct U32(pub i64);  // INT8

// For each of (U8, u8, i16), (U16, u16, i32), (U32, u32, i64):
impl From<u8> for U8
impl From<i16> for U8
impl From<&i16> for U8
impl TryFrom<U8> for u8 { type Error = carbon_core::error::Error; }
impl Deref for U8 { type Target = i16; }
```

**Details.** The inner field is the **signed** wider type, so `*U32` is an `i64`. `TryFrom` back to the unsigned type returns `Error::Custom("out of range")` if the stored value does not fit, which only happens when the column was written by something other than Carbon. Derives `Clone, Copy, PartialEq, Eq, Debug`.

**Example.**

```rust
let decimals: U8 = 255u8.into(); // stored as INT2 255
let back: u8 = decimals.try_into()?;
```

**See also.** [`U64`, `U128`](#u64-u128).

### `U64`, `U128`

Unsigned 64- and 128-bit integers stored as `NUMERIC`.

```rust
pub struct U64(pub u64);
pub struct U128(pub u128);

// For each of (U64, u64), (U128, u128):
impl From<u64> for U64
impl TryFrom<U64> for u64 { type Error = carbon_core::error::Error; } // always Ok
impl Deref for U64 { type Target = u64; }
impl From<U64> for sqlx::types::Decimal
impl From<sqlx::types::Decimal> for U64
impl From<&sqlx::types::Decimal> for U64
```

**Details.** Encodes through `BigDecimal` as a binary `NUMERIC`, so the full range survives (`u64::MAX` round-trips). Decoding fails with `Error::Custom` when the stored value has a fractional part or a minus sign (`"invalid sign/scale for unsigned integer"`) or exceeds the type's maximum (`"value exceeds maximum for target type"`). The `From<Decimal>` conversions are infallible: values that are negative, fractional or out of range become `0` silently. `From<U64> for Decimal` also returns `0` for values a `rust_decimal::Decimal` cannot hold (above about 7.9 × 10²⁸, only reachable with `U128`). Derives `Clone, Debug, PartialEq, Eq` (not `Copy`; bind with `&row.field`).

**Example.**

```rust
let supply: U64 = u64::MAX.into();
let raw: u64 = *supply;
```

**See also.** [`I128`](#i128).

### `I128`

Signed 128-bit integer stored as `NUMERIC`.

```rust
pub struct I128(pub i128);

impl From<i128> for I128
impl Deref for I128 { type Target = i128; }
```

**Details.** Same `NUMERIC` encoding as `U128`. Decoding fails with `Error::Custom("invalid scale for i128")` for fractional values and `"invalid numeric string"` when out of range. Unlike `U64`/`U128`, there is no `TryFrom<I128> for i128` and no `Decimal` conversion; use `*value` or `.0`.

**Example.**

```rust
let delta = I128::from(-1i128);
```

**See also.** [`U64`, `U128`](#u64-u128).

## Generated decoder types

A decoder built with its `postgres` feature (for example `carbon-meteora-dlmm-decoder`, generated by the [CLI](https://carbon.sevenlabs.org/reference/cli.md)) adds these public items. Names use the program's PascalCase name (`MeteoraDlmm`) and each type's snake_case name.

| Item                               | Path                     | Description                                                                                                                                                                                      |
| ---------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `<Program>AccountWithMetadata`     | `accounts::postgres`     | Tuple struct `(Account enum, AccountMetadata)`. `From`, `Insert`, `Upsert`; dispatches to the variant's row. Use as `W` in `PostgresAccountProcessor`.                                           |
| `<Program>AccountsMigration`       | `accounts::postgres`     | `sqlx_migrator` migration creating every `<account>_account` table. App name is the program's kebab-case name.                                                                                   |
| `<Account>Row`                     | `accounts::postgres`     | Typed row: flattened `AccountRowMetadata` plus one column per field. `from_parts(account, metadata)`, `TryFrom<Row> for <Account>`, `Insert`, `Upsert`, `Delete` and `Lookup` keyed by `Pubkey`. |
| `<Account>MigrationOperation`      | `accounts::postgres`     | Creates one account table.                                                                                                                                                                       |
| `<Program>InstructionWithMetadata` | `instructions::postgres` | Tuple struct `(Instruction enum, InstructionMetadata, Vec<AccountMeta>)`. Use as `W` in `PostgresInstructionProcessor`.                                                                          |
| `<Program>InstructionsMigration`   | `instructions::postgres` | Creates every `<instruction>_instruction` table.                                                                                                                                                 |
| `<Instruction>Row`                 | `instructions::postgres` | Flattened `InstructionRowMetadata`, one column per argument, and `__accounts JSONB`.                                                                                                             |
| `<Instruction>MigrationOperation`  | `instructions::postgres` | Creates one instruction table.                                                                                                                                                                   |
| `impl Table for <Type>`            | on the decoded type      | Table name (`swap_instruction`, `lb_pair_account`) and column list.                                                                                                                              |

Generated columns use `NUMERIC(20)` for `u64`, `NUMERIC(39)` for `u128`, `NUMERIC(38)` for `i128` (values with 39 digits, above 10³⁸, fail to insert), `INT2`/`INT4`/`INT8` for `u8`/`u16`/`u32`, `BYTEA` for pubkeys and byte arrays, and `JSONB` for nested structs and enums.

## Examples

### Example: typed tables

Indexes Meteora DLMM accounts and instructions into per-type tables. It uses the public RPC transaction crawler, so only `DATABASE_URL` is required.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["postgres"] }
carbon-log-metrics = "2.0.0"
carbon-meteora-dlmm-decoder = { version = "2.0.0", features = ["postgres"] }
carbon-rpc-transaction-crawler-datasource = "2.0.0"
env_logger = "0.11.8"
sqlx = { version = "0.8.6", features = ["runtime-tokio-rustls", "postgres"] }
sqlx_migrator = { version = "0.17.0", features = ["postgres"] }
tokio = { version = "1.48.0", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::{CarbonResult, Error},
        pipeline::Pipeline,
        postgres::processors::{PostgresAccountProcessor, PostgresInstructionProcessor},
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        accounts::{
            postgres::{MeteoraDlmmAccountWithMetadata, MeteoraDlmmAccountsMigration},
            MeteoraDlmmAccount,
        },
        instructions::{
            postgres::{MeteoraDlmmInstructionWithMetadata, MeteoraDlmmInstructionsMigration},
            MeteoraDlmmInstruction,
        },
        MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    sqlx_migrator::{Info, Migrate, Plan},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let database_url = std::env::var("DATABASE_URL")
        .map_err(|_| Error::Custom("DATABASE_URL must be set".to_string()))?;
    let pool = sqlx::PgPool::connect(&database_url)
        .await
        .map_err(|e| Error::Custom(e.to_string()))?;

    // Create one table per account type and per instruction type.
    let mut migrator = sqlx_migrator::Migrator::default();
    migrator
        .add_migration(Box::new(MeteoraDlmmAccountsMigration))
        .map_err(|e| Error::Custom(e.to_string()))?;
    migrator
        .add_migration(Box::new(MeteoraDlmmInstructionsMigration))
        .map_err(|e| Error::Custom(e.to_string()))?;
    let mut conn = pool.acquire().await.map_err(|e| Error::Custom(e.to_string()))?;
    migrator
        .run(&mut *conn, &Plan::apply_all())
        .await
        .map_err(|e| Error::Custom(e.to_string()))?;

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(
            MeteoraDlmmDecoder,
            PostgresAccountProcessor::<MeteoraDlmmAccount, MeteoraDlmmAccountWithMetadata>::new(
                pool.clone(),
            ),
        )
        .instruction(
            MeteoraDlmmDecoder,
            PostgresInstructionProcessor::<
                MeteoraDlmmInstruction,
                MeteoraDlmmInstructionWithMetadata,
            >::new(pool.clone()),
        )
        .build()?
        .run()
        .await
}
```

```sh
DATABASE_URL=postgres://postgres:postgres@localhost:5432/carbon cargo run
```

The transaction crawler emits transactions only, so the account tables stay empty here; pair the same processors with an account datasource such as [`rpc-program-subscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md) to fill them. Then query a table:

```sql
SELECT __signature, amount_in, min_amount_out FROM swap_instruction ORDER BY __slot DESC LIMIT 5;
```

### Example: generic JSONB tables

The same pipeline writing into the two generic tables. Replace the migrations and the processor:

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::{CarbonResult, Error},
        pipeline::Pipeline,
        postgres::{
            processors::PostgresJsonInstructionProcessor,
            rows::{GenericAccountsMigration, GenericInstructionMigration},
        },
    },
    carbon_log_metrics::LogMetrics,
    carbon_meteora_dlmm_decoder::{
        instructions::MeteoraDlmmInstruction, MeteoraDlmmDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    sqlx_migrator::{Info, Migrate, Plan},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let database_url = std::env::var("DATABASE_URL")
        .map_err(|_| Error::Custom("DATABASE_URL must be set".to_string()))?;
    let pool = sqlx::PgPool::connect(&database_url)
        .await
        .map_err(|e| Error::Custom(e.to_string()))?;

    // Creates the generic `accounts` and `instructions` tables.
    let mut migrator = sqlx_migrator::Migrator::default();
    migrator
        .add_migration(Box::new(GenericAccountsMigration))
        .map_err(|e| Error::Custom(e.to_string()))?;
    migrator
        .add_migration(Box::new(GenericInstructionMigration))
        .map_err(|e| Error::Custom(e.to_string()))?;
    let mut conn = pool.acquire().await.map_err(|e| Error::Custom(e.to_string()))?;
    migrator
        .run(&mut *conn, &Plan::apply_all())
        .await
        .map_err(|e| Error::Custom(e.to_string()))?;

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        ConnectionConfig::default(),
        Filters::new(None, None, None),
        None,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(
            MeteoraDlmmDecoder,
            PostgresJsonInstructionProcessor::<MeteoraDlmmInstruction>::new(pool.clone()),
        )
        .build()?
        .run()
        .await
}
```

Generated instruction enums serialize as `{"type": "<Variant>", "data": {"program_id": …, "data": {…}, "accounts": {…}}}`. Pubkeys inside the JSON are arrays of 32 numbers, not base58 strings.

```sql
SELECT __signature, decoded_instruction->'data'->'data'->>'amount_in' AS amount_in
FROM instructions
WHERE decoded_instruction->>'type' = 'Swap'
LIMIT 5;
```

### Example: custom row

A hand-written wrapper that stores only SPL Token mints in a table of your own. It shows `Table`, `Upsert`, `Lookup`, `Delete` and the primitives together. Create the table yourself:

```sql
CREATE TABLE mints (pubkey BYTEA PRIMARY KEY, slot NUMERIC NOT NULL, supply NUMERIC NOT NULL, decimals INT2 NOT NULL);
```

`src/mints.rs`:

```rust
use {
    carbon_core::{
        account::AccountMetadata,
        error::{CarbonResult, Error},
        postgres::{
            operations::{Delete, Lookup, Table, Upsert},
            primitives::{Pubkey, U64, U8},
            processors::PostgresAccountProcessor,
        },
    },
    carbon_token_program_decoder::accounts::TokenProgramAccount,
};

/// One row of a `mints` table. Non-mint accounts map to `None` and are skipped.
#[derive(sqlx::FromRow, Debug, Clone)]
pub struct MintRow {
    pub pubkey: Pubkey,
    pub slot: U64,
    pub supply: U64,
    pub decimals: U8,
}

pub struct MaybeMint(Option<MintRow>);

impl From<(TokenProgramAccount, AccountMetadata)> for MaybeMint {
    fn from((account, metadata): (TokenProgramAccount, AccountMetadata)) -> Self {
        match account {
            TokenProgramAccount::Mint(mint) => MaybeMint(Some(MintRow {
                pubkey: metadata.pubkey.into(),
                slot: metadata.slot.into(),
                supply: mint.supply.into(),
                decimals: mint.decimals.into(),
            })),
            _ => MaybeMint(None),
        }
    }
}

impl Table for MintRow {
    fn table() -> &'static str {
        "mints"
    }

    fn columns() -> Vec<&'static str> {
        vec!["pubkey", "slot", "supply", "decimals"]
    }
}

#[async_trait::async_trait]
impl Upsert for MaybeMint {
    async fn upsert(&self, pool: &sqlx::PgPool) -> CarbonResult<()> {
        let Some(row) = &self.0 else { return Ok(()) };
        sqlx::query(
            "INSERT INTO mints (pubkey, slot, supply, decimals) VALUES ($1, $2, $3, $4)
             ON CONFLICT (pubkey) DO UPDATE SET slot = $2, supply = $3, decimals = $4",
        )
        .bind(row.pubkey)
        .bind(&row.slot)
        .bind(&row.supply)
        .bind(row.decimals)
        .execute(pool)
        .await
        .map_err(|e| Error::Custom(e.to_string()))?;
        Ok(())
    }
}

#[async_trait::async_trait]
impl Lookup for MintRow {
    type Key = Pubkey;

    async fn lookup(key: Pubkey, pool: &sqlx::PgPool) -> CarbonResult<Option<Self>> {
        sqlx::query_as("SELECT * FROM mints WHERE pubkey = $1")
            .bind(key)
            .fetch_optional(pool)
            .await
            .map_err(|e| Error::Custom(e.to_string()))
    }
}

#[async_trait::async_trait]
impl Delete for MintRow {
    type Key = Pubkey;

    async fn delete(key: Pubkey, pool: &sqlx::PgPool) -> CarbonResult<()> {
        sqlx::query("DELETE FROM mints WHERE pubkey = $1")
            .bind(key)
            .execute(pool)
            .await
            .map_err(|e| Error::Custom(e.to_string()))?;
        Ok(())
    }
}

pub fn mint_processor(pool: sqlx::PgPool) -> PostgresAccountProcessor<TokenProgramAccount, MaybeMint> {
    PostgresAccountProcessor::new(pool)
}
```

Register it with `.account(TokenProgramDecoder, mints::mint_processor(pool.clone()))`. It needs `carbon-token-program-decoder = "2.0.0"` and `async-trait = "0.1.89"` in addition to the dependencies above.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/postgres · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# GraphQL API (Carbon 2.0)

> Every public item in carbon_core::graphql — the Juniper schema and Axum router helpers, and the GraphQL scalars for Solana types.

`carbon_core::graphql` serves indexed data over GraphQL with [Juniper](https://docs.rs/juniper/0.16) and [Axum](https://docs.rs/axum/0.8). It has two parts: [`server`](#server), which builds a schema and an Axum router with `/graphql` and `/graphiql`, and [`primitives`](#primitives), scalars that carry Solana pubkeys and wide integers without losing precision. For a walkthrough, see [Serve data over GraphQL](https://carbon.sevenlabs.org/guides/graphql.md).

The module is behind the `graphql` feature of `carbon-core`:

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["graphql"] }
juniper = "0.16.2"
axum = "0.8.7"
tokio = { version = "1.48.0", features = ["full"] }
```

> **Note: Not on docs.rs.** docs.rs builds `carbon-core` 2.0.0 with default features only, so the `graphql` module has no docs.rs page. The links below point to the source at `v2.0.0`. Run `cargo doc -p carbon-core --features graphql --open` for local API docs.

## Server

`carbon_core::graphql::server`. Source: [`server.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/graphql/server.rs).

### `build_schema`

Wraps a query root in a Juniper schema with no mutations and no subscriptions.

```rust
pub fn build_schema<Q, C>(
    query_root: Q,
) -> Arc<RootNode<'static, Q, DefaultMutation<C>, DefaultSubscription<C>>>
where
    Q: juniper::GraphQLType<Context = C, TypeInfo = ()> + Send + Sync + 'static,
    C: juniper::Context + Send + Sync + 'static,
```

**Details.**

| Parameter    | Type | Description                                                                                                                  |
| ------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------- |
| `query_root` | `Q`  | Your query type, usually a unit struct with a `#[graphql_object(context = C)]` impl. Generated decoders provide `QueryRoot`. |

Returns the schema in an `Arc`, ready for [`graphql_router`](#graphql_router). The schema is read-only: you cannot add mutations or subscriptions through this helper. To have them, build a `juniper::RootNode` yourself and mount it with `juniper_axum` directly.

**Example.**

```rust
let schema = build_schema(Query);
```

**See also.** [`graphql_router`](#graphql_router), [`DefaultMutation`](#defaultmutation-and-defaultsubscription).

### `graphql_router`

Builds an Axum `Router` that serves the schema.

```rust
pub fn graphql_router<Q, C>(
    schema: Arc<RootNode<'static, Q, DefaultMutation<C>, DefaultSubscription<C>>>,
    context: C,
) -> axum::Router
where
    Q: juniper::GraphQLTypeAsync<Context = C, TypeInfo = ()>
        + juniper::GraphQLValueAsync<Context = C, TypeInfo = ()>
        + Send + Sync + 'static,
    C: juniper::Context + Clone + Send + Sync + 'static,
```

**Details.**

| Parameter | Type               | Description                                                                       |
| --------- | ------------------ | --------------------------------------------------------------------------------- |
| `schema`  | `Arc<RootNode<…>>` | The schema from [`build_schema`](#build_schema).                                  |
| `context` | `C`                | Shared context, cloned into each request. Typically holds an `Arc<sqlx::PgPool>`. |

Routes:

| Path        | Methods       | Serves                                                                                                           |
| ----------- | ------------- | ---------------------------------------------------------------------------------------------------------------- |
| `/graphql`  | `GET`, `POST` | Executes a GraphQL request. `GET` reads `?query=…`; `POST` reads a JSON body `{"query": "…", "variables": {…}}`. |
| `/graphiql` | `GET`         | The GraphiQL playground, pointed at `/graphql`.                                                                  |

There is no subscriptions endpoint, no CORS layer and no authentication. Merge the router into your own app with `Router::merge` or `Router::nest` to add them. The router does not bind a port; pass it to `axum::serve`.

**Example.**

```rust
let app = graphql_router(schema, Context { slot: 380_000_000 });
let listener = tokio::net::TcpListener::bind("127.0.0.1:8080").await.unwrap();
axum::serve(listener, app).await.unwrap();
```

**See also.** [Example: a minimal server](#example-a-minimal-server).

### `DefaultMutation` and `DefaultSubscription`

Type aliases for the empty mutation and subscription roots `build_schema` uses.

```rust
pub type DefaultMutation<C> = juniper::EmptyMutation<C>;
pub type DefaultSubscription<C> = juniper::EmptySubscription<C>;
```

**Details.** Use them to name the schema type, for example in a function signature or a struct field.

**Example.**

```rust
use carbon_core::graphql::server::{DefaultMutation, DefaultSubscription};

type Schema = juniper::RootNode<'static, Query, DefaultMutation<Context>, DefaultSubscription<Context>>;
```

**See also.** [`build_schema`](#build_schema).

## Primitives

`carbon_core::graphql::primitives`. Source: [`primitives.rs`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/crates/core/src/graphql/primitives.rs).

GraphQL's built-in `Int` is 32-bit and JavaScript numbers lose precision above 2⁵³, so Carbon's scalars send wide integers as **strings**. Each is a newtype with one public field, derives `Clone, Debug, PartialEq, Eq, Serialize, Deserialize` and `juniper::GraphQLScalar`, and appears in the schema under its Rust name.

| Scalar                              | Wraps                   | Output                               | Accepted input           |
| ----------------------------------- | ----------------------- | ------------------------------------ | ------------------------ |
| [`Pubkey`](#pubkey)                 | `solana_pubkey::Pubkey` | base58 string                        | base58 string            |
| [`I64`](#i64-i128-u8-u32-u64-u128)  | `i64`                   | decimal string                       | decimal string           |
| [`I128`](#i64-i128-u8-u32-u64-u128) | `i128`                  | decimal string                       | decimal string           |
| [`U8`](#i64-i128-u8-u32-u64-u128)   | `u8`                    | decimal string                       | decimal string           |
| [`U32`](#i64-i128-u8-u32-u64-u128)  | `u32`                   | decimal string                       | decimal string           |
| [`U64`](#i64-i128-u8-u32-u64-u128)  | `u64`                   | decimal string                       | decimal string           |
| [`U128`](#i64-i128-u8-u32-u64-u128) | `u128`                  | decimal string                       | decimal string           |
| [`Json`](#json)                     | `serde_json::Value`     | GraphQL value (object, list, scalar) | a string containing JSON |

There is no `U16` or `I8`/`I16`/`I32` scalar; use GraphQL `Int` (`i32`) for those.

### `Pubkey`

A Solana public key as a base58 string.

```rust
#[derive(Clone, Debug, Deserialize, Eq, GraphQLScalar, PartialEq, Serialize)]
#[graphql(parse_token(String), description = "A Solana's public key")]
pub struct Pubkey(pub solana_pubkey::Pubkey);

impl Pubkey {
    pub fn new<S: Into<String>>(value: S) -> Result<Self, String>
}
impl Deref for Pubkey { type Target = solana_pubkey::Pubkey; }
impl Display for Pubkey
```

**Details.** `new` parses a base58 string and returns the parse error as a `String`. As an input value the scalar accepts only a base58 string; a number literal fails with `There needs to be an Int type`. It is a different type from `carbon_core::postgres::primitives::Pubkey`; convert with `graphql::primitives::Pubkey(*pg_pubkey)`.

**Example.**

```rust
let program = Pubkey::new("LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo").unwrap();
println!("{program}");
```

**See also.** [Postgres `Pubkey`](https://carbon.sevenlabs.org/reference/postgres.md#pubkey).

### `I64`, `I128`, `U8`, `U32`, `U64`, `U128`

Integers sent as decimal strings.

```rust
pub struct I64(pub i64);
pub struct I128(pub i128);
pub struct U8(pub u8);
pub struct U32(pub u32);
pub struct U64(pub u64);
pub struct U128(pub u128);
```

**Details.** Output is always a string, for example `"slot": "380000000"`. Input must be a string literal or variable. For a field `echo(v: U64!)`, `echo(v: "18446744073709551615")` works, `echo(v: 5)` fails with `There needs to be an Int type`, and a string that does not parse fails with `Invalid value "\"abc\"" for type "U64"`. These types have no `From` or `Deref` implementations: construct them with `U64(value)` and read with `.0`.

**Example.**

```rust
#[derive(juniper::GraphQLObject)]
struct Status {
    slot: U64,
    program: Pubkey,
}
```

**See also.** [Example: a minimal server](#example-a-minimal-server).

### `Json`

An arbitrary JSON value.

```rust
#[derive(Clone, Debug, Serialize, Deserialize, Eq, GraphQLScalar, PartialEq)]
#[graphql(parse_token(String), description = "Arbitrary JSON value")]
pub struct Json(pub serde_json::Value);
```

**Details.** Output converts the JSON value to a GraphQL value of the same shape. Numbers that fit in `i32` stay numbers; larger integers and every `u64` above `i64::MAX` become strings, and floats stay floats. As input, the scalar accepts only a string that contains JSON (`j(v: "{\"a\":1}")`); a GraphQL object literal (`j(v: {a: 1})`) is a parse error. Generated decoders use it for the `accounts` field of instructions and for nested types.

**Example.**

```rust
let accounts = Json(serde_json::json!({ "lb_pair": "…" }));
```

**See also.** [Generated decoder types](#generated-decoder-types).

## Generated decoder types

A decoder built with its `graphql` feature adds a schema that reads the decoder's [typed Postgres tables](https://carbon.sevenlabs.org/reference/postgres.md#generated-decoder-types).

> **Warning: Enable postgres too.** The generated GraphQL code queries the typed tables, but a decoder's `graphql` feature does not enable its `postgres` feature. With `features = ["graphql"]` alone, `carbon-meteora-dlmm-decoder` 2.0.0 fails to compile (`cannot find postgres in accounts`). Use `features = ["postgres", "graphql"]`.

| Item                   | Path                                          | Description                                                                                                                                                            |
| ---------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `QueryRoot`            | `graphql` (re-exported from `graphql::query`) | Query root with two fields per account type and two per instruction type.                                                                                              |
| `GraphQLContext`       | `graphql::context`                            | `#[derive(Clone)] pub struct GraphQLContext { pub pool: Arc<sqlx::PgPool> }`.                                                                                          |
| `<Account>GraphQL`     | `accounts::graphql`                           | GraphQL object for one account type, with an `account_metadata: AccountMetadataGraphQL { pubkey, slot }` field (`accountMetadata` in the schema).                      |
| `<Instruction>GraphQL` | `instructions::graphql`                       | GraphQL object for one instruction, with `instruction_metadata: InstructionMetadataGraphQL { signature, instruction_index, stack_height, slot }` and `accounts: Json`. |

`QueryRoot` fields (Juniper exposes them in camelCase):

| Field                                        | Arguments                                      | Returns                                          | Query                                                             |
| -------------------------------------------- | ---------------------------------------------- | ------------------------------------------------ | ----------------------------------------------------------------- |
| `<account>` (for example `lbPair`)           | `pubkey: String!`                              | the account or `null`                            | `Lookup` by pubkey                                                |
| `list<Account>` (for example `listLbPair`)   | `limit: Int!`, `offset: Int!`                  | list                                             | `ORDER BY __slot DESC`                                            |
| `<instruction>` (for example `swap`)         | `signature: String!`, `instructionIndex: Int!` | list of every row for that top-level instruction | `ORDER BY __stack_height ASC`                                     |
| `list<Instruction>` (for example `listSwap`) | `limit: Int!`, `offset: Int!`                  | list                                             | `ORDER BY __slot DESC, __signature DESC, __instruction_index ASC` |

Rows that fail to convert are skipped silently.

**Example.** Serve a generated decoder's schema over the tables written by [`PostgresInstructionProcessor`](https://carbon.sevenlabs.org/reference/postgres.md#postgresinstructionprocessor):

`src/main.rs`:

```rust
use {
    carbon_core::graphql::server::{build_schema, graphql_router},
    carbon_meteora_dlmm_decoder::graphql::{context::GraphQLContext, QueryRoot},
    std::sync::Arc,
};

#[tokio::main]
async fn main() {
    let database_url = std::env::var("DATABASE_URL").expect("DATABASE_URL must be set");
    let pool = sqlx::PgPool::connect(&database_url).await.unwrap();

    let schema = build_schema(QueryRoot);
    let app = graphql_router(schema, GraphQLContext { pool: Arc::new(pool) });

    let listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}
```

```graphql
{
  listSwap(limit: 5, offset: 0) {
    instructionMetadata { signature slot }
    amountIn
    minAmountOut
  }
}
```

Each generated `QueryRoot` is a distinct type, so one router serves one decoder. To serve several programs, wrap their query roots in a query type of your own that delegates to each.

## Example: a minimal server

A complete server with a custom query, no database required. Copy it to check your setup before wiring Postgres.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = { version = "2.0.0", features = ["graphql"] }
carbon-meteora-dlmm-decoder = "2.0.0"
juniper = "0.16.2"
axum = "0.8.7"
tokio = { version = "1.48.0", features = ["full"] }
```

`src/main.rs`:

```rust
use {
    carbon_core::graphql::{
        primitives::{Pubkey, U64},
        server::{build_schema, graphql_router},
    },
    juniper::{graphql_object, GraphQLObject},
};

#[derive(Clone)]
struct Context {
    slot: u64,
}

impl juniper::Context for Context {}

#[derive(GraphQLObject)]
struct Status {
    slot: U64,
    program: Pubkey,
}

struct Query;

#[graphql_object(context = Context)]
impl Query {
    fn status(context: &Context) -> Status {
        Status {
            slot: U64(context.slot),
            program: Pubkey(carbon_meteora_dlmm_decoder::PROGRAM_ID),
        }
    }
}

#[tokio::main]
async fn main() {
    let schema = build_schema(Query);
    let app = graphql_router(schema, Context { slot: 380_000_000 });

    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080").await.unwrap();
    println!("GraphiQL: http://127.0.0.1:8080/graphiql");
    axum::serve(listener, app).await.unwrap();
}
```

```sh
cargo run
```

Output:

```text
GraphiQL: http://127.0.0.1:8080/graphiql
```

In another terminal:

```sh
curl -s -X POST http://127.0.0.1:8080/graphql \
  -H 'content-type: application/json' \
  -d '{"query":"{ status { slot program } }"}'
```

Output:

```json
{"data":{"status":{"slot":"380000000","program":"LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo"}}}
```

The `slot` field is a string because `U64` is a string scalar.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/graphql · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Test utilities (Carbon 2.0)

> The carbon-test-utils API — JSON fixture loaders for instructions, accounts and transaction metadata, and a base58 helper for decoder tests.

`carbon-test-utils` loads Solana instructions, accounts and transaction metadata from JSON fixture files so you can unit-test decoders and processors without a network. Use this page when you write decoder tests; for a walkthrough of testing a pipeline, see [Testing](https://carbon.sevenlabs.org/guides/testing.md).

`Cargo.toml`:

```toml
[dev-dependencies]
carbon-test-utils = "2.0.0"
```

The crate exposes three loaders, three fixture types and one module. The loaders return `anyhow::Result`; add `anyhow` to your dev-dependencies only if your tests return `anyhow::Result<()>`.

| Item                                                                                                        | Purpose                                                   |
| ----------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| [`read_instruction`](#read_instruction)                                                                     | Load a `solana_instruction::Instruction` from JSON.       |
| [`read_account`](#read_account)                                                                             | Load a `solana_account::Account` from JSON.               |
| [`read_transaction_meta`](#read_transaction_meta)                                                           | Load a `UiTransactionStatusMeta` (the RPC `meta` object). |
| [`TestInstruction`](#testinstruction), [`TestAccountMeta`](#testaccountmeta), [`TestAccount`](#testaccount) | The JSON shapes the loaders accept.                       |
| [`base58_deserialize::ix_data`](#base58_deserializeix_data)                                                 | Decode a base58 string into bytes.                        |

## `read_instruction`

Reads a JSON file and returns the instruction it describes.

```rust
pub fn read_instruction<P: AsRef<Path>>(ix_path: P) -> anyhow::Result<solana_instruction::Instruction>
```

**Details.** The file must match [`TestInstruction`](#testinstruction): `program_id` and each `pubkey` as base58 strings, `data` as a **hex** string. Paths are relative to the current directory; `cargo test` runs tests from the package root, so `tests/fixtures/…` works. Errors are `Couldn't read fixture: …` when the file cannot be read and `Couldn't deserialize fixture: …` when the JSON does not match.

`tests/fixtures/transfer_ix.json`:

```json
{
  "program_id": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA",
  "accounts": [
    { "pubkey": "7VHUFJHWu2CuExkJcJrzhQPJ2oygupTWkL2A2For4BmE", "is_signer": false, "is_writable": true },
    { "pubkey": "3emsAVdmGKERbHjmGfQ6oZ1e35dkf5iYcS6U4CPKFVaa", "is_signer": false, "is_writable": true },
    { "pubkey": "5Q544fKrFoe6tsEbD7S8EmxGTJYAKtTVhAW5Q5pge4j1", "is_signer": true, "is_writable": false }
  ],
  "data": "03e803000000000000"
}
```

**Example.**

```rust
let instruction = carbon_test_utils::read_instruction("tests/fixtures/transfer_ix.json").unwrap();
```

**See also.** [Example: decoder tests](#example-decoder-tests), [`InstructionDecoder`](https://carbon.sevenlabs.org/reference/traits.md).

## `read_account`

Reads a JSON file and returns the account it describes.

```rust
pub fn read_account<P: AsRef<Path>>(acc_path: P) -> anyhow::Result<solana_account::Account>
```

**Details.** The file must match [`TestAccount`](#testaccount): `data` as **base64** (the encoding `getAccountInfo` returns with `"encoding": "base64"`), `owner` as base58, `lamports` and `rent_epoch` as JSON numbers. This is not the raw RPC response: copy `value.data[0]` into `data` and the other fields across, renaming `rentEpoch` to `rent_epoch`.

`tests/fixtures/mint_account.json`:

```json
{
  "data": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMqaOwAAAAAGAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==",
  "executable": false,
  "lamports": 1461600,
  "owner": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA",
  "rent_epoch": 18446744073709551615
}
```

**Example.**

```rust
let account = carbon_test_utils::read_account("tests/fixtures/mint_account.json").unwrap();
```

**See also.** [`AccountDecoder`](https://carbon.sevenlabs.org/reference/traits.md).

## `read_transaction_meta`

Reads a JSON file containing a transaction's status metadata in RPC form.

```rust
pub fn read_transaction_meta<P: AsRef<Path>>(
    tx_path: P,
) -> anyhow::Result<solana_transaction_status::UiTransactionStatusMeta>
```

**Details.** The file is the `meta` object of a `getTransaction` response (camelCase keys: `fee`, `preBalances`, `innerInstructions`, `logMessages`, …). Inner instructions must use the compiled form with base58 `data`, which is what `"encoding": "json"` returns. The conversion below replaces parsed (`jsonParsed`) inner instructions with empty placeholders and logs `Unsupported instruction type encountered`. To get the binary `TransactionStatusMeta` Carbon's pipeline uses, pass the result to `carbon_core::transformers::transaction_metadata_from_original_meta`.

`tests/fixtures/tx_meta.json`:

```json
{
  "err": null,
  "status": { "Ok": null },
  "fee": 5000,
  "preBalances": [1000000000, 0],
  "postBalances": [999995000, 0],
  "innerInstructions": [],
  "logMessages": ["Program 11111111111111111111111111111111 invoke [1]", "Program 11111111111111111111111111111111 success"],
  "preTokenBalances": [],
  "postTokenBalances": [],
  "rewards": [],
  "loadedAddresses": { "writable": [], "readonly": [] },
  "computeUnitsConsumed": 150
}
```

**Example.**

```rust
let meta = carbon_test_utils::read_transaction_meta("tests/fixtures/tx_meta.json").unwrap();
let meta = carbon_core::transformers::transaction_metadata_from_original_meta(meta).unwrap();
```

**See also.** [`TransactionMetadata`](https://carbon.sevenlabs.org/reference/processor-inputs.md).

## `TestInstruction`

The JSON shape [`read_instruction`](#read_instruction) accepts.

```rust
#[derive(Debug, Deserialize)]
pub struct TestInstruction {
    #[serde(with = "field_as_string")]
    pub program_id: Pubkey,
    pub accounts: Vec<TestAccountMeta>,
    #[serde(with = "hex_deserialize")]
    pub data: Vec<u8>,
}

impl From<TestInstruction> for solana_instruction::Instruction
```

**Details.**

| Field        | JSON type                                      | Description                    |
| ------------ | ---------------------------------------------- | ------------------------------ |
| `program_id` | base58 string                                  | Program the instruction calls. |
| `accounts`   | array of [`TestAccountMeta`](#testaccountmeta) | Accounts in instruction order. |
| `data`       | hex string, no `0x` prefix                     | Instruction data.              |

Deserialize-only: it does not implement `Serialize`, so you cannot write fixtures with it.

**Example.**

```rust
let fixture: carbon_test_utils::TestInstruction = serde_json::from_str(json)?;
let instruction: solana_instruction::Instruction = fixture.into();
```

**See also.** [`read_instruction`](#read_instruction).

## `TestAccountMeta`

One account of a [`TestInstruction`](#testinstruction).

```rust
#[derive(Debug, Deserialize)]
pub struct TestAccountMeta {
    #[serde(with = "field_as_string")]
    pub pubkey: Pubkey,
    pub is_signer: bool,
    pub is_writable: bool,
}

impl From<TestAccountMeta> for solana_instruction::AccountMeta
```

**Details.**

| Field         | JSON type     | Description                      |
| ------------- | ------------- | -------------------------------- |
| `pubkey`      | base58 string | Account address.                 |
| `is_signer`   | boolean       | Whether the account signs.       |
| `is_writable` | boolean       | Whether the account is writable. |

**Example.**

```json
{ "pubkey": "5Q544fKrFoe6tsEbD7S8EmxGTJYAKtTVhAW5Q5pge4j1", "is_signer": true, "is_writable": false }
```

**See also.** [`TestInstruction`](#testinstruction).

## `TestAccount`

The JSON shape [`read_account`](#read_account) accepts.

```rust
#[derive(Debug, Deserialize)]
pub struct TestAccount {
    #[serde(with = "base64_deserialize")]
    pub data: Vec<u8>,
    pub executable: bool,
    pub lamports: u64,
    #[serde(with = "field_as_string")]
    pub owner: Pubkey,
    pub rent_epoch: u64,
}

impl From<TestAccount> for solana_account::Account
```

**Details.**

| Field        | JSON type                                 | Description                                                               |
| ------------ | ----------------------------------------- | ------------------------------------------------------------------------- |
| `data`       | base64 string (standard alphabet, padded) | Account data.                                                             |
| `executable` | boolean                                   | Whether the account is a program.                                         |
| `lamports`   | number                                    | Balance in lamports.                                                      |
| `owner`      | base58 string                             | Owning program.                                                           |
| `rent_epoch` | number                                    | Rent epoch; `18446744073709551615` (`u64::MAX`) for rent-exempt accounts. |

**Example.**

```rust
let fixture: carbon_test_utils::TestAccount = serde_json::from_str(json)?;
let account: solana_account::Account = fixture.into();
```

**See also.** [`read_account`](#read_account).

## `base58_deserialize::ix_data`

Decodes a base58 string into bytes.

```rust
pub fn ix_data(ix: &str) -> Vec<u8>
```

**Details.** Use it to paste instruction data from an explorer or from a `getTransaction` response with `"encoding": "json"` (where `data` is base58) straight into a test. It panics on invalid base58; the panic message reads `hex decode failed` even though the input is base58. Despite the module name, it is a plain function, not a serde adapter.

**Example.**

```rust
use carbon_test_utils::base58_deserialize;

// [3, 232, 3, 0, 0, 0, 0, 0, 0]: an SPL Token Transfer of 1000.
let data = base58_deserialize::ix_data("3tGNFMqHiozw");
```

**See also.** [Example: decoder tests](#example-decoder-tests).

## Example: decoder tests

Integration tests for the SPL Token decoder, using the three fixture files above saved under `tests/fixtures/`.

`Cargo.toml`:

```toml
[dependencies]
carbon-core = "2.0.0"
carbon-token-program-decoder = "2.0.0"

[dev-dependencies]
carbon-test-utils = "2.0.0"
```

`tests/decode.rs`:

```rust
use {
    carbon_core::{account::AccountDecoder, instruction::InstructionDecoder},
    carbon_test_utils::{base58_deserialize, read_account, read_instruction, read_transaction_meta},
    carbon_token_program_decoder::{
        accounts::TokenProgramAccount, instructions::TokenProgramInstruction, TokenProgramDecoder,
    },
};

#[test]
fn decodes_transfer_instruction() {
    let instruction = read_instruction("tests/fixtures/transfer_ix.json").unwrap();

    let decoded = TokenProgramDecoder
        .decode_instruction(&instruction)
        .expect("not a token program instruction");

    let TokenProgramInstruction::Transfer { data, accounts, .. } = decoded else {
        panic!("expected Transfer, got {decoded:?}");
    };
    assert_eq!(data.amount, 1_000);
    assert_eq!(
        accounts.authority.to_string(),
        "5Q544fKrFoe6tsEbD7S8EmxGTJYAKtTVhAW5Q5pge4j1"
    );
}

#[test]
fn decodes_mint_account() {
    let account = read_account("tests/fixtures/mint_account.json").unwrap();

    let decoded = TokenProgramDecoder
        .decode_account(&account)
        .expect("not a token program account");

    let TokenProgramAccount::Mint(mint) = decoded.data else {
        panic!("expected Mint");
    };
    assert_eq!(mint.supply, 1_000_000_000);
    assert_eq!(mint.decimals, 6);
}

#[test]
fn loads_transaction_meta() {
    let meta = read_transaction_meta("tests/fixtures/tx_meta.json").unwrap();
    assert_eq!(meta.fee, 5000);

    // Convert the RPC (UI) shape into the binary meta Carbon's pipeline uses.
    let meta = carbon_core::transformers::transaction_metadata_from_original_meta(meta).unwrap();
    assert_eq!(meta.log_messages.unwrap().len(), 2);
}

#[test]
fn decodes_base58_instruction_data() {
    // Base58 of [3, 232, 3, 0, 0, 0, 0, 0, 0]: a token Transfer of 1000.
    let data = base58_deserialize::ix_data("3tGNFMqHiozw");
    assert_eq!(data, vec![3, 232, 3, 0, 0, 0, 0, 0, 0]);
}
```

```sh
cargo test --test decode
```

Output:

```text
running 4 tests
test decodes_base58_instruction_data ... ok
test decodes_mint_account ... ok
test decodes_transfer_instruction ... ok
test loads_transaction_meta ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/test-utils · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# CLI (Carbon 2.0)

> Every carbon-cli 2.0.0 command and flag, the scaffold datasources, the generated project layout, environment variables and exit codes.

`@sevenlabs-hq/carbon-cli` generates Carbon decoder crates from Anchor or Codama IDLs (`parse`) and scaffolds complete indexer workspaces around a generated decoder (`scaffold`). This page documents the published 2.0.0 package; its README lists only part of the flags.

Requirements: Node.js 20.18.0 or newer. `cargo fmt` (rustfmt) on your `PATH` to format the output; if it is missing the CLI prints a warning and leaves the code unformatted. The generated Rust code needs Rust 1.96.1 or newer.

## Run the CLI

Run it without installing, pinned to the version:

**npx**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 --help
```

**pnpm**

```sh
pnpm dlx @sevenlabs-hq/carbon-cli@2.0.0 --help
```

**bun**

```sh
bunx @sevenlabs-hq/carbon-cli@2.0.0 --help
```

**yarn**

```sh
yarn dlx @sevenlabs-hq/carbon-cli@2.0.0 --help
```

A global install (`npm install -g @sevenlabs-hq/carbon-cli@2.0.0`) provides two identical binaries, `carbon-cli` and `carbon`. Commands on this page follow the runner you pick in any runner tab.

## `carbon-cli`

The root command. It dispatches to `parse`, `scaffold` and `help`.

```text
carbon-cli [options] [command]
```

**Details.**

| Option / command     | Description                                                                         |
| -------------------- | ----------------------------------------------------------------------------------- |
| `-V`, `--version`    | Prints the version (`2.0.0`) and exits.                                             |
| `-h`, `--help`       | Prints help with usage examples and exits. Works on every command (`parse --help`). |
| `help [command]`     | Same as `--help` for the root or for `command`.                                     |
| `parse [options]`    | Generates a decoder crate. See [`parse`](#parse).                                   |
| `scaffold [options]` | Generates an indexer workspace. See [`scaffold`](#scaffold).                        |

Every `parse` and `scaffold` run prints an ASCII-art banner first. An unknown command or option exits with code 1 (`error: unknown command 'foo'`).

**Example.**

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 --version
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

Output:

```text
2.0.0
```

**See also.** [Exit codes](#exit-codes), [Interactive mode](#interactive-mode).

## `parse`

Generates a decoder crate from an IDL file or from the Anchor IDL stored on chain for a program address.

```text
carbon-cli parse [options]
```

**Details.** If `--idl` or `--out-dir` is missing, `parse` switches to [interactive mode](#interactive-mode) and prompts for them.

| Flag                                | Type                           | Default                                              | Description                                                                                                                                                                                                                                                                        |
| ----------------------------------- | ------------------------------ | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-i`, `--idl <fileOrAddress>`       | path or base58 address         | prompted                                             | A path ending in `.json`, or a program address (base58, 32 to 44 characters). Anything else exits with `Invalid IDL: must be a .json file or a Solana program address`.                                                                                                            |
| `-o`, `--out-dir <dir>`             | path                           | prompted (`./generated`)                             | Directory the crate is written to, resolved against the current directory. **Deleted and recreated** unless you pass `--no-clean`.                                                                                                                                                 |
| `-n`, `--name <string>`             | string                         | IDL program name                                     | Decoder name. Whitespace becomes `-`. Produces the crate `carbon-<name>-decoder` (kebab case) and the struct `<Name>Decoder` (Pascal case).                                                                                                                                        |
| `-s`, `--standard <anchor\|codama>` | `anchor` or `codama`           | `anchor`                                             | IDL format. Program addresses support `anchor` only.                                                                                                                                                                                                                               |
| `-u`, `--url <rpcUrl>`              | URL or alias                   | none                                                 | RPC endpoint for fetching the IDL; required when `--idl` is a program address. See [details](#parse-flag-details).                                                                                                                                                                 |
| `--program-id <address>`            | base58 address                 | none                                                 | Program ID for an IDL without one. **Required for every Codama IDL.** See [details](#parse-flag-details).                                                                                                                                                                          |
| `--event-hints <csv>`               | comma-separated names          | none                                                 | Codama only. Has no effect in 2.0.0. See [details](#parse-flag-details).                                                                                                                                                                                                           |
| `--event-cpi-discriminator <hex>`   | even-length hex, optional `0x` | `e445a52e51cb9a1d`                                   | Codama only. Envelope prefix of hidden-prefix event CPIs.                                                                                                                                                                                                                          |
| `--postgres-mode <generic\|typed>`  | `generic` or `typed`           | `typed`                                              | `typed` generates per-account and per-instruction row types, tables and migrations (`accounts::postgres`, `instructions::postgres`). `generic` skips them; you store data as JSON with the generic processors. See [Postgres](https://carbon.sevenlabs.org/reference/postgres.md). |
| `--with-postgres <boolean>`         | `true` or `false`              | `true`                                               | Generates the Postgres modules and a `postgres` Cargo feature.                                                                                                                                                                                                                     |
| `--with-graphql <boolean>`          | `true` or `false`              | `true`                                               | Generates the GraphQL modules and a `graphql` Cargo feature.                                                                                                                                                                                                                       |
| `--with-serde <boolean>`            | `true` or `false`              | `false`, or `true` when both flags above are `false` | Declares the `serde` feature. `postgres` and `graphql` enable it on their own.                                                                                                                                                                                                     |
| `--with-base58 <boolean>`           | `true` or `false`              | `false`                                              | Adds a `base58` feature (implies `serde`) that serializes `Pubkey` fields as base58 strings.                                                                                                                                                                                       |
| `--standalone <boolean>`            | `true` or `false`              | `true`                                               | `false` makes every dependency `{ workspace = true }`, for crates inside a Cargo workspace.                                                                                                                                                                                        |
| `--description <string>`            | string                         | derived from the name                                | Package description.                                                                                                                                                                                                                                                               |
| `--keywords <csv>`                  | comma-separated                | `solana`, `decoder` and the name parts               | Package keywords.                                                                                                                                                                                                                                                                  |
| `--categories <csv>`                | comma-separated                | `encoding`                                           | Package categories.                                                                                                                                                                                                                                                                |
| `--package-version <string>`        | semver                         | `0.1.0`                                              | `version` in the decoder's `Cargo.toml`.                                                                                                                                                                                                                                           |
| `--version-name <name>`             | string                         | none                                                 | Appends `-<name>` to the crate name (`carbon-dex-decoder-v1`) for [versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md).                                                                                                                                 |
| `--no-clean`                        | flag                           | cleaning on                                          | Keeps the existing contents of `--out-dir` and writes over them.                                                                                                                                                                                                                   |
| `-c`, `--as-crate`                  | flag                           | `false`                                              | Accepted but has no effect in 2.0.0: the output is always a crate.                                                                                                                                                                                                                 |

Boolean flags take a value: write `--with-postgres false`. Only the literal string `true` is true; `yes`, `1` or `TRUE` count as `false`. `--description`, `--keywords` and `--categories` are written to `Cargo.toml` only when at least one of them is passed.

#### Parse flag details

- **`--url`.** `mainnet-beta` and `devnet` expand to `https://api.mainnet-beta.solana.com` and `https://api.devnet.solana.com`; any other value is used as is.
- **`--program-id`.** For an Anchor IDL, the value is written to `PROGRAM_ID` when the IDL has no valid `address` (or `metadata.address`) and ignored when it has one. For a Codama IDL it's required, because the CLI checks only `address` and `metadata.address`, not `program.publicKey`. Without it, `parse` exits with code 2: `Program ID is required. IDL missing or invalid address field - provide via --program-id parameter`. The value is then ignored: `PROGRAM_ID` comes from `program.publicKey`.
- **`--event-hints`.** Accepted for compatibility but never passed to the renderer, so the output is byte-identical with and without it. Event types and the `CpiEvent` variant are generated only from native Codama `eventNode`s. Passing it with `-s anchor` exits with code 2.
- **`--standalone`.** `true` adds an empty `[workspace]` table so the crate builds on its own. `false` writes every dependency as `{ workspace = true }`, so the parent workspace must define them in `[workspace.dependencies]`.

When `--idl` is a program address, the CLI reads the program's Anchor IDL account with `@coral-xyz/anchor` at `confirmed` commitment. Programs that never ran `anchor idl init` exit with `No Anchor IDL found for program address.`

The generated crate depends on `carbon-core = { version = "2.0.0", features = ["macros"], default-features = false }`, `borsh`, `solana-pubkey`, `solana-account`, `solana-instruction` and `serde_json`, with `serde`, `sqlx`, `sqlx_migrator`, `async-trait` and `juniper` as optional dependencies behind its features. Its layout:

```text
<out-dir>/
├── Cargo.toml
└── src/
    ├── lib.rs            # <Name>Decoder, PROGRAM_ID, EVENT_CPI_DISCRIMINATOR (when the IDL has events)
    ├── accounts/         # <Name>Account enum + one file per account; postgres/ and graphql/ when enabled
    ├── instructions/     # <Name>Instruction enum + one file per instruction (and cpi_event.rs); postgres/, graphql/
    ├── events/           # one file per event, when the IDL has events
    ├── types/            # defined types; graphql/ when enabled
    └── graphql/          # context.rs, query.rs (QueryRoot), when --with-graphql true
```

Directories with nothing to generate are omitted. See [Feature flags](https://carbon.sevenlabs.org/reference/feature-flags.md) for the decoder's Cargo features.

**Example.** Generate a decoder for Pump.fun AMM from its on-chain IDL:

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse \
  --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA \
  --url mainnet-beta \
  --name pump-amm \
  --out-dir ./pump-amm-decoder
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

Output:

```text
✔ Decoder generated
✔ Code formatted successfully

✨ Decoder Generated Successfully! ✨
  • Output: /home/you/pump-amm-decoder
  • Source: program-address (anchor) @ https://api.mainnet-beta.solana.com
```

With `--postgres-mode generic`, the crate keeps its `postgres` feature but has no typed tables: compared with the default `typed` output above, `src/accounts/postgres/` and `src/instructions/postgres/` are not generated. Store decoded data as JSON with the [generic processors](https://carbon.sevenlabs.org/reference/postgres.md) instead:

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse \
  --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA \
  --url mainnet-beta \
  --name pump-amm \
  --out-dir ./pump-amm-decoder \
  --postgres-mode generic
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

From a local Codama IDL. `--program-id` is required for Codama files even though the generated `PROGRAM_ID` is read from the IDL's `program.publicKey`:

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse -i ./codama.json -s codama --program-id <ProgramAddress> -o ./decoder
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

**See also.** [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md), [Decoders](https://carbon.sevenlabs.org/concepts/decoders.md), [`scaffold`](#scaffold).

## `scaffold`

Generates a Cargo workspace with a decoder generated from your IDL and an indexer binary wired to a datasource, metrics and, optionally, Postgres and GraphQL.

```text
carbon-cli scaffold [options]
```

**Details.** If any of `--name`, `--out-dir`, `--data-source` or `--idl` is missing, `scaffold` switches to [interactive mode](#interactive-mode) and prompts for the missing values.

| Flag                                | Type                                     | Default                                                   | Description                                                                                                                                                                                                                                                                                                                       |
| ----------------------------------- | ---------------------------------------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-n`, `--name <string>`             | string                                   | prompted                                                  | Project directory name. The indexer package is named `<name>-indexer`. Not validated on the command line; the prompt accepts lowercase letters, digits and `-` only.                                                                                                                                                              |
| `-o`, `--out-dir <dir>`             | path                                     | prompted (`.`)                                            | Parent directory. The project goes to `<out-dir>/<name>`.                                                                                                                                                                                                                                                                         |
| `--idl <fileOrAddress>`             | path or base58 address                   | prompted                                                  | IDL file (`.json`) or program address. Same rules as `parse --idl`.                                                                                                                                                                                                                                                               |
| `--idl-standard <anchor\|codama>`   | `anchor` or `codama`                     | `anchor` (prompted in interactive mode)                   | IDL format. Program addresses support `anchor` only.                                                                                                                                                                                                                                                                              |
| `--idl-url <rpcUrl>`                | URL or alias                             | none                                                      | RPC endpoint for fetching the IDL of a program address; required in that case. Accepts the `mainnet-beta` and `devnet` aliases.                                                                                                                                                                                                   |
| `-d`, `--decoder <name>`            | string                                   | IDL program name                                          | Decoder name. Generates `carbon-<name>-decoder` in `decoder/` and the Rust names `<Name>Decoder`, `<Name>Instruction`, `<Name>Account` (each `-`-separated part capitalized: `pump-amm` gives `PumpAmm`). Without it the name comes from the IDL's `name`, `program.name` or `metadata.name`, kebab-cased, else `custom-decoder`. |
| `-s`, `--data-source <name>`        | see [Datasources](#scaffold-datasources) | prompted                                                  | Datasource the indexer uses. Any other value exits with code 2.                                                                                                                                                                                                                                                                   |
| `-m`, `--metrics <log\|prometheus>` | `log` or `prometheus`                    | `log`                                                     | `log` registers `LogMetrics::new_with_flush_interval(5)` (logs every 5 s); `prometheus` registers `PrometheusMetrics::new()` (scrape endpoint on `0.0.0.0:9464`). See [Metrics](https://carbon.sevenlabs.org/reference/metrics.md).                                                                                               |
| `--with-postgres <boolean>`         | `true` or `false`                        | `true`                                                    | Wires Postgres: connects to `DATABASE_URL`, runs migrations and stores accounts and instructions.                                                                                                                                                                                                                                 |
| `--with-graphql <boolean>`          | `true` or `false`                        | `true`                                                    | Serves a GraphQL API over the Postgres data on `0.0.0.0:8080`. Requires `--with-postgres true`.                                                                                                                                                                                                                                   |
| `--postgres-mode <generic\|typed>`  | `generic` or `typed`                     | `typed`                                                   | Table layout; see [`parse`](#parse).                                                                                                                                                                                                                                                                                              |
| `--with-serde <boolean>`            | `true` or `false`                        | `false`, or `true` when Postgres and GraphQL are both off | Enables the decoder's `serde` feature.                                                                                                                                                                                                                                                                                            |
| `--with-base58 <boolean>`           | `true` or `false`                        | `false`                                                   | Enables the decoder's `base58` feature.                                                                                                                                                                                                                                                                                           |
| `--event-hints <csv>`               | comma-separated names                    | none                                                      | Codama only. No effect in 2.0.0; see [`parse`](#parse).                                                                                                                                                                                                                                                                           |
| `--event-cpi-discriminator <hex>`   | even-length hex                          | `e445a52e51cb9a1d`                                        | Codama only; see [`parse`](#parse).                                                                                                                                                                                                                                                                                               |
| `--program-id <address>`            | base58 address                           | none                                                      | Program ID when an Anchor IDL has no valid address. Required for every Codama IDL file, where its value is ignored; see [`parse`](#parse).                                                                                                                                                                                        |
| `--package-version <string>`        | semver                                   | `0.1.0`                                                   | Version of the generated decoder crate. The indexer is always `0.0.1`.                                                                                                                                                                                                                                                            |
| `--version-name <name>`             | string                                   | none                                                      | Renames the decoder crate to `carbon-<name>-decoder-<version>`. Breaks the scaffold in 2.0.0; see [Known issues](#known-issues).                                                                                                                                                                                                  |
| `--force`                           | flag                                     | `false`                                                   | Writes into an existing `<out-dir>/<name>`. `decoder/` is deleted and regenerated; the other generated files are overwritten; files the CLI does not generate are kept. Without it an existing directory exits with code 2.                                                                                                       |

`scaffold` always generates the decoder in `decoder/` with `standalone: false` and pinned crates.io versions, so the workspace builds on its own. Because `--with-graphql` defaults to `true`, `--with-postgres false` alone exits with `GraphQL scaffolding requires Postgres. Enable --with-postgres or disable --with-graphql.`; pass both flags.

**Example.** A Postgres-free indexer that crawls Pump.fun AMM transactions over RPC:

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 scaffold \
  --name my-indexer \
  --out-dir . \
  --idl pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA \
  --idl-url mainnet-beta \
  --decoder pump-amm \
  --data-source rpc-transaction-crawler \
  --with-postgres false \
  --with-graphql false
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

Output:

```text
ℹ Using provided decoder name: pump-amm
✔ Project structure created
✔ Decoder generated successfully
✔ Code formatted successfully

✨ Scaffold Created Successfully! ✨
  • Location: /home/you/my-indexer
  • Decoder: pump-amm (generated from IDL)
```

Set the RPC endpoint in `.env`, then run the indexer from the workspace root:

`my-indexer/.env`:

```text
RPC_URL=https://api.mainnet-beta.solana.com
```

```sh
cd my-indexer
RUST_LOG=info cargo run -p my-indexer-indexer
```

Output:

```text
[2026-11-09T00:04:31Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 1, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 1, num_transaction_pipes: 0
[2026-11-09T00:04:31Z INFO  carbon_log_metrics] 00:00:00 (+3.354806ms) | 0 processed (0%), 0 successful, 0 failed (0%), 0 in queue
```

**See also.** [Generated project](#generated-project), [Environment variables](#environment-variables), [Datasources](https://carbon.sevenlabs.org/datasources.md), [Quickstart](https://carbon.sevenlabs.org/quickstart.md).

## Scaffold datasources

`--data-source` accepts exactly these five values. Each adds the datasource crate at `2.0.0` and the extra Solana crates its setup code needs.

| Value                     | Datasource                                                                                     | Reads                                                             | What the generated code subscribes to                                                                                                                                                                             |
| ------------------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `rpc-transaction-crawler` | [`RpcTransactionCrawler`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) | `RPC_URL` (no fallback: an empty URL)                             | Transactions that mention the decoder's `PROGRAM_ID`, newest first, at `finalized` commitment, 100 signatures per batch, polling every 5 s, 5 concurrent requests, default retries. Works on the public endpoint. |
| `rpc-block-subscribe`     | [`RpcBlockSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)         | `RPC_WS_URL` (falls back to `wss://api.mainnet-beta.solana.com/`) | Every transaction in every block (`RpcBlockSubscribeFilter::All`), `Base64` encoding, full details, transaction versions up to 1. The public endpoint rejects `blockSubscribe`; use a provider that supports it.  |
| `rpc-program-subscribe`   | [`RpcProgramSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md)     | `RPC_WS_URL` (falls back to `wss://api.mainnet-beta.solana.com/`) | Account updates for accounts owned by `PROGRAM_ID`, `Base64` encoding. Emits account updates only.                                                                                                                |
| `yellowstone-grpc`        | [`YellowstoneGrpcGeyserClient`](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md)  | `GEYSER_URL`, `X_TOKEN` (optional)                                | Accounts owned by `PROGRAM_ID`, plus successful non-vote transactions that include it, at `Confirmed` commitment.                                                                                                 |
| `helius-laserstream`      | [`LaserStreamGeyserClient`](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)    | `GEYSER_URL`, `X_TOKEN` (optional)                                | Same filters as `yellowstone-grpc`, through LaserStream with `LaserStreamClientConfig::default()`.                                                                                                                |

The two gRPC datasources also add `rustls` and `yellowstone-grpc-proto = "=12.7.0"`, and `main` installs the `aws-lc-rs` crypto provider before connecting.

## Generated project

`scaffold` writes this tree (shown for `--name my-indexer` with the defaults, Postgres and GraphQL on):

```text
my-indexer/
├── .env                 # placeholder values, see Environment variables
├── .gitignore           # debug/, target/, .env, .DS_Store
├── Cargo.toml           # [workspace] members = ["indexer", "decoder"]
├── README.md            # run command and the options you chose
├── decoder/             # carbon-<decoder>-decoder, same layout as `parse` output
│   ├── Cargo.toml
│   └── src/
└── indexer/
    ├── Cargo.toml       # package my-indexer-indexer, rust-version 1.96.1
    └── src/
        └── main.rs
```

`indexer/src/main.rs` does the following, in order:

1. Initializes `env_logger`, then loads `.env` with `dotenv`.
2. With Postgres: connects to `DATABASE_URL`, then runs the decoder's migrations (`<Name>AccountsMigration` and `<Name>InstructionsMigration`, or `GenericAccountsMigration` and `GenericInstructionMigration` in `generic` mode) through `sqlx_migrator`.
3. Builds the datasource as described in [Scaffold datasources](#scaffold-datasources).
4. Builds a pipeline with the metrics exporter and `ShutdownStrategy::Immediate`, and registers:
   - with Postgres: `.account(<Name>Decoder, PostgresAccountProcessor::<…>::new(pool))` and `.instruction(<Name>Decoder, PostgresInstructionProcessor::<…>::new(pool))` (`PostgresJsonAccountProcessor` and `PostgresJsonInstructionProcessor` in `generic` mode);
   - without Postgres: `.instruction(<Name>Decoder, <Name>InstructionProcessor)`, a stub processor in `main.rs` that logs `received the <Name> instruction, sig: <signature>` at `info` level and has an empty `match` for you to fill in.
5. With GraphQL: runs the pipeline and an Axum server side by side with `tokio::select!`, serving `http://0.0.0.0:8080/graphql` and GraphiQL at `/graphiql`. The process stops when either finishes.

Generated `indexer/Cargo.toml` dependencies: `carbon-core = "2.0.0"` (default features off; `postgres`/`graphql` features when enabled), the decoder by path, the datasource and metrics crates at `2.0.0`, `tokio` 1 (`rt-multi-thread`, `macros`), `dotenv` 0.15, `env_logger` 0.11.8, `log` 0.4.28, plus `sqlx` 0.8.6 and `sqlx_migrator` 0.17 with Postgres and `axum` 0.8.7 with GraphQL.

## Interactive mode

Run `parse` or `scaffold` without the required flags and the CLI prompts for what is missing. Flags you pass are used as answers and their prompts are skipped.

**`parse`** prompts for, in order:

1. IDL file or program address
2. Output directory (`./generated`)
3. RPC URL (`mainnet-beta`), for an address
4. Event hints, with `-s codama` (no effect)
5. Program ID, when the IDL has no address (every Codama IDL)

**`scaffold`** prompts for, in order:

1. Project name
2. Output directory (`.`)
3. Overwrite confirmation, if `<out-dir>/<name>` exists (no exits with code 1)
4. IDL file or program address
5. RPC URL (`mainnet-beta`) for an address, or IDL standard for a file
6. Program ID, when the IDL has no address
7. Decoder name (Enter auto-detects)
8. Data source
9. Add Postgres (`Y`)
10. Enable GraphQL (`Y`, only with Postgres)

Options that have a default never prompt: `parse` uses `anchor` for the standard and cleans the output directory, and both commands use `typed` for `--postgres-mode`; `scaffold` uses `log` metrics. Pass the flag to choose otherwise. Ctrl+C at a prompt exits with code 1 and a stack trace.

## Environment variables

The CLI itself reads no environment variables. The scaffolded indexer reads these, from the shell or from `.env` in the directory you run it from:

| Variable       | Used when                                      | Behaviour when unset                  |
| -------------- | ---------------------------------------------- | ------------------------------------- |
| `DATABASE_URL` | `--with-postgres true`                         | Panics: `DATABASE_URL must be set`.   |
| `RPC_URL`      | `rpc-transaction-crawler`                      | Empty URL; every request fails.       |
| `RPC_WS_URL`   | `rpc-block-subscribe`, `rpc-program-subscribe` | `wss://api.mainnet-beta.solana.com/`. |
| `GEYSER_URL`   | `yellowstone-grpc`, `helius-laserstream`       | Empty URL; the connection fails.      |
| `X_TOKEN`      | `yellowstone-grpc`, `helius-laserstream`       | No `x-token` header is sent.          |
| `RUST_LOG`     | always (`env_logger`)                          | Only errors are logged.               |

The generated `.env` contains `DATABASE_URL=postgres://user:password@localhost/dbname` (with Postgres) and one placeholder per datasource variable, such as `RPC_URL=your-rpc-url-here` or `X_TOKEN=your-x-token-here`. Replace them; delete `X_TOKEN` if your endpoint needs no token, or the placeholder is sent as the token.

> **Warning: Set RUST_LOG in the shell.** `main` calls `env_logger::init()` before it loads `.env`, so a `RUST_LOG` line in `.env` has no effect. Export it or prefix the command: `RUST_LOG=info cargo run -p my-indexer-indexer`.

## Exit codes

| Code | Meaning                                                                                                       |
| ---- | ------------------------------------------------------------------------------------------------------------- |
| `0`  | Success. A `cargo fmt` failure still exits 0 after a `Formatting failed` warning.                             |
| `1`  | A usage error from the argument parser, or an unhandled exception with a Node.js stack trace. Examples below. |
| `2`  | A validation error, printed as `✗ Error: <message>`.                                                          |

Code `1` covers usage errors such as `error: unknown option '--bogus'`, `error: option '-i, --idl <fileOrAddress>' argument missing` or an unknown command, and unhandled exceptions: an IDL file that doesn't exist (`ENOENT`), invalid JSON, a Codama run on an Anchor IDL, Ctrl+C at a prompt, or declining the overwrite prompt.

Messages that exit with code 2:

- `Invalid IDL: must be a .json file or a Solana program address`
- `IDL standard must be 'anchor' or 'codama'`
- `Codama standard is only supported with IDL files, not program addresses`
- `RPC URL is required when using a program address`
- `Event hints can only be used with Codama standard`
- `--event-cpi-discriminator can only be used with Codama standard`
- `Invalid --event-cpi-discriminator: expected an even-length hex string (e.g. e445a52e51cb9a1d)`
- `Program ID cannot be empty`, `Invalid Program ID format (must be base58, 32-44 characters)`
- `Program ID is required. IDL missing or invalid address field - provide via --program-id parameter`
- `No Anchor IDL found for program address.`, `Failed to fetch Anchor IDL: <reason>`
- `Invalid data source. Must be one of: helius-laserstream, rpc-block-subscribe, rpc-program-subscribe, rpc-transaction-crawler, yellowstone-grpc`
- `Metrics must be 'log' or 'prometheus'`
- `GraphQL scaffolding requires Postgres. Enable --with-postgres or disable --with-graphql.`
- `Output directory already exists: <path> (use --force to overwrite)`

## Known issues

These are 2.0.0 behaviours to work around:

- **`parse` deletes `--out-dir`.** Point it at a dedicated directory, never `.` or a directory with your own files, or pass `--no-clean`.
- **`scaffold --version-name` produces a workspace that does not build.** The decoder crate becomes `carbon-<name>-decoder-<version>` but `indexer/Cargo.toml` still depends on `carbon-<name>-decoder`, so Cargo fails with ``no matching package named `carbon-<name>-decoder` found``. Use `parse --version-name` for versioned decoders and wire them yourself ([Versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md)).
- **Pipes that never fire.** The scaffold registers the same pipes for every datasource. `rpc-transaction-crawler` and `rpc-block-subscribe` emit no account updates, so the account pipe stays idle. `rpc-program-subscribe` emits only account updates, so without Postgres (instruction pipe only) the indexer processes nothing. Add an `.account(...)` pipe for program-subscribe projects.
- **`rpc-block-subscribe` on the public endpoint.** The fallback `wss://api.mainnet-beta.solana.com/` rejects `blockSubscribe` with `Method not found`. Set `RPC_WS_URL` to a provider that supports it, and narrow the filter in `main.rs`: the generated `RpcBlockSubscribeFilter::All` streams every transaction on the network.
- **Help text says `--with-serde` defaults to `false`.** It defaults to `true` when both `--with-postgres` and `--with-graphql` are `false`.
- **`-c`, `--as-crate` does nothing.**
- **`--event-hints` does nothing.** It is validated (Codama only) but never reaches the renderer. Model events as Codama `eventNode`s instead.
- **Codama IDLs need `--program-id`.** The CLI looks for `address` or `metadata.address` and ignores `program.publicKey`, so a Codama file without `--program-id` exits with code 2: `Program ID is required. IDL missing or invalid address field - provide via --program-id parameter`. Pass the program address; the generated `PROGRAM_ID` still comes from `program.publicKey`, whatever value you pass.
- **Indexer package name.** It is always `<name>-indexer`, so `--name my-indexer` gives `my-indexer-indexer`.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/reference/cli · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Datasources (Carbon 2.0)

> Pick the right Carbon datasource for real-time streams, historical backfills or account snapshots, and compare all 14.

A datasource feeds updates (accounts, transactions, deletions, block details) into a Carbon pipeline. Carbon ships 12 datasources for Carbon 2.0 and two legacy ones that only work with Carbon 1.x. This page helps you choose; each datasource has its own page with a runnable example, every option, and its failure modes. For how datasources fit into a pipeline, see [Datasources](https://carbon.sevenlabs.org/concepts/datasources.md) and [Updates](https://carbon.sevenlabs.org/concepts/updates.md).

## Which datasource should I use?

| You need                                                          | You have                                                            | Use                                                                                            |
| ----------------------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Real-time transactions and accounts, lowest latency               | A Yellowstone gRPC endpoint (own node, Triton, Helius, QuickNode…)  | [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md)               |
| Real-time, no gaps after reconnects                               | A Helius plan with LaserStream                                      | [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)           |
| Real-time transactions over WebSocket                             | A Helius API key with enhanced WebSockets                           | [Helius enhanced WebSockets](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md)      |
| Real-time transactions and block details, no gRPC                 | An RPC node with `blockSubscribe` enabled (not the public endpoint) | [RPC blockSubscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)          |
| Real-time account changes for one program, free                   | Any RPC WebSocket, including the public one                         | [RPC programSubscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md)      |
| Current state of a program's accounts, once                       | Any RPC; filters for big programs                                   | [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md)                  |
| Current state of a large program, paginated or incremental        | A Helius API key                                                    | [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md)                     |
| Complete state of a huge program at one slot                      | Disk, RAM and a snapshot (or a node serving one)                    | [Validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md)           |
| History of one address, then new transactions                     | Any RPC, including the public one (slow)                            | [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) |
| History of one address, fast, oldest first                        | A Helius API key                                                    | [Helius getTransactionsForAddress](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)    |
| Every transaction in a slot range, or following the tip over HTTP | Any RPC with those blocks                                           | [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)             |
| Long historical ranges, very fast, no provider                    | A Carbon **1.x** application                                        | [Jetstreamer (legacy)](https://carbon.sevenlabs.org/datasources/jetstreamer.md)                |
| Transactions before execution, for latency-critical signals       | Jito Shredstream access and a Carbon **1.x** application            | [Jito Shredstream (legacy)](https://carbon.sevenlabs.org/datasources/jito-shredstream-grpc.md) |
| Updates from your own source (Kafka, files, tests)                | Your own producer                                                   | [Stream message](https://carbon.sevenlabs.org/datasources/stream-message.md)                   |

On a budget: the RPC datasources work on the free public endpoint, which is enough to learn and prototype but is heavily rate-limited. A common production setup is a snapshot datasource (GPA, gPA v2 or a validator snapshot) for initial state plus a streaming one (Yellowstone or LaserStream) in the same pipeline; see [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md) and [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md).

## Comparison

Every crate is named `carbon-<name>-datasource`, version 2.0.0, except the two legacy ones (1.0.0, Carbon 1.x only). "Emits" lists the update variants the datasource actually produces. Where the datasource's `update_types()` declares something different, the page says so; the pipeline doesn't check that list.

| Datasource                                                                                                            | Transport                | Emits                                 | Live / historical                | Finishes?            | Paid provider?                        | Latency                                  |
| --------------------------------------------------------------------------------------------------------------------- | ------------------------ | ------------------------------------- | -------------------------------- | -------------------- | ------------------------------------- | ---------------------------------------- |
| [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md)                                      | Geyser gRPC              | Account, AccountDeletion, Transaction | Live                             | No                   | Usually                               | Lowest (streamed as slots are processed) |
| [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md)                                  | Geyser gRPC (Helius)     | Account, AccountDeletion, Transaction | Live + replay from a recent slot | No                   | Yes                                   | Lowest                                   |
| [Helius enhanced WebSockets](https://carbon.sevenlabs.org/datasources/helius-atlas-ws.md)                             | WebSocket                | Transaction, Account, AccountDeletion | Live                             | No                   | Yes                                   | Low                                      |
| [RPC blockSubscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md)                                 | WebSocket                | BlockDetails, Transaction             | Live                             | No                   | Needs a node with block subscriptions | Low (per block)                          |
| [RPC programSubscribe](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md)                             | WebSocket                | Account, AccountDeletion              | Live                             | No                   | No                                    | Low                                      |
| [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md)                                         | HTTP, one call           | Account                               | Current state                    | Yes                  | No                                    | Batch                                    |
| [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md)                                            | HTTP, paginated          | Account                               | Current state                    | Yes                  | Yes                                   | Batch                                    |
| [Validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md)                                  | Snapshot archive         | Account                               | State at the snapshot slot       | Yes                  | No                                    | Batch (minutes to hours)                 |
| [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md)                        | HTTP polling             | Transaction                           | Historical, then polls           | No                   | No                                    | Seconds (5 s poll by default)            |
| [Helius getTransactionsForAddress](https://carbon.sevenlabs.org/datasources/helius-gtfa.md)                           | HTTP, paginated          | Transaction                           | Historical                       | Yes                  | Yes                                   | Batch                                    |
| [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)                                    | HTTP polling             | Transaction                           | Slot range, or follows the tip   | With `end_slot`      | No                                    | Seconds                                  |
| [Stream message](https://carbon.sevenlabs.org/datasources/stream-message.md)                                          | In-process channel       | Account, AccountDeletion, Transaction | Whatever you send                | When senders drop    | No                                    | Your producer's                          |
| [Jetstreamer](https://carbon.sevenlabs.org/datasources/jetstreamer.md) (legacy) — Carbon 1.x only                     | Old Faithful archive     | Transaction, BlockDetails             | Historical                       | Yes                  | No                                    | Batch (high throughput)                  |
| [Jito Shredstream gRPC](https://carbon.sevenlabs.org/datasources/jito-shredstream-grpc.md) (legacy) — Carbon 1.x only | gRPC (shredstream-proxy) | Transaction, without metadata         | Live, pre-execution              | When the stream ends | Jito access                           | Lowest (before execution)                |

> **Warning: Legacy datasources.** `carbon-jetstreamer-datasource` and `carbon-jito-shredstream-grpc-datasource` stay at 1.0.0 on `carbon-core` 1.0.0 and the Solana 3.x crates. They are excluded from the Carbon 2.0 workspace and can't be added to a 2.0 pipeline.

## Things every datasource page covers

- **Encoding.** Carbon rebuilds transactions and accounts from binary payloads. Every RPC and WebSocket datasource that lets you choose an encoding needs Base64; JSON encodings fail to decode.
- **Backpressure.** Most streaming datasources use `try_send` and drop updates when the pipeline's channel (`channel_buffer_size`, default 1000) is full. The pages say which ones, and the log line to look for.
- **Finishing.** A pipeline stops when every datasource has finished. Batch datasources finish on their own; streams run until cancelled. See [Shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).
- **Metrics.** Each datasource registers its own counters and histograms next to the pipeline's; see [Metrics](https://carbon.sevenlabs.org/concepts/metrics.md) and the [metrics reference](https://carbon.sevenlabs.org/reference/metrics.md).

Need something else? Write your own with the [custom datasource guide](https://carbon.sevenlabs.org/guides/custom-datasource.md), or push updates in through [Stream message](https://carbon.sevenlabs.org/datasources/stream-message.md).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Helius enhanced WebSockets (Carbon 2.0)

> Stream transactions and account changes from Helius enhanced (Atlas) WebSockets with transactionSubscribe and accountSubscribe.

`HeliusWebsocket` connects to Helius enhanced WebSockets (`wss://atlas-mainnet.helius-rpc.com`), runs one `transactionSubscribe` with your filter and one `accountSubscribe` per account you list, and emits transaction and account updates. It watches the Clock sysvar to detect a stalled connection and reconnects everything when it does. Use it for live transactions on Helius when you don't need gRPC.

|          |                                                                                                                                                  |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Crate    | `carbon-helius-atlas-ws-datasource` 2.0.0                                                                                                        |
| Install  | `cargo add carbon-helius-atlas-ws-datasource@2.0.0`                                                                                              |
| Emits    | `Transaction`, `AccountUpdate`, `AccountDeletion`                                                                                                |
| Mode     | Live. Runs until cancelled or until reconnects keep failing.                                                                                     |
| Provider | A Helius API key on a plan that includes enhanced WebSockets. You also need the `helius` crate (2.0.0) for the filter types.                     |
| API docs | [docs.rs](https://docs.rs/carbon-helius-atlas-ws-datasource/2.0.0/carbon_helius_atlas_ws_datasource/)                                            |
| Source   | [`datasources/helius-atlas-ws-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/helius-atlas-ws-datasource/src/lib.rs) |

## Quick example

This program subscribes to successful Orca Whirlpool transactions and prints each `swapV2`. It compiles as shown; to run it, set `HELIUS_API_KEY`.

```sh
cargo new helius-ws && cd helius-ws
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-helius-atlas-ws-datasource@2.0.0
cargo add helius@2.0.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_atlas_ws_datasource::{Filters, HeliusWebsocket},
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    helius::types::{
        enhanced_websocket::TransactionDetails, Cluster, RpcTransactionsConfig,
        TransactionCommitment, TransactionSubscribeFilter, TransactionSubscribeOptions,
        UiEnhancedTransactionEncoding,
    },
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let transactions = RpcTransactionsConfig {
        filter: TransactionSubscribeFilter {
            vote: Some(false),
            failed: Some(false),
            account_include: Some(vec![PROGRAM_ID.to_string()]),
            ..Default::default()
        },
        options: TransactionSubscribeOptions {
            commitment: Some(TransactionCommitment::Confirmed),
            // Required: the helius default is JsonParsed, which Carbon can't decode.
            encoding: Some(UiEnhancedTransactionEncoding::Base64),
            transaction_details: Some(TransactionDetails::Full),
            show_rewards: Some(false),
            max_supported_transaction_version: Some(1),
        },
    };

    let datasource = HeliusWebsocket::new(
        env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set"),
        Filters::new(vec![], Some(transactions))?,
        Cluster::MainnetBeta,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
HELIUS_API_KEY=your-helius-api-key RUST_LOG=info cargo run
```

Each matching swap prints one `slot … swapV2 pool=… amount=…` line. With an invalid key the connection is refused and retried every 3 seconds:

Output:

```text
[ERROR carbon_helius_atlas_ws_datasource] Failed to create Helius client: Unable to connect to server: HTTP error: 401 Unauthorized
```

`TransactionDetails` must be imported from `helius::types::enhanced_websocket`: `helius::types::TransactionDetails` is a different type used by Helius' HTTP API.

## Configuration

### `HeliusWebsocket`

```rust
pub const fn new(api_key: String, filters: Filters, cluster: Cluster) -> Self
pub fn with_ping_interval_secs(self, ping_interval_secs: u64) -> Self
pub fn with_pong_timeout_secs(self, pong_timeout_secs: u64) -> Self
pub fn with_transaction_idle_timeout_secs(self, transaction_idle_timeout_secs: u64) -> Self
```

| Field                           | Type                     | Default                 | Meaning                                                                                                          |
| ------------------------------- | ------------------------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `api_key`                       | `String`                 | —                       | Helius API key.                                                                                                  |
| `filters`                       | `Filters`                | —                       | What to subscribe to.                                                                                            |
| `cluster`                       | `helius::types::Cluster` | —                       | `MainnetBeta` or `Devnet`. `StakedMainnetBeta` has no enhanced WebSocket URL and fails to connect.               |
| `ping_interval_secs`            | `Option<u64>`            | `None` (10 s)           | Interval between WebSocket pings.                                                                                |
| `pong_timeout_secs`             | `Option<u64>`            | `None` (3 missed pongs) | Close the connection after this long without a pong, rounded up to whole ping intervals. `0` disables the check. |
| `transaction_idle_timeout_secs` | `Option<u64>`            | `None`                  | Reconnect when no transaction arrives for this long. Leave unset for quiet filters.                              |

For example:

```rust
let datasource = HeliusWebsocket::new(api_key, filters, Cluster::MainnetBeta)
    .with_ping_interval_secs(10)
    .with_pong_timeout_secs(30)
    .with_transaction_idle_timeout_secs(60);
```

### `Filters`

```rust
pub fn new(accounts: Vec<Pubkey>, transactions: Option<RpcTransactionsConfig>) -> CarbonResult<Self>
```

| Field          | Type                            | Meaning                                                                                                            |
| -------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `accounts`     | `Vec<Pubkey>`                   | One `accountSubscribe` per address. Opened without options, so the server's default encoding and commitment apply. |
| `transactions` | `Option<RpcTransactionsConfig>` | One `transactionSubscribe`.                                                                                        |

`new` returns `Err(Custom("Error creating Filters for the Helius WebSocket: accounts and transactions can't be both empty"))` when both are empty.

### `RpcTransactionsConfig` (from `helius::types`)

`filter: TransactionSubscribeFilter`:

| Field              | Type                  | Meaning                                                                          |
| ------------------ | --------------------- | -------------------------------------------------------------------------------- |
| `vote`             | `Option<bool>`        | Include or exclude vote transactions.                                            |
| `failed`           | `Option<bool>`        | Include or exclude failed transactions. Carbon drops failed transactions anyway. |
| `signature`        | `Option<String>`      | A single signature.                                                              |
| `account_include`  | `Option<Vec<String>>` | Any of these accounts (OR).                                                      |
| `account_exclude`  | `Option<Vec<String>>` | None of these accounts.                                                          |
| `account_required` | `Option<Vec<String>>` | All of these accounts (AND).                                                     |

`options: TransactionSubscribeOptions`. Its `Default` is not usable with Carbon, because it asks for `JsonParsed`:

| Field                               | Type                                    | Set to      | Meaning                                                                                           |
| ----------------------------------- | --------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------- |
| `commitment`                        | `Option<TransactionCommitment>`         | `Confirmed` | `Processed`, `Confirmed` or `Finalized`.                                                          |
| `encoding`                          | `Option<UiEnhancedTransactionEncoding>` | `Base64`    | **Required.** Any JSON encoding makes every transaction fail with `Failed to decode transaction`. |
| `transaction_details`               | `Option<TransactionDetails>`            | `Full`      | Other levels are skipped with `Skipping non-full Helius transaction notification`.                |
| `show_rewards`                      | `Option<bool>`                          | as needed   | Include rewards in the metadata.                                                                  |
| `max_supported_transaction_version` | `Option<u8>`                            | `Some(1)`   | Highest transaction version to receive.                                                           |

## Behaviour

- **Subscriptions.** Each connection runs a Clock sysvar subscription, one task per account, and one transaction task.
- **Stall detection.** If no Clock update arrives for 5 seconds, or the Clock jumps more than 10 slots, all subscriptions are closed and reopened. With `transaction_idle_timeout_secs`, a silent transaction stream does the same.
- **Reconnects.** Between connections the datasource waits 3 seconds. Building the client is retried up to 30 times in a row, then `consume` returns an error. Missed data is not replayed.
- **Partial failures.** If one `accountSubscribe` or the `transactionSubscribe` call fails, or its stream closes, only that task stops (with an error in the log); the others keep running until the next reconnect.
- **Transaction updates.** Failed transactions are dropped. `slot` comes from the notification; `index`, `block_time` and `block_hash` are `None`. `is_vote` is copied from your filter's `vote` value, not from the transaction. Return data in `meta.return_data` holds the base64 text as bytes rather than the decoded bytes.
- **Account updates.** `AccountUpdate` (or `AccountDeletion` at zero lamports) with the notification slot and `transaction_signature: None`.
- **Backpressure.** Updates use `try_send`. When the pipeline channel is full, the update is dropped and that subscription's task stops until the next reconnect.
- **Metrics.** Counters `helius_atlas_ws_transaction_updates_received_total`, `helius_atlas_ws_account_updates_received_total`, `helius_atlas_ws_account_deletions_received_total`; histograms `helius_atlas_ws_transaction_process_time_nanoseconds`, `helius_atlas_ws_account_process_time_nanoseconds`, `helius_atlas_ws_account_deletion_process_time_nanoseconds`, `helius_atlas_ws_clock_process_time_nanoseconds`.

## Troubleshooting

### `Failed to create Helius client: Unable to connect to server: HTTP error: 401 Unauthorized`

The API key is wrong, or your plan doesn't include enhanced WebSockets. After 30 attempts the datasource gives up.

### Every transaction logs `Failed to decode transaction`

`options.encoding` is `JsonParsed` (the helius default) or another JSON encoding. Set `Some(UiEnhancedTransactionEncoding::Base64)`.

### `Error decoding Helius WS Account event`

Account subscriptions use the server's default encoding, which Carbon can't always decode (for example for larger accounts). For program-owned accounts, use [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) or [`RpcProgramSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md) with Base64 instead.

### `No Clock updates received for 5 seconds, triggering reconnection`

The connection stalled; the datasource reconnects on its own. Frequent occurrences point to network problems or an overloaded process.

### Transactions stop after `Error sending transaction update`

The pipeline channel was full, and the transaction task stopped. It resumes at the next reconnect. Raise the pipeline's [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) or speed up processors.

## See also

- [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md): gRPC on Helius, with replay.
- [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/helius-atlas-ws · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Helius getTransactionsForAddress (Carbon 2.0)

> Backfill full transactions for an address in pages of 100 with Helius getTransactionsForAddress, filtered by slot, time, signature or status.

`HeliusGtfaDatasource` pages through Helius' `getTransactionsForAddress`, which returns full transactions with metadata in one call, and emits them as transaction updates. Compared with the [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md), it needs one request per 100 transactions instead of one per transaction, can go oldest-first, and filters by slot, block time, signature range or status on the server. Use it to backfill an address's history on Helius.

|          |                                                                                                                                          |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-helius-gtfa-datasource` 2.0.0                                                                                                    |
| Install  | `cargo add carbon-helius-gtfa-datasource@2.0.0`                                                                                          |
| Emits    | `Transaction`                                                                                                                            |
| Mode     | Historical. Finishes after the last page.                                                                                                |
| Provider | A Helius RPC URL with API key, on a plan that includes `getTransactionsForAddress`.                                                      |
| API docs | [docs.rs](https://docs.rs/carbon-helius-gtfa-datasource/2.0.0/carbon_helius_gtfa_datasource/)                                            |
| Source   | [`datasources/helius-gtfa-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/helius-gtfa-datasource/src/lib.rs) |

## Quick example

This program backfills 1,000 slots of Orca Whirlpool transactions, oldest first, prints each `swapV2`, and exits. It compiles as shown; to run it, set `HELIUS_API_KEY` and `START_SLOT`.

```sh
cargo new helius-gtfa && cd helius-gtfa
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-helius-gtfa-datasource@2.0.0
cargo add solana-commitment-config@3.1.1
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_gtfa_datasource::{
        HeliusGtfaConfig, HeliusGtfaDatasource, HeliusGtfaFilters, SlotFilter, SortOrder,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    solana_commitment_config::CommitmentConfig,
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let start_slot: u64 = env::var("START_SLOT").expect("START_SLOT must be set").parse().unwrap();

    let config = HeliusGtfaConfig::new(
        Some(SortOrder::Asc), // oldest first
        Some(100),            // transactions per page (max 100)
        Some(CommitmentConfig::finalized()),
        Some(HeliusGtfaFilters {
            slot: Some(SlotFilter {
                gte: Some(start_slot),
                lt: Some(start_slot + 1_000),
                gt: None,
                lte: None,
            }),
            block_time: None,
            signature: None,
            status: None, // successful transactions only
        }),
        None, // min_context_slot
    );

    let datasource = HeliusGtfaDatasource::new_with_config(
        format!(
            "https://mainnet.helius-rpc.com/?api-key={}",
            env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set")
        ),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
HELIUS_API_KEY=your-helius-api-key START_SLOT=467000000 RUST_LOG=info cargo run
```

Swaps print in slot order, followed by `Reached end of pagination` and pipeline shutdown. With an invalid key:

Output:

```text
[ERROR carbon_helius_gtfa_datasource] RPC request failed with status 401 Unauthorized: Unauthorized
[ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("HTTP error 401 Unauthorized: Unauthorized")
```

## Configuration

### Constructors

```rust
pub fn new(helius_rpc_url: String, address: Pubkey) -> Self
pub fn new_with_config(helius_rpc_url: String, address: Pubkey, config: HeliusGtfaConfig) -> Self
```

`helius_rpc_url` is the full URL including `?api-key=`. `address` is the account whose transactions you want: a program, wallet, pool or mint.

### `HeliusGtfaConfig`

```rust
pub fn new(
    sort_order: Option<SortOrder>,
    limit: Option<u32>,
    commitment: Option<CommitmentConfig>,
    filters: Option<HeliusGtfaFilters>,
    min_context_slot: Option<u64>,
) -> Self
```

| Field              | Type                        | `Default`   | Meaning                                                                                      |
| ------------------ | --------------------------- | ----------- | -------------------------------------------------------------------------------------------- |
| `sort_order`       | `Option<SortOrder>`         | `Some(Asc)` | `SortOrder::Asc` (oldest first) or `Desc`. `None` (possible with `new`) leaves it to Helius. |
| `limit`            | `Option<u32>`               | `Some(100)` | Transactions per page. Capped at 100; `None` in `new` also means 100.                        |
| `commitment`       | `Option<CommitmentConfig>`  | `confirmed` | `None` (possible with `new`) leaves it to Helius.                                            |
| `filters`          | `Option<HeliusGtfaFilters>` | `None`      | Server-side filters. A filter with every field empty is not sent.                            |
| `min_context_slot` | `Option<u64>`               | `None`      | Reject the request if Helius is behind this slot.                                            |

The datasource always requests `transactionDetails: "full"`, Base64 encoding and `maxSupportedTransactionVersion: 1`.

### `HeliusGtfaFilters`

| Field        | Type                              | Meaning                                                                        |
| ------------ | --------------------------------- | ------------------------------------------------------------------------------ |
| `slot`       | `Option<SlotFilter>`              | `RangeFilter<u64>` with `gte`, `gt`, `lte`, `lt`.                              |
| `block_time` | `Option<BlockTimeFilter>`         | Unix seconds: `gte`, `gt`, `lte`, `lt`, `eq`.                                  |
| `signature`  | `Option<SignatureFilter>`         | `RangeFilter<String>` of base58 signatures.                                    |
| `status`     | `Option<TransactionStatusFilter>` | `Succeeded`, `Failed` or `Any`. Applied by Helius and again by the datasource. |

The datasource's own status check: `None` and `Succeeded` drop failed transactions, `Failed` keeps only failed ones, `Any` keeps both.

## Behaviour

- **Paging.** Pages are fetched one at a time, following `paginationToken` until Helius returns none. Order within and across pages follows `sort_order`.
- **Updates.** `slot`, `block_time` and `index` (the transaction's position in its block, when Helius returns it) are set; `block_hash` is `None` and `is_vote` is `false`.
- **Backpressure.** Updates are sent with `send().await`; nothing is dropped.
- **Finishing.** `consume` returns after the last page. A pipeline with only this datasource then shuts down.
- **Errors.** There are no retries. A non-2xx status, a body that isn't JSON, an RPC error, or a transaction whose metadata can't be converted stops the datasource with `FailedToConsumeDatasource`. Transactions that fail to decode are skipped with `Failed to decode transaction`.
- **Cancellation.** Checked before each page and between transactions.
- **Metrics.** `helius_gtfa_pages_fetched_total`, `helius_gtfa_transactions_processed_total` (counters), `helius_gtfa_fetch_duration_milliseconds` (histogram).

## Troubleshooting

### `HTTP error 401 Unauthorized`

The API key is missing or invalid. The URL must end in `?api-key=<your key>`.

### `RPC error: … (code: …)`

Helius rejected the request: an unsupported method on your plan, a malformed filter, or a bad address. The message is Helius' own.

### The backfill stops halfway with an error

There are no retries. Restart with a `slot` filter (`gte` = the last slot you stored, with `Asc`) to resume, and make your writes idempotent so the overlap is harmless.

### Failed transactions are missing

By default the datasource drops them. Set `status: Some(TransactionStatusFilter::Failed)` or `Any`.

## See also

- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md)
- [RPC transaction crawler](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md)
- [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/helius-gtfa · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Helius gPA v2 (Carbon 2.0)

> Load every account of a program page by page with Helius getProgramAccountsV2, optionally only accounts changed since a slot.

`HeliusGpaV2Datasource` calls Helius' `getProgramAccountsV2` repeatedly, following `paginationKey` until the last page, and emits every account as an account update. Pages keep memory flat on programs with millions of accounts, and `changedSinceSlot` lets you fetch only accounts modified since your last load. Use it instead of [`GpaDatasource`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md) when you're on Helius or the program is large.

|          |                                                                                                                                              |
| -------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-helius-gpa-v2-datasource` 2.0.0                                                                                                      |
| Install  | `cargo add carbon-helius-gpa-v2-datasource@2.0.0`                                                                                            |
| Emits    | `AccountUpdate`                                                                                                                              |
| Mode     | One-shot snapshot of current state (or of changes since a slot). Finishes after the last page.                                               |
| Provider | A Helius RPC URL with API key, for example `https://mainnet.helius-rpc.com/?api-key=…`.                                                      |
| API docs | [docs.rs](https://docs.rs/carbon-helius-gpa-v2-datasource/2.0.0/carbon_helius_gpa_v2_datasource/)                                            |
| Source   | [`datasources/helius-gpa-v2-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/helius-gpa-v2-datasource/src/lib.rs) |

## Quick example

This program loads every Orca Whirlpool pool account, 5,000 per page, and prints each pool's mints. It compiles as shown; to run it, set `HELIUS_API_KEY`.

```sh
cargo new helius-gpa && cd helius-gpa
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-helius-gpa-v2-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_gpa_v2_datasource::{HeliusGpaV2Config, HeliusGpaV2Datasource},
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    solana_client::{rpc_config::RpcProgramAccountsConfig, rpc_filter::RpcFilterType},
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let config = HeliusGpaV2Config::new(
        Some(RpcProgramAccountsConfig {
            // Only `Whirlpool` pool accounts (653 bytes).
            filters: Some(vec![RpcFilterType::DataSize(653)]),
            ..Default::default()
        }),
        None,       // changed_since_slot: return every matching account
        Some(5000), // accounts per page (max 10000)
    );

    let datasource = HeliusGpaV2Datasource::new_with_config(
        format!(
            "https://mainnet.helius-rpc.com/?api-key={}",
            env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set")
        ),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, PoolLogger)
        .build()?
        .run()
        .await
}

struct PoolLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for PoolLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::Whirlpool(pool) = &input.decoded_account.data {
            println!(
                "{} {} / {} liquidity={}",
                input.metadata.pubkey, pool.token_mint_a, pool.token_mint_b, pool.liquidity
            );
        }
        Ok(())
    }
}
```

```sh
HELIUS_API_KEY=your-helius-api-key RUST_LOG=info cargo run
```

The program prints one line per pool, logs `Reached end of pagination`, and exits. With an invalid key it stops at once:

Output:

```text
[ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("Failed to parse response: error decoding response body")
```

## Configuration

### Constructors

```rust
pub fn new(helius_rpc_url: String, program_id: Pubkey) -> Self
pub fn new_with_config(helius_rpc_url: String, program_id: Pubkey, config: HeliusGpaV2Config) -> Self
```

`helius_rpc_url` is the full URL including `?api-key=`. `new` uses `HeliusGpaV2Config::default()`.

### `HeliusGpaV2Config`

```rust
pub fn new(
    program_accounts_config: Option<RpcProgramAccountsConfig>,
    changed_since_slot: Option<u64>,
    limit: Option<u32>,
) -> Self
```

| Field                     | Type                       | Default      | Meaning                                                                                                                                              |
| ------------------------- | -------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `program_accounts_config` | `RpcProgramAccountsConfig` | `default()`  | Only `filters` and `account_config.commitment` are used. Encoding is always Base64; `data_slice`, `min_context_slot` and `sort_results` are ignored. |
| `changed_since_slot`      | `Option<u64>`              | `None`       | Sent as `changedSinceSlot`: only accounts modified at or after this slot (see Helius' docs for the exact bound).                                     |
| `limit`                   | `Option<u32>`              | `Some(1000)` | Accounts per page. Values above 10000 are capped to 10000.                                                                                           |

Filters are the standard `RpcFilterType` values from `solana_client::rpc_filter`: `DataSize`, `Memcmp`, `TokenAccountState`.

## Behaviour

- **Paging.** Pages are fetched one after another. Each account update gets the slot of its page's response, so accounts from different pages can carry different slots.
- **Backpressure.** Updates are sent with `send().await`; nothing is dropped when the pipeline is busy.
- **Finishing.** `consume` returns after the page with no `paginationKey`. A pipeline with only this datasource then shuts down.
- **Errors.** There are no retries. An HTTP failure, a body that isn't JSON, an RPC error (`RPC error: … (code: …)`) or a response without `context`/`value` stops the datasource with `FailedToConsumeDatasource`; accounts from earlier pages have already been emitted. Accounts whose pubkey, owner or data can't be parsed are skipped without a log line.
- **Cancellation.** Checked before each page and between accounts.
- **Metrics.** `helius_gpa_v2_pages_fetched_total`, `helius_gpa_v2_accounts_processed_total` (counters), `helius_gpa_v2_fetch_duration_milliseconds` (histogram).

## Troubleshooting

### `Failed to parse response: error decoding response body`

Helius answered with something other than JSON-RPC, usually `401 Unauthorized` for a missing or invalid API key. Check that the URL ends in `?api-key=<your key>`.

### `RPC error: … (code: …)`

Helius rejected the request, for example an invalid filter or a plan without `getProgramAccountsV2`. The message is Helius' own.

### Incremental loads miss accounts

A `getProgramAccounts` query only returns accounts that still exist, so accounts closed since your last load never appear. Combine it with a stream (for example [LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md) account filters) to see deletions.

## See also

- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md)
- [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md)
- [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md) for transaction history
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/helius-gpa-v2 · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Helius LaserStream (Carbon 2.0)

> Stream accounts, transactions and blocks from Helius LaserStream gRPC, with automatic slot replay after a reconnect.

`LaserStreamGeyserClient` subscribes to Helius LaserStream, a hosted gRPC service that speaks the Yellowstone protocol, and emits account and transaction updates. Unlike the [Yellowstone datasource](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md), it tracks the last slot it saw and asks LaserStream to replay from that slot after a reconnect. Use it for real-time indexing on Helius when gaps after a disconnect are not acceptable.

|          |                                                                                                                                                        |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Crate    | `carbon-helius-laserstream-datasource` 2.0.0                                                                                                           |
| Install  | `cargo add carbon-helius-laserstream-datasource@2.0.0`                                                                                                 |
| Emits    | `AccountUpdate`, `AccountDeletion`, `Transaction`                                                                                                      |
| Mode     | Live, with replay from a recent slot. Runs until cancelled or until reconnects keep failing.                                                           |
| Provider | A Helius API key on a plan that includes LaserStream, and a regional LaserStream endpoint such as `https://laserstream-mainnet-ewr.helius-rpc.com`.    |
| API docs | [docs.rs](https://docs.rs/carbon-helius-laserstream-datasource/2.0.0/carbon_helius_laserstream_datasource/)                                            |
| Source   | [`datasources/helius-laserstream-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/helius-laserstream-datasource/src/lib.rs) |

## Quick example

This program streams confirmed, successful Orca Whirlpool transactions and prints each `swapV2`. It compiles as shown; to run it, set `LASERSTREAM_URL` and `HELIUS_API_KEY`.

```sh
cargo new laserstream && cd laserstream
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-helius-laserstream-datasource@2.0.0
cargo add yellowstone-grpc-proto@=12.7.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_helius_laserstream_datasource::{
        BlockFilters, LaserStreamClientConfig, LaserStreamGeyserClient,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    std::{collections::HashMap, env, sync::Arc},
    yellowstone_grpc_proto::geyser::{CommitmentLevel, SubscribeRequestFilterTransactions},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let mut transaction_filters = HashMap::new();
    transaction_filters.insert(
        "orca_whirlpool".to_string(),
        SubscribeRequestFilterTransactions {
            vote: Some(false),
            failed: Some(false),
            account_include: vec![PROGRAM_ID.to_string()],
            ..Default::default()
        },
    );

    let datasource = LaserStreamGeyserClient::new(
        env::var("LASERSTREAM_URL").expect("LASERSTREAM_URL must be set"),
        Some(env::var("HELIUS_API_KEY").expect("HELIUS_API_KEY must be set")),
        Some(CommitmentLevel::Confirmed),
        HashMap::new(), // account filters
        transaction_filters,
        BlockFilters::default(),
        LaserStreamClientConfig::default(), // replay_enabled: true
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
LASERSTREAM_URL=https://laserstream-mainnet-ewr.helius-rpc.com HELIUS_API_KEY=your-helius-api-key RUST_LOG=info cargo run
```

Each matching swap prints one `slot … swapV2 pool=… amount=…` line. The [Stream with LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md) guide covers endpoints and keys.

## Configuration

### `LaserStreamGeyserClient::new`

```rust
pub const fn new(
    endpoint: String,
    x_token: Option<String>,
    commitment: Option<CommitmentLevel>,
    account_filters: HashMap<String, SubscribeRequestFilterAccounts>,
    transaction_filters: HashMap<String, SubscribeRequestFilterTransactions>,
    block_filters: BlockFilters,
    geyser_config: LaserStreamClientConfig,
) -> Self
```

| Parameter             | Type                                                  | Meaning                                                                                                                                          |
| --------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `endpoint`            | `String`                                              | LaserStream gRPC URL for your region.                                                                                                            |
| `x_token`             | `Option<String>`                                      | Your Helius API key, sent as `x-token`.                                                                                                          |
| `commitment`          | `Option<CommitmentLevel>`                             | `Processed`, `Confirmed` or `Finalized`. Also decides how far back replay starts (see below).                                                    |
| `account_filters`     | `HashMap<String, SubscribeRequestFilterAccounts>`     | Named account filters, as for [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md#configuration).                         |
| `transaction_filters` | `HashMap<String, SubscribeRequestFilterTransactions>` | Named transaction filters, as for Yellowstone.                                                                                                   |
| `block_filters`       | `BlockFilters`                                        | `filters: HashMap<String, SubscribeRequestFilterBlocks>` and `failed_transactions: Option<bool>` (`None` keeps failed transactions from blocks). |
| `geyser_config`       | `LaserStreamClientConfig`                             | Connection and replay settings.                                                                                                                  |

There is no disconnect notifier or stream timeout on this datasource.

### `LaserStreamClientConfig`

```rust
pub const fn new(
    compression: Option<CompressionEncoding>,
    connect_timeout: Option<Duration>,
    timeout: Option<Duration>,
    max_decoding_message_size: Option<usize>,
    tls_config: Option<ClientTlsConfig>,
    tcp_nodelay: Option<bool>,
    replay_enabled: bool,
    from_slot: Option<u64>,
) -> Self
```

| Field                       | Type                          | Default                          | Meaning                                                                                                                                   |
| --------------------------- | ----------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `compression`               | `Option<CompressionEncoding>` | `None`                           | Send and accept compressed messages.                                                                                                      |
| `connect_timeout`           | `Option<Duration>`            | 15 s                             | Connection timeout.                                                                                                                       |
| `timeout`                   | `Option<Duration>`            | 15 s                             | Request timeout.                                                                                                                          |
| `max_decoding_message_size` | `Option<usize>`               | `None` (client default)          | Raise it if large messages fail to decode.                                                                                                |
| `tls_config`                | `Option<ClientTlsConfig>`     | `None` (system and webpki roots) | Custom TLS configuration.                                                                                                                 |
| `tcp_nodelay`               | `Option<bool>`                | `None`                           | Set `TCP_NODELAY`.                                                                                                                        |
| `replay_enabled`            | `bool`                        | `true`                           | Track slots and resume from the last one after a reconnect.                                                                               |
| `from_slot`                 | `Option<u64>`                 | `None`                           | Start the first subscription at this past slot. **Only honoured when `replay_enabled` is `true`**; with replay off it is reset to `None`. |

To start from a recent past slot:

```rust
let geyser_config = LaserStreamClientConfig {
    from_slot: Some(467_000_000),
    ..LaserStreamClientConfig::default()
};
```

## Behaviour

- **Connecting.** `consume` connects first and returns `FailedToConsumeDatasource` if that fails, which stops a pipeline that has no other datasource.
- **Replay.** With `replay_enabled`, the datasource adds an internal slot subscription and remembers the last slot it saw. On reconnect it sets `from_slot` to that slot for `Confirmed` and `Finalized`, or to 31 slots earlier for `Processed` (and when `commitment` is `None`). Messages from the resumed slot can arrive twice; add a [deduplication filter](https://carbon.sevenlabs.org/concepts/filters.md) if your processor isn't idempotent.
- **Updates.** Same mapping as [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md#behaviour): account messages become `AccountUpdate` or `AccountDeletion`, transaction messages become `Transaction` with `is_vote` and `index` from Geyser, and block messages expand into transactions and accounts. Failed and vote transactions pass unless your filter excludes them. No `BlockDetails` are emitted.
- **Reconnects.** After the stream ends, errors or fails to subscribe, the datasource waits 3 seconds and subscribes again. After 10 consecutive failed attempts it logs `Max reconnection attempts reached: 10` and stops, which shuts down a pipeline that has no other datasource. A successful subscription resets the count. There is no inactivity timeout: a connection that stays open but silent is not detected.
- **Backpressure.** Updates use `try_send` and are dropped with `Failed to send … ` when the pipeline channel is full.
- **Metrics.** Counters `laserstream_account_updates_received_total`, `laserstream_account_deletions_received_total`, `laserstream_transaction_updates_received_total`; histograms `laserstream_account_process_time_nanoseconds`, `laserstream_account_deletion_process_time_nanoseconds`, `laserstream_transaction_process_time_nanoseconds`.

## Troubleshooting

### The pipeline exits with `FailedToConsumeDatasource("gRPC transport error: …")`

The first connection failed. Check the regional endpoint URL, including `https://`, and your network.

### `Failed to subscribe, will retry after 5s delay`

The subscription was rejected, usually because the API key is wrong or the plan doesn't include LaserStream. The actual delay is 3 seconds; after 10 failures the datasource stops.

### `from_slot` has no effect

You set `replay_enabled: false`. The datasource only sends `from_slot` when replay is enabled.

### Duplicate transactions after a reconnect

Expected with replay. Deduplicate by signature, or make writes idempotent (for example, upserts keyed by signature).

### The stream stops without an error

There is no inactivity timeout. Monitor `laserstream_transaction_updates_received_total` (see [Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md)) and restart the process if it stops growing.

## See also

- [Stream with LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md)
- [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) for filter types and the full update mapping
- [Filters](https://carbon.sevenlabs.org/concepts/filters.md) for deduplication
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/helius-laserstream · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Jetstreamer (legacy) (Carbon 2.0)

> Replay historical slots or whole epochs at high speed from the Old Faithful archive with Jetstreamer. Carbon 1.x only.

> **Danger: Carbon 1.x only.** `carbon-jetstreamer-datasource` is not part of Carbon 2.0. The latest release is **1.0.0**, built on `carbon-core` **1.0.0** and the Solana 3.x crates, because Jetstreamer doesn't support the Solana 4 stack yet. It does not work in a Carbon 2.0 pipeline: use it in a Carbon 1.0 application, with 1.0.0 decoders and `carbon-log-metrics` 1.0.0. The source in the `v2.0.0` tag is identical to the published 1.0.0 crate but is excluded from the 2.0 workspace.

`JetstreamerDatasource` runs the Jetstreamer firehose over a slot range or an epoch, reading blocks from the public Old Faithful archive over HTTP with several threads in parallel, and emits transactions (and optionally block details). Use it for fast historical backfills over long ranges without an RPC provider.

|          |                                                                                                                                          |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-jetstreamer-datasource` 1.0.0 (**Carbon 1.x**: `carbon-core` 1.0.0)                                                              |
| Install  | `cargo add carbon-jetstreamer-datasource@1.0.0`                                                                                          |
| Emits    | `Transaction`, and `BlockDetails` when `include_blocks` is set. `update_types()` declares only `Transaction`.                            |
| Mode     | Historical. Finishes at the end of the range.                                                                                            |
| Provider | None: Old Faithful's public archive (`https://files.old-faithful.net/`) by default.                                                      |
| API docs | [docs.rs](https://docs.rs/carbon-jetstreamer-datasource/1.0.0/carbon_jetstreamer_datasource/)                                            |
| Source   | [`datasources/jetstreamer-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/jetstreamer-datasource/src/lib.rs) |

## Quick example

This Carbon 1.0 program replays 100 mainnet slots, keeps successful non-vote transactions that touch the Orca Whirlpool program, and prints each `swapV2`. It needs no API key. Building it pulls in RocksDB, so install a C++ compiler and libclang first (`sudo apt install build-essential clang libclang-dev`).

```sh
cargo new jetstreamer && cd jetstreamer
cargo add carbon-core@1.0.0 carbon-log-metrics@1.0.0 carbon-orca-whirlpool-decoder@1.0.0 carbon-jetstreamer-datasource@1.0.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
# jetstreamer-firehose 0.2.0 breaks with rseek 0.4 (reqwest 0.13); pin the older release
cargo update -p rseek --precise 0.3.0
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_jetstreamer_datasource::{
        filter::{JetstreamerFilter, TransactionFilter},
        range::JetstreamerRange,
        JetstreamerDatasource,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    std::{collections::HashSet, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let filter = JetstreamerFilter {
        include_transactions: true,
        include_blocks: false,
        transaction_filters: vec![TransactionFilter::new(
            Some(false),                    // vote
            Some(false),                    // failed
            HashSet::from([PROGRAM_ID]),    // account_include
            HashSet::new(),                 // account_exclude
            HashSet::new(),                 // account_required
        )],
    };

    let datasource = JetstreamerDatasource::new_with_old_faithful_mainnet(
        JetstreamerRange::new_with_slots(376_000_000, 376_000_100), // end slot is exclusive
        filter,
        4,    // firehose threads
        None, // stats tracking interval (slots)
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
RUST_LOG=info cargo run
```

The 100 slots took about 7 seconds. Four threads each replay 25 slots, so output from different threads interleaves:

Output:

```text
slot 376000050 swapV2 pool=2nJjE2ba3iGtefN4UNM1KN5FdYCwssU4Bzhc8URceqh6 amount=169535616
slot 376000001 swapV2 pool=44W73kGYQgXCTNkGxUmHv8DDBPCxojBcX49uuKmbFc9U amount=355597021341
slot 376000001 swapV2 pool=VPAZrfafQQerQjx4ZFHmKTXDXh4SStCdoFs4J3GfYjT amount=1617157367
slot 376000075 swapV2 pool=44W73kGYQgXCTNkGxUmHv8DDBPCxojBcX49uuKmbFc9U amount=364530054
...
[INFO  jetstreamer::firehose] 🚒 firehose finished successfully.
[INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[INFO  carbon_log_metrics]   jetstreamer_transactions_sent_total: 227
[INFO  carbon_log_metrics]   jetstreamer_transactions_filtered_out_total: 118827
```

## Configuration

### `JetstreamerDatasource`

```rust
pub fn new(
    range: JetstreamerRange,
    filter: JetstreamerFilter,
    threads: u64,
    tracking_interval_slots: Option<u64>,
    archive_url: Option<String>,
    network: Option<String>,
) -> Self

pub fn new_with_old_faithful_mainnet(
    range: JetstreamerRange,
    filter: JetstreamerFilter,
    threads: u64,
    tracking_interval_slots: Option<u64>,
) -> Self // archive_url and network = None
```

| Parameter                 | Type                | Meaning                                                                                                   |
| ------------------------- | ------------------- | --------------------------------------------------------------------------------------------------------- |
| `range`                   | `JetstreamerRange`  | Slots or epoch to replay.                                                                                 |
| `filter`                  | `JetstreamerFilter` | Which updates to emit.                                                                                    |
| `threads`                 | `u64`               | Firehose threads; the range is split evenly between them. Must be at least 1.                             |
| `tracking_interval_slots` | `Option<u64>`       | When set, the firehose reports progress every this many slots into the `jetstreamer_internal_*` gauges.   |
| `archive_url`             | `Option<String>`    | Sets `JETSTREAMER_COMPACT_INDEX_BASE_URL` for the process. `None` uses `https://files.old-faithful.net/`. |
| `network`                 | `Option<String>`    | Sets `JETSTREAMER_NETWORK` for the process (default `mainnet`).                                           |

### `JetstreamerRange` (`carbon_jetstreamer_datasource::range`)

| Constructor                                    | Meaning                                  |
| ---------------------------------------------- | ---------------------------------------- |
| `JetstreamerRange::new_with_slots(start, end)` | `start..end`: the end slot is exclusive. |
| `JetstreamerRange::new_with_epoch(epoch)`      | Every slot of the epoch.                 |

### `JetstreamerFilter` and `TransactionFilter` (`carbon_jetstreamer_datasource::filter`)

| Field                  | Type                     | Default | Meaning                                                                         |
| ---------------------- | ------------------------ | ------- | ------------------------------------------------------------------------------- |
| `include_transactions` | `bool`                   | `false` | Emit `Transaction` updates. Set it, or you receive no transactions.             |
| `include_blocks`       | `bool`                   | `false` | Emit a `BlockDetails` update per block.                                         |
| `transaction_filters`  | `Vec<TransactionFilter>` | empty   | A transaction passes if it matches any filter. Empty: every transaction passes. |

`TransactionFilter::new(vote, failed, account_include, account_exclude, account_required)`:

| Field              | Type              | Meaning                                                            |
| ------------------ | ----------------- | ------------------------------------------------------------------ |
| `vote`             | `Option<bool>`    | Match only votes (`Some(true)`) or only non-votes (`Some(false)`). |
| `failed`           | `Option<bool>`    | Match only failed or only successful transactions.                 |
| `account_include`  | `HashSet<Pubkey>` | At least one of these accounts (when non-empty).                   |
| `account_exclude`  | `HashSet<Pubkey>` | None of these accounts.                                            |
| `account_required` | `HashSet<Pubkey>` | All of these accounts.                                             |

Accounts are matched against static keys and lookup-table addresses.

## Behaviour

- **Fetching.** The firehose reads the Old Faithful compact indexes for the epoch, then streams blocks from the archive, one thread per sub-range.
- **Ordering.** Sequential within a thread, interleaved across threads. Use `threads: 1` for slot order.
- **Updates.** Transactions carry full metadata from the archive, `is_vote` and `index` (position in the slot); `block_time` and `block_hash` are `None`. Failed transactions are emitted unless a filter excludes them. `BlockDetails` carry hash, parent hash, rewards, partitions, block time and height.
- **Backpressure.** Updates are sent with `send().await`; nothing is dropped.
- **Finishing.** `consume` returns when the firehose finishes the range, and a pipeline with only this datasource shuts down. Firehose errors become `FailedToConsumeDatasource`.
- **Cancellation.** The datasource ignores the cancellation token. With the default `ShutdownStrategy::ProcessPending`, Ctrl-C does not stop the pipeline until the range is done; set `.shutdown_strategy(ShutdownStrategy::Immediate)` to exit at once.
- **Metrics.** Counters `jetstreamer_blocks_sent_total`, `jetstreamer_transactions_sent_total`, `jetstreamer_transactions_filtered_in_total`, `jetstreamer_transactions_filtered_out_total`; gauges `jetstreamer_internal_slots_processed`, `jetstreamer_internal_blocks_processed`, `jetstreamer_internal_transactions_processed` (filled only with `tracking_interval_slots`).

## Troubleshooting

### `expected F to return RequestBuilder, but it returns RequestBuilder` in `jetstreamer-firehose`

`rseek` 0.4 moved to `reqwest` 0.13 while `jetstreamer-firehose` 0.2.0 uses 0.12. Run `cargo update -p rseek --precise 0.3.0`.

### Trait or type mismatch errors between `carbon_core` versions

You mixed this crate with `carbon-core` 2.0.0 or 2.0.0 decoders. Use `carbon-core`, `carbon-log-metrics` and decoders at 1.0.0 in the same crate.

### Build fails in `clang-sys`

Install libclang (`sudo apt install clang libclang-dev`) or set `LIBCLANG_PATH`.

### No transactions are emitted

`include_transactions` defaults to `false`. Set it to `true`.

### Ctrl-C doesn't stop the pipeline

See Cancellation above: use `ShutdownStrategy::Immediate`.

## See also

- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md)
- [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md): the Carbon 2.0 alternative for slot ranges.
- [Migrate to v2](https://carbon.sevenlabs.org/guides/migrate-to-v2.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/jetstreamer · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Jito Shredstream gRPC (legacy) (Carbon 2.0)

> Receive transactions from a Jito shredstream-proxy as the leader produces them, before execution and without metadata. Carbon 1.x only.

> **Danger: Carbon 1.x only.** `carbon-jito-shredstream-grpc-datasource` is not part of Carbon 2.0. The latest release is **1.0.0**, built on `carbon-core` **1.0.0** and the Solana 3.x crates, because it doesn't support Transaction V1 yet. It does not work in a Carbon 2.0 pipeline: use it in a Carbon 1.0 application, with 1.0.0 decoders and `carbon-log-metrics` 1.0.0. The source in the `v2.0.0` tag is identical to the published 1.0.0 crate but is excluded from the 2.0 workspace.

`JitoShredstreamGrpcClient` subscribes to the `SubscribeEntries` stream of a Jito [shredstream-proxy](https://github.com/jito-labs/shredstream-proxy), deserializes the entries the leader is producing, and emits every transaction in them. Transactions arrive before they are executed, so they have **no status, logs, balances or inner instructions**. Use it only for latency-critical signals (such as reacting to a swap before the block lands), never for indexing state.

|          |                                                                                                                                                              |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Crate    | `carbon-jito-shredstream-grpc-datasource` 1.0.0 (**Carbon 1.x**: `carbon-core` 1.0.0)                                                                        |
| Install  | `cargo add carbon-jito-shredstream-grpc-datasource@1.0.0`                                                                                                    |
| Emits    | `Transaction` (unexecuted, without metadata)                                                                                                                 |
| Mode     | Live, pre-confirmation. Stops when the stream ends.                                                                                                          |
| Provider | A running shredstream-proxy with its gRPC service enabled, which in turn needs Shredstream access from Jito.                                                 |
| API docs | [docs.rs](https://docs.rs/carbon-jito-shredstream-grpc-datasource/1.0.0/carbon_jito_shredstream_grpc_datasource/)                                            |
| Source   | [`datasources/jito-shredstream-grpc-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/jito-shredstream-grpc-datasource/src/lib.rs) |

## Quick example

This Carbon 1.0 program prints every Orca Whirlpool `swapV2` the proxy sees. It compiles as shown; to run it, set `SHREDSTREAM_URL` to the proxy's gRPC address.

```sh
cargo new shredstream && cd shredstream
cargo add carbon-core@1.0.0 carbon-log-metrics@1.0.0 carbon-orca-whirlpool-decoder@1.0.0 carbon-jito-shredstream-grpc-datasource@1.0.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_jito_shredstream_grpc_datasource::JitoShredstreamGrpcClient,
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder},
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // The gRPC address of your shredstream-proxy, e.g. http://127.0.0.1:9999
    let datasource = JitoShredstreamGrpcClient::new(
        env::var("SHREDSTREAM_URL").expect("SHREDSTREAM_URL must be set"),
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        // No execution metadata: the transaction may still fail on chain.
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 (unconfirmed) pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
SHREDSTREAM_URL=http://127.0.0.1:9999 RUST_LOG=info cargo run
```

With no proxy listening, the pipeline stops at once:

Output:

```text
[ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("transport error")
[INFO  carbon_core::pipeline] update_receiver closed, shutting down.
```

## Configuration

### `JitoShredstreamGrpcClient::new`

```rust
pub fn new(endpoint: String) -> Self
```

`endpoint` is the proxy's gRPC URL, for example `http://127.0.0.1:9999`. There are no other options.

## Behaviour

- **Connecting.** `consume` connects first and returns `FailedToConsumeDatasource("transport error")` if the proxy isn't reachable.
- **Entries.** Each message holds a slot and bincode-encoded entries. Entries already seen (by entry hash, in a cache of about 4,096) are skipped and counted as duplicates. Messages are processed concurrently, so updates are not strictly ordered.
- **Updates.** One `Transaction` per transaction in each entry, with `slot` from the message and `meta` set to `TransactionStatusMeta { status: Ok(()), ..Default::default() }`: no fee, balances, logs, inner instructions or loaded addresses. `is_vote` is always `false` and votes are included. `index` and `block_hash` are `None`. `block_time` is the local receive time in **milliseconds** since the Unix epoch, not seconds.
- **What decodes.** Top-level instructions decode normally. CPIs, log events and anything that reads token balances or loaded addresses don't, because the metadata is empty. With no loaded addresses, accounts that come from address lookup tables are missing from the instruction, so decoders usually return nothing for those instructions.
- **Backpressure.** Updates use `try_send`. When the pipeline channel is full, the rest of that message's transactions are dropped with `Failed to send transaction update …`.
- **Reconnects.** None. If the subscription fails or the stream ends, the task logs `Failed to subscribe` or `Grpc stream error` and stops; a pipeline with only this datasource then shuts down. Run it under a supervisor that restarts the process.
- **Metrics.** Counters `jito_shredstream_grpc_entry_updates_received_total`, `jito_shredstream_grpc_duplicate_entries_total`; histogram `jito_shredstream_grpc_entry_process_time_nanoseconds`.

## Troubleshooting

### `FailedToConsumeDatasource("transport error")`

The proxy isn't running, or its gRPC service is disabled or on another port. Start shredstream-proxy with its gRPC service enabled and point `SHREDSTREAM_URL` at that port.

### Trait or type mismatch errors between `carbon_core` versions

You mixed this crate with `carbon-core` 2.0.0 or 2.0.0 decoders. Use `carbon-core`, `carbon-log-metrics` and decoders at 1.0.0 in the same crate.

### Decoded swaps never appear in the chain

Expected for some: shredstream shows transactions before execution, and some fail. Confirm with a [Yellowstone](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) or [RPC](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) datasource if you need final results.

### `block_time` values look 1000 times too large

They are milliseconds of local receive time. Divide by 1000 for Unix seconds.

## See also

- [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) for executed transactions with metadata
- [Migrate to v2](https://carbon.sevenlabs.org/guides/migrate-to-v2.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/jito-shredstream-grpc · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# RPC block crawler (Carbon 2.0)

> Fetch blocks slot by slot with getBlock, for a fixed slot range or following the chain tip, on any Solana RPC.

`RpcBlockCrawler` calls `getBlock` for every slot from `start_slot` onwards and emits each successful transaction in the block. With an `end_slot` it stops after that slot; without one it follows the chain tip. Use it to backfill a slot range for any number of programs at once, or to follow the chain from a plain HTTP RPC.

|          |                                                                                                                                                      |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-rpc-block-crawler-datasource` 2.0.0                                                                                                          |
| Install  | `cargo add carbon-rpc-block-crawler-datasource@2.0.0`                                                                                                |
| Emits    | `Transaction`                                                                                                                                        |
| Mode     | Historical slot range (finishes), or live by polling the tip (never finishes).                                                                       |
| Provider | Any Solana JSON-RPC HTTP endpoint with the blocks you want. The public endpoint works for short ranges.                                              |
| API docs | [docs.rs](https://docs.rs/carbon-rpc-block-crawler-datasource/2.0.0/carbon_rpc_block_crawler_datasource/)                                            |
| Source   | [`datasources/rpc-block-crawler-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/rpc-block-crawler-datasource/src/lib.rs) |

## Quick example

This program crawls five blocks starting at `START_SLOT`, prints the Orca Whirlpool `swapV2` and `swap` instructions in them, and exits.

```sh
cargo new block-crawler && cd block-crawler
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-block-crawler-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder},
    carbon_rpc_block_crawler_datasource::{RpcBlockConfig, RpcBlockCrawler},
    solana_client::rpc_config::{TransactionDetails, UiTransactionEncoding},
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let start_slot: u64 = env::var("START_SLOT").expect("START_SLOT").parse().unwrap();

    let block_config = RpcBlockConfig {
        encoding: Some(UiTransactionEncoding::Base64), // required: Carbon can't decode JSON payloads
        transaction_details: Some(TransactionDetails::Full),
        rewards: Some(false),
        commitment: None, // defaults to confirmed
        max_supported_transaction_version: Some(1),
    };

    let datasource = RpcBlockCrawler::new(
        env::var("RPC_URL").unwrap_or("https://api.mainnet-beta.solana.com".to_string()),
        start_slot,
        Some(start_slot + 4), // end_slot is inclusive: 5 blocks, then the pipeline exits
        None,                 // block_interval: 100 ms
        block_config,
        Some(2), // max_concurrent_requests
        None,    // channel_buffer_size: 1000
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            OrcaWhirlpoolInstruction::Swap { data, accounts, .. } => {
                println!("slot {} tx #{:?} swap pool={} amount={}", tx.slot, tx.index, accounts.whirlpool, data.amount);
            }
            OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } => {
                println!("slot {} tx #{:?} swapV2 pool={} amount={}", tx.slot, tx.index, accounts.whirlpool, data.amount);
            }
            _ => {}
        }
        Ok(())
    }
}
```

Pick a recent finalized slot, then run:

```sh
START_SLOT=467116060 RUST_LOG=info cargo run
```

Output:

```text
slot 467116060 tx #Some(188) swapV2 pool=FAqh648xeeaTqL7du49sztp9nfj5PjRQrfvaMccyd9cz amount=14868973271
slot 467116060 tx #Some(193) swapV2 pool=B5EwJVDuAauzUEEdwvbuXzbFFgEYnUqqS37TUM1c4PQA amount=1813589446
slot 467116060 tx #Some(199) swapV2 pool=3ndjN1nJVUKGrJBc1hhVpER6kWTZKHdyDrPyCJyX3CXK amount=3112157408
...
[INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[INFO  carbon_log_metrics]   block_crawler_blocks_fetched_total: 5
[INFO  carbon_log_metrics]   block_crawler_blocks_skipped_total: 0
[INFO  carbon_log_metrics]   block_crawler_transactions_processed_total: 1347
[INFO  carbon_core::pipeline] pipeline shutdown complete.
```

The five blocks held 1,347 successful transactions (votes included) and 63 Orca swaps; the run took about 6 seconds. The public endpoint only keeps recent blocks, so very old slots need an archival RPC.

## Configuration

### `RpcBlockCrawler::new`

```rust
pub fn new(
    rpc_url: String,
    start_slot: u64,
    end_slot: Option<u64>,
    block_interval: Option<Duration>,
    block_config: RpcBlockConfig,
    max_concurrent_requests: Option<usize>,
    channel_buffer_size: Option<usize>,
) -> Self
```

| Parameter                 | Type               | Default | Meaning                                                                                           |
| ------------------------- | ------------------ | ------- | ------------------------------------------------------------------------------------------------- |
| `rpc_url`                 | `String`           | —       | HTTP JSON-RPC endpoint.                                                                           |
| `start_slot`              | `u64`              | —       | First slot to fetch.                                                                              |
| `end_slot`                | `Option<u64>`      | `None`  | Last slot to fetch, inclusive. `None` follows the tip forever.                                    |
| `block_interval`          | `Option<Duration>` | 100 ms  | When following the tip: wait between `getSlot` polls once caught up, and after a `getSlot` error. |
| `block_config`            | `RpcBlockConfig`   | —       | Options for `getBlock`. See below.                                                                |
| `max_concurrent_requests` | `Option<usize>`    | `10`    | `getBlock` calls in flight at once.                                                               |
| `channel_buffer_size`     | `Option<usize>`    | `1000`  | Blocks buffered between fetching and processing. This is not the pipeline's channel.              |

### `RpcBlockConfig`

Re-exported from `solana_client::rpc_config`. `TransactionDetails` and `UiTransactionEncoding` are in `solana_client::rpc_config` too.

| Field                               | Type                            | Recommended    | Meaning                                                                                                                        |
| ----------------------------------- | ------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `encoding`                          | `Option<UiTransactionEncoding>` | `Some(Base64)` | **Required in practice.** `None` means JSON, which Carbon can't decode: every transaction logs `Failed to decode transaction`. |
| `transaction_details`               | `Option<TransactionDetails>`    | `Some(Full)`   | Anything other than `Full` returns no transactions to decode.                                                                  |
| `rewards`                           | `Option<bool>`                  | `Some(false)`  | Rewards are not emitted by this datasource, so skip them.                                                                      |
| `commitment`                        | `Option<CommitmentConfig>`      | `None`         | `None` means `confirmed`. Also used for `getSlot` when following the tip.                                                      |
| `max_supported_transaction_version` | `Option<u8>`                    | `Some(1)`      | Without it, blocks containing versioned transactions fail to fetch.                                                            |

## Behaviour

- **Fetching.** Slots are requested in increasing order, up to `max_concurrent_requests` at a time, and processed in the order the responses arrive. Blocks can therefore be emitted out of order; transactions inside a block keep their order.
- **Following the tip.** Without `end_slot`, the crawler calls `getSlot` whenever it reaches the last known slot and waits `block_interval` when there is nothing new.
- **Skipped slots.** Errors `-32004` (block not available), `-32007` and `-32009` (slot skipped or missing) are counted in `block_crawler_blocks_skipped_total` and the slot is not retried. Other errors are logged as `Error fetching block at slot …` and the slot is also not retried. When following the tip, a block requested before the node has it is lost; use `confirmed` or `finalized` commitment and expect occasional gaps.
- **Updates.** One `Transaction` update per successful transaction, with `index` set to its position in the block, the block's `block_time` and `block_hash`. Failed transactions are skipped. Vote transactions are included and all updates have `is_vote: false`. No `BlockDetails` updates are emitted.
- **Backpressure.** Updates are sent to the pipeline with `try_send`. When the pipeline channel is full, the datasource logs `Error sending transaction update` and drops the rest of that block. For large backfills, raise the pipeline's [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) and lower `max_concurrent_requests`.
- **Finishing.** With `end_slot`, the pipeline shuts down after the last block when this is its only datasource. Without it, it runs until cancelled.
- **Metrics.** Counters `block_crawler_blocks_fetched_total`, `block_crawler_blocks_skipped_total`, `block_crawler_blocks_received_total`, `block_crawler_blocks_processed_total`, `block_crawler_transactions_processed_total`; histograms `block_crawler_blocks_fetch_times_milliseconds`, `block_crawler_block_process_time_nanoseconds`, `block_crawler_transaction_process_time_nanoseconds`.

## Troubleshooting

### Every transaction logs `Failed to decode transaction`

`block_config.encoding` is `None` or a JSON encoding. Set `encoding: Some(UiTransactionEncoding::Base64)`.

### `Error fetching block at slot …` with "Transaction version (0) is not supported"

Set `max_supported_transaction_version: Some(1)`. A lower value fails on any block that contains a newer transaction version.

### `block_crawler_blocks_skipped_total` grows quickly

Normal mainnet skip rate is a few percent of slots. If almost every slot is skipped, your RPC doesn't keep blocks that old (`-32009`, missing in long-term storage); use an archival RPC.

### `Error sending transaction update` and missing transactions

The pipeline channel overflowed. Raise `Pipeline::builder().channel_buffer_size(…)` and reduce `max_concurrent_requests`.

## See also

- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md)
- [Multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md)
- [RPC blockSubscribe](https://carbon.sevenlabs.org/datasources/rpc-block-subscribe.md): the same data pushed over a WebSocket.
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/rpc-block-crawler · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# RPC blockSubscribe (Carbon 2.0)

> Receive every confirmed block, or only transactions that mention a program, over a Solana RPC WebSocket with blockSubscribe.

`RpcBlockSubscribe` opens a `blockSubscribe` WebSocket subscription and emits a block-details update for every block, followed by a transaction update for each successful transaction in it. Use it for live transactions from an RPC node that has block subscriptions enabled, when you don't have a Geyser gRPC endpoint.

> **Warning: Not available on the public endpoint.** `api.mainnet-beta.solana.com` rejects `blockSubscribe` with `Method not found (-32601)`. You need a node started with `--rpc-pubsub-enable-block-subscription`, or a provider that offers the method.

|          |                                                                                                                                                          |
| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-rpc-block-subscribe-datasource` 2.0.0                                                                                                            |
| Install  | `cargo add carbon-rpc-block-subscribe-datasource@2.0.0`                                                                                                  |
| Emits    | `BlockDetails`, `Transaction`. `update_types()` declares only `Transaction`.                                                                             |
| Mode     | Live. Runs until cancelled.                                                                                                                              |
| Provider | An RPC WebSocket endpoint that supports `blockSubscribe`.                                                                                                |
| API docs | [docs.rs](https://docs.rs/carbon-rpc-block-subscribe-datasource/2.0.0/carbon_rpc_block_subscribe_datasource/)                                            |
| Source   | [`datasources/rpc-block-subscribe-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/rpc-block-subscribe-datasource/src/lib.rs) |

## Quick example

This program subscribes to confirmed blocks that mention the Orca Whirlpool program, prints each block and each `swapV2` instruction. It compiles as shown; to run it, set `RPC_WS_URL` to an endpoint that supports `blockSubscribe`.

```sh
cargo new block-subscribe && cd block-subscribe
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-block-subscribe-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        datasource::BlockDetails, error::CarbonResult,
        instruction::InstructionProcessorInputType, pipeline::Pipeline, processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_rpc_block_subscribe_datasource::{Filters, RpcBlockSubscribe},
    solana_client::rpc_config::{
        CommitmentConfig, RpcBlockSubscribeConfig, RpcBlockSubscribeFilter, TransactionDetails,
        UiTransactionEncoding,
    },
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let config = RpcBlockSubscribeConfig {
        commitment: Some(CommitmentConfig::confirmed()),
        encoding: Some(UiTransactionEncoding::Base64), // required: Carbon can't decode JSON payloads
        transaction_details: Some(TransactionDetails::Full),
        show_rewards: Some(false),
        max_supported_transaction_version: Some(1),
    };

    let datasource = RpcBlockSubscribe::new(
        env::var("RPC_WS_URL").expect("RPC_WS_URL must be set"),
        Filters::new(
            RpcBlockSubscribeFilter::MentionsAccountOrProgram(PROGRAM_ID.to_string()),
            Some(config),
        ),
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .block_details(BlockLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}

struct BlockLogger;

impl Processor<BlockDetails> for BlockLogger {
    async fn process(&mut self, block: &BlockDetails) -> CarbonResult<()> {
        println!("block {} time={:?} hash={:?}", block.slot, block.block_time, block.block_hash);
        Ok(())
    }
}
```

```sh
RPC_WS_URL=wss://your-node.example.com RUST_LOG=info cargo run
```

Against the public endpoint the subscription is refused and retried every few seconds:

Output:

```text
[ERROR carbon_rpc_block_subscribe_datasource] Failed to subscribe to block updates: SubscribeFailed { reason: "Method not found (-32601)", message: "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":-32601,\"message\":\"Method not found\"},\"id\":1}" }
```

## Configuration

### Constructors

```rust
pub fn new(rpc_ws_url: String, filters: Filters) -> Self
pub fn with_disconnect_notifier(
    rpc_ws_url: String,
    filters: Filters,
    disconnect_notifier: tokio::sync::mpsc::Sender<DatasourceDisconnection>,
) -> Self
pub fn with_stream_timeout(self, timeout: Duration) -> Self
```

| Field                 | Type                                            | Default                              | Meaning                                                                        |
| --------------------- | ----------------------------------------------- | ------------------------------------ | ------------------------------------------------------------------------------ |
| `rpc_ws_url`          | `String`                                        | —                                    | WebSocket endpoint.                                                            |
| `filters`             | `Filters`                                       | —                                    | Block filter and subscription config.                                          |
| `disconnect_notifier` | `Option<mpsc::Sender<DatasourceDisconnection>>` | `None`                               | Receives one event per reconnect with the slot gap. See below.                 |
| `stream_timeout`      | `Duration`                                      | 30 s (`DEFAULT_STREAM_TIMEOUT_SECS`) | With no message for this long, the connection is treated as dead and reopened. |

### `Filters`

```rust
pub const fn new(
    block_filter: RpcBlockSubscribeFilter,
    block_subscribe_config: Option<RpcBlockSubscribeConfig>,
) -> Self
```

`RpcBlockSubscribeFilter` (from `solana_client::rpc_config`) is `All` or `MentionsAccountOrProgram(String)`, the latter taking a base58 address. `RpcBlockSubscribeConfig`:

| Field                               | Type                            | Recommended    | Meaning                                                                                                           |
| ----------------------------------- | ------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------- |
| `commitment`                        | `Option<CommitmentConfig>`      | `confirmed`    | `None` uses the server default (`finalized`).                                                                     |
| `encoding`                          | `Option<UiTransactionEncoding>` | `Some(Base64)` | **Required in practice.** JSON encodings can't be decoded: every transaction logs `Failed to decode transaction`. |
| `transaction_details`               | `Option<TransactionDetails>`    | `Some(Full)`   | Anything else yields no transactions to decode.                                                                   |
| `show_rewards`                      | `Option<bool>`                  | as needed      | Rewards are copied into `BlockDetails.rewards`.                                                                   |
| `max_supported_transaction_version` | `Option<u8>`                    | `Some(1)`      | Needed for blocks with versioned transactions.                                                                    |

### `DatasourceDisconnection`

Sent with `try_send` on the first block after a reconnect: `source` (`"rpc-websocket"`), `disconnect_time`, `last_slot_before_disconnect`, `first_slot_after_reconnect`, `missed_slots`. Use it to trigger a backfill of the gap with [`RpcBlockCrawler`](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md).

## Behaviour

- **Updates per block.** One `BlockDetails` (slot, block hash, previous block hash, rewards, reward partitions, block time, block height), then one `Transaction` per transaction whose status is `Ok`. Transactions carry the block's `block_time` and `block_hash`, `is_vote: false`, and `index` set to the position in the notification's transaction list. With `MentionsAccountOrProgram` that list contains only matching transactions, so `index` is not the position in the full block.
- **`update_types()`.** Declares only `Transaction`, although `BlockDetails` is emitted too. The pipeline does not check this list, so `.block_details(…)` processors receive the blocks.
- **Reconnects.** On stream close or `stream_timeout`, the datasource waits 3 seconds and reconnects. Connection failures are retried every 3 seconds up to 10 times, subscription failures up to 11 times, then `consume` returns an error. The counter resets after a successful subscription. Missed blocks are not replayed.
- **Backpressure.** Updates use `try_send`. If the `BlockDetails` send fails, the subscription is dropped and reopened. If a transaction send fails, the rest of that block's transactions are dropped. Both log `Error sending …`.
- **Finishing.** Never finishes on its own.
- **Metrics.** `block_subscribe_blocks_received_total`, `block_subscribe_transactions_processed_total` (counters), `block_subscribe_block_process_time_nanoseconds`, `block_subscribe_transaction_process_time_nanoseconds` (histograms).

## Troubleshooting

### `Failed to subscribe to block updates: … Method not found (-32601)`

The endpoint doesn't support `blockSubscribe`. The public mainnet endpoint never does. Use a node with `--rpc-pubsub-enable-block-subscription`, a provider that exposes it, or switch to [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md) or [the block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md).

### Every transaction logs `Failed to decode transaction`

Set `encoding: Some(UiTransactionEncoding::Base64)` in `RpcBlockSubscribeConfig`.

### `Block stream timeout - no messages for 30s`

The node stopped sending blocks, often because the filter matched nothing for 30 seconds. With a narrow `MentionsAccountOrProgram` filter on a quiet program, raise the timeout with `.with_stream_timeout(Duration::from_secs(300))`.

### Blocks are missing after a reconnect

Expected: nothing is replayed. Pass a `disconnect_notifier` and backfill `last_slot_before_disconnect..first_slot_after_reconnect` with the block crawler.

## See also

- [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md)
- [RPC block crawler](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md)
- [Processor inputs](https://carbon.sevenlabs.org/reference/processor-inputs.md) for `BlockDetails`
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/rpc-block-subscribe · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# RPC getProgramAccounts (Carbon 2.0)

> Load every account a program owns, once, with a single getProgramAccounts call on any Solana RPC.

`GpaDatasource` calls `getProgramAccounts` once for a program, emits every returned account as an account update at the response's slot, and finishes. Use it to load the current state of a program's accounts at startup, usually with filters that select one account type, and pair it with a streaming datasource to keep that state up to date.

|          |                                                                                                                                  |
| -------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-rpc-gpa-datasource` 2.0.0                                                                                                |
| Install  | `cargo add carbon-rpc-gpa-datasource@2.0.0`                                                                                      |
| Emits    | `AccountUpdate`                                                                                                                  |
| Mode     | One-shot snapshot of current state. Finishes after the last account.                                                             |
| Provider | Any Solana JSON-RPC HTTP endpoint. The public endpoint works for filtered queries on programs with few accounts.                 |
| API docs | [docs.rs](https://docs.rs/carbon-rpc-gpa-datasource/2.0.0/carbon_rpc_gpa_datasource/)                                            |
| Source   | [`datasources/rpc-gpa-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/rpc-gpa-datasource/src/lib.rs) |

## Quick example

This program loads every Orca Whirlpool `FeeTier` account from the public RPC. A `DataSize(44)` filter keeps the request small.

```sh
cargo new gpa && cd gpa
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-gpa-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_rpc_gpa_datasource::GpaDatasource,
    solana_client::{
        rpc_config::{RpcAccountInfoConfig, RpcProgramAccountsConfig},
        rpc_filter::RpcFilterType,
    },
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // Only Orca `FeeTier` accounts: 8-byte discriminator + 32 + 2 + 2 = 44 bytes.
    let config = RpcProgramAccountsConfig {
        filters: Some(vec![RpcFilterType::DataSize(44)]),
        account_config: RpcAccountInfoConfig::default(),
        ..Default::default()
    };

    let datasource = GpaDatasource::new_with_config(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, FeeTierLogger)
        .build()?
        .run()
        .await
}

struct FeeTierLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for FeeTierLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::FeeTier(fee_tier) = &input.decoded_account.data {
            println!(
                "{} tick_spacing={} default_fee_rate={}",
                input.metadata.pubkey, fee_tier.tick_spacing, fee_tier.default_fee_rate
            );
        }
        Ok(())
    }
}
```

```sh
RUST_LOG=info cargo run
```

The program prints one line per fee tier (825 when this page was written) and exits about a second later:

Output:

```text
1XuAqKHeX1in2pJubEA5yUkyqJj2mDCt9GQdUqVg9iN tick_spacing=8 default_fee_rate=10
12UZ2XU7Lv1aWHgu6DMsymhXEdBHQEuLaB9fPCatGo4Y tick_spacing=32896 default_fee_rate=3000
143Eu77adnubqv8j68fTDDYaRweJtrLU3zvB9pyudTeR tick_spacing=32896 default_fee_rate=3000
14HcGyeg7VqSundRvpi4Xj6GeZxWVqxeTjdi3pibFtdg tick_spacing=8 default_fee_rate=50
...
[INFO  carbon_rpc_gpa_datasource] Fetched 825 accounts for program whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc (slot: 467112536)
[INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[INFO  carbon_core::pipeline] pipeline shutdown complete.
```

## Configuration

### Constructors

```rust
pub fn new(rpc_url: String, program_id: Pubkey) -> Self
pub const fn new_with_config(rpc_url: String, program_id: Pubkey, config: RpcProgramAccountsConfig) -> Self
```

`new` uses `RpcProgramAccountsConfig::default()`: no filters, every account of the program. The struct fields `rpc_url`, `program_id` and `config` are public.

### `RpcProgramAccountsConfig`

The type comes from `solana_client::rpc_config` (`solana-client` 4.2.2). Filters come from `solana_client::rpc_filter`.

| Field                             | Type                         | Default              | Meaning                                                                                                                                                                                                             |
| --------------------------------- | ---------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `filters`                         | `Option<Vec<RpcFilterType>>` | `None`               | Server-side filters, all of which must match: `RpcFilterType::DataSize(u64)`, `RpcFilterType::Memcmp(Memcmp::new_base58_encoded(offset, &bytes))` (or `Memcmp::new_raw_bytes`), `RpcFilterType::TokenAccountState`. |
| `account_config.encoding`         | `Option<UiAccountEncoding>`  | —                    | **Ignored**: the datasource always requests `Base64`.                                                                                                                                                               |
| `account_config.commitment`       | `Option<CommitmentConfig>`   | `None` (`confirmed`) | Commitment of the snapshot.                                                                                                                                                                                         |
| `account_config.data_slice`       | `Option<UiDataSliceConfig>`  | `None`               | Return only part of each account's data. Decoders usually fail on sliced data.                                                                                                                                      |
| `account_config.min_context_slot` | `Option<Slot>`               | `None`               | Reject the request if the node is behind this slot.                                                                                                                                                                 |
| `with_context`                    | `Option<bool>`               | —                    | **Ignored**: always set to `true`, so every update carries the response slot.                                                                                                                                       |
| `sort_results`                    | `Option<bool>`               | `None`               | Passed through to the RPC.                                                                                                                                                                                          |

## Behaviour

- **One request.** The datasource sends a single `getProgramAccounts` and holds the whole response in memory before emitting anything. Large programs need filters, a generous RPC, or [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md), which pages.
- **Slot.** Every update gets the response's context slot and `transaction_signature: None`.
- **Backpressure.** Updates are sent with `send().await`, so nothing is dropped when the pipeline is busy.
- **Finishing.** `consume` returns after the last account. When this is the pipeline's only datasource, the pipeline logs `update_receiver closed, shutting down.` and `run()` returns. When it runs next to a streaming datasource, the pipeline keeps running. See [shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md).
- **Errors.** There are no retries. Any RPC or transport error returns `FailedToConsumeDatasource("Failed to fetch program accounts")`; the original error is not included.
- **Cancellation.** Checked between accounts.
- **Metrics.** `rpc_gpa_accounts_processed_total` (counter), `rpc_gpa_fetch_duration_milliseconds` (histogram).

## Troubleshooting

### `Failed to fetch program accounts`

The datasource hides the RPC's error message. Send the same request with `curl` to see it:

```sh
curl -s https://api.mainnet-beta.solana.com -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"getProgramAccounts","params":["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc",{"encoding":"base64","filters":[{"dataSize":44}]}]}'
```

Common causes: the program has too many accounts for the node (add `DataSize` or `Memcmp` filters), the public endpoint's data allowance is used up (`"You have used your data allowance"`, HTTP 413: wait or use your own RPC), or the request timed out.

### The processor receives no accounts

The decoder returned `None` for every account. Check that your filters select an account type the decoder knows, and that you did not set `data_slice`.

### The pipeline exits right after loading

Expected when this is the only datasource. Add a streaming datasource such as [`RpcProgramSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md) to keep running; see [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md).

## See also

- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md)
- [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md): paginated, with `changedSinceSlot`.
- [Validator snapshot](https://carbon.sevenlabs.org/datasources/validator-snapshot.md): every account of a program from a snapshot file.
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/rpc-gpa · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# RPC programSubscribe (Carbon 2.0)

> Stream live changes to every account a program owns over a Solana RPC WebSocket, including the free public endpoint.

`RpcProgramSubscribe` opens a `programSubscribe` WebSocket subscription and emits an account update each time an account owned by the program changes. Accounts that drop to zero lamports arrive as account deletions. Use it for live account state when you don't have a Geyser provider; it works on the public endpoint.

|          |                                                                                                                                                              |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Crate    | `carbon-rpc-program-subscribe-datasource` 2.0.0                                                                                                              |
| Install  | `cargo add carbon-rpc-program-subscribe-datasource@2.0.0`                                                                                                    |
| Emits    | `AccountUpdate`, `AccountDeletion`                                                                                                                           |
| Mode     | Live. Only changes after the subscription starts; no initial snapshot. Runs until cancelled.                                                                 |
| Provider | Any Solana RPC WebSocket endpoint (`wss://…`). The public `wss://api.mainnet-beta.solana.com` works.                                                         |
| API docs | [docs.rs](https://docs.rs/carbon-rpc-program-subscribe-datasource/2.0.0/carbon_rpc_program_subscribe_datasource/)                                            |
| Source   | [`datasources/rpc-program-subscribe-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/rpc-program-subscribe-datasource/src/lib.rs) |

## Quick example

This program streams changes to Orca Whirlpool pool accounts from the public endpoint and prints each pool's current tick and liquidity.

```sh
cargo new program-subscribe && cd program-subscribe
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-program-subscribe-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_rpc_program_subscribe_datasource::{Filters, RpcProgramSubscribe},
    solana_client::{
        rpc_config::{RpcAccountInfoConfig, RpcProgramAccountsConfig, UiAccountEncoding},
        rpc_filter::RpcFilterType,
    },
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let config = RpcProgramAccountsConfig {
        // Only `Whirlpool` pool accounts (653 bytes).
        filters: Some(vec![RpcFilterType::DataSize(653)]),
        account_config: RpcAccountInfoConfig {
            encoding: Some(UiAccountEncoding::Base64),
            ..Default::default()
        },
        ..Default::default()
    };

    let datasource = RpcProgramSubscribe::new(
        "wss://api.mainnet-beta.solana.com".to_string(),
        Filters::new(PROGRAM_ID, Some(config)),
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, PoolLogger)
        .build()?
        .run()
        .await
}

struct PoolLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for PoolLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::Whirlpool(pool) = &input.decoded_account.data {
            println!(
                "slot {} pool {} tick={} liquidity={}",
                input.metadata.slot, input.metadata.pubkey, pool.tick_current_index, pool.liquidity
            );
        }
        Ok(())
    }
}
```

```sh
cargo run
```

Updates start within a second. On mainnet this subscription produced about 2,300 updates in 25 seconds:

Output:

```text
slot 467112567 pool VY1ZQXjqBwvuWgVfTfhqanJe96GGoQrX7xZZDrWPGiT tick=-44384 liquidity=2559193909761
slot 467112567 pool HxA6SKW5qA4o12fjVgTpXdq2YnZ5Zv1s7SB4FFomsyLM tick=66176 liquidity=1192468585558
slot 467112567 pool 6R4r93V5fcMzc13CL2enEepDSYcr4Qx3ptZBDwudTXCo tick=5929 liquidity=225897662074
slot 467112567 pool 7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm tick=-24426 liquidity=565651282086
```

Press Ctrl-C to stop.

## Configuration

### `RpcProgramSubscribe::new`

```rust
pub const fn new(rpc_ws_url: String, filters: Filters) -> Self
```

| Parameter    | Type      | Description                              |
| ------------ | --------- | ---------------------------------------- |
| `rpc_ws_url` | `String`  | WebSocket endpoint, `ws://` or `wss://`. |
| `filters`    | `Filters` | Program and subscription config.         |

### `Filters`

```rust
pub const fn new(pubkey: Pubkey, program_subscribe_config: Option<RpcProgramAccountsConfig>) -> Self
```

| Field                      | Type                               | Default | Meaning                                                               |
| -------------------------- | ---------------------------------- | ------- | --------------------------------------------------------------------- |
| `pubkey`                   | `Pubkey`                           | —       | Program ID to subscribe to.                                           |
| `program_subscribe_config` | `Option<RpcProgramAccountsConfig>` | `None`  | Passed to `programSubscribe` unchanged (`solana_client::rpc_config`). |

The fields of `RpcProgramAccountsConfig` that matter here:

| Field                       | Meaning                                                                                                                                                       |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `filters`                   | `DataSize` and `Memcmp` filters, as for [`getProgramAccounts`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md#rpcprogramaccountsconfig).                 |
| `account_config.encoding`   | **Set `Some(UiAccountEncoding::Base64)`.** The datasource does not set it, and with the server default every event fails with `Error decoding account event`. |
| `account_config.commitment` | Commitment of notifications. `None` uses the server default (`finalized` on Agave).                                                                           |
| `account_config.data_slice` | Return part of the data. Decoders usually fail on sliced data.                                                                                                |

## Behaviour

- **Updates.** Each notification becomes an `AccountUpdate` with the notification's context slot and `transaction_signature: None`. An account with zero lamports becomes an `AccountDeletion` instead.
- **No snapshot.** Only accounts that change after the subscription starts are emitted. Load the initial state with [`GpaDatasource`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md) in the same pipeline; see [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md).
- **Ordering.** In the order the node sends notifications.
- **Reconnects.** When the stream closes, the datasource waits 3 seconds, opens a new client and subscribes again. Failing to connect is retried every 3 seconds, up to 10 attempts; failing to subscribe, up to 11. After that `consume` returns an error. The attempt counter resets after each successful subscription. Changes during the gap are not replayed.
- **Backpressure.** Updates are sent with `try_send`. If the pipeline channel is full, the datasource logs `Error sending account update`, drops the subscription and reconnects after 3 seconds. Raise the pipeline's [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) for busy programs.
- **Finishing.** Never finishes on its own; stops on cancellation or Ctrl-C.
- **Metrics.** `program_subscribe_accounts_processed_total` (counter), `program_subscribe_account_process_time_nanoseconds` (histogram).

## Troubleshooting

### Every event logs `Error decoding account event`

The subscription uses the server's default encoding, which Carbon cannot decode. Set `account_config.encoding: Some(UiAccountEncoding::Base64)` as in the example.

### `Failed to subscribe to program updates`

Logged every 3 seconds. The endpoint rejected `programSubscribe`: check the URL scheme (`wss://`, not `https://`), the program ID, and your filters (a `Memcmp` filter with invalid bytes is rejected). The error text after `Failed to subscribe to program updates:` is the server's reason.

### Updates stop and `Error sending account update` appears

The pipeline is slower than the stream. The datasource reconnects, but changes during the gap are lost. Speed up the processor, raise `channel_buffer_size`, or filter more narrowly.

### No updates for an account that exists

The account hasn't changed since you subscribed. Load current state first with [`GpaDatasource`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md).

## See also

- [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md)
- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md)
- [Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/rpc-program-subscribe · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# RPC transaction crawler (Carbon 2.0)

> Backfill and then poll every transaction that mentions an address, using getSignaturesForAddress and getTransaction on any Solana RPC.

`RpcTransactionCrawler` pages through `getSignaturesForAddress` for one address, fetches each transaction with `getTransaction`, and emits it as a transaction update. It walks history backwards from the newest signature (or from a signature you choose), then keeps polling for new ones. Use it to backfill a program's or wallet's history, or to index a low-traffic program from any RPC endpoint, including the free public one.

|          |                                                                                                                                                                  |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-rpc-transaction-crawler-datasource` 2.0.0                                                                                                                |
| Install  | `cargo add carbon-rpc-transaction-crawler-datasource@2.0.0`                                                                                                      |
| Emits    | `Transaction`                                                                                                                                                    |
| Mode     | Historical backfill, then polling for new transactions. Never finishes on its own.                                                                               |
| Provider | Any Solana JSON-RPC HTTP endpoint. The public `https://api.mainnet-beta.solana.com` works, slowly.                                                               |
| API docs | [docs.rs](https://docs.rs/carbon-rpc-transaction-crawler-datasource/2.0.0/carbon_rpc_transaction_crawler_datasource/)                                            |
| Source   | [`datasources/rpc-transaction-crawler-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/rpc-transaction-crawler-datasource/src/lib.rs) |

## Quick example

This program crawls the Orca Whirlpool program on the public RPC and prints every decoded swap.

```sh
cargo new crawler && cd crawler
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-transaction-crawler-datasource@2.0.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_rpc_transaction_crawler_datasource::{ConnectionConfig, Filters, RpcTransactionCrawler},
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let datasource = RpcTransactionCrawler::new(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,                  // crawl signatures that mention the Orca Whirlpool program
        ConnectionConfig::default(), // 100 signatures per page, 5 concurrent getTransaction calls
        Filters::new(None, None, None),
        None, // commitment: defaults to confirmed
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        match input.decoded_instruction {
            OrcaWhirlpoolInstruction::Swap { data, accounts, .. } => {
                println!("slot {} swap   pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
            }
            OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } => {
                println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
            }
            _ => {}
        }
        Ok(())
    }
}
```

```sh
cargo run
```

Output:

```text
slot 467115758 swap   pool=4v8ufj8Hj7UvFgtofQJAtzUud5xomwZfEqfCTHZ4wM72 amount=19999
slot 467115758 swapV2 pool=FpCMFDFGYotvufJ7HrFHsWEiiQCGbkLCtwHiDnh7o28Q amount=7639572
slot 467115758 swap   pool=Czfq3xZZDmsdGdUyrNLtRhGc47cXcZtLG4crryfu44zE amount=2476330084
slot 467115758 swap   pool=FpCMFDFGYotvufJ7HrFHsWEiiQCGbkLCtwHiDnh7o28Q amount=168756526
```

The public endpoint rate-limits `getTransaction` per IP (its `x-ratelimit-method-limit` header reported 10 when this page was written), so expect a few transactions per second at most. Run with `RUST_LOG=info` to see the 429 retries and the pipeline counters. The program runs until you press Ctrl-C.

## Configuration

### `RpcTransactionCrawler::new`

```rust
pub const fn new(
    rpc_url: String,
    account: Pubkey,
    connection_config: ConnectionConfig,
    filters: Filters,
    commitment: Option<CommitmentConfig>,
) -> Self
```

| Parameter           | Type                       | Description                                                                                                |
| ------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `rpc_url`           | `String`                   | HTTP JSON-RPC endpoint.                                                                                    |
| `account`           | `Pubkey`                   | Address passed to `getSignaturesForAddress`: a program ID, a wallet, a pool, any account.                  |
| `connection_config` | `ConnectionConfig`         | Paging, concurrency, retries and channel sizes. See below.                                                 |
| `filters`           | `Filters`                  | Optional account filter and signature range. See below.                                                    |
| `commitment`        | `Option<CommitmentConfig>` | Commitment for both RPC calls. `None` means `confirmed`. `processed` is not accepted by these RPC methods. |

All fields of `RpcTransactionCrawler` are public, so you can also build it with a struct literal.

### `Filters`

```rust
pub const fn new(
    accounts: Option<Vec<Pubkey>>,
    before_signature: Option<Signature>,
    until_signature: Option<Signature>,
) -> Self
```

| Field              | Type                  | Default | Meaning                                                                                                                                                                                                          |
| ------------------ | --------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `accounts`         | `Option<Vec<Pubkey>>` | `None`  | Keep only transactions whose account keys (static keys plus addresses loaded from lookup tables) include at least one of these. Applied after the transaction is fetched, so it saves processing, not RPC calls. |
| `before_signature` | `Option<Signature>`   | `None`  | Start the backfill immediately before this signature (exclusive) instead of at the newest one.                                                                                                                   |
| `until_signature`  | `Option<Signature>`   | `None`  | Stop the backfill when this signature is reached (exclusive).                                                                                                                                                    |

### `ConnectionConfig`

`ConnectionConfig::default()` is an inherent `const fn`, not the `Default` trait. Override single fields with `ConnectionConfig { batch_limit: 1000, ..ConnectionConfig::default() }`. The positional constructor takes the arguments in a different order from the fields:

```rust
pub const fn new(
    batch_limit: usize,
    polling_interval: Duration,
    max_concurrent_requests: usize,
    retry_config: RetryConfig,
    max_signature_channel_size: Option<usize>,
    max_transaction_channel_size: Option<usize>,
    blocking_send: bool,
) -> Self
```

| Field                          | Type            | Default                  | Meaning                                                                                                                                                             |
| ------------------------------ | --------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `batch_limit`                  | `usize`         | `100`                    | `limit` for each `getSignaturesForAddress` page. RPC nodes cap it at 1000.                                                                                          |
| `polling_interval`             | `Duration`      | 5 s                      | Sleep after a page comes back empty, before polling for new signatures.                                                                                             |
| `max_concurrent_requests`      | `usize`         | `5`                      | `getTransaction` calls in flight at once.                                                                                                                           |
| `retry_config`                 | `RetryConfig`   | `RetryConfig::default()` | Backoff for both RPC calls.                                                                                                                                         |
| `max_signature_channel_size`   | `Option<usize>` | `None` (1000)            | Buffer between the signature fetcher and the transaction fetcher.                                                                                                   |
| `max_transaction_channel_size` | `Option<usize>` | `None` (1000)            | Buffer between the transaction fetcher and the step that sends to the pipeline.                                                                                     |
| `blocking_send`                | `bool`          | `false`                  | `false` uses `try_send` into the pipeline channel and **drops** the update when it is full. `true` waits for space. Set `true` for backfills that must be complete. |

### `RetryConfig`

```rust
pub const fn new(max_retries: u32, initial_backoff_ms: u64, max_backoff_ms: u64, backoff_multiplier: f64) -> Self
pub const fn default() -> Self   // 3, 1000, 10000, 2.0
pub const fn no_retry() -> Self  // 0, 0, 0, 0.0
```

| Field                | Type  | Default | Meaning                                       |
| -------------------- | ----- | ------- | --------------------------------------------- |
| `max_retries`        | `u32` | `3`     | Retries after the first failed attempt.       |
| `initial_backoff_ms` | `u64` | `1000`  | Delay before the first retry.                 |
| `max_backoff_ms`     | `u64` | `10000` | Upper bound on the delay.                     |
| `backoff_multiplier` | `f64` | `2.0`   | Factor applied to the delay after each retry. |

## Behaviour

- **Pipeline.** Three tasks run in sequence: a signature fetcher, a transaction fetcher that runs up to `max_concurrent_requests` `getTransaction` calls (Base64 encoding, `maxSupportedTransactionVersion: 1`), and a processor that builds the update and sends it to the pipeline. `consume` returns as soon as the tasks are spawned.
- **Lifecycle.** The first pass pages backwards from `before_signature` (or the newest signature) until the RPC returns an empty page or `until_signature` is reached. The crawler then sleeps `polling_interval` and polls from the tip down to the newest signature it has seen, forever. It never finishes, so a pipeline that only has this datasource runs until Ctrl-C or until you cancel it (see [shutdown](https://carbon.sevenlabs.org/concepts/shutdown.md)). After the first pass it also picks up everything newer than `before_signature`.
- **Ordering.** Signatures arrive newest first, and transactions are emitted in the order their `getTransaction` calls complete. Updates are not in slot order. Sort by `slot` downstream if order matters.
- **What is skipped.** Failed transactions (non-`Ok` status) and transactions without metadata are dropped. Every update has `is_vote: false`, `index: None` and `block_hash: None`; `block_time` comes from the RPC.
- **Retries.** Each call retries with exponential backoff from `RetryConfig`. A signature page that still fails is retried from scratch on the next loop, so the crawler never skips a page. A transaction that still fails is logged with `Failed to fetch transaction … after N retries` and skipped. On top of that, Solana's `RpcClient` retries HTTP 429 responses up to 5 times, honouring `Retry-After`, and only logs at `debug` level.
- **Backpressure.** With `blocking_send: false`, updates are dropped with `Failed to send update` when the pipeline channel (`channel_buffer_size`, default 1000) is full.
- **Metrics.** `transaction_crawler_signatures_fetched_total`, `transaction_crawler_transactions_fetched_total` (counters), `transaction_crawler_signatures_fetch_times_milliseconds`, `transaction_crawler_transaction_fetch_times_milliseconds`, `transaction_crawler_transaction_process_time_milliseconds` (histograms). See [metrics](https://carbon.sevenlabs.org/concepts/metrics.md).

## Troubleshooting

### Signatures are fetched but nothing is processed

`transaction_crawler_signatures_fetched_total` grows while `transaction_crawler_transactions_fetched_total` stays at 0. At first nothing is logged, because `RpcClient` retries 429 responses silently; once those retries run out, `RUST_LOG=info` shows `Failed to fetch transaction … Status(429, None)`. The RPC is rate limiting `getTransaction`. On the public endpoint the limit is shared by everyone on your IP. Lower `max_concurrent_requests` to 1 or 2, or use a dedicated RPC provider.

### `Method does not support commitment below confirmed`

You passed `CommitmentConfig::processed()`. `getSignaturesForAddress` and `getTransaction` only accept `confirmed` and `finalized`; the signature fetcher retries the failing call forever.

### Transactions are missing from a backfill

Check the logs for `Failed to send update` (pipeline channel full) and `Failed to fetch transaction … after N retries` (RPC errors). Set `blocking_send: true`, raise `retry_config.max_retries`, and raise the pipeline's [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) if your processors are slow. Failed transactions are skipped by design.

### Transactions arrive out of order

Expected: `getTransaction` calls complete in any order. Use `slot` (and your own ordering) in the processor, or set `max_concurrent_requests: 1` for strictly newest-first order.

### The program never exits after the backfill

Expected: the crawler switches to polling. To stop at the end of a range, cancel the pipeline from your code with [`datasource_cancellation_token`](https://carbon.sevenlabs.org/concepts/shutdown.md), or use [`RpcBlockCrawler`](https://carbon.sevenlabs.org/datasources/rpc-block-crawler.md) with an `end_slot`.

## See also

- [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md)
- [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md)
- [Helius `getTransactionsForAddress`](https://carbon.sevenlabs.org/datasources/helius-gtfa.md): the same job in fewer calls, on Helius.
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Stream message (Carbon 2.0)

> Feed account and transaction updates from your own code into a Carbon pipeline through a Tokio channel.

`StreamMessageClient` wraps the receiving end of a Tokio `mpsc` channel. Anything you send into the channel as a `UnifiedMessage` reaches the pipeline as an account or transaction update. Use it to plug in a source Carbon doesn't ship (Kafka, a message queue, a file replay, your own Geyser plugin) or to drive a pipeline from tests, without implementing the [`Datasource` trait](https://carbon.sevenlabs.org/guides/custom-datasource.md) yourself.

|          |                                                                                                                                                |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-stream-message-datasource` 2.0.0                                                                                                       |
| Install  | `cargo add carbon-stream-message-datasource@2.0.0`                                                                                             |
| Emits    | `AccountUpdate`, `AccountDeletion`, `Transaction`                                                                                              |
| Mode     | Whatever your producer sends. Finishes when every sender is dropped.                                                                           |
| Provider | None.                                                                                                                                          |
| API docs | [docs.rs](https://docs.rs/carbon-stream-message-datasource/2.0.0/carbon_stream_message_datasource/)                                            |
| Source   | [`datasources/stream-message-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/stream-message-datasource/src/lib.rs) |

## Quick example

This program builds three Orca Whirlpool `FeeTier` accounts by hand, sends them through the channel, and lets the pipeline decode them. It runs offline.

```sh
cargo new stream-message && cd stream-message
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-stream-message-datasource@2.0.0
cargo add solana-account@4.3.2 solana-pubkey@4.2.1
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, datasource::AccountUpdate, error::CarbonResult,
        pipeline::Pipeline, processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_stream_message_datasource::{StreamMessageClient, UnifiedMessage},
    solana_account::Account,
    solana_pubkey::Pubkey,
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let (tx, rx) = tokio::sync::mpsc::channel::<UnifiedMessage>(1_000);
    let datasource = StreamMessageClient::new(rx);

    // Your producer: a Kafka consumer, a file replay, a test fixture...
    tokio::spawn(async move {
        for (slot, tick_spacing) in [(1_u64, 1_u16), (2, 64), (3, 128)] {
            let update = AccountUpdate {
                pubkey: Pubkey::new_unique(),
                account: fee_tier_account(tick_spacing),
                slot,
                transaction_signature: None,
            };
            tx.send(UnifiedMessage::Account(update)).await.unwrap();
        }
        // Dropping `tx` closes the stream; the pipeline exits once it drains.
    });

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, FeeTierLogger)
        .build()?
        .run()
        .await
}

/// Builds the raw bytes of an Orca `FeeTier` account: discriminator, config, tick spacing, fee rate.
fn fee_tier_account(tick_spacing: u16) -> Account {
    let mut data = vec![56, 75, 159, 76, 142, 68, 190, 105];
    data.extend_from_slice(Pubkey::new_unique().as_ref());
    data.extend_from_slice(&tick_spacing.to_le_bytes());
    data.extend_from_slice(&3000_u16.to_le_bytes());
    Account {
        lamports: 1_200_000,
        data,
        owner: PROGRAM_ID,
        executable: false,
        rent_epoch: u64::MAX,
    }
}

struct FeeTierLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for FeeTierLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::FeeTier(fee_tier) = &input.decoded_account.data {
            println!(
                "slot {} fee tier: tick_spacing={} default_fee_rate={}",
                input.metadata.slot, fee_tier.tick_spacing, fee_tier.default_fee_rate
            );
        }
        Ok(())
    }
}
```

```sh
cargo run
```

Output:

```text
slot 1 fee tier: tick_spacing=1 default_fee_rate=3000
slot 2 fee tier: tick_spacing=64 default_fee_rate=3000
slot 3 fee tier: tick_spacing=128 default_fee_rate=3000
```

The program exits on its own after the third line.

## Configuration

### `StreamMessageClient::new`

```rust
pub fn new(receiver: tokio::sync::mpsc::Receiver<UnifiedMessage>) -> Self
```

The channel's capacity is yours to choose; it buffers messages before they reach the pipeline's own channel.

### `UnifiedMessage`

```rust
pub enum UnifiedMessage {
    Account(AccountUpdate),
    Transaction(Box<TransactionUpdate>),
}
```

`AccountUpdate` and `TransactionUpdate` are the types from `carbon_core::datasource`; see the [updates reference](https://carbon.sevenlabs.org/reference/updates.md) for their fields. There is no variant for block details or explicit deletions.

### `handle_message_stream`

```rust
pub async fn handle_message_stream(
    receiver: Receiver<UnifiedMessage>,
    cancellation_token: CancellationToken,
    sender: Sender<(Update, DatasourceId)>,
    id: DatasourceId,
)
```

The loop `StreamMessageClient` runs internally, exported for use in your own `Datasource` implementation.

## Behaviour

- **Accounts.** Converted with `AccountUpdate::into_update()`: zero-lamport accounts become `AccountDeletion`, others `AccountUpdate`.
- **Transactions.** Forwarded unchanged, so you set `signature`, `meta`, `slot`, `is_vote` and the rest yourself. Carbon does not filter failed transactions here.
- **Ordering and backpressure.** Messages are forwarded in channel order with `send().await`; nothing is dropped.
- **Finishing.** When every `Sender` for your channel is dropped, the loop logs `Receiver closed` and ends. A pipeline with only this datasource then shuts down.
- **Single use.** The receiver is taken on the first `consume`. A second call logs `StreamMessageClient.consume() called more than once; receiver already taken` and does nothing.
- **Metrics.** The names start with `agave_grpc_`: counters `agave_grpc_account_updates_received_total`, `agave_grpc_account_deletions_received_total`, `agave_grpc_transaction_updates_received_total`; histograms `agave_grpc_account_process_time_nanoseconds`, `agave_grpc_account_deletion_process_time_nanoseconds`, `agave_grpc_transaction_process_time_nanoseconds`.

## Troubleshooting

### The pipeline never exits

A `Sender` clone is still alive somewhere, so the stream never closes. Drop every clone when your producer is done.

### The pipeline exits immediately

Every sender was dropped before anything was sent, or the producer task panicked. Check that the producer holds `tx` for as long as it sends.

### Accounts aren't decoded

The decoder checks `account.owner` and the account discriminator. Set `owner` to the program ID and put the raw on-chain bytes, discriminator included, in `data`.

### The README example doesn't compile

The crate's README refers to `StreamMessageDatasource` and `pipeline.register`; neither exists. The type is `StreamMessageClient`, added with `Pipeline::builder().datasource(…)` as above.

## See also

- [Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md)
- [Testing](https://carbon.sevenlabs.org/guides/testing.md) and [test utilities](https://carbon.sevenlabs.org/reference/test-utils.md)
- [Updates reference](https://carbon.sevenlabs.org/reference/updates.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/stream-message · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Validator snapshot (Carbon 2.0)

> Load every account owned by a program, or a list of accounts, from a Solana validator snapshot on disk or downloaded from a node.

`SnapshotDatasource` loads a full validator snapshot with Agave's own loader, scans every account in it, and emits the ones owned by programs you list (or with addresses you list) as account updates at the snapshot's slot. Use it to bootstrap complete account state for a large program without `getProgramAccounts` limits, when you can afford the disk, memory and time a mainnet snapshot needs.

|          |                                                                                                                                                                   |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-validator-snapshot-datasource` 2.0.0                                                                                                                      |
| Install  | `cargo add carbon-validator-snapshot-datasource@2.0.0`                                                                                                            |
| Emits    | `AccountUpdate`                                                                                                                                                   |
| Mode     | Point-in-time state at the snapshot slot. Finishes after the scan.                                                                                                |
| Provider | None: a snapshot archive on disk, or a node that serves `/genesis.tar.bz2` and `/snapshot.tar.bz2` (Agave RPC nodes do, including `api.mainnet-beta.solana.com`). |
| API docs | [docs.rs](https://docs.rs/carbon-validator-snapshot-datasource/2.0.0/carbon_validator_snapshot_datasource/)                                                       |
| Source   | [`datasources/validator-snapshot-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/validator-snapshot-datasource/src/lib.rs)            |

> **Warning: Heavy prerequisites.** The crate links Agave's ledger and RocksDB, so it needs a C++ compiler and libclang to build (Debian and Ubuntu: `sudo apt install build-essential clang libclang-dev`); without libclang the build fails in `clang-sys`. A mainnet full snapshot was 126 GB compressed when this page was written, and loading it needs several times that in disk and tens of GB of RAM.

## Quick example

This program loads every Orca Whirlpool pool account from a snapshot and prints its liquidity. It compiles as shown; to run it, set `SNAPSHOT_PATH` to a ledger directory or `SNAPSHOT_URL` to a node.

```sh
cargo new snapshot && cd snapshot
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-validator-snapshot-datasource@2.0.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_validator_snapshot_datasource::{SnapshotDatasource, SnapshotSource},
    std::{env, sync::Arc},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let source = match env::var("SNAPSHOT_PATH") {
        // A ledger directory with genesis.tar.bz2 and snapshots/<full snapshot archive>
        Ok(path) => SnapshotSource::LocalPath(path.into()),
        // A node that serves /genesis.tar.bz2 and /snapshot.tar.bz2, e.g. http://1.2.3.4:8899
        Err(_) => SnapshotSource::Remote {
            url: env::var("SNAPSHOT_URL").expect("set SNAPSHOT_PATH or SNAPSHOT_URL"),
        },
    };

    let datasource = SnapshotDatasource::new(
        source,
        vec![PROGRAM_ID], // owners: every account owned by the Orca Whirlpool program
        vec![],           // accounts: specific pubkeys, whatever their owner
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, PoolLogger)
        .build()?
        .run()
        .await
}

struct PoolLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for PoolLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::Whirlpool(pool) = &input.decoded_account.data {
            println!(
                "slot {} pool {} liquidity={}",
                input.metadata.slot, input.metadata.pubkey, pool.liquidity
            );
        }
        Ok(())
    }
}
```

```sh
cargo build --release
SNAPSHOT_PATH=/data/ledger RUST_LOG=info ./target/release/snapshot
```

Build in release mode: loading a snapshot in a debug build is much slower. With `RUST_LOG=info` you see `Snapshot loaded at slot … in …s, starting account scan`, one line per pool, then `Snapshot processing completed`. Pointed at a directory without a genesis archive, it stops at once:

Output:

```text
[ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("Genesis archive not found: \"/data/ledger/genesis.tar.bz2\". Expected genesis.tar.bz2")
```

## Configuration

### `SnapshotDatasource::new`

```rust
pub fn new(source: SnapshotSource, owners: Vec<Pubkey>, accounts: Vec<Pubkey>) -> Self
```

| Parameter  | Type             | Meaning                                            |
| ---------- | ---------------- | -------------------------------------------------- |
| `source`   | `SnapshotSource` | Where the snapshot comes from.                     |
| `owners`   | `Vec<Pubkey>`    | Emit every account owned by one of these programs. |
| `accounts` | `Vec<Pubkey>`    | Also emit these accounts, whatever their owner.    |

If both lists are empty, the datasource logs `No owners or accounts specified, skipping account scan` and emits nothing.

### `SnapshotSource`

```rust
pub enum SnapshotSource {
    LocalPath(PathBuf),
    Remote { url: String },
}
```

| Variant           | Meaning                                                                                                                                                                                                                                                      |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `LocalPath(path)` | A ledger directory containing `genesis.tar.bz2` and a `snapshots/` directory with a full snapshot archive (`snapshot-<slot>-<hash>.tar.zst`; an incremental archive next to it is also loaded). The loader unpacks accounts into `<path>/accounts`.          |
| `Remote { url }`  | A base URL. The datasource downloads `<url>/genesis.tar.bz2` and `<url>/snapshot.tar.bz2`, following up to 10 redirects, into `./ledger` under the current working directory, then loads it like a local path. Pass the node's address, not the archive URL. |

## Behaviour

- **Downloading.** Each file is tried 3 times, 2 seconds apart. The saved file name comes from the `Content-Disposition` header or the final redirect URL, so a node's redirect to `snapshot-<slot>-<hash>.tar.zst` keeps the name Agave expects. Progress is logged every 50 MB and 5 seconds. Only the full snapshot is downloaded.
- **Loading.** Agave's `bank_forks_utils` loads the snapshot in load-only mode, skipping the initial accounts hash. A loaded bank at slot 0 is rejected as invalid. Loading runs on the async task that called `consume` and can't be cancelled.
- **Scanning.** Every account is visited once on a blocking thread. Accounts with zero lamports are skipped. Matches are emitted as `AccountUpdate` with the snapshot slot and `transaction_signature: None`; no deletions are emitted.
- **Ordering.** The order of the accounts database, not by pubkey or slot.
- **Backpressure.** Updates are sent with `blocking_send`, so the scan waits for the pipeline and nothing is dropped.
- **Finishing.** `consume` returns after the scan. A pipeline with only this datasource then shuts down; next to a streaming datasource, the pipeline keeps running.
- **Errors.** Missing directories or archives, a failed download, or a snapshot Agave can't load return `FailedToConsumeDatasource` with the reason.
- **Metrics.** `validator_snapshot_accounts_processed_total` (counter), `validator_snapshot_load_duration_milliseconds`, `validator_snapshot_scan_duration_milliseconds` (histograms).

## Troubleshooting

### Build fails in `clang-sys`: `couldn't find any valid shared libraries matching: ['libclang.so', …]`

Install libclang (`sudo apt install clang libclang-dev` on Debian and Ubuntu), or point `LIBCLANG_PATH` at the directory that contains `libclang.so`.

### `Genesis archive not found: …/genesis.tar.bz2`

`LocalPath` must point at the ledger directory itself, with `genesis.tar.bz2` at its top level.

### `No loadable full snapshot archive found in …/snapshots`

Put the full snapshot archive in `<ledger>/snapshots/` and keep its original `snapshot-<slot>-<hash>.tar.zst` name; Agave finds archives by name.

### `Failed to download …/genesis.tar.bz2 after 3 retries`

The URL is not a node's base address, or the node doesn't serve snapshots. `Remote { url }` appends `/genesis.tar.bz2` itself: use `https://api.mainnet-beta.solana.com`, not `…/snapshot.tar.bz2`.

### The process is killed during loading

Out of memory or disk. Mainnet snapshots need a machine sized like an RPC node. For a single program, [`GpaDatasource`](https://carbon.sevenlabs.org/datasources/rpc-gpa.md) or [Helius gPA v2](https://carbon.sevenlabs.org/datasources/helius-gpa-v2.md) is far cheaper.

## See also

- [Load a snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md)
- [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md)
- [RPC getProgramAccounts](https://carbon.sevenlabs.org/datasources/rpc-gpa.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/validator-snapshot · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Yellowstone gRPC (Carbon 2.0)

> Stream accounts, transactions and blocks from any Yellowstone (Dragon's Mouth) Geyser gRPC endpoint with the lowest latency Carbon offers.

`YellowstoneGrpcGeyserClient` opens one Yellowstone Geyser gRPC subscription and turns its account, transaction and block messages into Carbon updates. Use it for production real-time indexing when you have a Yellowstone endpoint: your own validator with the plugin, or a provider such as Triton, Helius, QuickNode or Shyft. For Helius LaserStream with slot replay, use [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md) instead.

|          |                                                                                                                                                    |
| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| Crate    | `carbon-yellowstone-grpc-datasource` 2.0.0                                                                                                         |
| Install  | `cargo add carbon-yellowstone-grpc-datasource@2.0.0`                                                                                               |
| Emits    | `AccountUpdate`, `AccountDeletion`, `Transaction`                                                                                                  |
| Mode     | Live. Runs until cancelled.                                                                                                                        |
| Provider | A Yellowstone gRPC endpoint and, usually, an `x-token`. Paid with most providers.                                                                  |
| API docs | [docs.rs](https://docs.rs/carbon-yellowstone-grpc-datasource/2.0.0/carbon_yellowstone_grpc_datasource/)                                            |
| Source   | [`datasources/yellowstone-grpc-datasource`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/datasources/yellowstone-grpc-datasource/src/lib.rs) |

## Quick example

This program subscribes to confirmed, successful, non-vote transactions that include the Orca Whirlpool program and prints each `swapV2`. It compiles as shown; to run it, set `GEYSER_URL` (for example `https://your-endpoint:443`) and `X_TOKEN`.

```sh
cargo new yellowstone && cd yellowstone
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-yellowstone-grpc-datasource@2.0.0
cargo add yellowstone-grpc-proto@=12.7.0
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4
```

`src/main.rs`:

```rust
use {
    carbon_core::{
        error::CarbonResult, instruction::InstructionProcessorInputType, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        instructions::OrcaWhirlpoolInstruction, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_yellowstone_grpc_datasource::{
        BlockFilters, YellowstoneGrpcClientConfig, YellowstoneGrpcGeyserClient,
    },
    std::{collections::HashMap, env, sync::Arc},
    yellowstone_grpc_proto::geyser::{CommitmentLevel, SubscribeRequestFilterTransactions},
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    let mut transaction_filters = HashMap::new();
    transaction_filters.insert(
        "orca_whirlpool".to_string(),
        SubscribeRequestFilterTransactions {
            vote: Some(false),
            failed: Some(false),
            account_include: vec![PROGRAM_ID.to_string()],
            ..Default::default()
        },
    );

    let datasource = YellowstoneGrpcGeyserClient::new(
        env::var("GEYSER_URL").expect("GEYSER_URL must be set"),
        env::var("X_TOKEN").ok(),
        Some(CommitmentLevel::Confirmed),
        HashMap::new(), // account filters
        transaction_filters,
        BlockFilters::default(),
        YellowstoneGrpcClientConfig::default(),
        None, // disconnect notifier
        None, // stream timeout: 30 s
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .instruction(OrcaWhirlpoolDecoder, SwapLogger)
        .build()?
        .run()
        .await
}

struct SwapLogger;

impl Processor<InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>> for SwapLogger {
    async fn process(
        &mut self,
        input: &InstructionProcessorInputType<'_, OrcaWhirlpoolInstruction>,
    ) -> CarbonResult<()> {
        let tx = &input.metadata.transaction_metadata;
        if let OrcaWhirlpoolInstruction::SwapV2 { data, accounts, .. } = input.decoded_instruction {
            println!("slot {} swapV2 pool={} amount={}", tx.slot, accounts.whirlpool, data.amount);
        }
        Ok(())
    }
}
```

```sh
GEYSER_URL=https://your-yellowstone-endpoint:443 X_TOKEN=your-x-token RUST_LOG=info cargo run
```

Each matching swap prints one `slot … swapV2 pool=… amount=…` line as its slot is confirmed. For a step-by-step walkthrough, see [Stream with Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md).

## Configuration

### `YellowstoneGrpcGeyserClient::new`

```rust
pub fn new(
    endpoint: String,
    x_token: Option<String>,
    commitment: Option<CommitmentLevel>,
    account_filters: HashMap<String, SubscribeRequestFilterAccounts>,
    transaction_filters: HashMap<String, SubscribeRequestFilterTransactions>,
    block_filters: BlockFilters,
    geyser_config: YellowstoneGrpcClientConfig,
    disconnect_notifier: Option<mpsc::Sender<DatasourceDisconnection>>,
    stream_timeout: Option<Duration>,
) -> Self
```

| Parameter             | Type                                                  | Default                   | Meaning                                                                                                                                                                          |
| --------------------- | ----------------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `endpoint`            | `String`                                              | —                         | gRPC URL, including scheme and port.                                                                                                                                             |
| `x_token`             | `Option<String>`                                      | —                         | Sent as the `x-token` header. `None` for endpoints without auth.                                                                                                                 |
| `commitment`          | `Option<CommitmentLevel>`                             | —                         | `Processed`, `Confirmed` or `Finalized`. `None` leaves it to the server (Yellowstone defaults to `Processed`, which can include transactions from forks that are later dropped). |
| `account_filters`     | `HashMap<String, SubscribeRequestFilterAccounts>`     | —                         | Named account filters. The key is a label of your choice. Empty: no account stream.                                                                                              |
| `transaction_filters` | `HashMap<String, SubscribeRequestFilterTransactions>` | —                         | Named transaction filters. Empty: no transaction stream.                                                                                                                         |
| `block_filters`       | `BlockFilters`                                        | `BlockFilters::default()` | Block subscriptions. See below.                                                                                                                                                  |
| `geyser_config`       | `YellowstoneGrpcClientConfig`                         | `default()`               | Connection settings. See below.                                                                                                                                                  |
| `disconnect_notifier` | `Option<mpsc::Sender<DatasourceDisconnection>>`       | `None`                    | Receives one event per reconnect with the slot gap.                                                                                                                              |
| `stream_timeout`      | `Option<Duration>`                                    | 30 s                      | With no message for this long, the stream is treated as dead and reopened.                                                                                                       |

Filter types come from `yellowstone_grpc_proto::geyser` (`yellowstone-grpc-proto` 12.7.0). They are prost messages, so `..Default::default()` fills the fields you don't set.

`SubscribeRequestFilterTransactions`:

| Field                                      | Type             | Meaning                                                                                                           |
| ------------------------------------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------- |
| `vote`                                     | `Option<bool>`   | `Some(false)` excludes vote transactions, `Some(true)` keeps only votes.                                          |
| `failed`                                   | `Option<bool>`   | `Some(false)` excludes failed transactions. Carbon does not drop failed transactions from this stream on its own. |
| `signature`                                | `Option<String>` | A single transaction signature.                                                                                   |
| `account_include`                          | `Vec<String>`    | Match transactions that use any of these addresses.                                                               |
| `account_exclude`                          | `Vec<String>`    | Drop transactions that use any of these addresses.                                                                |
| `account_required`                         | `Vec<String>`    | Match only transactions that use all of these addresses.                                                          |
| `cuckoo_account_include`, `token_accounts` |                  | Provider extensions; see your provider's docs.                                                                    |

`SubscribeRequestFilterAccounts`:

| Field                    | Type                                        | Meaning                                                                                           |
| ------------------------ | ------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| `account`                | `Vec<String>`                               | Specific account addresses.                                                                       |
| `owner`                  | `Vec<String>`                               | Accounts owned by these programs.                                                                 |
| `filters`                | `Vec<SubscribeRequestFilterAccountsFilter>` | `Filter::Memcmp`, `Filter::Datasize(u64)`, `Filter::TokenAccountState(bool)`, `Filter::Lamports`. |
| `nonempty_txn_signature` | `Option<bool>`                              | Only updates caused by a transaction.                                                             |

For example, to stream every Orca Whirlpool pool account:

```rust
use yellowstone_grpc_proto::geyser::{
    subscribe_request_filter_accounts_filter::Filter, SubscribeRequestFilterAccounts,
    SubscribeRequestFilterAccountsFilter,
};

let mut account_filters = HashMap::new();
account_filters.insert(
    "whirlpools".to_string(),
    SubscribeRequestFilterAccounts {
        owner: vec![PROGRAM_ID.to_string()],
        filters: vec![SubscribeRequestFilterAccountsFilter {
            filter: Some(Filter::Datasize(653)),
        }],
        ..Default::default()
    },
);
```

### `BlockFilters`

| Field                 | Type                                            | Default       | Meaning                                                                                                |
| --------------------- | ----------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------ |
| `filters`             | `HashMap<String, SubscribeRequestFilterBlocks>` | empty         | Named block filters: `account_include`, `include_transactions`, `include_accounts`, `include_entries`. |
| `failed_transactions` | `Option<bool>`                                  | `None` (keep) | `Some(false)` drops failed transactions from block messages.                                           |

### `YellowstoneGrpcClientConfig`

```rust
pub const fn new(
    compression: Option<CompressionEncoding>,
    connect_timeout: Option<Duration>,
    timeout: Option<Duration>,
    max_decoding_message_size: Option<usize>,
    tls_config: Option<ClientTlsConfig>,
    tcp_nodelay: Option<bool>,
) -> Self
pub fn with_reconnect(self, reconnect: ReconnectConfig) -> Self
```

| Field                       | Type                          | Default                                          | Meaning                                                                                                                    |
| --------------------------- | ----------------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| `compression`               | `Option<CompressionEncoding>` | `None`                                           | Send and accept compressed messages (`tonic::codec::CompressionEncoding`, re-exported by `yellowstone_grpc_proto::tonic`). |
| `connect_timeout`           | `Option<Duration>`            | 15 s                                             | Connection timeout.                                                                                                        |
| `timeout`                   | `Option<Duration>`            | 15 s                                             | Request timeout.                                                                                                           |
| `max_decoding_message_size` | `Option<usize>`, bytes        | `None`: tonic's limit of 4 MiB (4,194,304 bytes) | Largest message the client accepts. Raise it when large blocks or accounts fail; see below.                                |
| `tls_config`                | `Option<ClientTlsConfig>`     | `None` (system and webpki roots)                 | Custom TLS configuration.                                                                                                  |
| `tcp_nodelay`               | `Option<bool>`                | `None`                                           | Set `TCP_NODELAY`.                                                                                                         |
| `reconnect`                 | `Option<ReconnectConfig>`     | `None`                                           | Reconnect inside the client library and replay missed data. Set with `with_reconnect`.                                     |

A message over `max_decoding_message_size` ends the stream with tonic's `OutOfRange` status, which the datasource logs as a stream error before resubscribing. The status message has this form, with your sizes:

```text
Error, decoded message length too large: found 5242880 bytes, the limit is: 4194304 bytes
```

Block subscriptions and large program accounts are the usual causes. Raise the limit when you build the config:

```rust
// Accept messages up to 64 MiB instead of tonic's 4 MiB default.
let config = YellowstoneGrpcClientConfig::new(None, None, None, Some(64 * 1024 * 1024), None, None);
```

### Reconnects

Without any configuration, the datasource resubscribes on its own after a stream error and never gives up, but data sent during the gap is lost (see [Behaviour](#behaviour)). To have the client library also replay what you missed, opt in with `with_reconnect`.

`ReconnectConfig` comes from `yellowstone-grpc-client`, which you add with `cargo add yellowstone-grpc-client@=13.5.0`. Its default retries 3 times with a 10 ms backoff and recovers missed messages for up to 250 slots:

```rust
use yellowstone_grpc_client::ReconnectConfig;

let geyser_config = YellowstoneGrpcClientConfig::default().with_reconnect(ReconnectConfig::default());
```

### Detect gaps with `DatasourceDisconnection`

With a `disconnect_notifier`, the datasource sends `{ source: "yellowstone-grpc", disconnect_time, last_slot_before_disconnect, first_slot_after_reconnect, missed_slots }` on the first message after a reconnect. Read it in your own task:

```rust
let (disconnect_tx, mut disconnect_rx) = tokio::sync::mpsc::channel::<DatasourceDisconnection>(16);
tokio::spawn(async move {
    while let Some(gap) = disconnect_rx.recv().await {
        log::warn!(
            "missed {} slots ({}..{})",
            gap.missed_slots, gap.last_slot_before_disconnect, gap.first_slot_after_reconnect
        );
    }
});
// pass Some(disconnect_tx) as `disconnect_notifier`
```

## Behaviour

- **Connecting.** `consume` connects before spawning the stream task. If the first connection fails, it returns `FailedToConsumeDatasource("gRPC transport error: …")`; a pipeline with only this datasource then exits.
- **Subscription.** One `SubscribeRequest` with your account, transaction and block filters and commitment. When the server sends a ping, the datasource answers by resending the request with a ping.
- **Account messages** become `AccountUpdate` (or `AccountDeletion` at zero lamports) with the message's slot and the causing `transaction_signature`.
- **Transaction messages** become `Transaction` updates with `is_vote` and `index` from Geyser, and `block_time`/`block_hash` set to `None`. Failed and vote transactions pass through unless your filter excludes them.
- **Block messages** produce a `Transaction` update for each transaction (with the block's `block_time`) and an account update for each account in the block. No `BlockDetails` updates are emitted.
- **Ordering.** As delivered by the server: slot order for a given commitment, but accounts and transactions from different filters interleave.
- **Reconnects.** When the stream errors, closes or is silent for `stream_timeout`, the datasource subscribes again on the same client right away. If subscribing fails, it backs off from 100 ms, doubling up to 3 s, and never gives up. Without `reconnect`, data during the gap is lost; `disconnect_notifier` tells you which slots.
- **Backpressure.** Every update uses `try_send`. When the pipeline channel is full the update is dropped with `Failed to send transaction update …` or `Failed to send account event …`. Size the pipeline's [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md) for your peak rate.
- **Finishing.** Never finishes on its own.
- **Metrics.** Counters `yellowstone_grpc_account_updates_received_total`, `yellowstone_grpc_account_deletions_received_total`, `yellowstone_grpc_transaction_updates_received_total`; histograms `yellowstone_grpc_account_process_time_nanoseconds`, `yellowstone_grpc_account_deletion_process_time_nanoseconds`, `yellowstone_grpc_transaction_process_time_nanoseconds`.

## Troubleshooting

### `error consuming datasource: FailedToConsumeDatasource("gRPC transport error: transport error")`

The first connection failed and the pipeline exited. Check the URL (scheme and port, for example `https://host:443`), that the host is reachable, and the provider's IP allowlist.

### The log fills with `Geyser stream error: Status { … }`

The server rejects the subscription; the `Status` code and message say why. A missing or wrong token is the most common cause: pass it as `x_token`. A filter the provider doesn't allow (too many accounts, an unsupported field) is the next. The datasource resubscribes immediately after each stream error, so a persistent problem produces many log lines per second.

### `Geyser stream error … invalid compression flag … status: 501 Not Implemented`

The URL points at something that isn't a Yellowstone gRPC server, such as a JSON-RPC endpoint. Use the provider's gRPC URL.

### Panic: `Could not automatically determine the process-level CryptoProvider`

Two `rustls` crypto backends ended up in your dependency tree. Add `rustls` (`cargo add rustls@0.23 --no-default-features --features ring`) and install one provider at the top of `main`, before building the pipeline:

```rust
rustls::crypto::ring::default_provider()
    .install_default()
    .expect("failed to install rustls crypto provider");
```

### Updates are missing under load

Look for `Failed to send transaction update` in the logs. Raise `Pipeline::builder().channel_buffer_size(…)`, make processors faster, or narrow your filters.

### Transactions that later disappear

With `CommitmentLevel::Processed` (or `None`) you also receive transactions from forks that never confirm. Use `Confirmed` unless you handle rollbacks.

## See also

- [Stream with Yellowstone](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md)
- [Helius LaserStream](https://carbon.sevenlabs.org/datasources/helius-laserstream.md): same protocol, with slot replay.
- [Production](https://carbon.sevenlabs.org/guides/production.md) and [Prometheus](https://carbon.sevenlabs.org/guides/prometheus.md)
- [Datasources concept](https://carbon.sevenlabs.org/concepts/datasources.md) and [update types](https://carbon.sevenlabs.org/concepts/updates.md)

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/datasources/yellowstone-grpc · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Decoders (Carbon 2.0)

> All 63 program decoders published with Carbon 2.0.0: crates, program IDs and what each one decodes.

Carbon 2.0.0 ships 63 decoder crates for Solana programs, covering 1895 instructions, 301 account types and 455 events. Each page lists the crate, program ID, every instruction with its discriminator, data fields and accounts, every account and event, and a runnable example.

## What a decoder is

A decoder crate turns raw instructions and account data of one program into Rust types. Every crate exports the same items:

| Item                                 | What it is                                                                                                     |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| `PROGRAM_ID`                         | The program's address, a `Pubkey` constant. Use it to point a datasource at the program.                       |
| `<Program>Decoder`                   | A unit struct that implements `InstructionDecoder` and `AccountDecoder`. Pass it to the pipeline builder.      |
| `instructions::<Program>Instruction` | One variant per instruction: `{ program_id, data, accounts }`, with typed instruction data and named accounts. |
| `accounts::<Program>Account`         | One variant per account type, wrapping the decoded struct.                                                     |
| `instructions::CpiEvent`             | For Anchor programs that emit events with `emit_cpi!`: one variant per event.                                  |
| `types`                              | Structs and enums used inside instructions, accounts and events.                                               |

Every crate has the optional features `serde`, `postgres` and `graphql`. See [decoders](https://carbon.sevenlabs.org/concepts/decoders.md) for how decoding fits into a pipeline.

## Use a decoder

Add the crate at the same version as `carbon-core`, then register the decoder with a processor for its instruction or account enum:

```sh
cargo add carbon-core@2.0.0 carbon-pump-swap-decoder@2.0.0
```

This fragment goes in your pipeline setup; `BuyLogger` and `PoolLogger` are your [processors](https://carbon.sevenlabs.org/concepts/processors.md):

```rust
use carbon_pump_swap_decoder::{PumpSwapDecoder, PROGRAM_ID};

Pipeline::builder()
    .datasource(datasource) // e.g. an RpcTransactionCrawler for PROGRAM_ID
    .instruction(PumpSwapDecoder, BuyLogger)
    .account(PumpSwapDecoder, PoolLogger)
    .build()?
    .run()
    .await
```

Each decoder page has a complete program in its **Usage** section, and the [quickstart](https://carbon.sevenlabs.org/quickstart.md) walks through one end to end. To decode several programs in one pipeline, see [multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## Generate your own

Every crate here is generated by the Carbon CLI from the program's Anchor or Codama IDL. For a program that is not listed, generate a decoder the same way: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). To write one by hand, see [custom decoder](https://carbon.sevenlabs.org/guides/custom-decoder.md).

## Catalog

Instruction counts exclude the `CpiEvent` wrapper. Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`.

### Core & SPL

See [core and SPL program decoders](https://carbon.sevenlabs.org/decoders/core-and-spl.md) for these programs together.

| Program                                                                                       | Crate                                     | Program ID                                     | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Address Lookup Table](https://carbon.sevenlabs.org/decoders/address-lookup-table.md)         | `carbon-address-lookup-table-decoder`     | `AddressLookupTab1e1111111111111111111111111`  |            5 |        1 |      0 |
| [Associated Token Account](https://carbon.sevenlabs.org/decoders/associated-token-account.md) | `carbon-associated-token-account-decoder` | `ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL` |            3 |        0 |      0 |
| [SPL Memo](https://carbon.sevenlabs.org/decoders/memo-program.md)                             | `carbon-memo-program-decoder`             | `MemoSq4gqABAXKb96qnH8TysNcWxMyWCqXgDLGmfcHr`  |            1 |        0 |      0 |
| [SPL Name Service](https://carbon.sevenlabs.org/decoders/name-service.md)                     | `carbon-name-service-decoder`             | `namesLPneVptA9Z5rqUDD9tMTWEJwofgaYwp8cawRkX`  |            5 |        1 |      0 |
| [SPL Token](https://carbon.sevenlabs.org/decoders/token-program.md)                           | `carbon-token-program-decoder`            | `TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA`  |           28 |        3 |      0 |
| [Stake Program](https://carbon.sevenlabs.org/decoders/stake-program.md)                       | `carbon-stake-program-decoder`            | `Stake11111111111111111111111111111111111111`  |           18 |        0 |      0 |
| [System Program](https://carbon.sevenlabs.org/decoders/system-program.md)                     | `carbon-system-program-decoder`           | `11111111111111111111111111111111`             |           13 |        1 |      0 |
| [Token-2022](https://carbon.sevenlabs.org/decoders/token-2022.md)                             | `carbon-token-2022-decoder`               | `TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb`  |           91 |        3 |      0 |

### DEX & AMM

See [dEX and AMM decoders](https://carbon.sevenlabs.org/decoders/dex-and-amm.md) for these programs together.

| Program                                                                                  | Crate                                  | Program ID                                     | Instructions | Accounts | Events |
| ---------------------------------------------------------------------------------------- | -------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [BonkSwap](https://carbon.sevenlabs.org/decoders/bonkswap.md)                            | `carbon-bonkswap-decoder`              | `BSwp6bEBihVLdqJRKGgzjcGLHkcTuzmSo1TQkHepzH8p` |           19 |        5 |      0 |
| [FluxBeam](https://carbon.sevenlabs.org/decoders/fluxbeam.md)                            | `carbon-fluxbeam-decoder`              | `FLUXubRmkEi2q6K3Y9kBPg9248ggaZVsoSFhtJHSrm1X` |            6 |        0 |      0 |
| [Gavel](https://carbon.sevenlabs.org/decoders/gavel.md)                                  | `carbon-gavel-decoder`                 | `srAMMzfVHVAtgSJc8iH6CfKzuWuUTzLHVCE81QU1rgi`  |           10 |        2 |      0 |
| [Heaven](https://carbon.sevenlabs.org/decoders/heaven.md)                                | `carbon-heaven-decoder`                | `HEAVENoP2qxoeuF8Dj2oT1GHEnu49U5mJYkdeC8BAX2o` |           54 |        6 |      7 |
| [Jupiter Limit Order](https://carbon.sevenlabs.org/decoders/jupiter-limit-order.md)      | `carbon-jupiter-limit-order-decoder`   | `jupoNjAxXgZ4rjzxzPMP4oxduvQsQtZzyknqvzYNrNu`  |            9 |        2 |      3 |
| [Jupiter Limit Order v2](https://carbon.sevenlabs.org/decoders/jupiter-limit-order-2.md) | `carbon-jupiter-limit-order-2-decoder` | `j1o2qRpjcyUwEvwtcfhEQefh773ZgjxcVRry7LDqg5X`  |            8 |        2 |      4 |
| [Kamino Limit Order](https://carbon.sevenlabs.org/decoders/kamino-limit-order.md)        | `carbon-kamino-limit-order-decoder`    | `LiMoM9rMhrdYrfzUCxQppvxCSG1FcrUK9G8uLq4A1GF`  |           15 |        3 |      2 |
| [Lifinity AMM v2](https://carbon.sevenlabs.org/decoders/lifinity-amm-v2.md)              | `carbon-lifinity-amm-v2-decoder`       | `2wT8Yq49kHgDzXuPxZSaeLaH1qbmGXtEyPy64bL7aD3c` |            3 |        1 |      0 |
| [Meteora DAMM v2](https://carbon.sevenlabs.org/decoders/meteora-damm-v2.md)              | `carbon-meteora-damm-v2-decoder`       | `cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG`  |           38 |        9 |     23 |
| [Meteora DLMM](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md)                    | `carbon-meteora-dlmm-decoder`          | `LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo`  |           76 |       12 |     30 |
| [Meteora Pools](https://carbon.sevenlabs.org/decoders/meteora-pools.md)                  | `carbon-meteora-pools-decoder`         | `Eo7WjKq67rjJQSZxS6z3YkapzY3eMj6Xy8X5EQVn5UaB` |           26 |        3 |     18 |
| [OpenBook v2](https://carbon.sevenlabs.org/decoders/openbook-v2.md)                      | `carbon-openbook-v2-decoder`           | `opnb2LAfJYbRMAHHvqjCwQxanZn7ReEHp1k81EohpZb`  |           29 |        6 |      8 |
| [Orca Whirlpools](https://carbon.sevenlabs.org/decoders/orca-whirlpool.md)               | `carbon-orca-whirlpool-decoder`        | `whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc`  |           66 |       12 |      6 |
| [PancakeSwap](https://carbon.sevenlabs.org/decoders/pancake-swap.md)                     | `carbon-pancake-swap-decoder`          | `HpNfyc2Saw7RKkQd8nEL4khUcuPhQ7WwY1B2qjx8jxFq` |           27 |       10 |     11 |
| [Phoenix v1](https://carbon.sevenlabs.org/decoders/phoenix-v1.md)                        | `carbon-phoenix-v1-decoder`            | `PhoeNiXZ8ByJGLkxNfZRnkUfjvmuYqLR89jjFHGqdXY`  |           28 |        0 |      0 |
| [PumpSwap](https://carbon.sevenlabs.org/decoders/pump-swap.md)                           | `carbon-pump-swap-decoder`             | `pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA`  |           25 |        7 |     22 |
| [Raydium AMM v4](https://carbon.sevenlabs.org/decoders/raydium-amm-v4.md)                | `carbon-raydium-amm-v4-decoder`        | `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8` |           18 |        3 |      0 |
| [Raydium CLMM](https://carbon.sevenlabs.org/decoders/raydium-clmm.md)                    | `carbon-raydium-clmm-decoder`          | `CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK` |           34 |       12 |     15 |
| [Raydium CPMM](https://carbon.sevenlabs.org/decoders/raydium-cpmm.md)                    | `carbon-raydium-cpmm-decoder`          | `CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C` |           14 |        4 |      2 |
| [Raydium Stable Swap](https://carbon.sevenlabs.org/decoders/raydium-stable-swap.md)      | `carbon-raydium-stable-swap-decoder`   | `5quBtoiQqxF9Jv6KYKctB59NT3gtJD2Y65kdnB1Uev3h` |            6 |        0 |      0 |
| [Stabble Stable Swap](https://carbon.sevenlabs.org/decoders/stabble-stable-swap.md)      | `carbon-stabble-stable-swap-decoder`   | `swapNyd8XiQwJ6ianp9snpu4brUqFxadzvHebnAXjJZ`  |           19 |        3 |      2 |
| [Stabble Weighted Swap](https://carbon.sevenlabs.org/decoders/stabble-weighted-swap.md)  | `carbon-stabble-weighted-swap-decoder` | `swapFpHZwjELNnjvThjajtiVmkz3yPQEHjLtka2fwHW`  |           13 |        2 |      2 |

### Lending

See [lending decoders](https://carbon.sevenlabs.org/decoders/lending.md) for these programs together.

| Program                                                                   | Crate                           | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Jupiter Lend](https://carbon.sevenlabs.org/decoders/jupiter-lend.md)     | `carbon-jupiter-lend-decoder`   | `jupeiUmn818Jg1ekPURTpr4mFo29p46vygyykFJ3wZC`  |           24 |        7 |     19 |
| [Kamino Lending](https://carbon.sevenlabs.org/decoders/kamino-lending.md) | `carbon-kamino-lending-decoder` | `KLend2g3cP87fffoy8q1mQqGKjrxjC8boSyAYavgmjD`  |           63 |       10 |      5 |
| [Kamino Vault](https://carbon.sevenlabs.org/decoders/kamino-vault.md)     | `carbon-kamino-vault-decoder`   | `kvauTFR8qm1dhniz6pYuBZkuene3Hfrs1VQhVRgCNrr`  |           12 |        2 |      0 |
| [marginfi v2](https://carbon.sevenlabs.org/decoders/marginfi-v2.md)       | `carbon-marginfi-v2-decoder`    | `MFv2hWf31Z9kbCa1snEPYctwafyhdvnV7FZnsebVacA`  |           91 |       15 |     30 |
| [Meteora Vault](https://carbon.sevenlabs.org/decoders/meteora-vault.md)   | `carbon-meteora-vault-decoder`  | `24Uqj9JCLxUeoC3hGfh5W3s9FM9uCHDS2SG3LYwBpyTi` |           14 |        2 |      8 |
| [Sharky](https://carbon.sevenlabs.org/decoders/sharky.md)                 | `carbon-sharky-decoder`         | `SHARKobtfF1bHhxD2eqftjHBdVSCbKo9JtgK71FhELP`  |           35 |        9 |      0 |

### Perpetuals & derivatives

See [perpetuals and derivatives decoders](https://carbon.sevenlabs.org/decoders/perpetuals.md) for these programs together.

| Program                                                                           | Crate                               | Program ID                                    | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------- | ----------------------------------- | --------------------------------------------- | -----------: | -------: | -----: |
| [Drift v2](https://carbon.sevenlabs.org/decoders/drift-v2.md)                     | `carbon-drift-v2-decoder`           | `dRiftyHA39MWEi3m9aunc5MzRF1JYuBsbn6VPcn33UH` |          249 |       27 |     26 |
| [Jupiter Perpetuals](https://carbon.sevenlabs.org/decoders/jupiter-perpetuals.md) | `carbon-jupiter-perpetuals-decoder` | `PERPHjGBqRHArX4DySjwM6UJHiR3sWAatqfdBS2qQJu` |           59 |        8 |     25 |
| [Zeta Markets](https://carbon.sevenlabs.org/decoders/zeta.md)                     | `carbon-zeta-decoder`               | `ZETAxsqBRek56DhiGXrn75yj2NHU3aYUnxvHXpkf3aD` |          107 |       23 |      8 |

### Launchpads

See [launchpad decoders](https://carbon.sevenlabs.org/decoders/launchpads.md) for these programs together.

| Program                                                                         | Crate                              | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------------- | ---------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Boop](https://carbon.sevenlabs.org/decoders/boop.md)                           | `carbon-boop-decoder`              | `boop8hVGQGqehUK2iVEMEnMrL5RbjywRzHKBmBE7ry4`  |           29 |        4 |     25 |
| [Meteora DBC](https://carbon.sevenlabs.org/decoders/meteora-dbc.md)             | `carbon-meteora-dbc-decoder`       | `dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN`  |           28 |        9 |     23 |
| [Moonshot](https://carbon.sevenlabs.org/decoders/moonshot.md)                   | `carbon-moonshot-decoder`          | `MoonCVVNZFSYkqNXP6bxHLPL6QQJiMagDL3qcqUQTrG`  |            6 |        2 |      2 |
| [Pump.fun](https://carbon.sevenlabs.org/decoders/pumpfun.md)                    | `carbon-pumpfun-decoder`           | `6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P`  |           38 |        6 |     23 |
| [Raydium Launchpad](https://carbon.sevenlabs.org/decoders/raydium-launchpad.md) | `carbon-raydium-launchpad-decoder` | `LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj`  |           25 |        5 |      4 |
| [Vertigo](https://carbon.sevenlabs.org/decoders/vertigo.md)                     | `carbon-vertigo-decoder`           | `vrTGoBuy5rYSxAfV3jaRJWHH6nN9WK4NRExGxsk1bCJ`  |            6 |        1 |      3 |
| [Virtuals](https://carbon.sevenlabs.org/decoders/virtuals.md)                   | `carbon-virtuals-decoder`          | `5U3EU2ubXtK84QcRjWVmYt9RaDyA8gKxdUrPFXmZyaki` |            8 |        1 |      4 |
| [Wavebreak](https://carbon.sevenlabs.org/decoders/wavebreak.md)                 | `carbon-wavebreak-decoder`         | `waveQX2yP3H1pVU8djGvEHmYg8uamQ84AuyGtpsrXTF`  |           64 |        5 |      0 |

### Liquid staking & restaking

See [liquid staking and restaking decoders](https://carbon.sevenlabs.org/decoders/liquid-staking.md) for these programs together.

| Program                                                                                 | Crate                                      | Program ID                                    | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------- | -----------: | -------: | -----: |
| [Marinade Finance](https://carbon.sevenlabs.org/decoders/marinade-finance.md)           | `carbon-marinade-finance-decoder`          | `MarBmsSgKXdrN1egZf5sqe1TMai9K1rChYNDJgjq7aD` |           28 |        2 |     25 |
| [Solayer Restaking](https://carbon.sevenlabs.org/decoders/solayer-restaking-program.md) | `carbon-solayer-restaking-program-decoder` | `sSo1iU21jBrU9VaJ8PJib1MtorefUV4fzC9GURa2KNn` |            4 |        1 |      0 |

### NFTs & metadata

See [nFT and metadata decoders](https://carbon.sevenlabs.org/decoders/nfts.md) for these programs together.

| Program                                                                                | Crate                               | Program ID                                     | Instructions | Accounts | Events |
| -------------------------------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Bubblegum](https://carbon.sevenlabs.org/decoders/bubblegum.md)                        | `carbon-bubblegum-decoder`          | `BGUMAp9Gq7iTEuizy4pqaxsTyUCBK68MDfK752saRPUY` |           17 |        2 |      0 |
| [Metaplex Core](https://carbon.sevenlabs.org/decoders/mpl-core.md)                     | `carbon-mpl-core-decoder`           | `CoREENxT6tW1HoK8ypY1SxRMZTcVPm7R94rH4PZNhX7d` |           42 |        6 |      0 |
| [Metaplex Token Metadata](https://carbon.sevenlabs.org/decoders/mpl-token-metadata.md) | `carbon-mpl-token-metadata-decoder` | `metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s`  |           58 |       14 |      0 |

### Bridges & messaging

See [bridge and messaging decoders](https://carbon.sevenlabs.org/decoders/bridges.md) for these programs together.

| Program                                                                                                      | Crate                                          | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Circle CCTP Message Transmitter v2](https://carbon.sevenlabs.org/decoders/circle-message-transmitter-v2.md) | `carbon-circle-message-transmitter-v2-decoder` | `CCTPV2Sm4AdWt5296sk4P66VBZ7bEhcARwFaaS9YPbeC` |           15 |        3 |     11 |
| [Circle CCTP Token Messenger v2](https://carbon.sevenlabs.org/decoders/circle-token-messenger-v2.md)         | `carbon-circle-token-messenger-v2-decoder`     | `CCTPV2vPZJS2u2BBsUoscuikbYjnpFmbFsvVuJdgUMQe` |           25 |        7 |     22 |

### Aggregators & routing

See [aggregator and routing decoders](https://carbon.sevenlabs.org/decoders/aggregators.md) for these programs together.

| Program                                                                             | Crate                                | Program ID                                     | Instructions | Accounts | Events |
| ----------------------------------------------------------------------------------- | ------------------------------------ | ---------------------------------------------- | -----------: | -------: | -----: |
| [DFlow Aggregator v4](https://carbon.sevenlabs.org/decoders/dflow-aggregator-v4.md) | `carbon-dflow-aggregator-v4-decoder` | `DF1ow4tspfHX9JwWJsAb9epbkA8hmpSEAtxXy1V27QBH` |           17 |        1 |      2 |
| [Jupiter Swap](https://carbon.sevenlabs.org/decoders/jupiter-swap.md)               | `carbon-jupiter-swap-decoder`        | `JUP6LkbZbjS1jKKwapdHNy74zcZ3tLUZoi5QNyVTaV4`  |           17 |        1 |      6 |
| [OKX DEX](https://carbon.sevenlabs.org/decoders/okx-dex.md)                         | `carbon-okx-dex-decoder`             | `6m2CDdhRgxpH4WjvdzxAYbGxwdGUz5MziiL5jek2kBma` |           14 |        0 |      2 |
| [OnChain Labs DEX v2](https://carbon.sevenlabs.org/decoders/onchain-labs-dex-v2.md) | `carbon-onchain-labs-dex-v2-decoder` | `proVF4pMXVaYqmy4NjniPh4pqKNfMmsihgd4wdkCX3u`  |           19 |        1 |      6 |

### Other

See [other program decoders](https://carbon.sevenlabs.org/decoders/other.md) for these programs together.

| Program                                                                                         | Crate                                      | Program ID                                     | Instructions | Accounts | Events |
| ----------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------- | -----------: | -------: | -----: |
| [Jupiter DCA](https://carbon.sevenlabs.org/decoders/jupiter-dca.md)                             | `carbon-jupiter-dca-decoder`               | `DCA265Vj8a9CEuX1eb1LWRnDT7uK6q1xMipnNyatn23M` |           12 |        1 |      6 |
| [Kamino Farms](https://carbon.sevenlabs.org/decoders/kamino-farms.md)                           | `carbon-kamino-farms-decoder`              | `FarmsPZpWu9i7Kky8tPN37rs2TpmMrAZrC7S7vJa91Hr` |           26 |        4 |      0 |
| [Pump Fees](https://carbon.sevenlabs.org/decoders/pump-fees.md)                                 | `carbon-pump-fees-decoder`                 | `pfeeUxB6jkeY1Hxd7CsFCAjcbHA9rWtchMGdZ6VojVZ`  |           17 |        7 |     14 |
| [Raydium Liquidity Locking](https://carbon.sevenlabs.org/decoders/raydium-liquidity-locking.md) | `carbon-raydium-liquidity-locking-decoder` | `LockrWmn6K5twhz3y9w1dQERbmgSaRkfnTeTKbpofwE`  |            4 |        2 |      1 |
| [Swig](https://carbon.sevenlabs.org/decoders/swig.md)                                           | `carbon-swig-decoder`                      | `swigypWHEksbC64pWKwah1WTeh9JXwx8H1rJHLdbQMB`  |           15 |        0 |      0 |

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Core and SPL program decoders (Carbon 2.0)

> Decode 8 Solana core and SPL programs in Rust, including SPL Token, Token-2022, System Program and Address Lookup Table, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 8 core and SPL programs on Solana: 164 instructions, 9 account types and 0 events in all. Use them to index SOL and token transfers, token accounts and mints, stake accounts and address lookup tables as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                       | Crate                                     | Program ID                                     | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Address Lookup Table](https://carbon.sevenlabs.org/decoders/address-lookup-table.md)         | `carbon-address-lookup-table-decoder`     | `AddressLookupTab1e1111111111111111111111111`  |            5 |        1 |      0 |
| [Associated Token Account](https://carbon.sevenlabs.org/decoders/associated-token-account.md) | `carbon-associated-token-account-decoder` | `ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL` |            3 |        0 |      0 |
| [SPL Memo](https://carbon.sevenlabs.org/decoders/memo-program.md)                             | `carbon-memo-program-decoder`             | `MemoSq4gqABAXKb96qnH8TysNcWxMyWCqXgDLGmfcHr`  |            1 |        0 |      0 |
| [SPL Name Service](https://carbon.sevenlabs.org/decoders/name-service.md)                     | `carbon-name-service-decoder`             | `namesLPneVptA9Z5rqUDD9tMTWEJwofgaYwp8cawRkX`  |            5 |        1 |      0 |
| [SPL Token](https://carbon.sevenlabs.org/decoders/token-program.md)                           | `carbon-token-program-decoder`            | `TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA`  |           28 |        3 |      0 |
| [Stake Program](https://carbon.sevenlabs.org/decoders/stake-program.md)                       | `carbon-stake-program-decoder`            | `Stake11111111111111111111111111111111111111`  |           18 |        0 |      0 |
| [System Program](https://carbon.sevenlabs.org/decoders/system-program.md)                     | `carbon-system-program-decoder`           | `11111111111111111111111111111111`             |           13 |        1 |      0 |
| [Token-2022](https://carbon.sevenlabs.org/decoders/token-2022.md)                             | `carbon-token-2022-decoder`               | `TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb`  |           91 |        3 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-address-lookup-table-decoder@2.0.0 carbon-associated-token-account-decoder@2.0.0 carbon-memo-program-decoder@2.0.0 carbon-name-service-decoder@2.0.0 carbon-token-program-decoder@2.0.0 carbon-stake-program-decoder@2.0.0 carbon-system-program-decoder@2.0.0 carbon-token-2022-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_token_program_decoder::TokenProgramDecoder;
use carbon_token_2022_decoder::Token2022Decoder;
use carbon_system_program_decoder::SystemProgramDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(TokenProgramDecoder, TokenProgramProcessor)
    .instruction(Token2022Decoder, Token2022Processor)
    .instruction(SystemProgramDecoder, SystemProgramProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## Guides

- [Index token transfers](https://carbon.sevenlabs.org/guides/index-token-transfers.md)

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/core-and-spl · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# DEX and AMM decoders (Carbon 2.0)

> Decode 22 Solana DEX and AMM programs in Rust, including Raydium AMM v4, Raydium CPMM, Raydium CLMM and PumpSwap, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 22 DEX and AMM programs on Solana: 543 instructions, 104 account types and 155 events in all. Use them to index swaps, liquidity deposits and withdrawals, pools, positions and order book markets as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                  | Crate                                  | Program ID                                     | Instructions | Accounts | Events |
| ---------------------------------------------------------------------------------------- | -------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [BonkSwap](https://carbon.sevenlabs.org/decoders/bonkswap.md)                            | `carbon-bonkswap-decoder`              | `BSwp6bEBihVLdqJRKGgzjcGLHkcTuzmSo1TQkHepzH8p` |           19 |        5 |      0 |
| [FluxBeam](https://carbon.sevenlabs.org/decoders/fluxbeam.md)                            | `carbon-fluxbeam-decoder`              | `FLUXubRmkEi2q6K3Y9kBPg9248ggaZVsoSFhtJHSrm1X` |            6 |        0 |      0 |
| [Gavel](https://carbon.sevenlabs.org/decoders/gavel.md)                                  | `carbon-gavel-decoder`                 | `srAMMzfVHVAtgSJc8iH6CfKzuWuUTzLHVCE81QU1rgi`  |           10 |        2 |      0 |
| [Heaven](https://carbon.sevenlabs.org/decoders/heaven.md)                                | `carbon-heaven-decoder`                | `HEAVENoP2qxoeuF8Dj2oT1GHEnu49U5mJYkdeC8BAX2o` |           54 |        6 |      7 |
| [Jupiter Limit Order](https://carbon.sevenlabs.org/decoders/jupiter-limit-order.md)      | `carbon-jupiter-limit-order-decoder`   | `jupoNjAxXgZ4rjzxzPMP4oxduvQsQtZzyknqvzYNrNu`  |            9 |        2 |      3 |
| [Jupiter Limit Order v2](https://carbon.sevenlabs.org/decoders/jupiter-limit-order-2.md) | `carbon-jupiter-limit-order-2-decoder` | `j1o2qRpjcyUwEvwtcfhEQefh773ZgjxcVRry7LDqg5X`  |            8 |        2 |      4 |
| [Kamino Limit Order](https://carbon.sevenlabs.org/decoders/kamino-limit-order.md)        | `carbon-kamino-limit-order-decoder`    | `LiMoM9rMhrdYrfzUCxQppvxCSG1FcrUK9G8uLq4A1GF`  |           15 |        3 |      2 |
| [Lifinity AMM v2](https://carbon.sevenlabs.org/decoders/lifinity-amm-v2.md)              | `carbon-lifinity-amm-v2-decoder`       | `2wT8Yq49kHgDzXuPxZSaeLaH1qbmGXtEyPy64bL7aD3c` |            3 |        1 |      0 |
| [Meteora DAMM v2](https://carbon.sevenlabs.org/decoders/meteora-damm-v2.md)              | `carbon-meteora-damm-v2-decoder`       | `cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG`  |           38 |        9 |     23 |
| [Meteora DLMM](https://carbon.sevenlabs.org/decoders/meteora-dlmm.md)                    | `carbon-meteora-dlmm-decoder`          | `LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo`  |           76 |       12 |     30 |
| [Meteora Pools](https://carbon.sevenlabs.org/decoders/meteora-pools.md)                  | `carbon-meteora-pools-decoder`         | `Eo7WjKq67rjJQSZxS6z3YkapzY3eMj6Xy8X5EQVn5UaB` |           26 |        3 |     18 |
| [OpenBook v2](https://carbon.sevenlabs.org/decoders/openbook-v2.md)                      | `carbon-openbook-v2-decoder`           | `opnb2LAfJYbRMAHHvqjCwQxanZn7ReEHp1k81EohpZb`  |           29 |        6 |      8 |
| [Orca Whirlpools](https://carbon.sevenlabs.org/decoders/orca-whirlpool.md)               | `carbon-orca-whirlpool-decoder`        | `whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc`  |           66 |       12 |      6 |
| [PancakeSwap](https://carbon.sevenlabs.org/decoders/pancake-swap.md)                     | `carbon-pancake-swap-decoder`          | `HpNfyc2Saw7RKkQd8nEL4khUcuPhQ7WwY1B2qjx8jxFq` |           27 |       10 |     11 |
| [Phoenix v1](https://carbon.sevenlabs.org/decoders/phoenix-v1.md)                        | `carbon-phoenix-v1-decoder`            | `PhoeNiXZ8ByJGLkxNfZRnkUfjvmuYqLR89jjFHGqdXY`  |           28 |        0 |      0 |
| [PumpSwap](https://carbon.sevenlabs.org/decoders/pump-swap.md)                           | `carbon-pump-swap-decoder`             | `pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA`  |           25 |        7 |     22 |
| [Raydium AMM v4](https://carbon.sevenlabs.org/decoders/raydium-amm-v4.md)                | `carbon-raydium-amm-v4-decoder`        | `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8` |           18 |        3 |      0 |
| [Raydium CLMM](https://carbon.sevenlabs.org/decoders/raydium-clmm.md)                    | `carbon-raydium-clmm-decoder`          | `CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK` |           34 |       12 |     15 |
| [Raydium CPMM](https://carbon.sevenlabs.org/decoders/raydium-cpmm.md)                    | `carbon-raydium-cpmm-decoder`          | `CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C` |           14 |        4 |      2 |
| [Raydium Stable Swap](https://carbon.sevenlabs.org/decoders/raydium-stable-swap.md)      | `carbon-raydium-stable-swap-decoder`   | `5quBtoiQqxF9Jv6KYKctB59NT3gtJD2Y65kdnB1Uev3h` |            6 |        0 |      0 |
| [Stabble Stable Swap](https://carbon.sevenlabs.org/decoders/stabble-stable-swap.md)      | `carbon-stabble-stable-swap-decoder`   | `swapNyd8XiQwJ6ianp9snpu4brUqFxadzvHebnAXjJZ`  |           19 |        3 |      2 |
| [Stabble Weighted Swap](https://carbon.sevenlabs.org/decoders/stabble-weighted-swap.md)  | `carbon-stabble-weighted-swap-decoder` | `swapFpHZwjELNnjvThjajtiVmkz3yPQEHjLtka2fwHW`  |           13 |        2 |      2 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-bonkswap-decoder@2.0.0 carbon-fluxbeam-decoder@2.0.0 carbon-gavel-decoder@2.0.0 carbon-heaven-decoder@2.0.0 carbon-jupiter-limit-order-decoder@2.0.0 carbon-jupiter-limit-order-2-decoder@2.0.0 carbon-kamino-limit-order-decoder@2.0.0 carbon-lifinity-amm-v2-decoder@2.0.0 carbon-meteora-damm-v2-decoder@2.0.0 carbon-meteora-dlmm-decoder@2.0.0 carbon-meteora-pools-decoder@2.0.0 carbon-openbook-v2-decoder@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-pancake-swap-decoder@2.0.0 carbon-phoenix-v1-decoder@2.0.0 carbon-pump-swap-decoder@2.0.0 carbon-raydium-amm-v4-decoder@2.0.0 carbon-raydium-clmm-decoder@2.0.0 carbon-raydium-cpmm-decoder@2.0.0 carbon-raydium-stable-swap-decoder@2.0.0 carbon-stabble-stable-swap-decoder@2.0.0 carbon-stabble-weighted-swap-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_raydium_amm_v4_decoder::RaydiumAmmV4Decoder;
use carbon_raydium_cpmm_decoder::RaydiumCpmmDecoder;
use carbon_raydium_clmm_decoder::RaydiumClmmDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(RaydiumAmmV4Decoder, RaydiumAmmV4Processor)
    .instruction(RaydiumCpmmDecoder, RaydiumCpmmProcessor)
    .instruction(RaydiumClmmDecoder, RaydiumClmmProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## Guides

- [Index swaps across DEXs](https://carbon.sevenlabs.org/guides/index-dex-swaps.md)

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/dex-and-amm · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Aggregator and routing decoders (Carbon 2.0)

> Decode 4 Solana aggregator and routing programs in Rust, including Jupiter Swap, DFlow Aggregator v4 and OKX DEX, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 4 aggregator and routing programs on Solana: 67 instructions, 3 account types and 16 events in all. Use them to index swaps routed across several DEXs, with their routes and amounts as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                             | Crate                                | Program ID                                     | Instructions | Accounts | Events |
| ----------------------------------------------------------------------------------- | ------------------------------------ | ---------------------------------------------- | -----------: | -------: | -----: |
| [DFlow Aggregator v4](https://carbon.sevenlabs.org/decoders/dflow-aggregator-v4.md) | `carbon-dflow-aggregator-v4-decoder` | `DF1ow4tspfHX9JwWJsAb9epbkA8hmpSEAtxXy1V27QBH` |           17 |        1 |      2 |
| [Jupiter Swap](https://carbon.sevenlabs.org/decoders/jupiter-swap.md)               | `carbon-jupiter-swap-decoder`        | `JUP6LkbZbjS1jKKwapdHNy74zcZ3tLUZoi5QNyVTaV4`  |           17 |        1 |      6 |
| [OKX DEX](https://carbon.sevenlabs.org/decoders/okx-dex.md)                         | `carbon-okx-dex-decoder`             | `6m2CDdhRgxpH4WjvdzxAYbGxwdGUz5MziiL5jek2kBma` |           14 |        0 |      2 |
| [OnChain Labs DEX v2](https://carbon.sevenlabs.org/decoders/onchain-labs-dex-v2.md) | `carbon-onchain-labs-dex-v2-decoder` | `proVF4pMXVaYqmy4NjniPh4pqKNfMmsihgd4wdkCX3u`  |           19 |        1 |      6 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-dflow-aggregator-v4-decoder@2.0.0 carbon-jupiter-swap-decoder@2.0.0 carbon-okx-dex-decoder@2.0.0 carbon-onchain-labs-dex-v2-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_jupiter_swap_decoder::JupiterSwapDecoder;
use carbon_dflow_aggregator_v4_decoder::DflowAggregatorV4Decoder;
use carbon_okx_dex_decoder::OkxDexDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(JupiterSwapDecoder, JupiterSwapProcessor)
    .instruction(DflowAggregatorV4Decoder, DflowAggregatorV4Processor)
    .instruction(OkxDexDecoder, OkxDexProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/aggregators · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Launchpad decoders (Carbon 2.0)

> Decode 8 Solana launchpad programs in Rust, including Pump.fun, Meteora DBC, Raydium Launchpad and Boop, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 8 launchpad programs on Solana: 204 instructions, 33 account types and 84 events in all. Use them to index token creations, bonding-curve buys and sells, and migrations to AMM pools as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                         | Crate                              | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------------- | ---------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Boop](https://carbon.sevenlabs.org/decoders/boop.md)                           | `carbon-boop-decoder`              | `boop8hVGQGqehUK2iVEMEnMrL5RbjywRzHKBmBE7ry4`  |           29 |        4 |     25 |
| [Meteora DBC](https://carbon.sevenlabs.org/decoders/meteora-dbc.md)             | `carbon-meteora-dbc-decoder`       | `dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN`  |           28 |        9 |     23 |
| [Moonshot](https://carbon.sevenlabs.org/decoders/moonshot.md)                   | `carbon-moonshot-decoder`          | `MoonCVVNZFSYkqNXP6bxHLPL6QQJiMagDL3qcqUQTrG`  |            6 |        2 |      2 |
| [Pump.fun](https://carbon.sevenlabs.org/decoders/pumpfun.md)                    | `carbon-pumpfun-decoder`           | `6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P`  |           38 |        6 |     23 |
| [Raydium Launchpad](https://carbon.sevenlabs.org/decoders/raydium-launchpad.md) | `carbon-raydium-launchpad-decoder` | `LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj`  |           25 |        5 |      4 |
| [Vertigo](https://carbon.sevenlabs.org/decoders/vertigo.md)                     | `carbon-vertigo-decoder`           | `vrTGoBuy5rYSxAfV3jaRJWHH6nN9WK4NRExGxsk1bCJ`  |            6 |        1 |      3 |
| [Virtuals](https://carbon.sevenlabs.org/decoders/virtuals.md)                   | `carbon-virtuals-decoder`          | `5U3EU2ubXtK84QcRjWVmYt9RaDyA8gKxdUrPFXmZyaki` |            8 |        1 |      4 |
| [Wavebreak](https://carbon.sevenlabs.org/decoders/wavebreak.md)                 | `carbon-wavebreak-decoder`         | `waveQX2yP3H1pVU8djGvEHmYg8uamQ84AuyGtpsrXTF`  |           64 |        5 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-boop-decoder@2.0.0 carbon-meteora-dbc-decoder@2.0.0 carbon-moonshot-decoder@2.0.0 carbon-pumpfun-decoder@2.0.0 carbon-raydium-launchpad-decoder@2.0.0 carbon-vertigo-decoder@2.0.0 carbon-virtuals-decoder@2.0.0 carbon-wavebreak-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_pumpfun_decoder::PumpfunDecoder;
use carbon_meteora_dbc_decoder::MeteoraDbcDecoder;
use carbon_raydium_launchpad_decoder::RaydiumLaunchpadDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(PumpfunDecoder, PumpfunProcessor)
    .instruction(MeteoraDbcDecoder, MeteoraDbcProcessor)
    .instruction(RaydiumLaunchpadDecoder, RaydiumLaunchpadProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## Guides

- [Track Pump.fun launches and trades](https://carbon.sevenlabs.org/guides/track-pumpfun-launches.md)

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/launchpads · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Lending decoders (Carbon 2.0)

> Decode 6 Solana lending programs in Rust, including Kamino Lending, marginfi v2, Jupiter Lend and Kamino Vault, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 6 lending programs on Solana: 239 instructions, 45 account types and 62 events in all. Use them to index deposits, borrows, repayments, liquidations, reserves, obligations and vaults as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                   | Crate                           | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Jupiter Lend](https://carbon.sevenlabs.org/decoders/jupiter-lend.md)     | `carbon-jupiter-lend-decoder`   | `jupeiUmn818Jg1ekPURTpr4mFo29p46vygyykFJ3wZC`  |           24 |        7 |     19 |
| [Kamino Lending](https://carbon.sevenlabs.org/decoders/kamino-lending.md) | `carbon-kamino-lending-decoder` | `KLend2g3cP87fffoy8q1mQqGKjrxjC8boSyAYavgmjD`  |           63 |       10 |      5 |
| [Kamino Vault](https://carbon.sevenlabs.org/decoders/kamino-vault.md)     | `carbon-kamino-vault-decoder`   | `kvauTFR8qm1dhniz6pYuBZkuene3Hfrs1VQhVRgCNrr`  |           12 |        2 |      0 |
| [marginfi v2](https://carbon.sevenlabs.org/decoders/marginfi-v2.md)       | `carbon-marginfi-v2-decoder`    | `MFv2hWf31Z9kbCa1snEPYctwafyhdvnV7FZnsebVacA`  |           91 |       15 |     30 |
| [Meteora Vault](https://carbon.sevenlabs.org/decoders/meteora-vault.md)   | `carbon-meteora-vault-decoder`  | `24Uqj9JCLxUeoC3hGfh5W3s9FM9uCHDS2SG3LYwBpyTi` |           14 |        2 |      8 |
| [Sharky](https://carbon.sevenlabs.org/decoders/sharky.md)                 | `carbon-sharky-decoder`         | `SHARKobtfF1bHhxD2eqftjHBdVSCbKo9JtgK71FhELP`  |           35 |        9 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-jupiter-lend-decoder@2.0.0 carbon-kamino-lending-decoder@2.0.0 carbon-kamino-vault-decoder@2.0.0 carbon-marginfi-v2-decoder@2.0.0 carbon-meteora-vault-decoder@2.0.0 carbon-sharky-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_kamino_lending_decoder::KaminoLendingDecoder;
use carbon_marginfi_v2_decoder::MarginfiV2Decoder;
use carbon_jupiter_lend_decoder::JupiterLendDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(KaminoLendingDecoder, KaminoLendingProcessor)
    .instruction(MarginfiV2Decoder, MarginfiV2Processor)
    .instruction(JupiterLendDecoder, JupiterLendProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/lending · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Perpetuals and derivatives decoders (Carbon 2.0)

> Decode 3 Solana perpetuals and derivatives programs in Rust, including Drift v2, Jupiter Perpetuals and Zeta Markets, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 3 perpetuals and derivatives programs on Solana: 415 instructions, 58 account types and 59 events in all. Use them to index orders, fills, positions, funding and liquidations as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                           | Crate                               | Program ID                                    | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------- | ----------------------------------- | --------------------------------------------- | -----------: | -------: | -----: |
| [Drift v2](https://carbon.sevenlabs.org/decoders/drift-v2.md)                     | `carbon-drift-v2-decoder`           | `dRiftyHA39MWEi3m9aunc5MzRF1JYuBsbn6VPcn33UH` |          249 |       27 |     26 |
| [Jupiter Perpetuals](https://carbon.sevenlabs.org/decoders/jupiter-perpetuals.md) | `carbon-jupiter-perpetuals-decoder` | `PERPHjGBqRHArX4DySjwM6UJHiR3sWAatqfdBS2qQJu` |           59 |        8 |     25 |
| [Zeta Markets](https://carbon.sevenlabs.org/decoders/zeta.md)                     | `carbon-zeta-decoder`               | `ZETAxsqBRek56DhiGXrn75yj2NHU3aYUnxvHXpkf3aD` |          107 |       23 |      8 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-drift-v2-decoder@2.0.0 carbon-jupiter-perpetuals-decoder@2.0.0 carbon-zeta-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_drift_v2_decoder::DriftV2Decoder;
use carbon_jupiter_perpetuals_decoder::JupiterPerpetualsDecoder;
use carbon_zeta_decoder::ZetaDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(DriftV2Decoder, DriftV2Processor)
    .instruction(JupiterPerpetualsDecoder, JupiterPerpetualsProcessor)
    .instruction(ZetaDecoder, ZetaProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/perpetuals · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Liquid staking and restaking decoders (Carbon 2.0)

> Decode 2 Solana liquid staking and restaking programs in Rust, including Marinade Finance and Solayer Restaking, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 2 liquid staking and restaking programs on Solana: 32 instructions, 3 account types and 25 events in all. Use them to index stake deposits, unstake orders, liquid staking token mints and restaking as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                 | Crate                                      | Program ID                                    | Instructions | Accounts | Events |
| --------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------- | -----------: | -------: | -----: |
| [Marinade Finance](https://carbon.sevenlabs.org/decoders/marinade-finance.md)           | `carbon-marinade-finance-decoder`          | `MarBmsSgKXdrN1egZf5sqe1TMai9K1rChYNDJgjq7aD` |           28 |        2 |     25 |
| [Solayer Restaking](https://carbon.sevenlabs.org/decoders/solayer-restaking-program.md) | `carbon-solayer-restaking-program-decoder` | `sSo1iU21jBrU9VaJ8PJib1MtorefUV4fzC9GURa2KNn` |            4 |        1 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-marinade-finance-decoder@2.0.0 carbon-solayer-restaking-program-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_marinade_finance_decoder::MarinadeFinanceDecoder;
use carbon_solayer_restaking_program_decoder::SolayerRestakingProgramDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(MarinadeFinanceDecoder, MarinadeFinanceProcessor)
    .instruction(SolayerRestakingProgramDecoder, SolayerRestakingProgramProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/liquid-staking · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# NFT and metadata decoders (Carbon 2.0)

> Decode 3 Solana NFT and metadata programs in Rust, including Metaplex Core, Bubblegum and Metaplex Token Metadata, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 3 NFT and metadata programs on Solana: 117 instructions, 22 account types and 0 events in all. Use them to index NFT and compressed NFT mints, transfers and burns, collections and token metadata as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                | Crate                               | Program ID                                     | Instructions | Accounts | Events |
| -------------------------------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Bubblegum](https://carbon.sevenlabs.org/decoders/bubblegum.md)                        | `carbon-bubblegum-decoder`          | `BGUMAp9Gq7iTEuizy4pqaxsTyUCBK68MDfK752saRPUY` |           17 |        2 |      0 |
| [Metaplex Core](https://carbon.sevenlabs.org/decoders/mpl-core.md)                     | `carbon-mpl-core-decoder`           | `CoREENxT6tW1HoK8ypY1SxRMZTcVPm7R94rH4PZNhX7d` |           42 |        6 |      0 |
| [Metaplex Token Metadata](https://carbon.sevenlabs.org/decoders/mpl-token-metadata.md) | `carbon-mpl-token-metadata-decoder` | `metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s`  |           58 |       14 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-bubblegum-decoder@2.0.0 carbon-mpl-core-decoder@2.0.0 carbon-mpl-token-metadata-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_mpl_core_decoder::MplCoreDecoder;
use carbon_bubblegum_decoder::BubblegumDecoder;
use carbon_mpl_token_metadata_decoder::MplTokenMetadataDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(MplCoreDecoder, MplCoreProcessor)
    .instruction(BubblegumDecoder, BubblegumProcessor)
    .instruction(MplTokenMetadataDecoder, MplTokenMetadataProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/nfts · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Bridge and messaging decoders (Carbon 2.0)

> Decode 2 Solana bridge and messaging programs in Rust, including Circle CCTP Token Messenger v2, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 2 bridge and messaging programs on Solana: 40 instructions, 10 account types and 33 events in all. Use them to index cross-chain burns, mints and messages, including Circle CCTP USDC transfers as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                                      | Crate                                          | Program ID                                     | Instructions | Accounts | Events |
| ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | ---------------------------------------------- | -----------: | -------: | -----: |
| [Circle CCTP Message Transmitter v2](https://carbon.sevenlabs.org/decoders/circle-message-transmitter-v2.md) | `carbon-circle-message-transmitter-v2-decoder` | `CCTPV2Sm4AdWt5296sk4P66VBZ7bEhcARwFaaS9YPbeC` |           15 |        3 |     11 |
| [Circle CCTP Token Messenger v2](https://carbon.sevenlabs.org/decoders/circle-token-messenger-v2.md)         | `carbon-circle-token-messenger-v2-decoder`     | `CCTPV2vPZJS2u2BBsUoscuikbYjnpFmbFsvVuJdgUMQe` |           25 |        7 |     22 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-circle-message-transmitter-v2-decoder@2.0.0 carbon-circle-token-messenger-v2-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_circle_token_messenger_v2_decoder::CircleTokenMessengerV2Decoder;
use carbon_circle_message_transmitter_v2_decoder::CircleMessageTransmitterV2Decoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(CircleTokenMessengerV2Decoder, CircleTokenMessengerV2Processor)
    .instruction(CircleMessageTransmitterV2Decoder, CircleMessageTransmitterV2Processor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/bridges · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Other program decoders (Carbon 2.0)

> Decode 5 more Solana programs in Rust, including Jupiter DCA, Kamino Farms, Pump Fees and Raydium Liquidity Locking, with Carbon decoder crates.

Carbon 2.0.0 has decoder crates for 5 more programs on Solana: 74 instructions, 14 account types and 21 events in all. Use them to index DCA orders, farms, fee configuration, liquidity locks and smart wallets as typed Rust values instead of parsing raw bytes. Each program's page lists every instruction with its discriminator, data fields and accounts, every account and event, and a complete program that runs against the public RPC.

## Programs

Program IDs are the mainnet addresses compiled into each crate as `PROGRAM_ID`. Instruction counts exclude the `CpiEvent` wrapper.

| Program                                                                                         | Crate                                      | Program ID                                     | Instructions | Accounts | Events |
| ----------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------- | -----------: | -------: | -----: |
| [Jupiter DCA](https://carbon.sevenlabs.org/decoders/jupiter-dca.md)                             | `carbon-jupiter-dca-decoder`               | `DCA265Vj8a9CEuX1eb1LWRnDT7uK6q1xMipnNyatn23M` |           12 |        1 |      6 |
| [Kamino Farms](https://carbon.sevenlabs.org/decoders/kamino-farms.md)                           | `carbon-kamino-farms-decoder`              | `FarmsPZpWu9i7Kky8tPN37rs2TpmMrAZrC7S7vJa91Hr` |           26 |        4 |      0 |
| [Pump Fees](https://carbon.sevenlabs.org/decoders/pump-fees.md)                                 | `carbon-pump-fees-decoder`                 | `pfeeUxB6jkeY1Hxd7CsFCAjcbHA9rWtchMGdZ6VojVZ`  |           17 |        7 |     14 |
| [Raydium Liquidity Locking](https://carbon.sevenlabs.org/decoders/raydium-liquidity-locking.md) | `carbon-raydium-liquidity-locking-decoder` | `LockrWmn6K5twhz3y9w1dQERbmgSaRkfnTeTKbpofwE`  |            4 |        2 |      1 |
| [Swig](https://carbon.sevenlabs.org/decoders/swig.md)                                           | `carbon-swig-decoder`                      | `swigypWHEksbC64pWKwah1WTeh9JXwx8H1rJHLdbQMB`  |           15 |        0 |      0 |

## Decode several in one pipeline

Add `carbon-core` and the decoder crates you need, all at the same version. For every program in this category:

```sh
cargo add carbon-core@2.0.0 carbon-jupiter-dca-decoder@2.0.0 carbon-kamino-farms-decoder@2.0.0 carbon-pump-fees-decoder@2.0.0 carbon-raydium-liquidity-locking-decoder@2.0.0 carbon-swig-decoder@2.0.0
```

Register each decoder with a processor for its instruction enum. This fragment goes in your pipeline setup; the processors are yours, one per program, each implementing `Processor` for that program's instruction input (see [processors](https://carbon.sevenlabs.org/concepts/processors.md)):

```rust
use carbon_jupiter_dca_decoder::JupiterDcaDecoder;
use carbon_kamino_farms_decoder::KaminoFarmsDecoder;
use carbon_pump_fees_decoder::PumpFeesDecoder;

Pipeline::builder()
    .datasource(datasource) // a datasource that emits these programs' transactions
    .instruction(JupiterDcaDecoder, JupiterDcaProcessor)
    .instruction(KaminoFarmsDecoder, KaminoFarmsProcessor)
    .instruction(PumpFeesDecoder, PumpFeesProcessor)
    .build()?
    .run()
    .await
```

Each decoder only returns instructions whose program ID matches its own `PROGRAM_ID`, so one pipeline can hold all of them. For datasource filters that follow several programs, and one processor that sees every program's instructions in a transaction, see [index multiple programs](https://carbon.sevenlabs.org/guides/multiple-programs.md).

## A program that is not listed

Generate a decoder from the program's Anchor or Codama IDL with the Carbon CLI: see [generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md). Every crate above was generated the same way. The [full catalog](https://carbon.sevenlabs.org/decoders.md) lists decoders in every category.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/decoders/other · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Examples (Carbon 2.0)

> Every runnable example in the Carbon 2.0.0 repository, what it shows, what it needs, the command to run it and the output to expect.

The Carbon repository ships nine example pipelines in [`examples/`](https://github.com/sevenlabs-hq/carbon/tree/v2.0.0/examples). Each one is a small, complete indexer for one pattern. Use this page to pick one, check what it needs, and run it from a checkout of the `v2.0.0` tag.

| Example                                               | Shows                                                 | Datasource                                                 | Decoder                               | Needs                                                  |
| ----------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------ |
| [`versioned-decoders`](#versioned-decoders)           | Two decoder versions routed by slot                   | Mock, in the example                                       | Two generated `simple-dex` decoders   | Nothing                                                |
| [`gpa-rpc`](#gpa-rpc)                                 | One-shot load of a program's accounts                 | `GpaDatasource`                                            | `carbon-marginfi-v2-decoder`          | Public RPC works                                       |
| [`transaction-crawler-rpc`](#transaction-crawler-rpc) | Backfill a program's transaction history              | `RpcTransactionCrawler`                                    | `carbon-meteora-dlmm-decoder`         | Public RPC works, slowly                               |
| [`custom-datasource`](#custom-datasource)             | Writing your own `Datasource`                         | `HttpPollDatasource`, in the example                       | `carbon-jupiter-swap-decoder`         | Public RPC works, slowly                               |
| [`block-subscribe-rpc`](#block-subscribe-rpc)         | Real-time blocks over WebSocket                       | `RpcBlockSubscribe`                                        | `carbon-raydium-clmm-decoder`         | An RPC provider with `blockSubscribe`                  |
| [`yellowstone-grpc`](#yellowstone-grpc)               | Real-time transactions over Geyser gRPC               | `YellowstoneGrpcGeyserClient` or `LaserStreamGeyserClient` | `carbon-jupiter-swap-decoder`         | A Yellowstone or LaserStream endpoint                  |
| [`postgres-graphql`](#postgres-graphql)               | Persist accounts to Postgres, serve them with GraphQL | `YellowstoneGrpcGeyserClient`                              | `carbon-token-program-decoder`        | Postgres and a Yellowstone endpoint                    |
| [`snapshot-validator`](#snapshot-validator)           | Load accounts from a validator snapshot               | `SnapshotDatasource`                                       | A pass-through decoder in the example | `libclang`, a snapshot, lots of disk                   |
| [`jetstreamer`](#jetstreamer)                         | Archive backfill over Old Faithful                    | `JetstreamerDatasource` (Carbon 1)                         | `carbon-token-2022-decoder`           | A Carbon 1 project; doesn't build in the 2.0 workspace |

## Run an example

Every example except `jetstreamer` is a member of the Carbon workspace, so you build it from a checkout of the repository. You need Rust 1.96.1 (the checkout's `rust-toolchain.toml` pins it, and `rustup` installs it on first use) and a few GB of disk for the first build.

```sh
git clone --branch v2.0.0 https://github.com/sevenlabs-hq/carbon.git
cd carbon/examples/gpa-rpc
RPC_URL=https://api.mainnet-beta.solana.com RUST_LOG=info cargo run --release -p gpa-rpc-carbon-example
```

Git prints a "detached HEAD" note after the clone; that's expected for a tag checkout. The first build compiles the Solana dependencies and takes a few minutes; later runs start in seconds.

Two things apply to every example:

- **Logging.** Examples print through `log` and `env_logger`, so nothing appears unless `RUST_LOG` is set. `RUST_LOG=info` shows decoded items and the [`LogMetrics`](https://carbon.sevenlabs.org/concepts/metrics.md) summary every five seconds; `RUST_LOG=debug` also shows the variants the processors log at debug level.
- **`.env` files.** Each example calls `dotenv::dotenv()`, which reads a `.env` file from the current directory or one of its parents. Put the file in the example's directory and run `cargo run` from there. Running from the repository root doesn't pick up `examples/<name>/.env`. Setting the variables inline, as in the commands on this page, works from anywhere.

## `versioned-decoders`

Indexes a program that was upgraded with a breaking instruction layout change. A fictional DEX adds a `fee_tier: u8` field to `Swap` at slot 500, transaction index 10. The example registers two generated decoders, `carbon-simple-dex-decoder-v1` and `-v2` (in `decoder-v1/` and `decoder-v2/`, generated from the IDLs in `idls/`), each with `instruction_with_filters` and a [`SlotRangeFilter`](https://carbon.sevenlabs.org/reference/filters.md) so every instruction reaches only the version that was live at its slot. One processor implements `Processor` for both instruction types and normalizes them to a shared struct.

The datasource is a mock inside `src/main.rs` that sends four synthetic transactions and returns, so the pipeline exits on its own. The pattern is explained in [Versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md).

**Needs:** nothing. No environment variables, no network.

```sh
cd carbon/examples/versioned-decoders
RUST_LOG=info cargo run --release -p versioned-decoders-carbon-example
```

Output:

```text
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] === Versioned Decoders Example ===
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] Program upgrade at slot=500, tx_index=10
[2026-11-08T23:48:10Z INFO  carbon_core::pipeline] starting pipeline. num_datasources: 1, num_exporters: 0, num_account_pipes: 0, num_account_deletion_pipes: 0, num_instruction_pipes: 2, num_transaction_pipes: 0
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] [V1] Swap | slot=499 tx_idx=Some(0) | amount_in=1000000 min_amount_out=990000
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] [V1] Swap | slot=500 tx_idx=Some(9) | amount_in=2000000 min_amount_out=1980000
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] [V2] Swap | slot=500 tx_idx=Some(10) | amount_in=3000000 min_amount_out=2970000 fee_tier=5
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] [V2] Swap | slot=501 tx_idx=Some(0) | amount_in=4000000 min_amount_out=3960000 fee_tier=10
[2026-11-08T23:48:10Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-08T23:48:10Z INFO  carbon_core::pipeline] pipeline shutdown complete.
[2026-11-08T23:48:10Z INFO  versioned_decoders_carbon_example] Done.
```

The filters do the routing. The decoders alone can't tell the versions apart reliably: generated decoders ignore trailing bytes, so the V1 decoder also accepts V2 data and silently drops `fee_tier`.

## `gpa-rpc`

Loads every account owned by the Marginfi v2 program once, with a single `getProgramAccounts` call, decodes each into `MarginfiV2Account`, and exits when the datasource has sent everything. It logs banks and groups at `info` and user accounts at `debug`. `src/variants.rs` also has a Helius gPA v2 variant (paginated, with `changed_since_slot` for incremental syncs); switch by editing the `let datasource = …` line in `main.rs`. See [Load account state](https://carbon.sevenlabs.org/guides/load-account-state.md).

**Needs:**

| Variable         | Required                         | Used by                                                           |
| ---------------- | -------------------------------- | ----------------------------------------------------------------- |
| `RPC_URL`        | Yes, for the default RPC variant | `GpaDatasource`. The public endpoint works for Marginfi's size.   |
| `HELIUS_RPC_URL` | Only for the Helius variant      | `HeliusGpaV2Datasource`. A Helius RPC URL with the API key in it. |

```sh
cd carbon/examples/gpa-rpc
RPC_URL=https://api.mainnet-beta.solana.com RUST_LOG=info cargo run --release -p gpa-rpc-carbon-example
```

Against the public endpoint the run took about 12 seconds and processed 171,340 accounts. The end of the output:

Output:

```text
[2026-11-08T23:48:31Z INFO  gpa_rpc_carbon_example] bank pubkey=J5mxC3hLXqmsJB4Nauf9YrtbNMSTHGz7FWfAmy1v6Fzr mint=So11111111111111111111111111111111111111112 group=4qp6Fx6tnZkY5Wropq9wUYgtFxXKwE6viZxFHg3rdAG8
[2026-11-08T23:48:31Z INFO  gpa_rpc_carbon_example] group pubkey=J9VZnaMGTELGCPsqMxk8aoyEGYcVzhorj48HvtDdEtc8 admin=J525txDGqSuPisS3NQwySwDEDExVhjJNUfx94PGQUNJR
[2026-11-08T23:48:31Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-08T23:48:31Z INFO  carbon_log_metrics] 00:00:11 (+1.82520643s) | 171340 processed (100%), 171340 successful, 0 failed (0%), 0 in queue
[2026-11-08T23:48:31Z INFO  carbon_log_metrics]   carbon_account_updates_processed_total: 171340
[2026-11-08T23:48:31Z INFO  carbon_log_metrics]   rpc_gpa_accounts_processed_total: 171340
[2026-11-08T23:48:31Z INFO  carbon_log_metrics]   rpc_gpa_fetch_duration_milliseconds: count=1
[2026-11-08T23:48:31Z INFO  carbon_core::pipeline] pipeline shutdown complete.
[2026-11-08T23:48:31Z INFO  gpa_rpc_carbon_example] snapshot complete
```

## `transaction-crawler-rpc`

Backfills Meteora DLMM activity by walking the program's signatures backwards from the newest (`getSignaturesForAddress`), fetching each transaction (`getTransaction`), and decoding it. It logs `Swap`, `AddLiquidity` and `RemoveLiquidity` at `info` and every other instruction at `debug`. After it reaches the oldest signature it can see, it keeps polling for new ones every five seconds. `src/variants.rs` also has a Helius GTFA variant that pages through a fixed slot window and then exits. See [Backfill transactions](https://carbon.sevenlabs.org/guides/backfill-transactions.md).

**Needs:**

| Variable          | Required                         | Used by                                                                                                                                   |
| ----------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `RPC_URL`         | Yes, for the default RPC variant | `RpcTransactionCrawler`. The public endpoint works but is heavily rate limited.                                                           |
| `UNTIL_SIGNATURE` | No                               | Stops the backward walk at this signature. The crawler still keeps polling for new transactions afterwards, so the pipeline doesn't exit. |
| `HELIUS_RPC_URL`  | Only for the GTFA variant        | `HeliusGtfaDatasource`.                                                                                                                   |

```sh
cd carbon/examples/transaction-crawler-rpc
RPC_URL=https://api.mainnet-beta.solana.com RUST_LOG=info cargo run --release -p transaction-crawler-rpc-carbon-example
```

On the public endpoint, expect a handful of swaps per minute, with pauses. This excerpt is from a 60-second run that processed 17 transactions:

Output:

```text
[2026-11-08T23:48:38Z ERROR carbon_rpc_transaction_crawler_datasource] Failed to fetch signatures after 0 retries: Error { request: Some(GetSignaturesForAddress), kind: Reqwest(reqwest::Error { kind: Status(413, Some(ReasonPhrase(b"Request Entity Too Large"))), url: "https://api.mainnet-beta.solana.com/" }) }
[2026-11-08T23:48:42Z INFO  transaction_crawler_rpc_carbon_example] swap slot=467105207 sig=5FVRVLgohHkAMTsqb5M5vCKJ8xpWGuQddfMBtdaifGFeDtnHQ3TFDSBdZAsUFezqimgh2i2RB94c3qtchJScyvWz data=Swap { amount_in: 367843, min_amount_out: 0 }
[2026-11-08T23:48:42Z INFO  transaction_crawler_rpc_carbon_example] swap slot=467105207 sig=2cFZ9cJkYpQGtnyLi9TMDTFErAkxktDRdwnjiHwz2BAEYLTFLYGzGx86mmPpMbSuarJxZeeFR4BXKVUWpPiMZKfC data=Swap { amount_in: 91821913, min_amount_out: 7958851 }
[2026-11-08T23:48:46Z INFO  carbon_log_metrics] 00:00:10 (+5.00022008s) | 8 processed (100%), 8 successful, 0 failed (0%), 0 in queue
…
[2026-11-08T23:49:36Z INFO  carbon_log_metrics]   transaction_crawler_signatures_fetched_total: 1000
[2026-11-08T23:49:36Z INFO  carbon_log_metrics]   transaction_crawler_transactions_fetched_total: 26
```

The example sets `RetryConfig::no_retry()`, so a rejected `getSignaturesForAddress` call is logged once and retried on the next poll, as in the first line above. The crawler fetches a thousand signatures quickly and then waits on `getTransaction`, which the public endpoint limits hard: it answers HTTP 429 with `retry-after: 10`, and the Solana RPC client sleeps without logging anything at `info`. See [Crawler prints nothing for seconds at a time](https://carbon.sevenlabs.org/troubleshooting.md#crawler-prints-nothing-for-seconds-at-a-time).

## `custom-datasource`

A complete hand-written [`Datasource`](https://carbon.sevenlabs.org/guides/custom-datasource.md), `HttpPollDatasource`, that polls `getSignaturesForAddress` for the Jupiter v6 program every five seconds, fetches each transaction with `getTransaction` in Base64, converts the metadata with `transaction_metadata_from_original_meta`, and sends `Update::Transaction`s. It shows the parts a real source needs: cancellation checks at every sleep, a watermark (`until`) so it doesn't re-send old transactions, backoff on failed calls, skipping failed transactions, and two custom [`Counter`](https://carbon.sevenlabs.org/reference/metrics.md) metrics registered in the global registry. The processor logs the discriminant of each decoded `JupiterSwapInstruction`.

**Needs:**

| Variable  | Required | Used by                                                  |
| --------- | -------- | -------------------------------------------------------- |
| `RPC_URL` | Yes      | `HttpPollDatasource`. The public endpoint works, slowly. |

```sh
cd carbon/examples/custom-datasource
RPC_URL=https://api.mainnet-beta.solana.com RUST_LOG=info cargo run --release -p custom-datasource-carbon-example
```

Output:

```text
[2026-11-08T23:50:51Z INFO  custom_datasource_carbon_example] ix slot=467105803 sig=2r4uo4VNLHHy9KZQi4KjCFqQM4ugBJgc5tQgMS1RGjKrMzsfYoAFG9AkiGdcRL24YpY9zUziRMXMhmMZuDmhXXcx variant=Discriminant(11)
[2026-11-08T23:50:51Z INFO  custom_datasource_carbon_example] ix slot=467105803 sig=2r4uo4VNLHHy9KZQi4KjCFqQM4ugBJgc5tQgMS1RGjKrMzsfYoAFG9AkiGdcRL24YpY9zUziRMXMhmMZuDmhXXcx variant=Discriminant(9)
[2026-11-08T23:50:51Z INFO  custom_datasource_carbon_example] ix slot=467105803 sig=2r4uo4VNLHHy9KZQi4KjCFqQM4ugBJgc5tQgMS1RGjKrMzsfYoAFG9AkiGdcRL24YpY9zUziRMXMhmMZuDmhXXcx variant=Discriminant(17)
[2026-11-08T23:50:55Z INFO  carbon_log_metrics] 00:00:05 (+4.999159689s) | 2 processed (100%), 2 successful, 0 failed (0%), 0 in queue
…
[2026-11-08T23:51:49Z INFO  carbon_log_metrics]   custom_http_poll_transactions_fetched_total: 12
[2026-11-08T23:51:49Z INFO  carbon_log_metrics]   custom_http_poll_fetch_failures_total: 0
```

One transaction produces several lines because the processor runs for every Jupiter instruction in it, including inner instructions and the events Jupiter emits through self-CPI ([nested instructions](https://carbon.sevenlabs.org/concepts/instructions.md)). It runs until you press Ctrl-C; the example uses `ShutdownStrategy::Immediate`.

## `block-subscribe-rpc`

Subscribes to confirmed blocks that mention the Raydium CLMM program over the RPC WebSocket method `blockSubscribe`, decodes Raydium CLMM instructions (`Swap`, `SwapV2` and `OpenPosition` at `info`), and logs every block's header with a `block_details` processor. The `RpcBlockSubscribeConfig` it passes asks for Base64 encoding, full transaction details and `max_supported_transaction_version: Some(1)`, which Carbon 2 requires. See [Stream with RPC](https://carbon.sevenlabs.org/guides/stream-with-rpc.md).

**Needs:** an RPC provider that enables `blockSubscribe`. Solana's public endpoint doesn't.

| Variable     | Required         | Used by                                                                                                           |
| ------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------- |
| `RPC_WS_URL` | In practice, yes | `RpcBlockSubscribe`. A `wss://` URL. Falls back to `wss://api.mainnet-beta.solana.com`, which rejects the method. |

```sh
cd carbon/examples/block-subscribe-rpc
RPC_WS_URL="wss://your-provider.example/?api-key=…" RUST_LOG=info cargo run --release -p block-subscribe-rpc-carbon-example
```

With a provider that supports the method, `block slot=… hash=… parent=… time=…` lines interleave with decoded `swap`, `swap_v2` and `open_position` lines. Without `RPC_WS_URL`, the run against the public endpoint looks like this and the process exits after about 40 seconds:

Output:

```text
[2026-11-08T23:51:57Z ERROR carbon_rpc_block_subscribe_datasource] Failed to subscribe to block updates: SubscribeFailed { reason: "Method not found (-32601)", message: "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":-32601,\"message\":\"Method not found\"},\"id\":1}" }
…
[2026-11-08T23:52:37Z ERROR carbon_core::pipeline] error consuming datasource: Custom("Failed to subscribe after 10 attempts: subscribe failed: Method not found (-32601)")
[2026-11-08T23:52:37Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-08T23:52:37Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

## `yellowstone-grpc`

The standard real-time setup: a Yellowstone Geyser gRPC subscription to confirmed, successful transactions that include the Jupiter v6 program (`account_required`, `vote: false`, `failed: false`), decoded with `JupiterSwapDecoder`. It logs `Route` and `SharedAccountsRoute` at `info`. `src/variants.rs` has a Helius LaserStream variant built with `LaserStreamClientConfig` and replay enabled. `main` installs the `aws_lc_rs` rustls crypto provider before connecting. See [Stream with Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) and [Stream with Helius LaserStream](https://carbon.sevenlabs.org/guides/stream-with-laserstream.md).

**Needs:** a Yellowstone gRPC endpoint (Triton, Helius, QuickNode, a self-hosted Geyser plugin) or a Helius LaserStream plan.

| Variable          | Required                         | Used by                                                                           |
| ----------------- | -------------------------------- | --------------------------------------------------------------------------------- |
| `GEYSER_URL`      | Yes, for the default variant     | `YellowstoneGrpcGeyserClient`. Include the scheme and port, e.g. `https://…:443`. |
| `X_TOKEN`         | If your provider requires it     | Sent as the `x-token` header.                                                     |
| `LASERSTREAM_URL` | Only for the LaserStream variant | `LaserStreamGeyserClient`.                                                        |
| `API_KEY`         | Only for the LaserStream variant | Your Helius API key.                                                              |

```sh
cd carbon/examples/yellowstone-grpc
GEYSER_URL=https://your-yellowstone-endpoint:443 X_TOKEN=your-x-token RUST_LOG=info cargo run --release -p yellowstone-grpc-carbon-example
```

The example compiles at `v2.0.0`; it wasn't run for this page because it needs a paid endpoint. With a working endpoint, `route slot=… sig=… data=Route { … }` and `shared_accounts_route …` lines stream continuously between `LogMetrics` summaries until you press Ctrl-C. If `GEYSER_URL` is missing the example panics with `GEYSER_URL must be set: NotPresent`.

## `postgres-graphql`

Streams SPL Token account updates from Yellowstone, stores each decoded account as JSON in Postgres with `PostgresJsonAccountProcessor<TokenProgramAccount>` (Carbon's `postgres` feature), and serves a small GraphQL API over the same table with Carbon's `graphql` feature: a `mints(limit)` query and a `mint(pubkey)` lookup. The migration in `migrations/001_init.sql` creates the `accounts` table (`__pubkey BYTEA`, `__slot BIGINT`, `data JSONB`) and runs at startup. The GraphQL server starts before the pipeline, on `/graphql`, with a GraphiQL playground on `/graphiql`. See [Postgres](https://carbon.sevenlabs.org/guides/postgres.md) and [GraphQL](https://carbon.sevenlabs.org/guides/graphql.md).

**Needs:** Postgres and a Yellowstone endpoint. The subscription covers every account owned by the Token program, which is a high-volume stream.

| Variable       | Required                     | Used by                                         |
| -------------- | ---------------------------- | ----------------------------------------------- |
| `DATABASE_URL` | Yes                          | `sqlx` connection pool.                         |
| `GEYSER_URL`   | Yes                          | `YellowstoneGrpcGeyserClient`.                  |
| `X_TOKEN`      | If your provider requires it | Yellowstone `x-token`.                          |
| `BIND_ADDR`    | No                           | GraphQL server address. Default `0.0.0.0:8080`. |

The bundled `compose.yaml` starts Postgres 16 with user, password and database `carbon` / `carbon` / `carbon_example`, published on host port **5433**. The example's README uses port 5432 in `DATABASE_URL`; with the bundled database, use 5433:

```sh
cd carbon/examples/postgres-graphql
docker compose up -d
DATABASE_URL=postgres://carbon:carbon@localhost:5433/carbon_example \
GEYSER_URL=https://your-yellowstone-endpoint:443 X_TOKEN=your-x-token \
RUST_LOG=info cargo run --release -p postgres-graphql-carbon-example
```

The first log line is `graphiql at http://0.0.0.0:8080/graphiql`. Open `http://localhost:8080/graphiql` and run:

```graphql
{
  mints(limit: 5) {
    pubkey
    slot
    decimals
    supply
  }
}
```

The example compiles at `v2.0.0`; it wasn't run end to end for this page (no Postgres or Geyser endpoint was available). Without a reachable database it fails at startup with:

Output:

```text
Error: Custom("failed to connect to Postgres: pool timed out while waiting for an open connection")
```

## `snapshot-validator`

Loads accounts from a Solana validator snapshot, either downloaded or already extracted on disk, filtered by owner program or account address. A pass-through `AccountDecoder` hands each raw `Account` to the processor, which logs pubkey, slot, lamports, data size and owner; an `account_deletions` processor logs deletions. It uses `ShutdownStrategy::ProcessPending` and exits when the snapshot has been read. See [Load a snapshot](https://carbon.sevenlabs.org/guides/load-a-snapshot.md).

**Needs:**

- `libclang` at build time: `carbon-validator-snapshot-datasource` depends on `solana-ledger`, which builds RocksDB with `bindgen`. Install `libclang-dev` on Debian and Ubuntu or `llvm` with Homebrew. It's the only example that needs it.
- Disk and memory for a snapshot. The current mainnet full snapshot was about 126 GB on 2026-11-09. The remote source downloads `genesis.tar.bz2` and the snapshot into a `ledger/` directory under the current directory.

| Variable         | Required                  | Used by                                                                                                                                               |
| ---------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `VALIDATOR_URL`  | One of these two          | Base URL of an RPC node that serves snapshots, e.g. `https://api.mainnet-beta.solana.com`. Carbon appends `/genesis.tar.bz2` and `/snapshot.tar.bz2`. |
| `SNAPSHOT_PATH`  | One of these two          | An extracted ledger directory containing `genesis.tar.bz2` and `snapshots/`.                                                                          |
| `PROGRAM_OWNERS` | At least one of these two | Comma-separated owner program IDs.                                                                                                                    |
| `ACCOUNT_IDS`    | At least one of these two | Comma-separated account addresses.                                                                                                                    |

> **Warning: The example README's VALIDATOR_URL is wrong.** The README sets `VALIDATOR_URL=https://api.mainnet-beta.solana.com/snapshot.tar.bz2`. Carbon treats the value as a base URL, so it requests `…/snapshot.tar.bz2/genesis.tar.bz2`, gets `HTTP error: 405 Method Not Allowed` three times and gives up. Set the base URL only.

```sh
cd carbon/examples/snapshot-validator
VALIDATOR_URL=https://api.mainnet-beta.solana.com \
PROGRAM_OWNERS=TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA \
RUST_LOG=info cargo run --release -p snapshot-validator-carbon-example
```

The example compiles at `v2.0.0` with `libclang` installed; a full mainnet load wasn't run for this page. Loaded accounts print as `account pubkey=… slot=… lamports=… data_size=… owner=…`, followed by `snapshot bootstrap complete`. That last line also prints when loading failed, because the pipeline returns `Ok` after a datasource error; check the log for `error consuming datasource` (see [Pipeline exits with status 0 after a datasource error](https://carbon.sevenlabs.org/troubleshooting.md#pipeline-exits-with-status-0-after-a-datasource-error)).

## `jetstreamer`

Backfills Token-2022 activity over a slot range from Old Faithful through the Jetstreamer datasource, with server-side transaction filters and parallel workers, then exits.

This is a Carbon 1 / Solana 3 example. Jetstreamer doesn't support the Solana 4 crates yet, so `carbon-jetstreamer-datasource` was not published on the 2.0 line (crates.io has 1.0.0 only), and the workspace `Cargo.toml` excludes `examples/jetstreamer`. It doesn't build from a `v2.0.0` checkout: from the repository root, `cargo run -p jetstreamer-carbon-example` fails with ``package(s) `jetstreamer-carbon-example` not found in workspace``, and from its own directory with ``error inheriting `edition` from workspace root manifest's `workspace.package.edition` ``. To use it, copy the pattern into a project that depends on Carbon 1.0.0 crates throughout. Don't combine it with Carbon 2 crates ([why](https://carbon.sevenlabs.org/troubleshooting.md#instructiondecoder-is-not-implemented-for-a-decoder)).

It reads `START_SLOT` (default `415500000`), `END_SLOT` (`415931999`), `WORKER_COUNT` (`4`) and `PAGE_SIZE` (`100`), all optional.

## Next steps

- [Quickstart](https://carbon.sevenlabs.org/quickstart.md): Build a pipeline from an empty project instead of a checkout.
- [Datasources](https://carbon.sevenlabs.org/datasources.md): Choose the datasource for your use case.
- [Troubleshooting](https://carbon.sevenlabs.org/troubleshooting.md): Errors you'll see while running these examples, and fixes.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/examples · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Troubleshooting (Carbon 2.0)

> Fixes for the errors and surprises you hit building and running Carbon 2.0 pipelines, from compile errors to rate limits, missing output and shutdown.

Problems are grouped by the stage where they show up. Each heading is the symptom or the message you see; the error text quoted under it was produced by Carbon 2.0.0. For the variants of Carbon's own `Error` enum, see the [errors reference](https://carbon.sevenlabs.org/reference/errors.md).

## Installing and compiling

### `rustc 1.95.0 is not supported by the following packages`

Output:

```text
error: rustc 1.95.0 is not supported by the following packages:
  carbon-core@2.0.0 requires rustc 1.96.1
  carbon-macros@2.0.0 requires rustc 1.96.1
  carbon-proc-macros@2.0.0 requires rustc 1.96.1
```

Carbon 2.0 sets `rust-version = "1.96.1"`. Older Cargo versions word the same error as `cannot be built because it requires rustc 1.96.1 or newer`. Update the toolchain and build again:

```sh
rustup update stable
rustc --version
```

If the project has a `rust-toolchain.toml` that pins an older version, raise it to at least `1.96.1`.

### `cargo add` adds Carbon 1.0.0 instead of 2.0.0

On a toolchain older than 1.96.1, Cargo's resolver picks the newest version your compiler supports, and says so in passing:

Output:

```text
     Locking 276 packages to latest Rust 1.95.0 compatible versions
      Adding carbon-core v1.0.0 (available: v2.0.0, requires Rust 1.96.1)
```

You end up on Carbon 1 without an error, and the code on this site doesn't compile against it. Update Rust, then pin the version when you add crates: `cargo add carbon-core@2.0.0`. Check what resolved with `cargo tree -i carbon-core`.

### `InstructionDecoder` is not implemented for a decoder

Output:

```text
error[E0277]: the trait bound `for<'a> MeteoraDlmmDecoder: InstructionDecoder<'a>` is not satisfied
...
note: there are multiple different versions of crate `carbon_core` in the dependency graph
   --> ~/.cargo/registry/src/index.crates.io-…/carbon-core-2.0.0/src/instruction.rs:187:1
    |
187 | pub trait InstructionDecoder<'a> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this is the expected trait
    |
   ::: ~/.cargo/registry/src/index.crates.io-…/carbon-core-1.0.0/src/instruction.rs:188:1
    |
188 | pub trait InstructionDecoder<'a> {
    | -------------------------------- this is the found trait
```

A Carbon 1 crate is in the same build as Carbon 2: here `carbon-meteora-dlmm-decoder = "1"` next to `carbon-core = "2"`. Each pulls its own `carbon-core`, and their traits are unrelated types. The same happens with `Datasource` or `AccountDecoder`. Set every `carbon-*` dependency to `2.0.0` and run `cargo update`. `carbon-jetstreamer-datasource` and `carbon-jito-shredstream-grpc-datasource` have no 2.0 release, so they can't be used with Carbon 2 at all. See [Installation](https://carbon.sevenlabs.org/installation.md#keep-one-carbon-version-and-one-solana-cohort).

### `expected Address, found Pubkey`

Output:

```text
error[E0308]: mismatched types
   --> src/main.rs:20:9
    |
 18 |     let datasource = RpcTransactionCrawler::new(
    |                      -------------------------- arguments to this function are incorrect
 19 |         "https://api.mainnet-beta.solana.com".to_string(),
 20 |         solana_pubkey::Pubkey::from_str_const("LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo"),
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `Address`, found `Pubkey`
```

Your crate depends on a Solana crate from an older cohort (here `solana-pubkey = "2"`; `solana-sdk` 2.x does the same) and passes its type to Carbon, which uses the Agave 4.2 cohort. Variants of this error read `expected Pubkey, found Pubkey` or mention `Signature`, `Hash` or `VersionedTransaction`. Use the versions Carbon's workspace pins, for example `solana-pubkey = "4.2.1"`, `solana-signature = "3.4.1"`, `solana-client = "=4.2.2"`, `solana-transaction-status = "=4.2.2"` and `solana-commitment-config = "3.1.1"`, or use the re-exported types from the Carbon crate you're calling. Find the duplicates with:

```sh
cargo tree --duplicates --edges normal | grep solana
```

The full list of pinned versions is in the [workspace `Cargo.toml`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/Cargo.toml).

### Build fails in `clang-sys`: `couldn't find any valid shared libraries`

Output:

```text
error: failed to run custom build command for `clang-sys v1.8.1`
...
  called `Result::unwrap()` on an `Err` value: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
```

[`carbon-validator-snapshot-datasource`](https://carbon.sevenlabs.org/datasources/validator-snapshot.md) depends on `solana-ledger`, which builds RocksDB with `bindgen`, and `bindgen` needs libclang. It's the only Carbon 2 crate that does; you also hit this building the whole Carbon workspace. Install the development package, which provides `libclang.so`:

```sh
# Debian and Ubuntu
sudo apt-get install libclang-dev
# macOS
brew install llvm
```

If libclang is installed somewhere non-standard, point `LIBCLANG_PATH` at the directory that contains `libclang.so`.

### `the crate carbon-…-datasource@2.0.0 could not be found in registry index`

Output:

```text
error: the crate `carbon-jito-shredstream-grpc-datasource@2.0.0` could not be found in registry index.
```

Jito ShredStream and Jetstreamer datasources stayed on Carbon 1 (they don't support Transaction V1 and the Solana 4 crates yet). The [datasources page](https://carbon.sevenlabs.org/datasources.md) lists the twelve that exist for 2.0.

## Connecting to a datasource

### `Method not found (-32601)` from `blockSubscribe`

Output:

```text
[2026-11-08T23:51:57Z ERROR carbon_rpc_block_subscribe_datasource] Failed to subscribe to block updates: SubscribeFailed { reason: "Method not found (-32601)", message: "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":-32601,\"message\":\"Method not found\"},\"id\":1}" }
...
[2026-11-08T23:52:37Z ERROR carbon_core::pipeline] error consuming datasource: Custom("Failed to subscribe after 10 attempts: subscribe failed: Method not found (-32601)")
```

The endpoint doesn't enable `blockSubscribe`. Solana's public endpoint (`wss://api.mainnet-beta.solana.com`) never does, and many providers enable it only on paid plans. `RpcBlockSubscribe` retries every three seconds and gives up after ten failures, after which the pipeline stops. Use a provider that supports the method, or switch datasource: [`RpcProgramSubscribe`](https://carbon.sevenlabs.org/datasources/rpc-program-subscribe.md) works on the public endpoint for account updates, [`RpcTransactionCrawler`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md) for transactions, and [Yellowstone gRPC](https://carbon.sevenlabs.org/guides/stream-with-yellowstone.md) for production streaming.

### HTTP 429 `Too Many Requests` from the RPC endpoint

Solana's public endpoint limits requests per IP and per method. `getTransaction` is the tightest: the response headers on a throttled call show `x-ratelimit-method-limit: 10` and `retry-after: 10`. The Solana RPC client inside Carbon's RPC datasources retries a 429 up to five times on its own, sleeping for the `retry-after` time, and logs this only at debug level:

Output:

```text
[2026-11-08T23:49:57Z DEBUG solana_rpc_client::http_sender] Too many requests: server responded with Response { url: "https://api.mainnet-beta.solana.com/", status: 429, headers: {… "retry-after": "10", … "x-ratelimit-tier": "free", "x-ratelimit-method-limit": "10", "x-ratelimit-method-remaining": "-1", …} }, 4 retries left, pausing for 10s
```

When those retries run out, the call fails with `Status(429, …)` and the datasource's own retry policy takes over. For the transaction crawler that's `RetryConfig` (three retries with backoff by default); after it, the crawler logs `Failed to fetch transaction <signature> after 3 retries: …` and skips that transaction. Fixes, in order of effect:

- Use your own RPC provider. Every RPC datasource takes the URL as its first argument.
- Lower the request rate: for the crawler, reduce `max_concurrent_requests` and `batch_limit` in [`ConnectionConfig`](https://carbon.sevenlabs.org/datasources/rpc-transaction-crawler.md#configuration).
- To see the throttling, add the RPC client's logger: `RUST_LOG=info,solana_rpc_client=debug`.

### `Status(413, … "Request Entity Too Large")` from `getSignaturesForAddress`

Output:

```text
[2026-11-08T23:48:38Z ERROR carbon_rpc_transaction_crawler_datasource] Failed to fetch signatures after 0 retries: Error { request: Some(GetSignaturesForAddress), kind: Reqwest(reqwest::Error { kind: Status(413, Some(ReasonPhrase(b"Request Entity Too Large"))), url: "https://api.mainnet-beta.solana.com/" }) }
```

The public endpoint sometimes rejects `getSignaturesForAddress` for busy programs with HTTP 413; the same request succeeds a few seconds later. With `RetryConfig::no_retry()` (as in the `transaction-crawler-rpc` example) the crawler logs it and tries again on the next poll, so it's harmless. With `RetryConfig::default()` it backs off and retries up to three times first. A dedicated RPC provider doesn't do this.

### `gRPC transport error: transport error` and the pipeline stops

Output:

```text
[2026-11-09T00:05:46Z ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("gRPC transport error: transport error")
[2026-11-09T00:05:46Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
```

`YellowstoneGrpcGeyserClient` couldn't open the first connection: the host doesn't resolve, the port is closed, TLS failed, or the URL has no scheme. The first connection isn't retried, so the pipeline stops at once. Check that the URL is the full gRPC endpoint your provider gave you, with scheme and port (`https://your-endpoint:443`), and that `X_TOKEN` is set if the provider needs one. Reconnection after a connection has been established is configured separately; see [Yellowstone gRPC](https://carbon.sevenlabs.org/datasources/yellowstone-grpc.md). Error variant: [`FailedToConsumeDatasource`](https://carbon.sevenlabs.org/reference/errors.md#failedtoconsumedatasource).

### `Failed to receive updates(…)`

This is the message of [`Error::FailedToReceiveUpdates`](https://carbon.sevenlabs.org/reference/errors.md#failedtoreceiveupdates). No datasource or pipeline code in Carbon 2.0.0 returns it, so if it's in your log, it comes from a custom datasource or a third-party crate; the text in parentheses is that code's own reason. Search your code for `FailedToReceiveUpdates`. Carbon's own datasources report connection failures as [`FailedToConsumeDatasource`](https://carbon.sevenlabs.org/reference/errors.md#failedtoconsumedatasource) or [`Custom`](https://carbon.sevenlabs.org/reference/errors.md#custom), logged as `error consuming datasource: …`.

### `Geyser stream error … 501 Not Implemented`, many times per second

Output:

```text
[2026-11-09T00:05:47Z ERROR carbon_yellowstone_grpc_datasource] Geyser stream error: Status { code: Internal, message: "protocol error: received message with invalid compression flag: 123 (valid flags are 0 and 1) while receiving response with status: 501 Not Implemented", source: None }
[2026-11-09T00:05:47Z ERROR carbon_yellowstone_grpc_datasource] Disconnected at slot 0
```

The URL points at a server that accepts the connection but doesn't speak Yellowstone gRPC, typically a JSON-RPC URL such as `https://api.mainnet-beta.solana.com` put in `GEYSER_URL`. After a stream error the datasource resubscribes immediately with no delay, so the error repeats about twenty times a second until you stop the process. Use your provider's gRPC endpoint, which is a different host or port from its RPC URL.

### `Transaction version (1) is not supported by the requesting client`

Output:

```text
[2026-11-09T00:12:01Z ERROR carbon_rpc_block_crawler_datasource] Error fetching block at slot 467111665: Error { request: Some(GetBlock), kind: RpcError(RpcResponseError { code: -32015, message: "Transaction version (1) is not supported by the requesting client. Please try the request again with the following configuration parameter: \"maxSupportedTransactionVersion\": 1", data: Empty }) }
```

Your `RpcBlockConfig` (for `RpcBlockCrawler`) or `RpcBlockSubscribeConfig` (for `RpcBlockSubscribe`) leaves `max_supported_transaction_version` unset, so the node refuses blocks that contain versioned transactions, which is almost every mainnet block. The block is skipped. Set `max_supported_transaction_version: Some(1)`, as shown in [Transactions are logged as `Failed to decode transaction`](#transactions-are-logged-as-failed-to-decode-transaction). The transaction crawler and Helius GTFA datasources set it for you.

### `… must be set: NotPresent` although the variable is in `.env`

Output:

```text
thread 'main' panicked at examples/yellowstone-grpc/src/variants.rs:16:32:
GEYSER_URL must be set: NotPresent
```

`dotenv::dotenv()` reads `.env` from the current directory and its parents, not from the crate being run. Running an example from the repository root with `cargo run -p …` doesn't load `examples/<name>/.env`. Run from the example's directory, or set the variables in the shell.

## Decoding

### Transactions are logged as `Failed to decode transaction`

Output:

```text
[2026-11-09T00:12:18Z ERROR carbon_rpc_block_crawler_datasource] Failed to decode transaction: EncodedTransactionWithStatusMeta { transaction: Json(UiTransaction { signatures: ["4JBnX1dGq94vApHPA8MsjcGb8akiw8WwcuokdMWBaZP2z2JktVdtv5wYro8e6v8kJJgMN8a4Z66V9S7EjRhxV8n8"], message: Raw(UiRawMessage { header: MessageHeader { …
```

The datasource asked the node for transactions in `Json` or `JsonParsed` encoding. Carbon needs the original binary transaction to rebuild instructions and accounts, so it skips every transaction it can't decode (all 251 in the block above). Request Base64 and Transaction V1 in the config you pass to the datasource:

```rust
RpcBlockConfig {
    encoding: Some(UiTransactionEncoding::Base64),
    transaction_details: Some(TransactionDetails::Full),
    max_supported_transaction_version: Some(1),
    ..RpcBlockConfig::default()
}
```

`RpcBlockSubscribeConfig` takes the same three fields. For Helius Atlas `TransactionSubscribeOptions`, set `encoding: Some(UiEnhancedTransactionEncoding::Base64)` and `max_supported_transaction_version: Some(1)`; its SDK defaults to JSON-parsed. A custom datasource that calls `getTransaction` must also request `UiTransactionEncoding::Base64`. See [Migrate to 2.0](https://carbon.sevenlabs.org/guides/migrate-to-v2.md).

Looking for transactions that **failed on chain** rather than ones Carbon couldn't decode? See [Decode or skip failed transactions](https://carbon.sevenlabs.org/concepts/filters.md#decode-or-skip-failed-transactions).

### The processor never runs for a program's instructions

The pipeline receives transactions (`carbon_updates_received_total` grows) but your instruction processor is never called. Causes, most common first:

- **Wrong program ID.** Every generated decoder starts with `if instruction.program_id != PROGRAM_ID { return None; }`. A decoder for one program ignores a fork or redeployment of it at another address, and devnet deployments often use different IDs. Compare the decoder's `PROGRAM_ID` with the program in an explorer. For a program at another address, generate a decoder from its IDL with [`carbon-cli parse --program-id`](https://carbon.sevenlabs.org/reference/cli.md).
- **The datasource doesn't deliver that program's transactions.** Check the datasource filter: the crawler's account, Yellowstone's `account_include` / `account_required`, `blockSubscribe`'s `MentionsAccountOrProgram`.
- **The transactions failed.** RPC datasources drop transactions whose `meta.status` is an error before they reach the pipeline; Yellowstone does so when the filter sets `failed: Some(false)`.
- **A [filter](https://carbon.sevenlabs.org/concepts/filters.md) rejects them**, for example a `SlotRangeFilter` with the wrong bounds.

To see what the decoder returns, log every variant with `{other:?}` in a catch-all match arm and run with `RUST_LOG=debug`.

### Some instructions stop decoding after a program upgrade

After a program upgrade, a decoder generated from the old IDL returns `None` for instructions it can't match, and Carbon skips them silently. A generated decoder rejects an instruction when:

- its discriminator is new (an instruction added in the upgrade);
- its data is shorter than the old layout (a field removed or narrowed);
- it has fewer accounts than the old IDL lists.

Data that is longer than the old layout still decodes: the extra bytes are ignored, so new trailing fields are dropped without an error. Regenerate the decoder from the current IDL ([Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md)). If you index history from before the upgrade too, register both versions with slot filters: [Versioned decoders](https://carbon.sevenlabs.org/guides/versioned-decoders.md).

### Closed accounts arrive as `AccountDeletion`, not as account updates

In Carbon 2.0, every account update with zero lamports becomes `Update::AccountDeletion`, carrying the account's final state. Account processors registered with `.account(…)` don't see it. Register `.account_deletions(processor)` as well. See [Updates](https://carbon.sevenlabs.org/concepts/updates.md).

## Processing

### Nothing is printed

The pipeline runs, no error appears, and nothing is printed. Carbon, its datasources, `LogMetrics` and the examples all write through the `log` crate, which prints nothing unless a logger is initialized and enabled:

1. Call `env_logger::init()` (or initialize another logger) at the start of `main`.
2. Set `RUST_LOG` when you run. `env_logger` shows only errors when it's unset, and Carbon's progress messages are `info`:

```sh
RUST_LOG=info cargo run
```

If you use `println!` in a processor and still see nothing, the processor isn't being called; see [The processor never runs for a program's instructions](#the-processor-never-runs-for-a-programs-instructions).

### `error processing update` in the log

The line is logged by `carbon_core::pipeline` at `ERROR` and reads `error processing update (<the update>): <the error>`, with the full `Debug` output of the update. A processor or a decoder step returned an error for it. The pipeline logs it with the whole update, increments `carbon_updates_failed_total`, and moves on to the next update; it doesn't stop. The error also ends processing of that update: pipes after the failing one, and the remaining instructions of the same transaction, aren't run. Return `Ok(())` for conditions you can tolerate and log them yourself; return `Err` only when skipping the rest of the update is what you want. The error variants are in the [errors reference](https://carbon.sevenlabs.org/reference/errors.md); [Error handling](https://carbon.sevenlabs.org/concepts/error-handling.md) covers the patterns.

### The processor runs several times for one transaction

An instruction processor is called once per matching instruction, including instructions nested inside other programs' calls (CPIs) and the event logs that programs such as Meteora DLMM and Jupiter emit through self-CPI. A single Jupiter swap can produce a top-level route, several inner instructions and an event. Check `input.metadata.stack_height` (`1` for top-level instructions) or match only the variants you need. To handle a transaction once as a whole, use a [transaction processor](https://carbon.sevenlabs.org/concepts/transactions.md). See [Instructions](https://carbon.sevenlabs.org/concepts/instructions.md).

## Performance

### Crawler prints nothing for seconds at a time

`RpcTransactionCrawler` on the public endpoint fetches a thousand signatures in a few seconds, then stalls on `getTransaction`: throttled calls sleep for ten seconds inside the RPC client without logging at `info` (see [HTTP 429](#http-429-too-many-requests-from-the-rpc-endpoint)). In a 60-second run against Meteora DLMM, the `LogMetrics` line read:

Output:

```text
[2026-11-08T23:49:36Z INFO  carbon_log_metrics]   transaction_crawler_signatures_fetched_total: 1000
[2026-11-08T23:49:36Z INFO  carbon_log_metrics]   transaction_crawler_transactions_fetched_total: 26
```

Failed transactions are fetched but skipped, so a busy stretch of failed bot transactions also prints nothing. Use your own RPC endpoint for real backfills, or [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md) for history with server-side filters.

### `Full(..)` send errors and missing updates

Output:

```text
[2026-11-09T00:12:19Z ERROR carbon_rpc_block_crawler_datasource] Error sending transaction update: "Full(..)"
```

The channel between the datasources and the pipeline is full, and the datasource dropped the update instead of waiting. Almost every Carbon 2.0 datasource sends with `try_send`, so a full channel means lost data, each with its own message:

| Datasource                                                                              | Logged message                                                                                                | What's dropped                                |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
| `RpcBlockCrawler`, `RpcBlockSubscribe`                                                  | `Error sending transaction update: "Full(..)"`                                                                | The rest of that block                        |
| `YellowstoneGrpcGeyserClient`, `LaserStreamGeyserClient`                                | `Failed to send transaction update with signature … at slot …` or `Failed to send account event for pubkey …` | That update                                   |
| `RpcProgramSubscribe`                                                                   | `Error sending account update: …`                                                                             | That update, then the subscription reconnects |
| `RpcTransactionCrawler` with `blocking_send: false` (the `ConnectionConfig::default()`) | `Failed to send update: …` (warning)                                                                          | That transaction                              |

The channel fills when processors are slower than the datasource or the buffer is small: with `channel_buffer_size(10)`, the block crawler above delivered 10 of a block's 251 transactions. Fixes:

- Keep the default buffer of 1,000 or raise it with [`channel_buffer_size`](https://carbon.sevenlabs.org/reference/pipeline-builder.md#channel_buffer_size). A bigger buffer absorbs bursts, not a sustained gap.
- For the transaction crawler, set the last `ConnectionConfig::new` argument, `blocking_send`, to `true`, so it waits for space instead of dropping.
- Make processors faster (next entry), and watch `carbon_updates_queued` in [metrics](https://carbon.sevenlabs.org/reference/metrics.md).

### `in queue` keeps growing in the `LogMetrics` line

`LogMetrics` prints `… | N processed (…), N successful, 0 failed (0%), M in queue` every five seconds. If `M` keeps growing, processors are slower than the datasource. The pipeline handles one update at a time and runs every pipe for it before taking the next, so a slow processor delays everything. Once the queue reaches the channel size, updates are dropped (previous entry). Move slow work out of the processing path: batch database writes, hand work to a spawned task through a channel, and filter at the datasource so fewer updates arrive. See [Production](https://carbon.sevenlabs.org/guides/production.md).

## Shutdown

### Pipeline exits immediately after starting

Output:

```text
[2026-11-08T23:48:31Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-08T23:48:31Z INFO  carbon_core::pipeline] pipeline shutdown complete.
```

`run()` returns when every datasource has finished and dropped its sender. Some datasources are finite by design: `GpaDatasource`, `HeliusGpaV2Datasource`, `SnapshotDatasource`, `HeliusGtfaDatasource` and `RpcBlockCrawler` with an `end_slot` stop when they've sent everything. A custom datasource whose `consume` returns without keeping a sender alive also ends the pipeline. If you expected the pipeline to keep running, pair the finite source with a streaming one in the same pipeline ([Multiple datasources](https://carbon.sevenlabs.org/guides/multiple-datasources.md)), or check the log for an error immediately before these lines (next entry). See [Finite datasources](https://carbon.sevenlabs.org/concepts/shutdown.md#finite-datasources).

### Pipeline exits with status 0 after a datasource error

Output:

```text
[2026-11-09T00:14:58Z ERROR carbon_core::pipeline] error consuming datasource: FailedToConsumeDatasource("Failed to download https://api.mainnet-beta.solana.com/snapshot.tar.bz2/genesis.tar.bz2 after 3 retries: Failed to consume datasource (HTTP error: 405 Method Not Allowed)")
[2026-11-09T00:14:58Z INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[2026-11-09T00:14:58Z INFO  carbon_core::pipeline] pipeline shutdown complete.
[2026-11-09T00:14:58Z INFO  snapshot_validator_carbon_example] snapshot bootstrap complete
```

When a datasource's `consume` returns an error, the pipeline only logs it. Once no datasource is left, `run()` returns `Ok(())`, so your code after it runs as if the work succeeded and the process exits with status 0. If a supervisor or a job runner must see the failure, check a counter you increment in your processors after `run()` returns, or wrap the datasource in your own `Datasource` that records the error before returning it. Error variant: [`FailedToConsumeDatasource`](https://carbon.sevenlabs.org/reference/errors.md#failedtoconsumedatasource).

### The transaction crawler never exits, even with `until_signature`

`RpcTransactionCrawler` walks backwards until it reaches `until_signature` (or the oldest signature the node returns), then keeps polling for new signatures every `polling_interval`. Its `consume` never finishes, so the pipeline runs until it's cancelled. To backfill a fixed range and stop, cancel the pipeline yourself ([Stop with a cancellation token](https://carbon.sevenlabs.org/concepts/shutdown.md#stop-with-a-cancellation-token)) when your processor sees the slot or signature you want, or use [Helius GTFA](https://carbon.sevenlabs.org/datasources/helius-gtfa.md) with a slot range, which exits when drained.

### Ctrl-C does nothing the first time

The pipeline only notices Ctrl-C while it waits for the next update. A Ctrl-C that arrives while a processor is running is lost, and the process doesn't exit, because Tokio has replaced the default signal handler. In a test with a processor that takes 100 ms per update, the first Ctrl-C was ignored and the pipeline processed all 100 queued updates; a second Ctrl-C, sent while it was idle, stopped it. Handle Ctrl-C in your own task and cancel the pipeline's token from there: [Ctrl-C](https://carbon.sevenlabs.org/concepts/shutdown.md#ctrl-c).

### Pending updates are lost with `ShutdownStrategy::ProcessPending`

`ProcessPending` is meant to drain the queue on Ctrl-C, but in 2.0.0 the pipeline loop exits as soon as it sees the cancellation it triggered, so queued updates are usually dropped. Don't rely on it to drain; see [`ShutdownStrategy`](https://carbon.sevenlabs.org/concepts/shutdown.md#shutdownstrategy) for the details and alternatives.

### `Failed to send signature: SendError { .. }` when stopping

Output:

```text
[2026-11-08T23:49:36Z ERROR carbon_rpc_transaction_crawler_datasource] Failed to send signature: SendError { .. }
```

The crawler's internal tasks were still sending when the pipeline closed its channels. It's logged during shutdown only and is harmless.

## Still stuck

Run with `RUST_LOG=debug` to see decoder and datasource detail, and compare your setup with the runnable [examples](https://carbon.sevenlabs.org/examples.md). If you think you've found a bug in Carbon, open an issue on [GitHub](https://github.com/sevenlabs-hq/carbon/issues) with the Carbon version, your `Cargo.toml`, and the log.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/troubleshooting · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Contributing (Carbon 2.0)

> Set up the Carbon repository, run the same checks as CI, add a decoder or a datasource, and open a pull request. Also how to improve these docs.

Carbon is MIT-licensed and developed in the open at [sevenlabs-hq/carbon](https://github.com/sevenlabs-hq/carbon). This page covers working on the repository itself: setting it up, the checks a pull request must pass, and how to add a decoder or a datasource. It follows [`CONTRIBUTING.md`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/CONTRIBUTING.md) at `v2.0.0`, with the commands checked against that tag.

## Set up the repository

You need:

- **Rust 1.96.1.** The repository's `rust-toolchain.toml` pins it, so `rustup` installs and selects it the first time you run `cargo` in the checkout. Published crates declare the same MSRV in the workspace `Cargo.toml` and `clippy.toml`.
- **libclang**, for `carbon-validator-snapshot-datasource` (RocksDB is built with `bindgen`). `sudo apt-get install libclang-dev` on Debian and Ubuntu, `brew install llvm` on macOS. Without it, building the whole workspace fails in `clang-sys` ([details](https://carbon.sevenlabs.org/troubleshooting.md#build-fails-in-clang-sys-couldnt-find-any-valid-shared-libraries)).
- **Node.js 20.18.0 or newer and pnpm 9.6.0**, only if you work on the TypeScript packages in `packages/` (the CLI, the Codama renderer and the version registry).

```sh
git clone https://github.com/sevenlabs-hq/carbon.git
cd carbon
cargo build
cargo test
```

A full workspace build compiles every decoder and datasource with the Solana 4.2 crates, so the first one takes a while and needs several GB of disk. While you work on one crate, build and test only that crate with `-p`:

```sh
cargo test -p carbon-core
```

### Repository layout

| Directory      | Contents                                                                                                       |
| -------------- | -------------------------------------------------------------------------------------------------------------- |
| `crates/`      | `carbon-core` (pipeline, traits, filters, metrics), `carbon-macros`, `carbon-proc-macros`, `carbon-test-utils` |
| `datasources/` | One crate per datasource, `carbon-<name>-datasource`                                                           |
| `decoders/`    | One crate per program decoder, `carbon-<name>-decoder`                                                         |
| `metrics/`     | `carbon-log-metrics`, `carbon-prometheus-metrics`                                                              |
| `examples/`    | Runnable example pipelines (see [Examples](https://carbon.sevenlabs.org/examples.md))                          |
| `packages/`    | The TypeScript CLI (`@sevenlabs-hq/carbon-cli`), the Codama renderer and the version registry                  |
| `misc/`        | Vendored Jito protobufs (a git submodule), used only by the Carbon 1 ShredStream datasource                    |
| `scripts/`     | Formatting, linting, publishing and CLI version scripts                                                        |

The workspace members are `crates/*`, `datasources/*`, `decoders/*`, `examples/*` and `metrics/*`. `datasources/jetstreamer-datasource`, `datasources/jito-shredstream-grpc-datasource` and `examples/jetstreamer` are excluded: they stay on Carbon 1 and Solana 3.

## Run the checks

Every pull request runs [`.github/workflows/check.yml`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/.github/workflows/check.yml) on Rust 1.96.1. All jobs must pass before a PR can merge:

| Check                           | Command                                                             |
| ------------------------------- | ------------------------------------------------------------------- |
| Format                          | `cargo fmt --check`                                                 |
| Lint                            | `cargo clippy --locked --all-targets --all-features -- -D warnings` |
| `Cargo.toml` ordering           | `cargo sort -c -g` (cargo-sort 2.0.2)                               |
| Unused dependencies             | `cargo machete` (cargo-machete 0.7.0)                               |
| Tests                           | `cargo test --locked --all-targets --all-features`                  |
| Published dependency resolution | `cargo package -p carbon-test-utils --exclude-lockfile`             |

Install the two extra Cargo tools at the versions CI uses:

```sh
cargo install --locked cargo-sort@2.0.2
cargo install --locked cargo-machete@0.7.0
```

To run the same checks before every commit, link the pre-commit script as a git hook. It asks before linking, installs any missing tools, and runs `cargo machete`, `cargo fmt -- --check`, `cargo sort -c -g`, `cargo test --all-targets --all-features` and `cargo clippy --all-targets --all-features -- -D warnings`:

```sh
./.pre-commit.sh
```

`./scripts/cargo-fmt.sh` runs `cargo fmt --all` to format everything. `rustfmt.toml` sets a few nightly-only options (`wrap_comments`, `comment_width`, `imports_granularity`, `group_imports`); stable `rustfmt` prints a warning for each and ignores them, which is expected.

> **Warning: scripts/cargo-clippy.sh doesn't pass at v2.0.0.** `CONTRIBUTING.md` suggests `./scripts/cargo-clippy.sh`, which adds `--deny` flags for `arithmetic_side_effects`, `manual_let_else`, `default_trait_access` and `used_underscore_binding`. At `v2.0.0`, `carbon-core` itself fails it with 12 errors. CI doesn't use the script; run the clippy command from the table above instead.

If you change the TypeScript packages, run what [`check-typescript.yml`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/.github/workflows/check-typescript.yml) runs:

```sh
pnpm install --frozen-lockfile
pnpm build
pnpm test
pnpm format:check
pnpm type-check
```

## Add a decoder

A decoder crate turns one program's instructions, accounts and events into Rust types. Generate it from the program's IDL with the Carbon CLI rather than writing it by hand, so it matches the other decoders in `decoders/`. [Generate a decoder](https://carbon.sevenlabs.org/guides/generate-a-decoder.md) explains the generated code; [the CLI reference](https://carbon.sevenlabs.org/reference/cli.md) lists every flag.

### Step 1: Generate the crate into `decoders/`

From the repository root, point `--idl` at an IDL file or at the program address (the CLI fetches the on-chain IDL from `--url`). `--standalone false` makes the generated `Cargo.toml` use the workspace's dependency versions instead of adding its own `[workspace]` section.

```sh
npx -y @sevenlabs-hq/carbon-cli@2.0.0 parse \
  --idl <path-to-idl.json-or-program-address> \
  --url mainnet-beta \
  --standard anchor \
  --name my-program \
  --out-dir decoders/my-program-decoder \
  --standalone false \
  --package-version 2.0.0 \
  --description "My Program Decoder" \
  --keywords solana,decoder,my-program
```

With pnpm, bun or yarn, replace `npx -y` with `pnpm dlx`, `bunx` or `yarn dlx`.

Use `--standard codama` for a Codama IDL. Postgres and GraphQL modules are generated by default, as in the existing decoders, behind the crate's `postgres` and `graphql` features. With `--description`, the CLI writes the `license`, `readme` and `repository` fields from the workspace and a `README.md`.

Output:

```text
✔ Decoder generated
- Formatting generated code...
✔ Code formatted successfully

✨ Decoder Generated Successfully! ✨
  • Output: …/carbon/decoders/my-program-decoder
  • Source: file (anchor)
```

### Step 2: Wire it into the workspace

`decoders/*` is already a workspace member, so Cargo picks up the new crate. Add it to `[workspace.dependencies]` in the root `Cargo.toml`, in alphabetical order (`cargo sort -c -g` checks this):

`Cargo.toml`:

```toml
carbon-my-program-decoder = { path = "decoders/my-program-decoder", version = "2.0.0" }
```

Add the crate name to the `workspace_crates` list in `scripts/publish-crate.sh` so it's published with the next release.

### Step 3: Add tests

Capture real instructions and accounts from mainnet as JSON fixtures under `tests/fixtures/` and replay them through the decoder with [`carbon-test-utils`](https://carbon.sevenlabs.org/reference/test-utils.md), which is already a dev-dependency of the generated crate. Test at least one instruction per common variant and each account type. [Testing](https://carbon.sevenlabs.org/guides/testing.md) shows the pattern. At `v2.0.0`, only `decoders/token-2022-decoder/tests` exists, so new decoders with fixtures are welcome.

### Step 4: Check the crate

```sh
cargo fmt -p carbon-my-program-decoder --check
cargo clippy -p carbon-my-program-decoder --all-targets --all-features -- -D warnings
cargo test -p carbon-my-program-decoder --all-features
```

A freshly generated decoder passes all three. Run the full checks from [Run the checks](#run-the-checks) before you push.

When a program upgrades its IDL, regenerate its decoder with the same command (the CLI deletes the output directory before rendering unless you pass `--no-clean`) and review the diff for removed or reordered fields.

## Add a datasource

A datasource crate produces `Update`s for the pipeline. It lives in `datasources/<name>-datasource/` with a `Cargo.toml`, a `README.md` and `src/lib.rs`, and implements the `Datasource` trait from `carbon-core`:

`src/lib.rs`:

```rust
use {
    async_trait::async_trait,
    carbon_core::{
        datasource::{Datasource, DatasourceId, Update, UpdateType},
        error::CarbonResult,
    },
    tokio::sync::mpsc::Sender,
    tokio_util::sync::CancellationToken,
};

pub struct YourDatasource;

#[async_trait]
impl Datasource for YourDatasource {
    async fn consume(
        &self,
        id: DatasourceId,
        sender: Sender<(Update, DatasourceId)>,
        cancellation_token: CancellationToken,
    ) -> CarbonResult<()> {
        // Send updates with `sender.send((update, id.clone())).await`
        // until `cancellation_token` is cancelled.
        let _ = (id, sender);
        cancellation_token.cancelled().await;
        Ok(())
    }

    fn update_types(&self) -> Vec<UpdateType> {
        vec![UpdateType::Transaction]
    }
}
```

[Custom datasource](https://carbon.sevenlabs.org/guides/custom-datasource.md) walks through a complete one, and [`examples/custom-datasource`](https://carbon.sevenlabs.org/examples.md#custom-datasource) is a runnable reference. A datasource merged into Carbon should also:

- **Declare every update type it can emit** in `update_types`: `AccountUpdate`, `Transaction`, `AccountDeletion`, `BlockDetails`.
- **Normalize accounts through `AccountUpdate::into_update()`**, so zero-lamport accounts become `Update::AccountDeletion` like in every other datasource.
- **Request binary transactions with Transaction V1**: `UiTransactionEncoding::Base64` and `max_supported_transaction_version: Some(1)`. Carbon can't rebuild a transaction from JSON.
- **Stop promptly on cancellation**, including during sleeps and backoff, and return when the source is exhausted if it's finite.
- **Retry transient failures with backoff**, and make the retry policy configurable.
- **Register its metrics** with `MetricsRegistry::global()` so they appear in every exporter ([Metrics](https://carbon.sevenlabs.org/concepts/metrics.md)).
- **Include tests with mocked data** and a README that covers setup, configuration and tradeoffs.

Wire it in like a decoder: an entry in `[workspace.dependencies]`, the crate name in `scripts/publish-crate.sh`, and a line in the datasource list in `CONTRIBUTING.md`.

## Open a pull request

1. Fork the repository and create a branch from `main`.
2. Make the change, with tests and updated READMEs and doc comments.
3. Run [the checks](#run-the-checks).
4. Open a pull request against `main`. Describe what changed, why, and how you tested it; link related issues; mark breaking changes clearly.

Use [conventional commit](https://www.conventionalcommits.org/) messages and PR titles, `type(scope): description`, with one of `feat`, `fix`, `docs`, `style`, `refactor`, `test` or `chore`:

```text
feat(decoders): add my-program decoder
fix(yellowstone-grpc-datasource): back off between failed subscribe attempts
```

Report bugs and request features in [GitHub issues](https://github.com/sevenlabs-hq/carbon/issues). By contributing, you agree that your contributions are licensed under the MIT License.

## Improving these docs

To fix or improve a page:

- **Rate the page.** Answer **Was this page helpful?** at the bottom of each page, with an optional comment. It goes to the docs maintainers and needs no GitHub account.
- **Report a problem.** The **Report an issue** link on each page opens a new issue in [sevenlabs-hq/carbon](https://github.com/sevenlabs-hq/carbon/issues), labelled `documentation`, with the page's URL filled in. Use it for anything wrong, missing or unclear, such as a command that doesn't run or an output that doesn't match.

If a page disagrees with Carbon's behaviour at `v2.0.0`, include the command you ran and its output.

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/contributing · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt


---

# Changelog (Carbon 2.0)

> Carbon's release history, from 0.3.0 to 2.0.0, with the breaking changes and migration notes for each version.

What changed in each Carbon release. 2.0.0 comes from [`CHANGELOG.md`](https://github.com/sevenlabs-hq/carbon/blob/v2.0.0/CHANGELOG.md) at the `v2.0.0` tag; earlier versions come from the [GitHub release notes](https://github.com/sevenlabs-hq/carbon/releases), which are the only record for them. Numbers like (#587) are pull requests in [sevenlabs-hq/carbon](https://github.com/sevenlabs-hq/carbon/pulls). To upgrade from Carbon 1, follow [Migrate to 2.0](https://carbon.sevenlabs.org/guides/migrate-to-v2.md).

## 2.0.0 (2026-09-04)

Carbon 2.0 is the Agave 4 compatibility release. It upgrades the supported Solana dependency cohort, adds end-to-end Transaction V1 handling, and standardizes account-closure delivery without introducing the larger datasource and pipeline redesign planned for Carbon 3.

### Highlights

- Upgraded the workspace to the Agave 4.2-compatible Solana crates and Rust 1.96.1 (#587).
- Added Transaction V1 conversion and instruction traversal, including runtime inner instructions and V1 account metadata.
- Added strict Yellowstone protobuf conversion for legacy, V0 and V1 transactions.
- Classified zero-lamport account updates as `Update::AccountDeletion` consistently while preserving the final `Account` state.
- Added an optional starting slot to Helius LaserStream (#585) and opt-in tonic client reconnection to Yellowstone (#589).
- Added native Codama event rendering (#575, #578), event-CPI discriminator exports (#583), encoded discriminator handling, and base58 instruction-metadata serialization (#576).
- Hardened instruction handling for short log events (#582) and invalid CPI stack heights (#584), and sanitized exported Prometheus metric names (#570).
- Updated projects generated by `@sevenlabs-hq/carbon-cli` and `@sevenlabs-hq/carbon-codama-renderer` to depend on Carbon 2 crates.
- Corrected the CLI package's ESM and CommonJS export paths.
- Made the CLI reject GraphQL scaffolds without the Postgres storage layer they require.
- Fixed template lookup through the renderer's CommonJS entry point.
- Raised the npm packages' runtime requirement to Node.js 20.18.0.

> **Note: CLI version number.** `CHANGELOG.md` and the `packages/*/package.json` files at the tag call the companion npm packages 0.13. They were published to npm as **2.0.0** (on 2026-09-10); there is no 0.13 release. Use `@sevenlabs-hq/carbon-cli@2.0.0`.

### Also in this release

- `carbon-token-program-decoder` decodes accounts with the SPL `Pack` layout (#569).
- Yellowstone gRPC datasource: accepts a ping as the first event after a disconnect (#573), backs off between failed subscribe attempts (#581), and keeps the subscription filters on the ping resubscribe (#588).
- `carbon-helius-atlas-ws-datasource` uses the latest published Helius SDK crate (#567).

### Migration from Carbon 1

#### Toolchain and dependencies

Carbon 2 requires Rust 1.96.1. The companion CLI, renderer and version-registry packages require Node.js 20.18.0 or newer. Update every directly declared Carbon crate to `2.0.0` and keep direct Solana dependencies compatible with the Agave 4.2 cohort. Do not mix Carbon 2 with Carbon 1 datasource or decoder crates in one dependency graph.

If Carbon types appear in your public API, update imports and fix type mismatches after resolving the new Solana crates. A single version of each Solana type crate should resolve in the final graph.

#### Transaction source configuration

Caller-owned RPC block configurations must opt into Transaction V1 and request a binary transaction encoding. Carbon cannot reconstruct the original transaction from `Json` or `JsonParsed` payloads.

```rust
RpcBlockSubscribeConfig {
    encoding: Some(UiTransactionEncoding::Base64),
    transaction_details: Some(TransactionDetails::Full),
    max_supported_transaction_version: Some(1),
    ..RpcBlockSubscribeConfig::default()
}
```

Use the equivalent fields on `RpcBlockConfig` when constructing `RpcBlockCrawler`. Carbon's RPC transaction crawler and Helius GTFA datasource set V1 and Base64 internally.

For Helius Atlas transaction subscriptions, override the SDK's JSON-parsed default and keep V1 enabled:

```rust
TransactionSubscribeOptions {
    encoding: Some(UiEnhancedTransactionEncoding::Base64),
    max_supported_transaction_version: Some(1),
    ..TransactionSubscribeOptions::default()
}
```

#### Account deletions

`AccountDeletion` now contains the source's final `account: Account`. Custom datasources should construct an `AccountUpdate` and call `into_update()` so every zero-lamport account is normalized at the same boundary.

The former `account_deletions_tracked` constructor argument and fields were removed. Remove that argument when constructing:

- `HeliusWebsocket`
- `LaserStreamGeyserClient`
- `YellowstoneGrpcGeyserClient`
- `StreamMessageClient`

If you construct `AccountDeletion` directly, provide its new `account` field.

#### Datasource configuration changes

- `LaserStreamClientConfig::new` has a new trailing `from_slot: Option<u64>` argument. Pass `None` to preserve the previous starting behavior, or use `LaserStreamClientConfig::default()` and set the field.
- Yellowstone tonic client reconnection is disabled by default. Opt in with `YellowstoneGrpcClientConfig::with_reconnect(ReconnectConfig)` when the upstream supports replay appropriate for your pipeline.

#### SPL Token account helpers

The `Mint::decode`, `Multisig::decode` and `Token::decode` convenience methods in `carbon-token-program-decoder` were removed. Direct callers should unpack with `solana_program_pack::Pack` and convert the result:

```rust
let token = spl_token_interface::state::Account::unpack(data)
    .ok()
    .map(carbon_token_program_decoder::accounts::token::Token::from);
```

Normal decoding through `TokenProgramDecoder` requires no special migration.

The step-by-step version of this section, with before-and-after code, is [Migrate to 2.0](https://carbon.sevenlabs.org/guides/migrate-to-v2.md).

### Carbon 2 datasource set

The Carbon 2 publication contains these twelve datasource crates:

- Helius Atlas WS, GPA V2, GTFA and LaserStream
- RPC block crawler, block subscribe, GPA, program subscribe and transaction crawler
- Stream message
- Validator snapshot
- Yellowstone gRPC

`carbon-jito-shredstream-grpc-datasource` and `carbon-jetstreamer-datasource` are not published on the 2.0 line. Jito ShredStream does not yet support Transaction V1, while Jetstreamer does not yet support the Solana v4 stack. Existing versions remain available, but they must not be selected as Carbon 2 dependencies.

## 1.0.0 (2026-06-04)

The first stable release: a core refactor with atomic metrics, a filtering redesign, versioned decoders and CPI events, four new datasources, and regenerated decoders. [Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v1.0.0).

### Core

- Atomic metrics (#531) and datasource-level metrics flushing (#538).
- Filtering redesign (#534).
- Versioned decoders (#535).
- Transaction pipe refactor (#537) and performance updates (#532).
- `ArrangeAccounts` refactor (#529).
- CPI events support (#550).
- Transaction index field on `TransactionUpdate` and `TransactionMetadata` (#503).
- Datasource disconnection tracking with gap detection (#471).
- Better identification of writable accounts (#542).
- Precompile offsets handled in instruction log extraction (#526).
- Consistency pass and workspace doc comments (#562).
- Rust toolchain 1.88.0 (#488).

### Datasources

- New: Jetstreamer (#487), RPC GPA (#492), Helius GPA v2 (#493), Helius GTFA (#505) and validator snapshot (#497).
- Helius WebSocket: configurable timeouts and reconnection on idle transaction streams (#507).

### Decoders

- New: Onchain Labs DEX router v1 (#518) and v2 (#519), Jupiter Lend Liquidity (#524).
- Optional base58 serialization for `Pubkey` fields in decoder structs (#514).
- All decoders regenerated with the new generator (#540) and updated from the latest IDLs (#564).
- Fixes: Token-2022 accounts without extensions (#510) and an extensions panic (#513); Pump.fun, PumpSwap and Pump fees decoders (#517, #545), including `OptionBool` deserialization (#555, #556); program ID validation in the Meteora DBC decoder (#516); Meteora DAMM v2 from the latest IDL (#522); Orca Whirlpool event discriminator (#547); Jupiter swap (#490); bugs found in a decoder survey (#565).

### CLI and tooling

- The CLI generates workspace decoders (#539) and passes clippy on generated decoders (#536); CI fixes for generated decoders and indexers (#489).
- Packages: removed an unused `base64` dependency and made `serde-big-array` conditional (#523).
- New examples (#544, #558).
- Corrected path typos in `workspace.dependencies` (#557).

## 0.12.0 (2025-11-27)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.12.0).

- Upgraded the Solana crates (#435, #470, #482) and other outdated dependencies (#474).
- Added the `instruction_decoder_collection_fast` macro (#447).
- Fixed the filter in `decode_log_events()` (#448).
- New decoders: Bonkswap (#450), PancakeSwap (#462), DFlow aggregator v4 (#480), Swig (#483).
- Pump.fun: `create_v2` (#465) and `buy_exact_sol_in` (#467) instructions. PumpSwap: `buy_exact_quote_in` (#466, #473) and the `is_mayhem_mode` pool field (#472).
- Updated Meteora DAMM v2 (#457) and fixed the Token-2022 decoder with extensions (#479).
- CLI: code generation enhancements and fixes (#453, #458, #460), `BorshDeserialize` instead of `CarbonDeserialize` in generated code (#485), README (#451).

## 0.11.0 (2025-10-18)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.11.0).

- New Codama renderer and a new Carbon CLI (#386).
- Helius LaserStream datasource with replay (#433).
- Circle CCTP decoders (#444).
- Raydium AMM v4 `SwapBaseInV2` and `SwapBaseOutV2` (#441); Jupiter swap decoder update (#438).
- Rust tuple structs in IDLs; `track_volume` field type updated in Pump.fun and PumpSwap (#442).
- Attributes required by `instruction_decoder_collection` (#439).
- Associated Token Account decoder accepts empty data as `Create` (#437).

## 0.10.0 (2025-09-16)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.10.0).

- Generic stream message datasource (#372).
- Log events extraction (#413).
- Optional accounts in decoders (#370).
- Account updates carry the transaction signature (#398).
- Configurable Yellowstone gRPC client (#371, #405).
- RPC transaction crawler fix for newly sent transactions (#380).
- New decoders: WaveBreak (#390), Bubblegum (#393), Meteora DBC (#407, #409, #410, #424), Heaven (#414), Meteora Vault (#415), Vertigo (#418), Pump fees (#420).
- Updated decoders: PumpSwap (#385, #389, #430), Pump.fun (#392), Raydium CPMM (#403), Raydium Launchpad (#404), Orca Whirlpool (#406, #408, #416), Token-2022 Scaled UI Amount and Pausable extensions (#382), System program legacy accounts (#401), Metaplex Token Metadata account discriminators (#426).
- CLI: `m` and `d` shortcuts for mainnet-beta and devnet (#431).

## 0.9.1 (2025-07-18)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.9.1).

- The "not all bytes read" warning is now logged at debug level (#361).
- Discriminators are exported from decoders (#374).
- Missing derives on Raydium AMM v4 accounts and other types (#373, #379).
- Boop decoder update (#364).
- Solana crates locked to `~2.2` (#381).

## 0.9.0 (2025-07-06)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.9.0).

- The `Filter` trait and `DatasourceFilter` (#355).
- The pipeline stops automatically after all datasources complete (#317).
- Configurable channel sizes (#340).
- `InstructionMetadata::absolute_path` (#293).
- Raw account and instructions returned to processors (#330).
- Retries in the RPC transaction crawler (#337).
- Configurable Prometheus metrics port (#350).
- Yellowstone gRPC block filters (#324) and pipeline cancellation fix (#348).
- Shutdown clean-up (#341).
- `carbon-fetch-ix` for generating transaction fixtures (#305); fixtures and tests for Raydium CLMM, Meteora DLMM and Orca Whirlpool (#303, #297, #347).
- New decoders: Meteora DAMM v2 (#327, #329), Gavel (#318), Raydium stable swap (#351).
- Decoder updates and fixes: Pump.fun IDL (#301), Meteora DLMM 0.9.0 (#308), Meteora virtual curve events (#328), PumpSwap owner check (#333), Raydium CPMM `PoolState` (#339).
- Linting and a pre-commit hook (#306); examples read `RUST_LOG` from the environment (#335); `rand` no longer required for `solana-signature` and `solana-pubkey` (#310).

## 0.8.1 (2025-05-12)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.8.1).

- GraphQL server (#284).
- Postgres support in the Token program decoder (#229).
- `block_hash` on transactions for fork detection (#278).
- Block-only subscriptions (#283).
- `Serialize` for `DecodedInstruction` (#252); `IntoIter` for `NestedInstructions` (#289).
- Nested instructions parsed inside `TransactionPipe` (#267); faster `extract_instructions_with_metadata` (#273) and gRPC transaction metadata handling (#261).
- Lenient deserialization that warns about remaining bytes (#291).
- Vendored Jito protos (#253); Jito ShredStream deduplication (#257) and `is_vote: false` (#254).
- Fixed a duplicate WebSocket connection (#270), account meta `is_signer` (#274) and the `log` export used by the proc macro (#292).
- New decoders: Meteora virtual curve (#256), Boop (#280), Address Lookup Table (#290).
- Fixes: Jupiter nested empty instructions (#268), Raydium CLMM event discriminators (#294), Meteora DLMM `addLiquidityByStrategy2` (#295), doc tests (#281).
- Pinned rustls 0.23 with the `aws_lc_rs` crypto provider in an example to stop a startup panic (#260).

## 0.8.0 (2025-04-18)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.8.0).

- Jito ShredStream gRPC datasource (#249).
- The RPC transaction crawler discovers newly sent transactions (#248).
- CPI instructions parsed (#221), including the third depth level (#242).
- Direct Solana crates instead of `solana-sdk` (#233, #245).
- `serde` `Serialize` and `Deserialize` derived for generated accounts (#238), with `BigArray` support (#239).
- Raydium Launchpad decoder (#250); Pump.fun account layouts and instructions updated (#236).

## 0.7.0 (2025-03-30)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.7.0).

- Bounded update channel (#219), default size 1,000 (#220).
- RPC block crawler datasource (#222).
- Meteora Pools (#224) and Virtuals (#225) decoders.
- Started the move from `solana-sdk` to direct dependencies (#226).
- Fixes to the Associated Token Account and System program decoders (#204).

## 0.6.2 (2025-03-21)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.6.2).

- Instruction index in instruction metadata (#216).
- PumpSwap decoder (#214).
- Project scaffolding in the CLI (#197).
- Corrected Raydium CLMM program address (#206).

## 0.6.1 (2025-03-17)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.6.1).

- Interactive CLI (#184).
- Datasource cancellation token on the pipeline (#198).
- Program IDs exported from decoders (#203).
- Several new program decoders (#199); Fluxbeam (#192) and Pump.fun (#195) fixes; Jupiter DCA fixtures and tests (#191).

## 0.6.0 (2025-03-12)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.6.0).

- `block_time` on `TransactionUpdate` and `TransactionMetadata` (#181).
- `carbon-test-utils` crate (#167).
- New decoders: Associated Token Account (#178), OKX DEX (#183), Stake program (#186).
- Corrected the Jupiter Limit Order 2 program address (#177).
- A README for every crate (#182), crate keywords and categories (#188), release scripts (#189), Lifinity fixtures and tests (#190).

## 0.5.1 (2025-02-22)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.5.1).

- Decoders pre-filter by program ID (#175) and check accounts (#176).
- New decoders: Zeta (#168), Phoenix v1 (#169), Lifinity AMM v2 (#170), Stabble stable swap (#171) and weighted swap (#172), Raydium liquidity locking (#173).

## 0.5.0 (2025-02-20)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.5.0).

- Performance optimizations (#120).
- Codama IDL parsing in the CLI (#136).
- Dependency fixes, Yellowstone bump and pinned Solana crates (#144, #166).
- New decoders: Moonshot (#148), Name Service (#151), Token-2022 (#150), Raydium CPMM (#154), Jupiter Perpetuals (#158), Fluxbeam (#159), Drift v2 (#160).
- Fixes to slice handling and extra remaining-account fields for swaps (#152).
- Workspace dependencies, lints and formatting (#156).

## 0.4.1 (2025-02-18)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.4.1).

- Reconnection logic for datasources and a cluster option for the Atlas WebSocket datasource (#112).
- Conventional `From` traits for instruction extraction (#114).
- Removed the "pipes against update types" checks (#135).
- Anchor PDA IDL parsing in the CLI (#124).
- New decoders: Raydium CLMM (#118), OpenBook v2 (#140); Sharky decoders moved into the repository (#131).
- Fixes: regenerated Meteora DLMM decoders (#130), Raydium AMM account parsing (#134), mint account in `TransferChecked` (#138).

## 0.4.0 (2024-12-19)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.4.0).

- Access to the transaction's `VersionedMessage` on transaction metadata (#106).

## 0.3.1 (2024-12-02)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.3.1).

- CLI code generation for IDLs of native Rust programs (#101).
- Helius Atlas WebSocket datasource on Helius SDK 0.2.2 (#104).
- Metaplex decoders regenerated with the fixed CLI for non-Anchor programs (#103).

## 0.3.0 (2024-11-21)

[Release notes](https://github.com/sevenlabs-hq/carbon/releases/tag/v0.3.0).

### Added

- A `solana_transaction_status::TransactionStatusMeta` field on `TransactionMetadata` (#92).
- The transaction pipe processor type includes `Vec<DecodedInstruction>` in its instruction type (#95).
- Type aliases for processor input types: `AccountProcessorInputType`, `InstructionProcessorInputType`, `TransactionProcessorInputType` (#95).
- `carbon-kamino-lending-decoder` and `carbon-kamino-vault-decoder`.

### Changed

- Schemas are optional inside transaction pipes (#95).
- Pipeline-level `updates_process_time` is measured in both milliseconds and nanoseconds (#96).
- The RPC block subscribe, RPC program subscribe, Yellowstone gRPC and Helius Atlas WebSocket datasources measure time in nanoseconds instead of milliseconds (#96).
- `ArrangeAccounts::arrange_accounts` is a static function on instruction data structs (#97).

### Fixed

- `carbon-raydium-amm-v4-decoder` uses the right instruction discriminators and size matching for accounts (#82).
- System program decoder structs have public fields (#84).
- Inner instructions are schema-matched (#89).

---

Carbon 2.0.0 documentation · Web: https://carbon.sevenlabs.org/changelog · Last updated 2026-11-09 · Index of all pages: https://carbon.sevenlabs.org/llms.txt