Datasources
Pick the right Carbon datasource for real-time streams, historical backfills or account snapshots, and compare all 14.
A datasource feeds updates (accounts, transactions, deletions, block details) into a Carbon pipeline. Carbon ships 12 datasources for Carbon 2.0 and two legacy ones that only work with Carbon 1.x. This page helps you choose; each datasource has its own page with a runnable example, every option, and its failure modes. For how datasources fit into a pipeline, see Datasources and Updates.
Which datasource should I use?#
| You need | You have | Use |
|---|---|---|
| Real-time transactions and accounts, lowest latency | A Yellowstone gRPC endpoint (own node, Triton, Helius, QuickNode…) | Yellowstone gRPC |
| Real-time, no gaps after reconnects | A Helius plan with LaserStream | Helius LaserStream |
| Real-time transactions over WebSocket | A Helius API key with enhanced WebSockets | Helius enhanced WebSockets |
| Real-time transactions and block details, no gRPC | An RPC node with blockSubscribe enabled (not the public endpoint) | RPC blockSubscribe |
| Real-time account changes for one program, free | Any RPC WebSocket, including the public one | RPC programSubscribe |
| Current state of a program's accounts, once | Any RPC; filters for big programs | RPC getProgramAccounts |
| Current state of a large program, paginated or incremental | A Helius API key | Helius gPA v2 |
| Complete state of a huge program at one slot | Disk, RAM and a snapshot (or a node serving one) | Validator snapshot |
| History of one address, then new transactions | Any RPC, including the public one (slow) | RPC transaction crawler |
| History of one address, fast, oldest first | A Helius API key | Helius getTransactionsForAddress |
| Every transaction in a slot range, or following the tip over HTTP | Any RPC with those blocks | RPC block crawler |
| Long historical ranges, very fast, no provider | A Carbon 1.x application | Jetstreamer (legacy) |
| Transactions before execution, for latency-critical signals | Jito Shredstream access and a Carbon 1.x application | Jito Shredstream (legacy) |
| Updates from your own source (Kafka, files, tests) | Your own producer | Stream message |
On a budget: the RPC datasources work on the free public endpoint, which is enough to learn and prototype but is heavily rate-limited. A common production setup is a snapshot datasource (GPA, gPA v2 or a validator snapshot) for initial state plus a streaming one (Yellowstone or LaserStream) in the same pipeline; see Load account state and Multiple datasources.
Comparison#
Every crate is named carbon-<, version 2.0.0, except the two legacy ones (1.0.0, Carbon 1.x only). "Emits" lists the update variants the datasource actually produces. Where the datasource's update_ declares something different, the page says so; the pipeline doesn't check that list.
| Datasource | Transport | Emits | Live / historical | Finishes? | Paid provider? | Latency |
|---|---|---|---|---|---|---|
| Yellowstone gRPC | Geyser gRPC | Account, AccountDeletion, Transaction | Live | No | Usually | Lowest (streamed as slots are processed) |
| Helius LaserStream | Geyser gRPC (Helius) | Account, AccountDeletion, Transaction | Live + replay from a recent slot | No | Yes | Lowest |
| Helius enhanced WebSockets | WebSocket | Transaction, Account, AccountDeletion | Live | No | Yes | Low |
| RPC blockSubscribe | WebSocket | BlockDetails, Transaction | Live | No | Needs a node with block subscriptions | Low (per block) |
| RPC programSubscribe | WebSocket | Account, AccountDeletion | Live | No | No | Low |
| RPC getProgramAccounts | HTTP, one call | Account | Current state | Yes | No | Batch |
| Helius gPA v2 | HTTP, paginated | Account | Current state | Yes | Yes | Batch |
| Validator snapshot | Snapshot archive | Account | State at the snapshot slot | Yes | No | Batch (minutes to hours) |
| RPC transaction crawler | HTTP polling | Transaction | Historical, then polls | No | No | Seconds (5 s poll by default) |
| Helius getTransactionsForAddress | HTTP, paginated | Transaction | Historical | Yes | Yes | Batch |
| RPC block crawler | HTTP polling | Transaction | Slot range, or follows the tip | With end_slot | No | Seconds |
| Stream message | In-process channel | Account, AccountDeletion, Transaction | Whatever you send | When senders drop | No | Your producer's |
| Jetstreamer (legacy) — Carbon 1.x only | Old Faithful archive | Transaction, BlockDetails | Historical | Yes | No | Batch (high throughput) |
| Jito Shredstream gRPC (legacy) — Carbon 1.x only | gRPC (shredstream-proxy) | Transaction, without metadata | Live, pre-execution | When the stream ends | Jito access | Lowest (before execution) |
Things every datasource page covers#
- Encoding. Carbon rebuilds transactions and accounts from binary payloads. Every RPC and WebSocket datasource that lets you choose an encoding needs Base64; JSON encodings fail to decode.
- Backpressure. Most streaming datasources use
try_sendand drop updates when the pipeline's channel (channel_, default 1000) is full. The pages say which ones, and the log line to look for.buffer_ size - Finishing. A pipeline stops when every datasource has finished. Batch datasources finish on their own; streams run until cancelled. See Shutdown.
- Metrics. Each datasource registers its own counters and histograms next to the pipeline's; see Metrics and the metrics reference.
Need something else? Write your own with the custom datasource guide, or push updates in through Stream message.