RPC getProgramAccounts

GpaDatasource calls getProgramAccounts once for a program, emits every returned account as an account update at the response's slot, and finishes. Use it to load the current state of a program's accounts at startup, usually with filters that select one account type, and pair it with a streaming datasource to keep that state up to date.

Quick example#

This program loads every Orca Whirlpool FeeTier account from the public RPC. A DataSize(44) filter keeps the request small.

cargo new gpa && cd gpa
cargo add carbon-core@2.0.0 carbon-log-metrics@2.0.0 carbon-orca-whirlpool-decoder@2.0.0 carbon-rpc-gpa-datasource@2.0.0
cargo add solana-client@=4.2.2
cargo add tokio@1 --features full
cargo add env_logger@0.11 log@0.4

use {
    carbon_core::{
        account::AccountProcessorInputType, error::CarbonResult, pipeline::Pipeline,
        processor::Processor,
    },
    carbon_log_metrics::LogMetrics,
    carbon_orca_whirlpool_decoder::{
        accounts::OrcaWhirlpoolAccount, OrcaWhirlpoolDecoder, PROGRAM_ID,
    },
    carbon_rpc_gpa_datasource::GpaDatasource,
    solana_client::{
        rpc_config::{RpcAccountInfoConfig, RpcProgramAccountsConfig},
        rpc_filter::RpcFilterType,
    },
    std::sync::Arc,
};

#[tokio::main]
async fn main() -> CarbonResult<()> {
    env_logger::init();

    // Only Orca `FeeTier` accounts: 8-byte discriminator + 32 + 2 + 2 = 44 bytes.
    let config = RpcProgramAccountsConfig {
        filters: Some(vec![RpcFilterType::DataSize(44)]),
        account_config: RpcAccountInfoConfig::default(),
        ..Default::default()
    };

    let datasource = GpaDatasource::new_with_config(
        "https://api.mainnet-beta.solana.com".to_string(),
        PROGRAM_ID,
        config,
    );

    Pipeline::builder()
        .datasource(datasource)
        .metrics(Arc::new(LogMetrics::new()))
        .account(OrcaWhirlpoolDecoder, FeeTierLogger)
        .build()?
        .run()
        .await
}

struct FeeTierLogger;

impl Processor<AccountProcessorInputType<'_, OrcaWhirlpoolAccount>> for FeeTierLogger {
    async fn process(
        &mut self,
        input: &AccountProcessorInputType<'_, OrcaWhirlpoolAccount>,
    ) -> CarbonResult<()> {
        if let OrcaWhirlpoolAccount::FeeTier(fee_tier) = &input.decoded_account.data {
            println!(
                "{} tick_spacing={} default_fee_rate={}",
                input.metadata.pubkey, fee_tier.tick_spacing, fee_tier.default_fee_rate
            );
        }
        Ok(())
    }
}

RUST_LOG=info cargo run

The program prints one line per fee tier (825 when this page was written) and exits about a second later:

1XuAqKHeX1in2pJubEA5yUkyqJj2mDCt9GQdUqVg9iN tick_spacing=8 default_fee_rate=10
12UZ2XU7Lv1aWHgu6DMsymhXEdBHQEuLaB9fPCatGo4Y tick_spacing=32896 default_fee_rate=3000
143Eu77adnubqv8j68fTDDYaRweJtrLU3zvB9pyudTeR tick_spacing=32896 default_fee_rate=3000
14HcGyeg7VqSundRvpi4Xj6GeZxWVqxeTjdi3pibFtdg tick_spacing=8 default_fee_rate=50
...
[INFO  carbon_rpc_gpa_datasource] Fetched 825 accounts for program whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc (slot: 467112536)
[INFO  carbon_core::pipeline] update_receiver closed, shutting down.
[INFO  carbon_core::pipeline] pipeline shutdown complete.

Configuration#

Constructors#

pub fn new(rpc_url: String, program_id: Pubkey) -> Self
pub const fn new_with_config(rpc_url: String, program_id: Pubkey, config: RpcProgramAccountsConfig) -> Self

new uses RpcProgramAccountsConfig::default(): no filters, every account of the program. The struct fields rpc_url, program_id and config are public.

RpcProgramAccountsConfig#

The type comes from solana_client::rpc_config (solana-client 4.2.2). Filters come from solana_client::rpc_filter.

Behaviour#

• One request. The datasource sends a single getProgramAccounts and holds the whole response in memory before emitting anything. Large programs need filters, a generous RPC, or Helius gPA v2, which pages.
• Slot. Every update gets the response's context slot and transaction_signature: None.
• Backpressure. Updates are sent with send().await, so nothing is dropped when the pipeline is busy.
• Finishing. consume returns after the last account. When this is the pipeline's only datasource, the pipeline logs update_receiver closed, shutting down. and run() returns. When it runs next to a streaming datasource, the pipeline keeps running. See shutdown.
• Errors. There are no retries. Any RPC or transport error returns FailedToConsumeDatasource("Failed to fetch program accounts"); the original error is not included.
• Cancellation. Checked between accounts.
• Metrics. rpc_gpa_accounts_processed_total (counter), rpc_gpa_fetch_duration_milliseconds (histogram).

Troubleshooting#

Failed to fetch program accounts#

The datasource hides the RPC's error message. Send the same request with curl to see it:

curl -s https://api.mainnet-beta.solana.com -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"getProgramAccounts","params":["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc",{"encoding":"base64","filters":[{"dataSize":44}]}]}'

Common causes: the program has too many accounts for the node (add DataSize or Memcmp filters), the public endpoint's data allowance is used up ("You have used your data allowance", HTTP 413: wait or use your own RPC), or the request timed out.

The processor receives no accounts#

The decoder returned None for every account. Check that your filters select an account type the decoder knows, and that you did not set data_slice.

The pipeline exits right after loading#

Expected when this is the only datasource. Add a streaming datasource such as RpcProgramSubscribe to keep running; see Load account state.

See also#

• Load account state
• Helius gPA v2: paginated, with changedSinceSlot.
• Validator snapshot: every account of a program from a snapshot file.
• Datasources concept and update types