SVM Series 04: TransactionBatchProcessor

What is TransactionBatchProcessor?

The TransactionBatchProcessor handles validation and execution of sanitized batched transactions within the SVM acting as the central interface for the whole transaction execution pipeline.

// TransactionBatchProcessor struct as declared in SVM code
pub struct TransactionBatchProcessor<FG: solana_program_runtime::loaded_programs::ForkGraph> {
    /// Bank slot (i.e. block)
    slot: Slot,

    /// Bank epoch
    epoch: Epoch,

    /// SysvarCache is a collection of system variables that are
    /// accessible from on chain programs. It is passed to SVM from
    /// client code (e.g. Bank) and forwarded to process_message.
    sysvar_cache: RwLock<SysvarCache>,

    /// Programs required for transaction batch processing
    pub program_cache: Arc<RwLock<ProgramCache<FG>>>,

    /// Builtin program ids
    pub builtin_program_ids: RwLock<HashSet<Pubkey>>,

    execution_cost: SVMTransactionExecutionCost,
}

Fields of TransactionBatchProcessor struct

1. slot - a fixed timeframe when a validator can produce a block (can be as low as 150ms for alpenglow consensus). It’s Rust data type is a u64.

2. epoch - the timeframe when a leader schedule is valid divided into slots. The leader schedule is the mapping of validator public keys to each slot within the given epoch. It’s Rust data type is also a u64.

3. sysvar_cache - defined by the Rust type SysvarCache, it contains system variables like the Clock (network time), Epoch Schedule, Epoch Rewards, Rent, Stake History and Last Network Restart Slot.

4. program_cache - defined by the Rust type ProgramCache, it contains loaded, compiled and verified programs required for transaction batch processing, optimizing shared data across validator forks. It tracks program information such as usage stats, verification status, and handles deployments, evictions, and tombstones for unloadable programs. Executable programs that are frequently used are kept in memory to improve execution speed through cooperative batch loading and eager preloading. The program cache isn’t saved to disk, so it must be rebuilt after each SVM restart.

5. builtin_program_ids - It contains builtin program IDs like BPF Loader that own and execute smart contract programs and program configs like the supported SBF versions (all SBF versions are enabled by default).

6. execution_cost - configures the CUs used by various operations like how much computing a SHA256 hash costs, cost of logging (msg!()), CPI, verifying Ed25519 signatures, CUs used for additional heap allocations above the default and more.

Processing batched sanitized transactions

The method on TransactionBatchProcessor for processing sanitized transaction batches is TransactionBatchProcessor:: load_and_execute_sanitized_transactions. It acts as the entry point to the SVM and takes the following parameters:

1. callbacks: This parameter allows the transaction processor to load information about accounts required for transaction execution. It requires the value supplied to this argument to implement the TransactionProcessingCallback trait which is a supertrait of InvokeContextCallback trait (the callback used by solana_program_runtime::invoke_context::InvokeContext in SVM to manage the main pipeline from runtime to program execution). By defining this trait developers can maintain full control of their application while ensuring integration with SVM according to the specification. This trait has three methods:

   • get_account_shared_data() which takes &self, a public key (the address to fetch information for) as parameters and returns an optional AccountSharedData struct which is the in-memory representation of an Account. An Option::None can be returned if the account does not exist.

   • account_matches_owner which takes &self, a public key and a slice of public keys (&[Pubkey]) representing potential owners of an account. The goal of this method is to get the index of the owner of the public key within the slice of owners &[Pubkey], therefore, it returns an Option of type usize. An Option::None can be returned if the account public key does not exist in the Bank or if the lamports balance is zero.

   • add_builtin_account which takes a &self, name of type &str and a program_id public key. This method is used to add a program to the available programs in the execution pipeline, for example adding the native loader to the bank, that would load a smart contract program into the execution pipeline.

2. sanitized_txs: This parameter contains a slice of sanitized Solana transactions (&[impl SVMTransaction]). A sanitized Solana transaction can be any Rust type as long as it implements the SVMTransaction trait which is a supertrait of SVMMessage. These traits allow any type implementing them to provide convenience methods that can fetch information such as the fee payer, the first signature, all the signatures in a transaction, number of transaction signatures, the recent blockhash of the transaction, the number of write locks, the account keys and more. An example of a type implementing this trait is SanitizedTransaction.

3. check_results - this parameter contains a list of the results of checking the validity of a transaction. Some checks performed that produce these results include the validity of a blockhash or nonce, whether an account matches it’s owner and whether the number of sanitized transactions equals the number of transactions that underwent these checks.

4. environment - This parameter is the runtime environment for transaction batch processing instantiated by TransactionProcessingEnvironment struct. It configures the blockhash to use for the current transaction batch, determine rent due for the account and collect rent, the runtime features to use, lamports per signature for nonce accounts and the total stake for the current epoch.

5. config - this parameter configures customization of transaction processing behavior using TransactionProcessingConfig struct. Custom configurations include account overrides, log message byte size limits, configuring of the recording capabilities for transaction execution, whether to limit the number of programs loaded for the transaction batch and whether or not to check a program’s modification slot when replenishing a program cache instance.

The load_and_execute_sanitized_transactions returns a LoadAndExecuteSanitizedTransactionsOutput struct which contains the fields:

1. error_metrics - the error metrics for the processed transactions

2. execute_timings - timings for transaction batch execution

3. processing_results - a vector containing type alias TransactionProcessingResult indicating whether a transaction executed successfully or the transaction execution failed and needs to be rolled back

4. balance_collector - a Option<BalanceCollector> containing balances accumulated for TransactionStatusSender (a channel within the Solana ledger) when transaction balance recording is enabled.

ForkGraph Trait

The struct TransactionBatchProcessor contains a generic FG which requires the passed type to implement the ForkGraph trait. The trait maps the block’s relationship between two slots returning the BlockRelation enum.

// Relationship between two fork IDs
pub enum BlockRelation {
    // The slot is on the same fork and is an ancestor of the other slot
    Ancestor,
    // The two slots are equal and are on the same fork
    Equal,
    // The slot is on the same fork and is a descendant of the other slot
    Descendant,
    // The slots are on two different forks and may have had a common ancestor at some point
    Unrelated,
    // Either one or both of the slots are either older than the latest root, or are in future
    Unknown,
}

The ForkGraph trait requires the relationship(&self, a: Slot, b: Slot) method to be implemented for the generic type. This method requires the two slots arguments a: Slot and b: Slot to be compared in a manner that returns the relationship (BlockRelation enum) between them. The Slot is just a type alias for a Rust u64 representing an entry that estimates wallclock duration (a tick) . An example of implementing this trait:

# Add `solana-clock` dependency used in SVM ecosystem, it contains the `Slot` type
cargo add solana-clock

pub struct SlotRelationships;

impl ForkGraph for SlotRelationships {
    fn relationship(&self, slot_a: Slot, slot_b: Slot) -> BlockRelation {
        // Since a `Slot` is just a Rust `u64`, and a `u64` implement
        // Rust `PartialCmp` and `Cmp`, we can use `.cmp()` method
        // on `slot_a` to compare against `slot_b` and derive a relationship
        // checking if `slot_a` is less than, equal to or greater than `slot_b`
        match slot_a.cmp(&slot_b) {
            std::cmp::Ordering::Less => BlockRelation::Ancestor,
            std::cmp::Ordering::Equal => BlockRelation::Equal,
            std::cmp::Ordering::Greater => BlockRelation::Descendant,
        }
    }
}

The TransactionBatchProcessor has some more methods to configure of get information about the transaction execution pipeline that you can explore.

< Previous Article: SVM Series 03: Compute Budget & Syscalls

Resources

1. Understanding Slots, Blocks, and Epochs on Solana
   https://www.helius.dev/blog/solana-slots-blocks-and-epochs

2. Solana SVM TransactionBatchProcessor module docs
   https://docs.rs/solana-svm/latest/solana_svm/transaction_processor/struct.TransactionBatchProcessor.html