Time-Based Transaction Ordering: How Shardeum Ensures Chronological Consistency and Fairness

What is Fairness in Blockchain?

Fairness in blockchain refers to the principle that all participants in a blockchain-based system should have an equal opportunity to participate in the transaction processing, consensus process, governance, resource allocation, and receiving rewards for their contributions. It ensures the network is secure and resistant to attacks, promoting the credibility and legitimacy of the blockchain system, which is crucial for its adoption and success. We will take a detailed look at the topic that is often ignored by many – the time-based transaction ordering process – which is the entry point for issues like MEV (Miner Extractable Value) and unfairness in the blockchain industry at large.

Achieving fairness in a blockchain is not a straightforward task. Challenges such as limited scalability, high latency, weak consensus algorithms, Sybil attacks, and economic incentives can create power imbalances and give some participants an unfair advantage. At the same time, we need to get the context right here because front running, for instance, was an unintended consequence of blockchain's transparency and self-imposed limits to keep a public blockchain network secure and decentralized simultaneously.

Compare this to the enforcement of car seat belt regulations and issuing tickets for non-compliance while driving. While most law enforcement officers ticket individuals for simply not following the rules, a few officers may see it as a chance to gain from the system designed to reward them for enforcing the law. While their motivation may not always be perfect, it's clear that this offers an incentive to safeguard the lives of drivers and passengers and encourage compliance with the rules. A new technology may face obstacles on its path to broader usage, and until it advances through research and development, certain compromises must be taken into account and put into practice.

The Development of Blockchain Technology

Blockchain as a Decentralized Payment System

It's important to stress that blockchain networks, which were introduced in the aftermath of the 2008 financial crisis, were never designed to meet the diverse needs of every individual on the planet. Bitcoin is rightfully regarded as the premier peer-to-peer payment network with the highest levels of security, transparency, and decentralization, all while preserving user privacy. It subsequently revolutionized the efficiency of the industry in terms of cross-border payments, making them significantly cheaper and faster compared to Web2 platforms. However, it's worth noting that Bitcoin can only process up to 10 transactions per second (TPS), whereas Web2 platforms like Paypal and Twitter can handle over 10,000 TPS. A capacity of 10 TPS was sufficient back when public blockchains were rarely used before a significant shift occurred in the middle of the last decade.

Greater practical applications in the real world result in higher levels of acceptance.

Global adoption will only occur when a technology serves multiple purposes. If AI were used solely for language translation, it would have been limited to translation services with limited acceptance. However, today, AI powers various applications such as Siri, chatbots, voice recognition, cybersecurity, IoT devices, exchange order books, fraud detection, and more. Although blockchain is still a relatively new industry, one of the earliest signs of its potential popularity emerged with the introduction of smart contracts by Ethereum and lighter consensus algorithms in 2016. Apart from functioning as a payment network, smart contracts have enabled numerous uses for blockchains across different industries.

Blockchains can now link their data with external networks, enabling the creation of products and services without intermediaries. Public blockchains have demonstrated their ability to perform a wide range of functions in a Web2 world, all while maintaining transparency. Similar to how apps are developed on Android or iOS, anyone can create decentralized applications dapps on layer 1 blockchains like Shardeum. Unlike traditional Web2 entities, every transaction is openly recorded on a public network's digital ledger. This has motivated industries such as finance, supply chain, retail, consumer goods, manufacturing, and more to adopt blockchains to varying degrees. DappRadar, a global app store for dapps, reported that as of May 2023, there are over 12,000 dapps with a Total Value Locked (TVL) of approximately $55 billion, despite facing several downturns and negative perceptions within the industry.

Why is it difficult for blockchains to achieve high fairness?

The Trilemma of Blockchain Transparency and Scalability

The Scalability trilemma refers to the public blockchain's inability to be scalable, secure, and decentralized all at the same time. Blockchain pioneers primarily focused on security and decentralization, imposing self-imposed scaling limits. Eventually, blockchain networks couldn't keep up with the adoption rate over the last 4+ years. Users encountered a persistent problem of network latency, delayed/failed transactions, and outages due to network congestion caused by a backlog of transactions submitted by users. When demand on the network becomes too high, average transaction fees rapidly increase. Maintaining accurate time-based ordering becomes challenging as the network grows with limited scalability, especially when processing a large volume of transactions simultaneously.

Another unwelcome consequence of blockchain's efficiency lies in its transparency, as mentioned earlier. Public blockchains cryptographically record every transaction on a public ledger validated by unrelated network participants (or nodes/validators) distributed worldwide. While user information is encrypted for security, block explorers, similar to search engines, can be used to track and identify the status of a transaction from its initiation by a user until the network confirms and processes the transaction at each stage. Essentially, this information is open to anyone, including users, data analytics firms, law enforcement, and, of course, the validators.

Validators Making Money from Manually Arranging Transactions

Most PoS networks typically follow a somewhat similar process to validate and process transactions. When a user submits a transaction, it lands in the network's transaction or mempool. The mempool acts as a storage place for unconfirmed transactions within blockchain networks, waiting for validation from node validators. Once chosen, individual validators verify these transactions before they are included in a block. As the block nears its maximum capacity, it gets added to the network chain. The validated transactions within the block are then shared with other validators on the network to reach a majority consensus, a critical step in ensuring the security of public networks. Once a majority consensus is achieved for a block, the transactions within it are confirmed, and the block becomes a permanent part of the network chain, preserving its integrity and immutability. It's worth noting that a consensus mechanism also plays a significant role in the transaction ordering process in blockchain networks, and we'll delve deeper into that in the following paragraphs.

Source | Equity in Blockchain

As you can see here, validators play a significant role in the operations of a public blockchain network. You might expect that transactions are selected and processed on a "first-come, first-served" (FCFS) basis ideally. However, due to the transparent nature of blockchains and the scalability issues discussed in the preceding paragraphs, transactions in the mempool compete for limited block space, and validators prioritize them based on factors such as transaction fees, transaction size, and network congestion. Typically, transactions that offer higher fees have a better chance of being included in the next block by validators, encouraging users to attach higher fees to their transactions for quicker confirmation.

In this context, validators give preference to higher-value or more profitable transactions while including lower-value or smaller transactions to meet the block space limit and achieve consensus and block confirmation swiftly. The ordering of transactions is carried out manually, rather than through an autonomous or independent process, to maintain high fairness, free from any manipulation or bias.

Network Latency

Significant or irregular network latency can create discrepancies in the perceived order of transactions, even when the network uses a clock synchronization protocol. Even the newer sharded blockchains face challenges with inconsistent latency issues, especially during periods of high demand. This is because they reach consensus at the block level, which often makes it difficult to process transactions in parallel.

During a demand spike, these blockchains typically rely on a predefined group of shards within the network. Transactions on such sharded platforms are either processed one after the other once a minimum number of nodes join the network to create a new shard, or they are processed after a waiting period for the new shard validators to synchronize with the latest state of the network. This leads to increased network latency, which directly affects the finality of transactions.

When the time required to confirm transactions and guarantee their irreversibility varies depending on the network's demand, it becomes challenging to maximize the efficiency of a timestamp-based ordering protocol. The significant delay can also potentially enable malicious individuals, in the meantime, to initiate hard forks, leading to multiple unsuccessful or delayed transactions. Users would then need to resubmit these transactions, disrupting the fairness of the blockchain system.

Absence of Atomic Composability & Immediate Finality

Some transactions may rely on other transactions, such as in the case of interactions with smart contracts, token transfers, or chain reorganizations in the form of hard forks. This reorganization occurs as a consequence of deterministic finality taking an extended period to confirm transactions, as discussed earlier. Blockchains are increasingly facing challenges in achieving seamless and simultaneous execution of transactions across different parts of the network.

Without seamless execution, transactions may potentially fail or leave the blockchain in an inconsistent state, leading to security risks and decreased reliability. Moreover, without cross-shard communication, transactions will be unable to access and use data and state information from various shards, limiting the execution of complex transactions and smart contracts in a partitioned environment. In essence, maintaining a correct time-based order on the network is impossible without ensuring that dependencies are correctly synchronized across shards.

Principal Node Election

The selection of primary nodes in consensus mechanisms, such as Proof of Stake (PoS) or Delegated Proof of Stake (DPoS), which are employed by several blockchain networks today, is also susceptible to issues like MEV crises, front-running, and sandwich attacks. In a consensus mechanism based on primary node elections, where a single node or a small group of nodes are chosen as validators or leaders for creating blocks, there exists a risk that these nodes may gain an advantage in extracting MEV. They might be able to prioritize their own transactions or engage in other manipulative practices, leading to unfair economic advantages. What's even worse is that they could become targets for bot attacks and takeover attempts by malicious actors, which could disrupt the chronological order of transaction processing.

Furthermore, apart from the MEV-related risks, if a consensus mechanism relies on the authority of primary nodes to validate transactions and establish a consistent order, the efficiency and speed of these specific nodes in reaching consensus can affect the accuracy of time-based ordering. The election of primary nodes can also impact the network's stability. If the primary node is not elected or replaced promptly, it can result in disruptions to the network or delays in processing transactions, which would distort the accuracy of time-based ordering.

Block Level Consensus

While consensus algorithms, or mechanisms, primarily facilitate agreement among validators for approved transactions, they also significantly impact how transactions are organized on a blockchain network. For instance, conducting consensus at the block level introduces complications when determining the actual timestamp of transactions. Many blockchains have predefined limits on block sizes to ensure network scalability and performance. When the number of transactions exceeds this limit, choosing and including transactions in a block becomes challenging. This process is often influenced by prioritization mechanisms employed by networks, which prioritize transactions based on criteria other than their chronological order. This is done to prevent network congestion and disruptions.

Furthermore, conducting consensus at the block level often leads to transactions being processed sequentially, unless blockchain networks scale vertically. However, both sequential processing and vertical scalability can result in slower processing speeds and stability issues, which can hinder the chronological ordering of transactions within a block.

Effect of Layer 2 Scaling Solutions and Rollups

Layer 2 blockchains are brought in as a way to deal with scalability issues on top of L1 platforms to some extent. L2 solutions, which include rollups, aim to reduce the strain on the main blockchain by handling a significant portion of transactions off-chain. Typically, these solutions group multiple transactions together into one batch and then send the results of that batch to the main chain. It's important to note that the final order of transactions on the main chain is often determined by the timestamp when the rollups or layer 2 solution submits them!

While transactions within the layer 2 solution might get confirmed or settled faster, they may take some time to get included in the main chain, which can affect the precise chronological order of transactions on the main chain, especially if there are conflicts or changes in order during the submission process. Additionally, we need to consider that the communication between the layer 2 solution and the main chain can introduce delays that impact the chronological order unless the network achieves atomic composability.

How Does Shardeum Attain High Fairness?

Bitcoin brought about decentralization. Ethereum expanded the decentralized economy. Alongside other recently launched blockchains and utilities, the industry reached a value of over $2 trillion at its all-time high (nearly equal to the largest publicly traded Web2 company – Apple – in terms of market capitalization).

Shardeum aims to make Web3 and decentralization widely accepted by solving the scalability trilemma and maintaining consistently low gas fees indefinitely. The network's inherent design includes innovative mechanisms to tackle both unfairness and the MEV crisis that is common in blockchain networks.

Dynamic State Sharding & Consensus at Transaction Level

Shardeum will employ a time-based ordering of transactions before network validators validate and achieve consensus on each of them. The L1 smart contract platform will expand proportionally through dynamic state sharding. It will divide its state by evenly distributing computing tasks, storage, and data transmission capacity among all the nodes, while also automatically adjusting the number and size of these partitions based on the prevailing workload.

It's also crucial to note that on Shardeum, processing and consensus occur at the transaction level rather than the block level, allowing for parallel processing of transactions across shards while preserving the ability to combine them atomically and across different shards. Shardeum will ensure that complex transactions and smart contracts can be executed effectively in a sharded environment while upholding the blockchain's consistency. Once the transactions are individually validated and processed across shards, they will be grouped without any restrictions on the size or limit of these groups or partitions. These grouped partitions will be sent to archive nodes on Shardeum, which are responsible for storing transaction history.

Linear Scalability & Immediate Finality

Every time a new node joins the network, especially during periods of high demand, it will immediately create partitions (shards), increasing its processing capacity in direct proportion. For example, if one node can handle one transaction per second (TPS), then 1000 nodes can handle 1000 TPS, and so forth. Because Shardeum scales in a linear fashion, transaction fees on the network will always be very low and predictable. Consequently, network transactions will be finalized instantly, and the network's latency, which is consistently 0.2 seconds across the entire network, will remain unchanged, with a 0% chance of reversing a transaction once it's been finalized.

In addition to its linear scalability, Shardeum's reliability in terms of compatibility and instant finalization will eliminate delays and challenges associated with Miner Extractable Value (MEV) when using a time-based ordering protocol. This will also eliminate the need to exclusively develop Layer 2 solutions for scalability while maintaining the network's time-based ordering system independently from the decentralized applications (dApps) built on it.

Consensus Algorithm Preventing Sybil Attacks

The consensus algorithms implemented in Shardeum will significantly contribute to achieving a fair and secure network. Shardeum utilizes Proof of Quorum (PoQ) and Proof of Stake (PoS) consensus algorithms to process transactions on the network. PoQ allows for the trustless and leaderless collection of votes following validations and subsequent consensus on individual transactions. PoS will require validators to stake a minimum amount of network coins to participate in the consensus process, and misbehaving validators will face penalties. Importantly, the consensus algorithm will periodically rotate validator and standby nodes randomly to enhance security, regardless of the network's load.

Achieving High Fairness

The efficiency and outcomes resulting from dynamic state sharding, automatic scaling, and the consensus mechanism on the network will enable Shardeum to maintain independent and autonomous clock synchronization. This, in turn, will enable the network to process transactions on a "first-come, first-served" (FCFS) basis. No transaction will receive preferential treatment or be prioritized over others based on the sender's identity, wealth, or any other discriminatory criteria. Every transaction is treated equally and processed in the order it enters the transaction pool. As you can imagine, a time-based ordering system can only be effective when the network scales linearly, incorporates a robust consensus algorithm and offers atomic composability.