Shardeum’s Innovative Approach to Achieve Atomic and Cross Shard Composability

What is Composability in Blockchain?

Note: We have a separate blog that is dedicated to discussing what composability is especially from a blockchain standpoint. However let’s summarize this here so it will be easier for you to understand and connect the various discussion points we will be discussing in this blog about the significance of atomic and cross shard composability in public blockchain networks.

Composability is a powerful concept in blockchain that enables developers to combine and integrate smart contracts, decentralized applications (dapps), and different networks to create innovative and complex systems. It promotes rapid development, as existing building blocks can be reused, reducing time and effort.

Source | How does composability work?

Although this proposition seems to be a blessing for smart contract platforms, its success depends largely on the robustness of the platform’s underlying protocol layer in terms of security, scalability, communication, storage and DX. While composability looks at how a system works from the inside and how its parts can be rearranged to make new functions, its success also depends on few external factors to reinforce its utility like security audits and thorough testing. This essentially renders ‘composability’ a make or break feature for blockchain platforms.

What is Atomicity in Blockchain?

Atomicity and atomic composability are closely related. Atomicity refers to the property of a transaction or operation being indivisible or all-or-nothing. In other words, it means that a transaction is either executed completely or not executed at all, without any partial or intermediate states. If any part of the transaction fails or encounters an error, the entire transaction is rolled back or aborted, ensuring that the system remains in a consistent state. You would see atomicity as part of the ACID (Atomicity, Consistency, Isolation, Durability) concept in computer and data science.

Source: https://erainnovator.com/acid-properties-in-dbms/

What is Atomic Composability in Blockchain?

Atomic composability refers to the ability to combine and execute multiple operations or transactions in distributed database systems like blockchain and DAGs in a way that they either all succeed or all fail together typically within the boundaries of a single shard or a blockchain. It extends the concept of atomicity to the composability of operations within the blockchain context.

Similar to how blockchain networks employ a consensus mechanism to establish agreement on transaction validity and ordering, blockchains employ specific mechanisms such as messaging, coordinator, and routing protocols to uphold composability within these networks. These are more technical level details which we will learn more in our next sections. At a functional level though, it must be emphasized that blockchains can ensure atomic composability only when they have a pitch perfect design/deployment of their consensus mechanism, smart contracts, scalability, transaction verification and roll back mechanism. That is to say, they must have the fundamentals strong in the form of the protocol and application layer.

Source | What is atomic composability, and how does composability work?

As mentioned earlier, atomic composability is typically achieved within a single partition, cluster, or blockchain. However, ensuring atomic composability becomes more complex when dealing with multiple partitions and states, such as in a sharded network.

Sharding is a way to improve the scalability of distributed systems, like blockchains, by dividing data into smaller, easier-to-manage pieces called shards. Sharding does not necessarily break composability, but it can potentially limit the ability of different system parts to communicate and interact with each other. Sharding introduces complexity to the network architecture and implementation and often requires mechanisms for cross-shard communication and composability. Sharding also can result in uneven distribution of data and transactions across shards due to lack of optimal design and testing, or in the other words, a solid protocol layer. Before understanding how cross-shard composability can be achieved, let’s take a brief look at what cross-sharding in itself means?

Cross-Shard Composability in Blockchain Systems

Source | What is cross-shard composability: Importance of cross-shard composability

In blockchain technology, sharding is a way to divide a network into smaller, easier-to-manage pieces called shards. Each shard contains a subset of the network’s total data and transactions, which enables more efficient processing and higher throughput. By dividing the network this way, nodes only need to process the data in their assigned shard, reducing the computational load and improving overall performance.

Cross-shard composability refers to the ability of different shards in a blockchain network to interact seamlessly. Since each shard operates independently, cross-shard composability requires the implementation of protocols and mechanisms like universal account system, message-passing protocols, or implementing side chains or bridges to connect different shards. This will enable data and asset transfers between shards, as well as maintaining a consistent state across all shards further. By enabling cross-shard composability, blockchain networks can process more transactions and achieve higher throughput without sacrificing security or decentralization.

Key Considerations for Achieving Cross-Shard Composability in Blockchain Networks

Cross-Shard Transactions

Cross-shard transactions involve executing a single transaction that involves multiple shards in a blockchain network. They are essential for achieving scalability and enabling more complex smart contracts. Cross-shard transactions require mechanisms for ensuring atomicity, consistency, isolation, and durability (ACID) across all shards.

The Need for Composability

Composability is crucial for enabling cross-shard composability and ensuring the seamless interaction of smart contracts and transactions across different shards. Without composability, blockchain networks would be limited to operating within their shards, reducing their utility and adoption.

Two-Phase Commit Protocol for Cross-Shard Transactions

The Two-Phase Commit protocol is a mechanism for ensuring atomicity in cross-shard transactions. It involves a coordinator who initiates the transaction and communicates with all involved shards to ensure they are ready to commit. If all shards agree, the transaction is committed; otherwise, it is aborted.

Routing Cross-Shard Transactions

Routing is the mechanism used to direct cross-shard transactions to the appropriate shards. This involves identifying the shards involved in the transaction and determining the optimal route for executing the transaction. This is achieved through routing tables and message-passing protocols enabling cross-shard composability communication between the shards.

Coordinators in Cross-Shard Composability

Coordinators manage cross-shard transactions and ensure their atomicity, consistency, isolation, and durability. They facilitate communication between shards involved in a transaction and are essential for achieving cross-shard composability. Coordinators initiate cross-shard transactions, communicate with all involved shards to ensure they are ready to commit, and perform a two-phase commit protocol to ensure atomicity. Coordinators also maintain state information about the transaction and communicate with other coordinators if the transaction spans multiple blockchain networks.

Keep in mind though, Coordinators can also introduce centralization into the blockchain network, reducing its security and decentralization. They also require additional computational resources and can introduce latency into the transaction processing. Moreover, a single point of failure may emerge if the coordinator fails or is compromised.

Ensuring Composability Between Shards

For ensuring or enabling cross-shard composability in a blockchain network, it is essential to establish protocols and mechanisms for communication and asset transfer between different shards. This involves implementing routing mechanisms to direct transactions to the appropriate shards and creating a universal account system that all shards can access.

Additionally, it requires using message-passing protocols, such as inter-shard communication (ISC), to enable communication between shards. Maintaining a consistent state is also essential to ensure consistency between all involved shards.

Metadata Management in Cross-Shard Transactions

In a cross-shard transaction, metadata refers to information associated with the transaction but not included in the payload. Metadata can include transaction status, routing information, and other data necessary for ensuring atomicity, consistency, isolation, and durability in cross-shard transactions.

Metadata management involves designing protocols and mechanisms for storing, accessing, and updating metadata across all shards. This includes implementing metadata storage systems accessible by all involved shards, creating metadata standards and formats, and ensuring that metadata is protected and secured to prevent unauthorized access or manipulation.

Shardeum Achieves Parallel Transaction Execution with Atomic & Cross-Shard Composability

About Shardeum

Shardeum is a layer 1 blockchain network with an EVM based smart contract platform that scales linearly to maintain low transaction fees permanently, along with high decentralization and security. Transactions are ordered in a time based way on Shardeum (first come first serve basis) to maintain chronological consistency. Transaction verification and consensus are done individually i.e. they are done at the transaction level instead of block level as you see with typical blockchain networks. Shardeum uses a unique/combined consensus mechanism – Proof of Quorum and Proof of Stake.

While we know how Proof of Stake generally works, Proof of Quorum enables Shardeum to generate a receipt showing that a majority of the consensus group has voted for a particular transaction. Each node in the consensus group signs the transaction hash and gossips it to other nodes in the consensus group. Nodes collect these votes, and when the number of votes is more than 50%, these votes form a receipt that can prove consensus on the transaction. Notably, the consensus algorithm on Shardeum also plays a key role in assigning a ‘node ID’ randomly to validator nodes before they join the network. With the help of node IDs, the network will constantly auto-rotate the validator and standby nodes (nodes that are awaiting for their turn to be auto-rotated into the network as active validators) to make it even more difficult for bad actors to take over at any given point in time.

Processed transactions, as such, are grouped together and passed onto archive nodes on the network who are responsible for storing historical data (validator nodes will only store the state of accounts they are involved in across shards, allowing average users to operate a node on Shardeum). And, through dynamic state sharding, Shardeum implements a 3-dimensional sharding approach by sharding State, Network, and Transaction on top of dynamically autoscaling the network capacity up or down relative to the traffic on the network.

Every node on the network will be assigned dynamic account spaces across multiple shards with sufficient levels of overlaps between address ranges. This allows for parallel processing of transactions while maintaining atomic and cross shard composability. Autoscaling will allow Shardeum to make efficient use of resources and respond to demand changes dynamically resulting in immediate finality and low latency while retaining low transaction fees forever. Further, developers can deploy and interact with Solidity or Vyper contracts on Shardeum without special considerations for sharding, since contracts are deployed to unique shards automatically while retaining atomic composability across all shards.

How Does Shardeum Achieve Atomic & Cross-Shard Composability?

While we can research and benefit from the network’s innovation and further development once whitepaper release and protocol open sourcing happens ahead of mainnet in Q3/Q4 of 2023, let’s see how Shardeum is able to retain atomic and cross-shard composability at a high level based on what we have discussed above.

• Shardeum orders and processes transactions on a time-based way individually

• Shardeum’s consensus mechanism involves leaderless and rotated nodes

• Consensus is done at the transaction level (and not at the block level)

• Validators on the network are assigned dynamic account spaces across multiple shards. Although dynamic state sharding requires each node to hold a different address range, there will be a significant overlap between the addresses covered by nodes with an adequate redundancy of at least 128 validators

• Because transactions are processed individually, transactions that affect multiple shards does not risk the possibility of being confirmed in one shard, but getting rolled back in another shard.

• Further transaction level consensus allows a transaction that affects multiple shards to be processed simultaneously by these shards which not only reduces the time to process the transaction even if it affects multiple shards, but also ensures atomic processing with very low latency and immediate finality

• When a multi-step transaction is initiated in the case of complex smart contracts, it involves executing all those steps across multiple shards simultaneously. With Shardeum’s transaction-level consensus, these steps can be grouped into a single atomic transaction to ensure atomic composability

• With the help of dynamic state sharding, autoscaling, optimal consensus mechanism, time based transaction ordering, Shardeum will scale linearly allowing average users to run a node on the network with negligible computational requirements

• Linear scaling and auto-scaling will keep the operational cost of the network sustainable which essentially means transaction fees on the network will remain very low and constant permanently

• Low gas fees enable users and developers to interact with multiple shards and execute cross-shard transactions cost-effectively while promoting increased user participation, which in turn facilitates seamless cross-shard composability

• Every node/shard joining the network will increase the throughput of the network proportionally which allows Shardeum to scale limitlessly preventing congestion and transaction backlog. This directly enhances the composability on the network

• Linear scaling, further, facilitates distributing the workload evenly across shards. By ensuring a balanced distribution of transactions and operations, it prevents any single shard from becoming a bottleneck or experiencing excessive congestion to allow for smooth cross shard composability

• Linear scaling allows inter-shard communication protocols developed to facilitate the secure and reliable transfer of data and state information between shards, enabling seamless coordination for cross-shard composability

• Since Shardeum introduces operational efficiency through linear and auto scaling, it enhances the network’s capacity to handle complex operations and interactions across multiple shards, enabling a more interconnected and interoperable blockchain ecosystem

Conclusion

In conclusion, the importance of cross-shard composability is immense in blockchain systems, enabling scalability and the development of more complex smart contracts. It involves executing transactions across multiple shards and maintaining atomicity, consistency, isolation, and durability (ACID) across all involved shards.

While achieving cross-shard composability requires the implementation of various mechanisms, such as routing protocols, inter-shard communication, universal account systems, and metadata management systems, they are only complementary to the underlying protocol’s efficiency and scalability. Shardeum is uniquely positioned to onboard complex dapps/smart contracts while facilitating cross-chain interactions and interoperability as a result of its path breaking technology. Stay tuned to the project in its final stretch of production readiness.

To stay updated on Shardeum's latest developments, you can join the Telegram channel, follow the Twitter channel, or join the Discord channel. You can also visit the Shardeum blog page to read the latest blog posts and learn more about the project.

Lastly, Shardeum offers community reward programs for developers who contribute to the project. You can visit the Community Reward Programs page to learn more about the available programs.

Also, don't forget to check out the Shardeum Innovations Blog Page to stay tuned to the latest updates from us #ShardeumIsBorderless.