What is Sharding in Blockchain?

An obstacle that has always been there for blockchain (the much-hyped distributed ledger technology) is Scalability. This is because as the blockchain grows, so does the data stored in each computer in the peer-to-peer network. It is a critical issue as blockchain has outgrown its cryptocurrency roots and is now poised to reshape the supply chain, parts of the fintech world, real estate, and a host of other industries. To achieve its full potential, blockchain has to grow exponentially without becoming too slow or bogging the computers on which it runs. That means making it scalable. Sharding is one of the several methods being tested by start-ups, developers, and current blockchain platforms such as Ethereum to see if developers can overcome the scalability roadblock.

Consensus Protocol

One of the main problems with public blockchains involves something called consensus protocol, which is based on proof of work. This is what underlies how transactions are authenticated. A majority of the blockchain users agree that the proposed transactions are authentic and can be added to the chain. In other words, there has to be a consensus. However, the consensus algorithms used by the likes of Bitcoin and Ethereum payment networks are highly compute-intensive. They use a lot of CPU cycles.

Further complicating things, proof of work-based blockchains require each authenticating computer or node to record all the chain data because it is part of the consensus process. But as more transactions occur and the blockchain grows, more computing cycles are needed, and everything slows down. How slow? Bitcoin can process only 3.3 to 7 transactions per second, and a single transaction can take at least ten minutes to complete. Ethereum is a little faster – it can process from 12 to 30 transactions per second. But that’s nothing compared to Visa’s 50-year-old electronic payment network, VisaNet. It processes around 1700 transactions a second. To compete with VisaNet and other conventional payment networks in terms of scalability and performance, blockchain needs turbocharging. That’s where sharding comes in.

How does sharding work?

Sharding has been around for a while and was originally designed for horizontal data partitioning. It is the process of spreading the computing and storage workload from a blockchain network so that each node no longer has to process the entire network’s transactional load. Each node only maintains the information related to its specific partition or shard. Sharding allows the blockchain to remain decentralized and secure – two things that make it so popular. The information in a shard can still be shared, and everyone can see all the ledger entries, but every node is freed from recording and storing all the data on every other node. That allows data to be stored more quickly and easier to find because its location is mapped on the blockchain. Because fewer nodes now see and process the transactions, more transactions can be processed in parallel.

Hurdles Sharding has to overcome:

While sharding could be the key to allowing blockchains to scale securely, some hurdles remain. For one, if you are going to maintain blockchain security, you have to guard against shard takeovers. Corrupting a node in the given shard will lead to permanent loss of data. Ethereum tackles this issue by randomly assigning a node to a shard and then randomly reassigning those nodes to other shards. A second challenge involves thin clients, also called SPV (Simplified Payment Verification wallets). Ensuring that the nodes have a full picture of the blockchain’s current state while divided amongst shards. To address that issue, Thin Clients communicate via separate networks and maintain local state copies of each shard.

And finally, it is important to know that interns hard communication, while good for security, still poses a challenge because each shard appears as a separate blockchain network.

So you can say that while sharding has the potential to eliminate a lot of the scaling problems that blockchain has, it still very much in the development and testing phase, pretty much like the technology itself.

https://www.youtube.com/watch?v=WI6pE5nFeFI