What is Sharding?
Sharding is a very important concept that helps the system to keep data in different resources according to the sharding process. The word “Shard” means “a small part of a whole“. Sharding means dividing a larger part into smaller parts. In DBMS, Sharding is a type of database partitioning in which a large Database is divided or partitioned into smaller data and different nodes. These shards are not only smaller, but also faster and hence easily manageable.
Illustration: Now let us consider two scenarios where there is no sharding and in other, we will be having simple sharding via medias to understand it better as follows:
- Case 1: No Sharding
No sharding
- Case 2: Simple Sharding
Simple Sharding
Need for Sharding
Consider a very large database whose sharding has not been done. For example, let’s take a Database of a college in which all the student records (present and past) in the whole college are maintained in a single database. So, it would contain a very large number of data, say 100, 000 records. Now when we need to find a student from this Database, each time around 100, 000 transactions have to be done to find the student, which is very very costly.
Sharding Of A Database
Now consider the same college students’ records, divided into smaller data shards based on years. Now each data shard will have around 1000-5000 students’ records only. So not only the database became much more manageable, but also the transaction cost each time also reduces by a huge factor, which is achieved by Sharding. Hence this is why Sharding is needed.
Features of sharding:
- Sharding makes the Database smaller
- Sharding makes the Database faster
- Sharding makes the Database much more easily manageable
- Sharding can be a complex operation sometimes
- Sharding reduces the transaction cost of the Database
- Each shard reads and writes its own data.
- Many NoSQL databases offer auto-sharding.
- Failure of one shard doesn’t affect the data processing of other shards.
Complete Reference to Databases in Designing Systems – Learn System Design
Previous Parts of this System Design Tutorial
Contact Us