Why Uber created their Schemaless DB and Why you should too.

Not so long ago, Uber created their Database and they named it Schemaless DB. It is not every day that organizations choose to reinvent the wheel. This is an interesting trend in the industry. I found it fascinating and below are reasons why they opt to do that.

See Uber wanted a highly-available write operations database. This was a difficult feat to achieve as all the writes were directed to the leader node. You can't ensure high availability if the leader node goes down.

Their tremendous success was attributed to a technique called Buffered Writes. This means that every write request is stored on at least two nodes- Primary Leader and Secondary Leader. Below is how it works:

• The client requests the request handler.

• The request handler sends the write requests to the Secondary Leader. The data is stored in a special Buffer Table on the secondary leader.

• Then, it also sends the write request to the primary Leader. Only if both writes are successful then the client receives a successful write confirmation.

• The primary leader's job is to replicate the data.

• But if the leader goes down before the async replication is successful, the secondary Leader serves as a temporary backup of data.

• A background Worker monitors the primary Follower for when the record appears after replication.

• Once the record appears on the Primary follower, the background worker goes ahead and deletes the record from the buffer table.

A few important points to note:

• The number of secondary leaders is configurable.

• The secondary leader is chosen at random.

• The buffered writes support idempotency. If there are multiple writes with the same identity fields, it doesn't matter how many times the request is made.

Thank you for your time and see you in the next.

Special shoutout to @ProgressiveCod2 on X(Formerly Twitter) for his support.