how i built redis rival caching database dustdb?
Muhammad yasir
I built dustdb and you can see here Dustdb.
I’m Muhammad Yasir a engineer in pakistan and i was nerd since my studies and i got punished by teachers for learning extra thing and when my journey started as a developer i first stuck at database connection and the programming language i was using for the first time was PHP and i thought it is impossible to build something like a database and i became very sad about that i am terrible at coding like a coding muffin.
It was coded in 5 years and researched for month san di struggled days and nights reading different books on internals of database and then i saw something in every database that they were un optimized and not multithreaded.
I started this as my hobby project in golang and C language but after some time i came at point that if i used Assembly rather than C or C++ i can make it unbeatable and it is true that assembly is machine language but it is also true that if you don’t have knowledge about assembler or compiler you will end up writing code which will be slower than compiler or maybe interpreter?
I used LRU and LFU based caching mechanisms in this database and it has web ui and you can set TTL(time to live for caching and purge for it ).
I made docs for it which are available in docs/index.html.
Databases are hardest pill to take not because they are hard but rather than it is hard to build something in a way which whiplashes performance aor something better than existing ones could be hard.
It takes only 22mb of ram which is considered pretty good for old servers and if you see redis it takes 512 mb ram and all other database takes this ior near this much amount of ram except memcached which uses 256+ ram .
I used almost 99.5 efficiency of memory and avoided segmentation at all coast i f you put i bit on disk database like sqlite or postgres or memcache or redis it will take 1 byte /8 bits and 7 bits will be held as useless thing and i solved this problem in Dustdb and it is first database which achieved this thing.
Multithreading is built-in in dustdb.
It supports sam commands like redis and example is given below.
- setting values(setting key and value in database)
SET name "gujjar"
OK
- Getting values(getting certain key will show its value)
GET name
"gujjar"
- Deleting values
DEL key
PING compatibility with redis
PING
PONG
- Deceasing value of a key integer
DECR key
value deleted
| here is detailed comparison overview of redis,memcached,dragonfly and dustdb. i developed dustdb while studying Operating systems books and databases research papers and made optimized algorithms. |
| Redis | Memcached | DragonflyDB | DustDB | |
| Type | In-memory data store | In-memory cache | In-memory data store | In-memory data store |
| Performance | High | High | Higher than both | Above all three |
| Data Structures | Rich data types | Simple key-value | Compatible with Redis | Advanced key-value data structures |
| Scalability | Clustering available | Limited | Highly scalable | Linear horizontal scaling |
| Latency | Low | Low | Lower than both | Ultra-low latency |
| Use Cases | Caching, messaging | Caching | Caching, real-time apps | Caching, real-time apps |
| Memory Efficiency | Moderate (~70–80%) | High (~90%) | Superior (~95%) | Exceptional (99.5%) |
| Eviction Policy | LRU, LFU | LRU | Custom algorithm | Custom yet OS level optimized algorithms |
| Minimum RAM Required | 512 MB+ | 256 MB+ | 512 MB+ | Only 22 MB — runs like a charm on old mobile servers |
| Threading Model | Mostly single-threaded | Multi-threaded but not optimal | Fully multithreaded, async | Multithreaded, async-optimized |
| Algorithms | Event loop, RDB/AOF | Slab allocator, hash table | Custom scheduler, smart locking | OS-level schedulers, zero-copy pipelines |
| Garbage Handling | Can leave memory fragmentation | Can leak/fragment memory | Efficient GC avoidance | No garbage buildup — runs clean on legacy servers |
| License | BSD 3-Clause (open source) | BSD 3-Clause (open source) (paid hosting) | Proprietary (source-available) | GNU Public License (100% free) |
| Ownership | Corporate | Corporate | Corporate | Individually maintained |
| Pricing | Free | Free | Free (limited) / Paid | Fully free |
Conclusion:
Anything is possible if you mess with something it will be solved but if you don’t do this and enjoy life like everyone you will be fade by time.
Subscribe to my newsletter
Read articles from Muhammad yasir directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by