Databases to keep an eye on
Last updated: Jan 5, 2022
Introduction
Database market is huge and is constantly evolving. Databases being the centre of all applications, is one of the key component whose performance directly impact the product. We not only find startups trying to push this limit but can see talented individuals writing new datastores. In this blog, I would like to mention few of those which I am planning to keep an eye on in coming years.
Databases
DuckDB
It is an embedded database for OLAP requirements. Think of it as an intersection of SQLite and ClickHouse. Benchmarks results are very promising. Even ClickHouse maintainers has plans to build something like this within ClickHouse in their 2022 roadmap.
Rqlite
It is a distributed cluster of SQLite databases which uses raft for consensus. Think of it like Etcd/Consul but with SQLite in the backend.
Sled
It is another embedded database written in Rust and uses many new research papers as reference. It falls under the same category as RocksDB, LevelDB and LMDB. It uses Bw-Tree which is designed for modern hardware, multi-core with lock-free algorithms, cache friendly (avoiding cache invalidations) and is optimized for SSDs with very fast random reads but very slow random writes. Performance Guide from their documentation is a very good read.
EdgeDB
It is built on PostgreSQL and provides nice query language, strict typed schema and intelligent query caching capabilities. It has first class support for GraphQL and for schema migrations.
SeaweedFS
It is a distributed filesystem with support for object store. It is written inspired from Facebook’s papers: Haystack, f4, Tectonic. It falls in the same category as any other distributed filesystem like HDFS, GlusterFS etc and also object stores like AWS S3, Ceph, Minio etc. It is mainly optimized for serving lots of small files.
TensorBase
It is a ClickHouse compatible datastore which claims to be 5x to 30x faster than ClickHouse. It is built in Rust and uses Apache Arrow as the data format. It is still in quite early stage but the result are very promising.
Conclusion
All the above databases are written either by a single person or a very small team. In case you use them in production, please do consider to sponsor them either personally or via your employer and also encourage others to do the same.