During an internship, I met an senior developer who built his company’s internal database engine from scratch over several decades, and was happy to share how it worked: every architectural decision, every trade-off between performance and complexity. Seeing someone master something they had built entirely, at that level of depth, raised a simple question: how does a database engine actually work? The only way to find out was to build one.
Flint is an experimental database engine I’m building in Rust. The goal: an ultra-modular architecture where each layer (on-disk storage format, in-memory organization, query parsing) is an independent interface, implementable and replaceable without touching the rest.
Why Rewrite from C++
The project originally existed in C++, with a classic row-oriented implementation. After reading a research paper on modular database architectures and discovering Rust, I rewrote everything. Rust is the right choice here for two concrete reasons: memory safety without a garbage collector (critical for an engine managing its own allocations), and a trait system that makes interface-based architecture natural rather than forced.
Architectural Decision: Why Ultra-Modular
The hypothesis to test: the data access format (row-based, column-based, hybrid) should be dynamically chosen based on the incoming query type to minimize processing time. An analytical query over an entire column shouldn’t pay the cost of row-oriented access.
Making each layer swappable has a second benefit: it makes this a good open source project. Anyone can implement their own storage format or memory access strategy without modifying the engine core.
Current State
- Base modular architecture implemented
CREATE,SELECT,INSERToperations functional in memory- On-disk storage and parser under development
- Open source publication planned once maturity threshold is reached