Technology Fridays: BoltDB Wants to be the Storage Layer of the Edge Computing World

Welcome to Technology Fridays! Today, we are going to talk about embedded databases and one of the most popular technologies in that market: BoltDB.

Embedded databases became really relevant in the industry with the evolution of mobile computing. Even though there have been models of embedded databases for decades it was the emergence of mobile apps which brought the space into prominence. Most mobile developer have certain level of familiarity with SQLLite which can be arguably considered the best-known embedded database in the market. However, SQLLite often results limited in many mobile or internet of things(IOT ) scenarios due to its dependence on relational models. This is where the new generation of embedded databases led by BoltDB comes in. If SQLLite is the OracleDB of the embedded data storage world then BoltDB can be considered the equivalent of DynamoDB of in-application databases.

Originally built on Go, BoltDB can be executed as a library bundled into a process such as a website or mobile app. Obviously, that characteristic entails that only a small number of processes can access the BoltDB runtime concurrently but it simplifies aspects such as backups or portability.

Key-value pairs are the main model used to represent information in BoltDB. From that perspective, BoltDB can be used to store semi-structured or unstructured data using a consistent model. The platform also introduces the concept of Buckets to group key-value pairs into logical units within the same database. Buckets can be dynamically created using the different client SDKs available with the platform. BoltDB can combine multiple Buckets into more complex hierarchical structures called Nested Buckets. A Nested Bucket can be composed by several layers of Buckets. For instance, a social network application can have a root User Bucket containing individual Buckets for each user which are formed by other Buckets that represent user-specific information such as friends or photos.

Transactions play an important role in the BoltDB experience. The platform allows both read-only and read-write transactions but the runtime considerations are fairly different. BoltDB supports many concurrent read-only transaction but only one read-write transaction can be active at any given time. The constraint makes complete sense if you factor in that BoltDB does not run in a server side environment but rather inside an application process.

Internally, BoltDB uses a B+Tree structure to store the data using a single file. That feature, together with the serializable ACID transaction semantics, make BoltDB excel in scenarios that involve high-volume reads and consistent writes. Developers can start interacting with BoltDB using the different client libraries available with the platform. The SDKs covers different runtimes such as Android or IOS as well as general-purpose programming languages such as Go.

BoltDB enjoys a vibrant developer community that has contributed valuable tools to the platform such as BoltDBWeb(management or BoltStore( session persistence). Additionally, BoltDB is the underlying data storage runtime behind popular platforms like Consul.io( discovery) or InfluxDB( time-series database).

Competition?

BoltDB is an embedded database platform which many times is positioned as an alternative to the ultra-popular SQLLite. However, competition for BoltDB does not stop there. LevelDB, RocksDB and HyperLevelDB are alternative to BoltDB that have a stronger focus on writing performance. LMDB is another embedded database that shares a lot of similarities with BoltDB even though the developer experience is a bit more complex.