Table of contents
Open Table of contents
What’s Next?
I think at this time, semidbm has more than exceeded it’s original
goal, which was to be a pure python cross platform key value storage
that had reasonable performance. So what’s next for semidbm? In a
nutshell, higher level abstractions (aka the “fun stuff”). Code that
builds on the simple key value storage of semidbm.db and provides
additional features. And as we get higher level, I think it makes sense
to reevaluate the original goals of semidbm and whether or not it makes
sense to carry those goals forward:
- Cross platform. I’m inclined to not support windows for these higher level abstractions.
- Pure python. I think the big reason for remaining pure python was for ease of installation. Especially on windows, pip installing a package should just work. With C extensions, this becomes much harder on windows. If semidbm isn’t going to support windows for these higher level abstractions, then C extensions are fair game.
Some ideas I’ve been considering:
- A C version of
_Semidbm. - A dict like interface that is concurrent (possibly single writer multiple reader).
- A sorted version of semidbm (supporting things like range queries).
- Caching reads (need an efficient LRU cache).
- Automatic background compaction of data file.
- Batched writes
- Transactions
- Compression (I played around with this earlier. Zlib turned out to be too slow for the smaller sized values (~100 bytes) but it might be worth being able to configure this on a per db basis.