Based on all the numbers I've seen, it looks like bcachefs's b-tree might actually be the fastest ordered key value store around (there are faster persistent hash tables).
If anyone knows...
2018-02-04 23:42:01 +0000 UTC View Postrand_mixed is 3/4 lookups, 1/4 update
Lookups, both sequential and random, are beautifully fast.
Random updates ought to scale better than that, but I haven't profiled it yet so I'm not sure w...
2018-01-28 03:32:51 +0000 UTC View PostI should have done this ages ago...
bcachefs: btree_perf_test() doing 10.0M rand_insert:
bcachefs: btree_perf_test() done in 27 sec, 2587 nsec per iter, 377k per sec
bcachefs:...
2018-01-24 20:07:19 +0000 UTC View PostYou can now expand a filesystem on a device - shrink isn't implemented just yet. The command is bcachefs device resize, and it takes the same arguments as resize2fs:
bcachefs device resize /dev/sdb ...
2018-01-02 23:41:21 +0000 UTC View PostReplication tests are finally all passing! This means that device removal and write error handling (for replicated writes) should finally be fully working.
Those two codepaths have in common that th...
2017-12-24 21:21:48 +0000 UTC View PostJust pushed out a patch to add per-inode options for some options that could previously only be set globally. Currently this is just checksum type and compression type, but more will be added in the f...
2017-12-15 16:21:18 +0000 UTC View PostI've been back to work for several months but procrastinating and neglecting the updates...
The biggest thing I've been spending my time on lately has been improving the test infrastructure and test...
2017-12-13 16:07:29 +0000 UTC View Post
Took this earlier today in Nashville :)
On the bcachefs front - might be announcing a corporate sponsor in the next few days! Stay tuned.
If you've noticed things have been quieter lately, you haven't been imagining things - I've been busy with getting ready for a rather big move, and in the process I'm taking an extended road trip. I'm...
2017-06-13 23:48:51 +0000 UTC View PostAs I think I mentioned awhile ago, for replication the last big item left was IO error handling - that is, handling IO errors without just going read only when we've got another replica to read from (...
2017-05-15 08:12:50 +0000 UTC View PostRecently pushed a patch to add prefetching of btree nodes. It's a rather minor change compared to the stuff I'm still working on for replication, but it does improve both mount and fsck times by aroun...
2017-04-25 14:49:53 +0000 UTC View PostDebugging, debugging, more debugging...
If you've been wondering at the slow progress, that's where all my time's been going. The unfortunate reality about creating a filesystem is that a files...
http://bcachefs.org/
Don't have any _new_ content there yet, it's just all the existing stuff in one place. Would love to have peopl...
It's been far too long since the last announcement - lots of stuff has been
happening. The biggest milestone has been all the breaking on disk format
changes finally landing, but there's been ...
2017-03-16 00:04:34 +0000 UTC View PostFirst off, some background on where we're at currently, regarding metadata IO:
- A userspace process will never block on IO - i.e., wait for a journal write or a btree node write - unnecessari...
2016-12-03 03:59:18 +0000 UTC View PostLately, the big bottleneck is getting to be testing - I really need more people willing to try out the latest code and make sure it isn't going to eat anyone's data before I push it out for general co...
2016-11-06 08:27:02 +0000 UTC View PostThese patches haven't landed yet, and the numbers should be higher when I'm done - but the fsmark numbers are now looking really nice. Delete performance is massively improved, too.
Time to completi...
2016-10-24 10:48:01 +0000 UTC View PostTiering should finally be working with the last big batch of fixes I pushed.
Chris Halse Rogers (RAOF in the #bcache IRC channel) has been testing it. He has been seeing an intermittent deadlock whi...
2016-09-13 02:10:00 +0000 UTC View PostFirst off, a word about definitions. In bcachefs, tiering is caching by another name: storage devices can be assigned to different tiers, and we can use a faster tier to cache a slower tier.
In some...
2016-09-13 01:40:07 +0000 UTC View PostJust saw this really excellent article about disk encryption - this explains better than I could the issues with encryption at the block layer:
- Encryption's mostly done, got some useful feedback from the design doc.
- Starting to work on multiple devices and replication again. Found some "there's no way this could have possibl...
2016-09-05 02:40:02 +0000 UTC View Posthttps://bcache.evilpiepirate.org/Encryption/
2016-09-02 04:19:07 +0000 UTC View PostBeen studying random papers/RFCs/Dan Bernstein's code and figuring out the plan for adding encryption to bcachefs... doing crypto right is hard. In storage land, I'm not sure anyone really gets it rig...
2016-08-07 16:10:52 +0000 UTC View PostI'm kicking myself for not noticing this sooner (most likely I saw it months ago and then forgot about it because I'm terrible about taking notes...). Do not use bcachefs with compression enabled yet ...
2016-08-06 05:12:49 +0000 UTC View PostThis is my current project, so I thought I'd write something about it and how this area of bcachefs works.
So, for some background: every remotely modern filesystem has some sort of facil...
2016-08-02 12:40:59 +0000 UTC View PostThe last bit - disk space accounting - is finished, so it should actually be useful now.
Please test it out - I'd like to hear how well it's working for people.
Currently lz4 and gzip ...
2016-08-02 10:39:12 +0000 UTC View PostThank you so much for the support this month!
2016-07-31 23:59:00 +0000 UTC View PostFinally figured out how to make compressed disk usage accounting work. It's a surprisingly thorny issue - I'll have to write more about it later.
The TL;DR is - disk usage is only allowed to inc...
2016-07-26 11:34:27 +0000 UTC View Post