bcachefs

More expensive on disk format upgrades

6.11 was an expensive forced on disk format upgrade, for the disk accounting rewrite: while bcachefs is still marked as experimental, I'm making on disk format changes that would not be feasible if...

Update: no bcachefs updates for 6.13 confirmed

Fixes will have to come from my repository.

This came just as the CoC board seemed to be relenting, and we started to be having a public conversation. That's now been cut off, and I think aft...

Trouble in the kernel

TLDR: the future of bcachefs in the kernel is uncertain, and lots of things aren't looking good.

Linus has said he isn't accepting my 6.13 pull request, per "an open issue with the CoC board"...

Repeat after me: you do not fuck around in filesystem land

So, kernel programming is known for being a high stakes, high pressure, toxic environment to work in. There's a reason for that: the consequences for screwing up are really high, and you can make y...

Online fsck

Initial support for online fsck is merged - it's in my master branch, and will be in Linux 6.8. To use it, just run the normal fsck command; if the filesystem is mounted, it'll use the online codep...

Telemetry

Recently, I added a new superblock section for tracking counts of every distinct filesystem error (i.e. fsck error) since filesystem creation, as well as the date of the most recent error.

Th...

Out west

Note on the phoronix numbers

Phoronix recently posted some bcachefs benchmarks, and their results looked a bit... off.
Here's what I just got, testing 4k random writes with a similar fio configuration. SSD is a Samsung 970 ...

Recent work

Some recent features that have landed:

 * Rebalance work btree: Rebalance no longer has to scan for extents that need the background_target or background_compression option applied. Inst...

Your irregular status update: upstreaming, growing the team, and a funding situation update

So, the upstreaming process has been rocky - but bcachefs is in linux-next! Fingers crossed for 6.7. Since being merged into the linux-next tree we've also been getting a lot more assorted bugfixes...

Your irregular status update

Let's see, what is there to talk about since the last:

Mostly, it's been a whole lot of bug fixing and stabilizing, grinding away at test failures in the CI: 2023-03-29 14:57:29 +0000 UTC View Post

Thanksgiving update

There's so much that goes into developing a real filesystem. Especially one that's intended to be good enough to replace our existing filesystems, codebases that have had decades of refinement by t...

New test dashboard

We _finally_ have a server continuously running tests and outputting to a nice dashboard. I'm pretty excited - this is going to make my life a lot easier, and it's another thing people can look at ...

Backpointers has been merged!

New on disk format, required upgrade. Your existing filesystem will be automagically upgraded when you upgrade kernel & tools to the new version.

What this means: much more efficient copy...

.plan file

Reorganized my todo list last night, now automatically synced from my home directory:

https://evilpiepirate.org/~kent/.plan....

Zoned device support

I've been starting to work on support for zoned devices, laying out what needs to be done. This will get us native support for SMR (shingled magnetic recording) hard drives, and (even more exciting...

New allocator has been merged!

It's a mandatory disk format upgrade; when switching to the new version on an

existing filesystem you'll see it initialize the freespace btree when you mount.

What...

Status update

Bacon and eggs, tea, sitting down to work on finishing the BTREE_ITER_WITH_JOURNAL patch, but before I can get to the interesting and necessary algorithmic work I've got like half a dozen bug repor...

User manual

Bcachefs now has a user manual - check it out!

https://bcachefs.org/bcachefs-principles-of-operat...

Short update

Just finished fixing a whole bunch of i_sectors accounting bugs - the count of how many sectors are in a given inode/file. It turns out our accounting on disk, in the btree was completely fine - fs...

Snapshots have been merged

They work similarly to btrfs snapshots. There's new bcachefs subcommands for creating subvolumes and snapshots.

We've got writable snapshots, snapshots of snapshots, and snapshot and subvolum...

Status update - new on disk format change

There's a new on disk format change, and it's a required upgrade - you'll want to upgrade your kernel and tools at the same time to stay in sync.

t was The new update closes a hole in our abi...

Quick note

Please report it _any_ time errors are reported on a filesystem, fsck errors or otherwise. There's a couple of nasty bugs I'm trying to track down right now (one of them appears to be btree writes ...

Disk format change for snapshots

Turns out, there was a hiccup after all in the on disk format changes for snapshots: it turns out we were generating packed bkey formats where the snapshot field was too big, and this breaks the lo...

Snapshots are working!

They're still in a very early state, and nothing but the bare minimum is complete - but they're up, for people to look at and try out. You'll need to use the snapshots branch from both the kernel a...

New snapshots design doc

Snapshots are really coming together - the whole thing is starting to look shockingly elegant and simple (you wouldn't be able to guess from the design doc how many years it took to get to that poi...

Status update

Snapshots are coming :)

Upcoming changes

Recently, Dave Chinner's been helping out with some performance testing of bcachefs on his big dual socket machine, which has been great for finding scalability issues and he's also been able to pr...

Status update - erasure coding

Erasure coding is ready for wider testing!

I just finished reworking disk space accounting for parity blocks - they're broken out from user data now, so bcachefs fs usage will show you that o...

Erasure coding, bcache2

Finally been making some good progress on erasure coding: currently debugging the new code to update existing stripes with new blocks, which is the main piece that was missing - this is needed to d...