linux.git
13 months agobcachefs: ptr_stale() -> dev_ptr_stale()
Kent Overstreet [Wed, 1 May 2024 00:56:54 +0000 (20:56 -0400)]
bcachefs: ptr_stale() -> dev_ptr_stale()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: extent_ptr_durability() -> bch2_dev_rcu()
Kent Overstreet [Wed, 1 May 2024 00:54:20 +0000 (20:54 -0400)]
bcachefs: extent_ptr_durability() -> bch2_dev_rcu()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_extent_merge() -> bch2_dev_rcu()
Kent Overstreet [Wed, 1 May 2024 00:49:22 +0000 (20:49 -0400)]
bcachefs: bch2_extent_merge() -> bch2_dev_rcu()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: ec_validate_checksums() -> bch2_dev_tryget()
Kent Overstreet [Wed, 1 May 2024 00:42:50 +0000 (20:42 -0400)]
bcachefs: ec_validate_checksums() -> bch2_dev_tryget()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: ob_dev()
Kent Overstreet [Wed, 1 May 2024 00:38:05 +0000 (20:38 -0400)]
bcachefs: ob_dev()

Wrapper around bch2_dev_have_ref() for open_buckets; we do guarantee
that the device an open_bucket points to exists.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: move replica_set from bch_dev to bch_fs
Kent Overstreet [Wed, 1 May 2024 00:32:44 +0000 (20:32 -0400)]
bcachefs: move replica_set from bch_dev to bch_fs

This is needed for the next patch - the write submit path has to be able
to allocate a replica bio even when we weren't able to get a ref on the
device.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Kill bch2_dev_bkey_exists() in backpointer code
Kent Overstreet [Tue, 30 Apr 2024 20:50:28 +0000 (16:50 -0400)]
bcachefs: Kill bch2_dev_bkey_exists() in backpointer code

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: PTR_BUCKET_POS() now takes bch_dev
Kent Overstreet [Tue, 30 Apr 2024 23:34:28 +0000 (19:34 -0400)]
bcachefs: PTR_BUCKET_POS() now takes bch_dev

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_dev_iterate()
Kent Overstreet [Tue, 30 Apr 2024 19:37:25 +0000 (15:37 -0400)]
bcachefs: bch2_dev_iterate()

New helper for getting refs to devices as we iterate.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_evacuate_bucket() -> bch2_dev_tryget()
Kent Overstreet [Tue, 30 Apr 2024 19:46:45 +0000 (15:46 -0400)]
bcachefs: bch2_evacuate_bucket() -> bch2_dev_tryget()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_bucket_ref_update() now takes bch_dev
Kent Overstreet [Tue, 30 Apr 2024 20:20:49 +0000 (16:20 -0400)]
bcachefs: bch2_bucket_ref_update() now takes bch_dev

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_trigger_alloc() -> bch2_dev_tryget()
Kent Overstreet [Tue, 30 Apr 2024 19:53:03 +0000 (15:53 -0400)]
bcachefs: bch2_trigger_alloc() -> bch2_dev_tryget()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_check_alloc_key() -> bch2_dev_tryget_noerror()
Kent Overstreet [Tue, 30 Apr 2024 19:43:20 +0000 (15:43 -0400)]
bcachefs: bch2_check_alloc_key() -> bch2_dev_tryget_noerror()

More elimination of bch2_dev_bkey_exists() usage.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Convert to bch2_dev_tryget_noerror()
Kent Overstreet [Fri, 3 May 2024 21:48:23 +0000 (17:48 -0400)]
bcachefs: Convert to bch2_dev_tryget_noerror()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_dev_tryget()
Kent Overstreet [Tue, 30 Apr 2024 19:30:35 +0000 (15:30 -0400)]
bcachefs: bch2_dev_tryget()

Most uses of bch2_dev_bkey_exists() are going away, where we assume that
because a key references a device the device most exist - instead, we'll
be explicitly checking if the device exists and getting a reference to
it.

This adds the new helpers.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_have_enough_devs() checks for nonexistent device
Kent Overstreet [Tue, 30 Apr 2024 19:45:05 +0000 (15:45 -0400)]
bcachefs: bch2_have_enough_devs() checks for nonexistent device

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: journal_replay_entry_early() checks for nonexistent device
Kent Overstreet [Tue, 30 Apr 2024 19:45:26 +0000 (15:45 -0400)]
bcachefs: journal_replay_entry_early() checks for nonexistent device

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_dev_btree_bitmap_marked() -> bch2_dev_rcu()
Kent Overstreet [Tue, 30 Apr 2024 20:11:15 +0000 (16:11 -0400)]
bcachefs: bch2_dev_btree_bitmap_marked() -> bch2_dev_rcu()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Pass device to bch2_bucket_do_index()
Kent Overstreet [Tue, 30 Apr 2024 19:41:48 +0000 (15:41 -0400)]
bcachefs: Pass device to bch2_bucket_do_index()

Eliminating bch2_dev_bkey_exists() uses and replacing them with proper
checks; this one was unnecessary since the caller already has it.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Pass device to bch2_alloc_write_key()
Kent Overstreet [Tue, 30 Apr 2024 19:44:24 +0000 (15:44 -0400)]
bcachefs: Pass device to bch2_alloc_write_key()

More elimating bch2_dev_bkey_exists()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_dev_safe() -> bch2_dev_rcu()
Kent Overstreet [Tue, 30 Apr 2024 19:30:35 +0000 (15:30 -0400)]
bcachefs: bch2_dev_safe() -> bch2_dev_rcu()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Debug asserts for ca->ref
Kent Overstreet [Fri, 3 May 2024 22:07:40 +0000 (18:07 -0400)]
bcachefs: Debug asserts for ca->ref

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: New helpers for device refcounts
Kent Overstreet [Fri, 3 May 2024 21:39:16 +0000 (17:39 -0400)]
bcachefs: New helpers for device refcounts

This will be used in the next patch for adding some new debug mode
asserts.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_print_allocator_stuck()
Kent Overstreet [Fri, 3 May 2024 18:49:23 +0000 (14:49 -0400)]
bcachefs: bch2_print_allocator_stuck()

If we block on the allocator for more than 10 seconds, print out some
useful debugging info.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agoclosures: closure_sync_timeout()
Kent Overstreet [Fri, 3 May 2024 18:43:54 +0000 (14:43 -0400)]
closures: closure_sync_timeout()

Add a new variant of closure_sync_timeout() that takes a timeout.

Note that when this returns -ETIME the closure will still be waiting on
something, i.e. it's not safe to return if you've got a stack allocated
closure.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_bkey_drop_ptrs() declares loop iter
Kent Overstreet [Wed, 1 May 2024 23:04:13 +0000 (19:04 -0400)]
bcachefs: bch2_bkey_drop_ptrs() declares loop iter

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: x-macroize journal flags enums
Kent Overstreet [Tue, 30 Apr 2024 10:20:37 +0000 (06:20 -0400)]
bcachefs: x-macroize journal flags enums

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: On device add, prefer unused slots
Kent Overstreet [Tue, 30 Apr 2024 08:24:58 +0000 (04:24 -0400)]
bcachefs: On device add, prefer unused slots

We can't strictly guarantee that no pointers refer to nonexistent
devices - we attempt to, but we need to be safe when the filesystem is
corrupt.

Therefore, change device_add to try to pick a slot that's never been
used, or the slot that's been unused the longest.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Kill opts.buckets_nouse
Kent Overstreet [Sat, 6 Apr 2024 04:07:46 +0000 (00:07 -0400)]
bcachefs: Kill opts.buckets_nouse

Now explicitly allocate and free the buckets_nouse bitmap - this is
going to be used for online fsck.

To go RW when we haven't check allocations, we'll do a much slimmed down
version that just initializes the buckets_nouse bitmaps.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: CodingStyle
Kent Overstreet [Mon, 29 Apr 2024 07:15:21 +0000 (03:15 -0400)]
bcachefs: CodingStyle

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: simplify bch2_trans_start_alloc_update()
Kent Overstreet [Tue, 30 Apr 2024 07:27:30 +0000 (03:27 -0400)]
bcachefs: simplify bch2_trans_start_alloc_update()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: __mark_stripe_bucket() now takes bch_alloc_v4
Kent Overstreet [Tue, 30 Apr 2024 06:41:16 +0000 (02:41 -0400)]
bcachefs: __mark_stripe_bucket() now takes bch_alloc_v4

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: __mark_pointer now takes bch_alloc_v4
Kent Overstreet [Tue, 30 Apr 2024 06:10:57 +0000 (02:10 -0400)]
bcachefs: __mark_pointer now takes bch_alloc_v4

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: kill bch2_dev_usage_update_m()
Kent Overstreet [Tue, 30 Apr 2024 04:29:24 +0000 (00:29 -0400)]
bcachefs: kill bch2_dev_usage_update_m()

by using bucket_m_to_alloc() more, we can get some nice code cleanup.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: alloc_data_type_set()
Kent Overstreet [Tue, 30 Apr 2024 06:47:30 +0000 (02:47 -0400)]
bcachefs: alloc_data_type_set()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: dirty_sectors -> replicas_sectors
Kent Overstreet [Tue, 30 Apr 2024 05:49:59 +0000 (01:49 -0400)]
bcachefs: dirty_sectors -> replicas_sectors

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: delete old gen check bch2_alloc_write_key()
Kent Overstreet [Tue, 30 Apr 2024 04:41:42 +0000 (00:41 -0400)]
bcachefs: delete old gen check bch2_alloc_write_key()

this was from metadata only gc - we don't need it anymore

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Correct the FS_IOC_GETFLAGS to FS_IOC32_GETFLAGS in bch2_compat_fs_ioctl()
Youling Tang [Tue, 30 Apr 2024 03:27:44 +0000 (11:27 +0800)]
bcachefs: Correct the FS_IOC_GETFLAGS to FS_IOC32_GETFLAGS in bch2_compat_fs_ioctl()

It should be FS_IOC32_GETFLAGS instead of FS_IOC_GETFLAGS in
compat ioctl.

Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Fix error path of bch2_link_trans()
Youling Tang [Tue, 30 Apr 2024 03:28:39 +0000 (11:28 +0800)]
bcachefs: Fix error path of bch2_link_trans()

In bch2_link_trans(), if bch2_inode_nlink_inc() fails, it needs to
call bch2_trans_iter_exit() in the error path.

Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Change destroy_inode to free_inode
Youling Tang [Fri, 26 Apr 2024 07:29:56 +0000 (15:29 +0800)]
bcachefs: Change destroy_inode to free_inode

The vfs[1] documentation describes free_inode as follows:
```
free_inode
    this method is called from RCU callback. If you use call_rcu()
    in ->destroy_inode to free ‘struct inode’ memory, then it’s
    better to release memory in this method.
```

free_inode will be called by the RCU callback, so it might be better
to move the inode free operation to destroy_inode.

Similar to commit ae6b47b5653e ("fs/ntfs3: Change destroy_inode to
free_inode").

Link:
[1]: https://www.kernel.org/doc/html/latest/filesystems/vfs.html

Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Simplify resuming of journal position
Kent Overstreet [Fri, 26 Apr 2024 04:31:14 +0000 (00:31 -0400)]
bcachefs: Simplify resuming of journal position

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: check inode backpointer in bch2_lookup()
Kent Overstreet [Thu, 25 Apr 2024 23:54:03 +0000 (19:54 -0400)]
bcachefs: check inode backpointer in bch2_lookup()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: check for inodes that should have backpointers in fsck
Kent Overstreet [Fri, 26 Apr 2024 02:11:49 +0000 (22:11 -0400)]
bcachefs: check for inodes that should have backpointers in fsck

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch_member.last_journal_bucket
Kent Overstreet [Fri, 26 Apr 2024 04:32:56 +0000 (00:32 -0400)]
bcachefs: bch_member.last_journal_bucket

On recovery from clean shutdown we don't typically read the journal, but
we still want to avoid overwriting existing entries in the journal for
list_journal debugging.

Thus, add some fields to the member info section so we can remember
where we left off.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: uninline set_btree_iter_dontneed()
Kent Overstreet [Thu, 25 Apr 2024 06:20:20 +0000 (02:20 -0400)]
bcachefs: uninline set_btree_iter_dontneed()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: eliminate the uninitialized compilation warning in bch2_reconstruct_snapshots
Hongbo Li [Fri, 26 Apr 2024 03:21:35 +0000 (11:21 +0800)]
bcachefs: eliminate the uninitialized compilation warning in bch2_reconstruct_snapshots

When compiling the bcachefs-tools, the following compilation warning
is reported:
    libbcachefs/snapshot.c: In function ‘bch2_reconstruct_snapshots’:
    libbcachefs/snapshot.c:915:19: warning: ‘tree_id’ may be used uninitialized in this function [-Wmaybe-uninitialized]
      915 |  snapshot->v.tree = cpu_to_le32(tree_id);
    libbcachefs/snapshot.c:903:6: note: ‘tree_id’ was declared here
      903 |  u32 tree_id;
       |      ^~~~~~~

This is a false alert, because @tree_id is changed in
bch2_snapshot_tree_create after it returns 0. And if this function
returns other value, @tree_id wouldn't be used. Thus there should
be nothing wrong in logical.

Although the report itself is a false alert, we can still make it more
explicit by setting the initial value of @tree_id to 0 (an invalid
tree ID).

Fixes: a292be3b68f3 ("bcachefs: Reconstruct missing snapshot nodes")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: fix btree_path_clone() ip_allocated
Kent Overstreet [Thu, 25 Apr 2024 06:15:08 +0000 (02:15 -0400)]
bcachefs: fix btree_path_clone() ip_allocated

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Fix format specifiers in bch2_btree_key_cache_to_text()
Nathan Chancellor [Tue, 23 Apr 2024 20:12:27 +0000 (13:12 -0700)]
bcachefs: Fix format specifiers in bch2_btree_key_cache_to_text()

When building for a 32-bit target, for which 'size_t' is 'unsigned int',
there are two warnings around mismatched format specifiers and argument
types:

  In file included from fs/bcachefs/vstructs.h:5,
                   from fs/bcachefs/bcachefs_format.h:79,
                   from fs/bcachefs/bcachefs.h:207,
                   from fs/bcachefs/btree_key_cache.c:3:
  fs/bcachefs/btree_key_cache.c: In function 'bch2_btree_key_cache_to_text':
  fs/bcachefs/btree_key_cache.c:1046:25: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=]
   1046 |         prt_printf(out, "nonpcpu freelist:\t%lu\r\n",   bc->nr_freed_nonpcpu);
        |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~    ~~~~~~~~~~~~~~~~~~~~
        |                                                           |
        |                                                           size_t {aka unsigned int}
  fs/bcachefs/util.h:192:63: note: in definition of macro 'prt_printf'
    192 | #define prt_printf(_out, ...)           bch2_prt_printf(_out, __VA_ARGS__)
        |                                                               ^~~~~~~~~~~
  fs/bcachefs/btree_key_cache.c:1046:47: note: format string is defined here
   1046 |         prt_printf(out, "nonpcpu freelist:\t%lu\r\n",   bc->nr_freed_nonpcpu);
        |                                             ~~^
        |                                               |
        |                                               long unsigned int
        |                                             %u
  fs/bcachefs/btree_key_cache.c:1047:25: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=]
   1047 |         prt_printf(out, "pcpu freelist:\t%lu\r\n",      bc->nr_freed_pcpu);
        |                         ^~~~~~~~~~~~~~~~~~~~~~~~~       ~~~~~~~~~~~~~~~~~
        |                                                           |
        |                                                           size_t {aka unsigned int}
  fs/bcachefs/util.h:192:63: note: in definition of macro 'prt_printf'
    192 | #define prt_printf(_out, ...)           bch2_prt_printf(_out, __VA_ARGS__)
        |                                                               ^~~~~~~~~~~
  fs/bcachefs/btree_key_cache.c:1047:44: note: format string is defined here
   1047 |         prt_printf(out, "pcpu freelist:\t%lu\r\n",      bc->nr_freed_pcpu);
        |                                          ~~^
        |                                            |
        |                                            long unsigned int
        |                                          %u
  cc1: all warnings being treated as error

Use the proper 'size_t' specifier, '%zu', to clear up the warnings for
these platforms.

Fixes: f2d47ec26af5 ("bcachefs: Btree key cache instrumentation")
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Fix type of flags parameter for some ->trigger() implementations
Nathan Chancellor [Tue, 23 Apr 2024 18:58:09 +0000 (11:58 -0700)]
bcachefs: Fix type of flags parameter for some ->trigger() implementations

When building with clang's -Wincompatible-function-pointer-types-strict
(a warning designed to catch potential kCFI failures at build time),
there are several warnings along the lines of:

  fs/bcachefs/bkey_methods.c:118:2: error: incompatible function pointer types initializing 'int (*)(struct btree_trans *, enum btree_id, unsigned int, struct bkey_s_c, struct bkey_s, enum btree_iter_update_trigger_flags)' with an expression of type 'int (struct btree_trans *, enum btree_id, unsigned int, struct bkey_s_c, struct bkey_s, unsigned int)' [-Werror,-Wincompatible-function-pointer-types-strict]
    118 |         BCH_BKEY_TYPES()
        |         ^~~~~~~~~~~~~~~~
  fs/bcachefs/bcachefs_format.h:394:2: note: expanded from macro 'BCH_BKEY_TYPES'
    394 |         x(inode,                8)                      \
        |         ^~~~~~~~~~~~~~~~~~~~~~~~~~
  fs/bcachefs/bkey_methods.c:117:41: note: expanded from macro 'x'
    117 | #define x(name, nr) [KEY_TYPE_##name]   = bch2_bkey_ops_##name,
        |                                           ^~~~~~~~~~~~~~~~~~~~
  <scratch space>:277:1: note: expanded from here
    277 | bch2_bkey_ops_inode
        | ^~~~~~~~~~~~~~~~~~~
  fs/bcachefs/inode.h:26:13: note: expanded from macro 'bch2_bkey_ops_inode'
     26 |         .trigger        = bch2_trigger_inode,           \
      |                           ^~~~~~~~~~~~~~~~~~

There are several functions that did not have their flags parameter
converted to 'enum btree_iter_update_trigger_flags' in the recent
unification, which will cause kCFI failures at runtime because the
types, while ABI compatible (hence no warning from the non-strict
version of this warning), do not match exactly.

Fix up these functions (as well as a few other obvious functions that
should have it, even if there are no warnings currently) to resolve the
warnings and potential kCFI runtime failures.

Fixes: 31e4ef3280c8 ("bcachefs: iter/update/trigger/str_hash flag cleanup")
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Kill gc_init_recurse()
Kent Overstreet [Sun, 7 Apr 2024 03:58:01 +0000 (23:58 -0400)]
bcachefs: Kill gc_init_recurse()

This unifies the online and offline btree gc passes; we're not yet
running it online.

We now iterate over one level of the btree at a time - the same as
check_extents_to_backpointers(); this ordering preserves order of keys
regardless of btree splits and merges, which will be important when we
re-enable online gc.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: do reflink_p repair from BTREE_TRIGGER_check_repair
Kent Overstreet [Sun, 7 Apr 2024 23:47:31 +0000 (19:47 -0400)]
bcachefs: do reflink_p repair from BTREE_TRIGGER_check_repair

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Run bch2_check_fix_ptrs() via triggers
Kent Overstreet [Sun, 7 Apr 2024 23:07:09 +0000 (19:07 -0400)]
bcachefs: Run bch2_check_fix_ptrs() via triggers

Currently, the reflink_p gc trigger does repair as well - turning a
reflink_p key into an error key if the reflink_v it points to doesn't
exist.

This won't work with online check/repair, because the repair path once
online will be subject to transaction restarts, but BTREE_TRIGGER_gc is
not idempotant - we can't run it multiple times if we get a transaction
restart.

So we need to split these paths; to do so this patch calls
check_fix_ptrs() by a new general path - a new trigger type,
BTREE_TRIGGER_check_repair.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: kill gc looping for bucket gens
Kent Overstreet [Wed, 17 Apr 2024 02:35:02 +0000 (22:35 -0400)]
bcachefs: kill gc looping for bucket gens

looping when we change a bucket gen is not ideal - it means we risk
failing if we'd go into an infinite loop, and it's better to make
forward progress even if fsck doesn't fix everything.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_bucket_ref_update()
Kent Overstreet [Fri, 19 Apr 2024 23:03:58 +0000 (19:03 -0400)]
bcachefs: bch2_bucket_ref_update()

If we hit an inconsistency when updating allocation information, we
don't want to fail the update if it's for a deletion - only if it's for
a new key.

Rename check_bucket_ref() -> bucket_ref_update() so we can centralize
the logic to do this.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Consolidate mark_stripe_bucket() and trans_mark_stripe_bucket()
Kent Overstreet [Mon, 22 Apr 2024 23:01:40 +0000 (19:01 -0400)]
bcachefs: Consolidate mark_stripe_bucket() and trans_mark_stripe_bucket()

This eliminates some duplicated logic, and the gc path now handles
stripe updates and deletions - we need this since soon we're bringing
back runtime gc.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: mark_stripe_bucket cleanup
Kent Overstreet [Sat, 20 Apr 2024 04:04:07 +0000 (00:04 -0400)]
bcachefs: mark_stripe_bucket cleanup

Start to work on unifying mark_stripe_bucket() and
trans_mark_stripe_bucket(); first, clean up all the unnecessary and
gratuitious differences.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bucket_data_type_mismatch()
Kent Overstreet [Mon, 22 Apr 2024 21:21:35 +0000 (17:21 -0400)]
bcachefs: bucket_data_type_mismatch()

We're working on potentially unifying bch2_check_bucket_ref() and
bch2_check_fix_ptrs() - or at least eliminating gratuitious differences.

Most immediately, there's a bunch of cleanups to be done regarding
BCH_DATA_stripe.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Clean up inode alloc
Kent Overstreet [Sun, 21 Apr 2024 02:03:09 +0000 (22:03 -0400)]
bcachefs: Clean up inode alloc

There's no need to be using new_inode(); we can skip all that
indirection and make the code easier to follow.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: journal seq blacklist gc no longer has to walk btree
Kent Overstreet [Sun, 21 Apr 2024 02:19:48 +0000 (22:19 -0400)]
bcachefs: journal seq blacklist gc no longer has to walk btree

Since btree_ptr_v2, we no longer require the journal seq blacklist table
for skipping blacklisted bsets (btree node entries); the pointer to a
given node indicates how much data is present.

Therefore there's no longer any need for journal seq blacklist gc to
walk the btree - we can prune entries older than journal last_seq.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: plumb data_type into bch2_bucket_alloc_trans()
Kent Overstreet [Sat, 20 Apr 2024 20:10:40 +0000 (16:10 -0400)]
bcachefs: plumb data_type into bch2_bucket_alloc_trans()

prep work for making the allocator try to keep btree nodes within the
existing member info btree allocated bitmap

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Add btree_allocated_bitmap to member_to_text()
Kent Overstreet [Sat, 20 Apr 2024 20:50:46 +0000 (16:50 -0400)]
bcachefs: Add btree_allocated_bitmap to member_to_text()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Btree key cache instrumentation
Kent Overstreet [Sat, 20 Apr 2024 18:49:22 +0000 (14:49 -0400)]
bcachefs: Btree key cache instrumentation

It turns out the btree key cache shrinker wasn't actually reclaiming
anything, prior to the previous patch. This adds instrumentation so that
if we have further issues we can see what's going on.

Specifically, sysfs internal/btree_key_cache is greatly expanded with
new counters, and the SRCU sequence numbers of the first 10 entries on
each pending freelist, and we also add trigger_btree_key_cache_shrink
for testing without having to prune all the system caches.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Remove calls to folio_set_error
Matthew Wilcox (Oracle) [Sat, 20 Apr 2024 02:50:01 +0000 (03:50 +0100)]
bcachefs: Remove calls to folio_set_error

Common code doesn't test the error flag, so we don't need to set it in
bcachefs.  We can use folio_end_read() to combine the setting (or not)
of the uptodate flag and clearing the lock flag.

Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Brian Foster <bfoster@redhat.com>
Cc: linux-bcachefs@vger.kernel.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Move gc of bucket.oldest_gen to workqueue
Kent Overstreet [Sat, 20 Apr 2024 02:44:12 +0000 (22:44 -0400)]
bcachefs: Move gc of bucket.oldest_gen to workqueue

This is a nice cleanup - and we've also been having problems with
kthread creation in the mount path.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: fix flag printing in journal_buf_to_text()
Kent Overstreet [Sat, 20 Apr 2024 00:23:32 +0000 (20:23 -0400)]
bcachefs: fix flag printing in journal_buf_to_text()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Sync journal when we complete a recovery pass
Kent Overstreet [Wed, 17 Apr 2024 06:03:22 +0000 (02:03 -0400)]
bcachefs: Sync journal when we complete a recovery pass

Make things easier when we're debugging long fsck runs - persist the
work that successful recovery passes did.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: make btree read errors silent during scan
Kent Overstreet [Wed, 17 Apr 2024 05:26:02 +0000 (01:26 -0400)]
bcachefs: make btree read errors silent during scan

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Rip bch2_snapshot_equiv() out of fsck
Kent Overstreet [Wed, 17 Apr 2024 03:08:39 +0000 (23:08 -0400)]
bcachefs: Rip bch2_snapshot_equiv() out of fsck

Originally, when deleting snapshots we didn't collapse redundant
snapshot nodes; thus, the notion of a class of equivalent snapshot nodes
leaked into fsck.

Now we do, so snapshot ID equivalence classes are purely local to
snapshot deletion.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Check for writing btree_ptr_v2.sectors_written == 0
Kent Overstreet [Tue, 16 Apr 2024 23:33:56 +0000 (19:33 -0400)]
bcachefs: Check for writing btree_ptr_v2.sectors_written == 0

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Add asserts to bch2_dev_btree_bitmap_marked_sectors()
Kent Overstreet [Tue, 16 Apr 2024 23:33:51 +0000 (19:33 -0400)]
bcachefs: Add asserts to bch2_dev_btree_bitmap_marked_sectors()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: fs_alloc_debug_to_text()
Kent Overstreet [Tue, 16 Apr 2024 04:11:33 +0000 (00:11 -0400)]
bcachefs: fs_alloc_debug_to_text()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: assert that online_reserved == 0 on shutdown
Kent Overstreet [Sun, 14 Apr 2024 23:43:12 +0000 (19:43 -0400)]
bcachefs: assert that online_reserved == 0 on shutdown

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_trans_verify_not_unlocked()
Kent Overstreet [Wed, 10 Apr 2024 00:14:21 +0000 (20:14 -0400)]
bcachefs: bch2_trans_verify_not_unlocked()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_btree_path_can_relock()
Kent Overstreet [Thu, 11 Apr 2024 01:51:37 +0000 (21:51 -0400)]
bcachefs: bch2_btree_path_can_relock()

With the new assertions, we shouldn't be holding locks when
trans->locked is false, thus, we shouldn't use relock when we just want
to check if we can relock.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: trans->locked
Kent Overstreet [Tue, 9 Apr 2024 23:57:08 +0000 (19:57 -0400)]
bcachefs: trans->locked

Add a field for tracking whether a transaction object holds btree locks,
and assertions to verify state.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_btree_root_alloc_fake_trans()
Kent Overstreet [Sun, 14 Apr 2024 22:42:42 +0000 (18:42 -0400)]
bcachefs: bch2_btree_root_alloc_fake_trans()

We're starting to be more strict about transaction locked state, and
multiple transactions in a task.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_trans_unlock() must always be followed by relock() or begin()
Kent Overstreet [Wed, 10 Apr 2024 03:23:08 +0000 (23:23 -0400)]
bcachefs: bch2_trans_unlock() must always be followed by relock() or begin()

We're about to add new asserts for btree_trans locking consistency, and
part of that requires that aren't using the btree_trans while it's
unlocked.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Use bch2_btree_path_upgrade() in key cache traverse
Kent Overstreet [Wed, 10 Apr 2024 04:35:24 +0000 (00:35 -0400)]
bcachefs: Use bch2_btree_path_upgrade() in key cache traverse

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_btree_path_upgrade() checks nodes_locked, not uptodate
Kent Overstreet [Thu, 11 Apr 2024 02:19:40 +0000 (22:19 -0400)]
bcachefs: bch2_btree_path_upgrade() checks nodes_locked, not uptodate

In the key cache fill path, we use path_upgrade() on a path that isn't
uptodate yet but should be locked.

This change makes bch2_btree_path_upgrade() slightly looser so we can
use it in key cache upgrade, instead of the __ version.

Also, make the related assert - that path->uptodate implies nodes_locked
- slightly clearer.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: maintain lock invariants in btree_iter_next_node()
Kent Overstreet [Thu, 11 Apr 2024 22:04:00 +0000 (18:04 -0400)]
bcachefs: maintain lock invariants in btree_iter_next_node()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_trans_commit_flags_to_text()
Kent Overstreet [Sat, 13 Apr 2024 21:49:23 +0000 (17:49 -0400)]
bcachefs: bch2_trans_commit_flags_to_text()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: prefer drop_locks_do()
Kent Overstreet [Sat, 13 Apr 2024 21:40:06 +0000 (17:40 -0400)]
bcachefs: prefer drop_locks_do()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: get_unlocked_mut_path -> bch2_path_get_unlocked_mut
Kent Overstreet [Fri, 12 Apr 2024 19:44:09 +0000 (15:44 -0400)]
bcachefs: get_unlocked_mut_path -> bch2_path_get_unlocked_mut

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: fix typo in reference to BCACHEFS_DEBUG
Lukas Bulwahn [Thu, 11 Apr 2024 08:29:31 +0000 (10:29 +0200)]
bcachefs: fix typo in reference to BCACHEFS_DEBUG

Commit ec9cc18fc2e6 ("bcachefs: Add checks for invalid snapshot IDs")
intends to check the sanity of a snapshot and panic when
BCACHEFS_DEBUG is set, but that conditional has a typo.

Fix the typo to refer to the actual existing Kconfig symbol.

This was found with ./scripts/checkkconfigsymbols.py.

Signed-off-by: Lukas Bulwahn <lukas.bulwahn@redhat.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: chardev: make bch_chardev_class constant
Ricardo B. Marliere [Fri, 8 Mar 2024 12:12:47 +0000 (09:12 -0300)]
bcachefs: chardev: make bch_chardev_class constant

Since commit 43a7206b0963 ("driver core: class: make class_register() take
a const *"), the driver core allows for struct class to be in read-only
memory, so move the bch_chardev_class structure to be declared at build
time placing it into read-only memory, instead of having to be dynamically
allocated at boot time. Also, correctly clean up after failing paths in
bch2_chardev_init().

Cc: Hongbo Li <lihongbo22@huawei.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: member helper cleanups
Kent Overstreet [Fri, 12 Apr 2024 03:31:55 +0000 (23:31 -0400)]
bcachefs: member helper cleanups

Some renaming for better consistency

bch2_member_exists -> bch2_member_alive
bch2_dev_exists -> bch2_member_exists
bch2_dev_exsits2 -> bch2_dev_exists
bch_dev_locked -> bch2_dev_locked
bch_dev_bkey_exists -> bch2_dev_bkey_exists

new helper - bch2_dev_safe

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bucket_valid()
Kent Overstreet [Fri, 12 Apr 2024 01:18:35 +0000 (21:18 -0400)]
bcachefs: bucket_valid()

cut out a branch from doing it the obvious way

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_trans_relock_fail() - factor out slowpath
Kent Overstreet [Tue, 9 Apr 2024 23:45:41 +0000 (19:45 -0400)]
bcachefs: bch2_trans_relock_fail() - factor out slowpath

Factor out slowpath into a separate helper

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_dir_emit() - drop_locks_do() conversion
Kent Overstreet [Wed, 10 Apr 2024 17:18:22 +0000 (13:18 -0400)]
bcachefs: bch2_dir_emit() - drop_locks_do() conversion

Add a new helper that calls dir_emit() and updates ctx->pos on success;
this lets us convert bch2_readdir() to drop_locks_do().

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_btree_insert_trans() no longer specifies BTREE_ITER_cached
Kent Overstreet [Sun, 7 Apr 2024 23:13:45 +0000 (19:13 -0400)]
bcachefs: bch2_btree_insert_trans() no longer specifies BTREE_ITER_cached

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: iter/update/trigger/str_hash flag cleanup
Kent Overstreet [Sun, 7 Apr 2024 22:05:34 +0000 (18:05 -0400)]
bcachefs: iter/update/trigger/str_hash flag cleanup

Combine iter/update/trigger/str_hash flags into a single enum, and
x-macroize them for a to_text() function later.

These flags are all for a specific iter/key/update context, so it makes
sense to group them together - iter/update/trigger flags were already
given distinct bits, this cleans up and unifies that handling.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: __BTREE_ITER_ALL_SNAPSHOTS -> BTREE_ITER_SNAPSHOT_FIELD
Kent Overstreet [Sun, 7 Apr 2024 21:54:00 +0000 (17:54 -0400)]
bcachefs: __BTREE_ITER_ALL_SNAPSHOTS -> BTREE_ITER_SNAPSHOT_FIELD

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: mark_superblock cleanup
Kent Overstreet [Sun, 7 Apr 2024 06:11:03 +0000 (02:11 -0400)]
bcachefs: mark_superblock cleanup

Consolidate mark_superblock() and trans_mark_superblock(), like we did
with the other trigger paths.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: gc_btree_init_recurse() uses gc_mark_node()
Kent Overstreet [Sun, 7 Apr 2024 03:39:12 +0000 (23:39 -0400)]
bcachefs: gc_btree_init_recurse() uses gc_mark_node()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: move root node topo checks to node_check_topology()
Kent Overstreet [Sun, 7 Apr 2024 03:26:36 +0000 (23:26 -0400)]
bcachefs: move root node topo checks to node_check_topology()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: move topology repair kick to gc_btrees()
Kent Overstreet [Sun, 7 Apr 2024 03:20:49 +0000 (23:20 -0400)]
bcachefs: move topology repair kick to gc_btrees()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: kill metadata only gc
Kent Overstreet [Sun, 7 Apr 2024 02:40:12 +0000 (22:40 -0400)]
bcachefs: kill metadata only gc

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: Finish converting reconstruct_alloc to errors_silent
Kent Overstreet [Sun, 7 Apr 2024 21:05:01 +0000 (17:05 -0400)]
bcachefs: Finish converting reconstruct_alloc to errors_silent

with errors_silent, reconstruct_alloc no longer requires fsck and
fix_errors to work

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: bch2_gc() is now private to btree_gc.c
Kent Overstreet [Sat, 6 Apr 2024 04:11:01 +0000 (00:11 -0400)]
bcachefs: bch2_gc() is now private to btree_gc.c

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
13 months agobcachefs: for_each_btree_key_continue()
Kent Overstreet [Sun, 7 Apr 2024 04:52:47 +0000 (00:52 -0400)]
bcachefs: for_each_btree_key_continue()

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>