Kent Overstreet [Tue, 4 Jan 2022 04:38:50 +0000 (23:38 -0500)]
bcachefs: Improved superblock-related error messages
This patch converts bch2_sb_validate() and the .validate methods for the
various superblock sections to take printbuf, to which they can print
detailed error messages, including printing the entire section that was
invalid.
This is a great improvement over the previous situation, where we could
only return static strings that didn't have precise information about
what was wrong.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 3 Jan 2022 09:17:02 +0000 (04:17 -0500)]
bcachefs: Use kvmalloc() for array of sorted keys in journal replay
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 3 Jan 2022 05:22:29 +0000 (00:22 -0500)]
bcachefs: Make eytzinger size parameter more conventional
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 3 Jan 2022 05:14:39 +0000 (00:14 -0500)]
bcachefs: Kill bch2_bset_fix_invalidated_key()
Was dead code, so delete it.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 3 Jan 2022 03:24:43 +0000 (22:24 -0500)]
bcachefs: Fix an assertion
bch2_trans_commit() can legitimately return -ENOSPC with
BTREE_INSERT_NOFAIL set if BTREE_INSERT_NOWAIT was also set.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 3 Jan 2022 02:45:35 +0000 (21:45 -0500)]
bcachefs: bch_dev->dev
Add a field to bch_dev for the dev_t of the underlying block device -
this fixes a null ptr deref in tracepoints.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 04:10:06 +0000 (23:10 -0500)]
bcachefs: Simplify journal replay
With BTREE_ITER_WITH_JOURNAL, there's no longer any restrictions on the
order we have to replay keys from the journal in, and we can also start
up journal reclaim right away - and delete a bunch of code.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 11 Sep 2023 03:35:25 +0000 (23:35 -0400)]
fixup! bcachefs: Factor out __bch2_btree_iter_set_pos()
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 26 Dec 2021 01:07:00 +0000 (20:07 -0500)]
bcachefs: BTREE_ITER_WITH_JOURNAL
This adds a new btree iterator flag, BTREE_ITER_WITH_JOURNAL, that is
automatically enabled when initializing a btree iterator before journal
replay has completed - it overlays the contents of the journal with the
btree.
This lets us delete bch2_btree_and_journal_walk() and just use the
normal btree iterator interface instead - which also lets us delete a
significant amount of duplicated code.
Note that BTREE_ITER_WITH_JOURNAL is still unoptimized in this patch -
we're redoing the binary search over keys in the journal every time we
call bch2_btree_iter_peek().
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 2 Jan 2022 01:45:30 +0000 (20:45 -0500)]
bcachefs: Tweak journal reclaim order
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 2 Jan 2022 00:46:12 +0000 (19:46 -0500)]
bcachefs: Make sure BCH_FS_FSCK_DONE gets set
If we're not running fsck we still want to set BCH_FS_FSCK_DONE, so that
bch2_fsck_err() calls are interpreted as bch2_inconsistent_error()
calls().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 2 Jan 2022 00:04:33 +0000 (19:04 -0500)]
bcachefs: Improve error messages in superblock write path
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 1 Jan 2022 23:27:50 +0000 (18:27 -0500)]
bcachefs: Log what we're doing when repairing
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 31 Dec 2021 22:54:13 +0000 (17:54 -0500)]
bcachefs: Fix race between btree updates & journal replay
Add a flag to indicate whether a journal replay key has been
overwritten, and set/test it with appropriate btree locks held.
This fixes a race between the allocator - invalidating buckets, and
doing btree updates - and journal replay, which before this patch could
clobber the allocator thread's update with an older version of the key
from the journal.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 31 Dec 2021 22:06:29 +0000 (17:06 -0500)]
bcachefs: bch2_journal_entry_to_text()
This adds a _to_text() pretty printer for journal entries - including
every subtype - which will shortly be used by the 'bcachefs
list_journal' subcommand.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 31 Dec 2021 21:12:54 +0000 (16:12 -0500)]
bcachefs: BCH_JSET_ENTRY_log
Add a journal entry type for logging messages, and add an option to use
it to log the transaction name - this makes for a very handy debugging
tool, as with it we can use the 'bcachefs list_journal' command to see
not only what updates were done, but what was doing them.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 31 Dec 2021 01:14:52 +0000 (20:14 -0500)]
bcachefs: Add error messages for memory allocation failures
This adds some missing diagnostics from rare but annoying to debug
runtime allocation failure paths.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 29 Dec 2021 20:55:25 +0000 (15:55 -0500)]
bcachefs: Journal replay does't resort main list of keys
The upcoming BTREE_ITER_WITH_JOURNAL patch will require journal keys to
stay in sorted order, so the btree iterator code can overlay them over
btree keys.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 29 Dec 2021 18:49:34 +0000 (13:49 -0500)]
bcachefs: Add iter_flags arg to bch2_btree_delete_range()
Will be used by the new snapshot tests, to pass in
BTREE_ITER_ALL_SNAPSHOTS.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 02:28:50 +0000 (21:28 -0500)]
bcachefs: Add an error message for copygc spinning
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 29 Dec 2021 16:27:47 +0000 (11:27 -0500)]
bcachefs: Fix keylist size in btree_update
This fixes a buffer overrun, fortunately caught by a BUG_ON().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 21:31:57 +0000 (16:31 -0500)]
bcachefs: Improve error messages in device add path
This converts the error messages in the device add to a better style,
and adds some missing ones.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 21:01:25 +0000 (16:01 -0500)]
bcachefs: bch2_hprint(): don't print decimal if conversion was exact
There's places where we parse these numbers, and our parsing doesn't
cope with decimals currently - this is a hack to get the device_add path
working again where for the device blocksize there doesn't ever need to
be a decimal.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 04:56:13 +0000 (23:56 -0500)]
bcachefs: Optimize bucket reuse
If the btree updates pointing to a bucket were never flushed by the
journal before the bucket became empty again, we can reuse the bucket
without a journal flush.
This tweaks the tracking of journal sequence numbers in alloc keys to
implement this optimization: now, we only update the journal sequence
number in alloc keys on transitions to and from empty. When a bucket
becomes empty, we check if we can tell the journal not to flush entries
starting from when the bucket was used.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 12 Feb 2022 07:32:11 +0000 (02:32 -0500)]
bcachefs: Always check for bucket reuse after read
Since dirty extents can be moved or overwritten, it's not just cached
data that we need the ptr_stale() check in bc2h_read_endio for - this
fixes data checksum errors seen in the tiering ktest tests.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 04:51:48 +0000 (23:51 -0500)]
bcachefs: bch2_journal_noflush_seq()
Add bch2_journal_noflush_seq(), for telling the journal that entries
before a given sequence number should not be flushes - to be used by an
upcoming allocator optimization.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 01:45:07 +0000 (20:45 -0500)]
bcachefs: Add a tracepoint for the btree cache shrinker
This is to help with diagnosing why the btree node can doesn't seem to
be shrinking - we've had issues in the past with granularity/batch size,
since btree nodes are so big.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 01:05:07 +0000 (20:05 -0500)]
bcachefs: Run scan_old_btree_nodes after version upgrade
In the recovery path, we scan for old btree nodes if we don't have
certain compat bits set. If we do this, we should be doing it after we
upgraded to the newest on disk format.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 28 Dec 2021 00:58:12 +0000 (19:58 -0500)]
bcachefs: Update sysfs compression_stats for snapshots
- BTREE_ITER_ALL_SNAPSHOTS flag is required here
- change it to also walk the reflink btree
- change it to accumulate stats for all pointers in an extent
- change it to account for incompressible data
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 27 Dec 2021 03:27:10 +0000 (22:27 -0500)]
bcachefs: Kill bch2_ec_mem_alloc()
bch2_ec_mem_alloc() was only used by GC, and there's no real need to
preallocate the stripes radix tree since we can cope fine with memory
allocation failure when we use the radix tree. This deletes a fair bit
of code, and it's also needed for the upcoming patch because
bch2_btree_iter_peek_prev() won't be working before journal replay
completes (and using it was incorrect previously, as well).
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 27 Dec 2021 02:41:09 +0000 (21:41 -0500)]
bcachefs: Fix allocator + journal interaction
The allocator needs to wait until the last update touching a bucket has
been commited before writing to it again. However, the code was checking
against the last dirty journal sequence number, not the last flushed
journal sequence number.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 00:55:34 +0000 (19:55 -0500)]
bcachefs: New in-memory array for bucket gens
The main in-memory bucket array is going away, but we'll still need to
keep bucket generations in memory, at least for now - ptr_stale() needs
to be an efficient operation.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 03:37:19 +0000 (22:37 -0500)]
bcachefs: Separate out gc_bucket()
Since the main in memory bucket array is going away, we don't want to be
calling bucket() or __bucket() when what we want is the GC in-memory
bucket.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 02:43:29 +0000 (21:43 -0500)]
bcachefs: Put open_buckets in a hashtable
This is so that the copygc code doesn't have to refer to
bucket_mark.owned_by_allocator - assisting in getting rid of the in
memory bucket array.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 02:21:46 +0000 (21:21 -0500)]
bcachefs: Refactor open_bucket code
Prep work for adding a hash table of open buckets - instead of embedding
a bch_extent_ptr, we need to refer to the bucket directly so that we're
not calling sector_to_bucket() in the hash table lookup code, which has
an expensive divide.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 02:14:49 +0000 (21:14 -0500)]
bcachefs: bch2_alloc_sectors_append_ptrs() now takes cached flag
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Sun, 26 Dec 2021 21:59:36 +0000 (16:59 -0500)]
bcachefs: Delete some obsolete journal_seq_blacklist code
Since metadata version bcachefs_metadata_version_btree_ptr_sectors_written,
we haven't needed the journal seq blacklist mechanism for ignoring
blacklisted btree node writes - we now only need it for ignoring journal
entries that were written after the newest flush journal entry, and then
we only need to keep those blacklist entries around until journal replay
is finished.
That means we can delete the code for scanning btree nodes to GC
journal_seq_blacklist entries.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 20 Dec 2021 21:55:49 +0000 (16:55 -0500)]
bcachefs: Journal initialization fixes
This fixes a rare bug when mounting & unmounting RO - flushing a clean
filesystem that never went RO should be a no op.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 26 Dec 2021 01:13:47 +0000 (20:13 -0500)]
bcachefs: Use BTREE_ITER_NOPRESERVE in bch2_btree_iter_verify_ret()
This fixes a transaction path overflow.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 25 Dec 2021 23:40:15 +0000 (18:40 -0500)]
bcachefs: Fix bch2_journal_meta()
This patch ensures that the journal entry written gets written as flush
entry, which is important for the shutdown path - the last entry written
needs to be a flush entry.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 24 Dec 2021 02:35:28 +0000 (21:35 -0500)]
bcachefs: bch2_journal_key_insert() no longer transfers ownership
bch2_journal_key_insert() used to assume that the key passed to it was
allocated with kmalloc(), and on success took ownership. This patch
deletes that behaviour, making it more similar to
bch2_trans_update()/bch2_trans_commit().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 24 Dec 2021 08:08:06 +0000 (03:08 -0500)]
bcachefs: Kill ptr_bucket_mark()
Only used in one place, we can just delete it.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 31 Dec 2021 00:30:42 +0000 (19:30 -0500)]
bcachefs: Don't start allocator threads too early
If the allocator threads start before journal replay has finished
replaying alloc keys, journal replay might overwrite the allocator's
btree updates.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 24 Dec 2021 09:27:01 +0000 (04:27 -0500)]
bcachefs: bch2_bucket_alloc_new_fs() no longer depends on bucket marks
Now that bch2_bucket_alloc_new_fs() isn't looking at bucket marks to
decide what buckets are eligible to allocate, we can clean up the
filesystem initialization and device add paths. Previously, we had to
use ancient code to mark superblock/journal buckets in the in memory
bucket marks as we allocated them, and then zero that out and re-do that
marking using the newer transational bucket mark paths. Now, we can
simply delete the in-memory bucket marking.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 24 Dec 2021 09:22:20 +0000 (04:22 -0500)]
bcachefs: Rewrite bch2_bucket_alloc_new_fs()
This changes bch2_bucket_alloc_new_fs() to a simple bump allocator that
doesn't need to use the in memory bucket array, part of a larger patch
series to entirely get rid of the in memory bucket array, except for
gc/fsck.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 24 Dec 2021 07:55:11 +0000 (02:55 -0500)]
bcachefs: Kill non-lru cache replacement policies
Prep work for persistent LRUs and getting rid of the in memory bucket
array.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Thu, 23 Dec 2021 03:39:50 +0000 (22:39 -0500)]
bcachefs: Fix a null ptr deref in bch2_inode_delete_keys()
Similarly to bch2_btree_delete_range_trans(), bch2_inode_delete_keys()
may sometimes split compressed extents, and needs to pass in a disk
reservation.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 14 Dec 2021 19:34:03 +0000 (14:34 -0500)]
bcachefs: Turn encoded_extent_max into a regular option
It'll now be handled at format time and in sysfs like other options - it
still can only be set at format time, though.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 14 Dec 2021 19:24:41 +0000 (14:24 -0500)]
bcachefs: Option improvements
This adds flags for options that must be a power of two (block size and
btree node size), and options that are stored in the superblock as a
power of two (encoded extent max).
Also: options are now stored in memory in the same units they're
displayed in (bytes): we now convert when getting and setting from the
superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 22 Dec 2021 02:57:10 +0000 (21:57 -0500)]
bcachefs: Fix debugfs -bfloat-failed
It wasn't updated for snapshots - it's iterating across keys in all
snapshots, so needs to be specifying BTREE_ITER_ALL_SNAPSHOTS.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 22 Dec 2021 01:48:26 +0000 (20:48 -0500)]
bcachefs: BTREE_ITER_NOPRESERVE
This adds a flag to not mark the initial btree_path as preserve, for
paths that we expect to be cheap to reconstitute if necessary - this
solves a btree_path overflow caused by need_whiteout_for_snapshot().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 20 Dec 2021 23:18:35 +0000 (18:18 -0500)]
bcachefs: Fix some shutdown path bugs
This fixes some bugs when we hit an error very early in the filesystem
startup path, before most things have been initialized.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 20 Dec 2021 00:02:50 +0000 (19:02 -0500)]
bcachefs: Optimize memory accesses in bch2_btree_node_get()
This puts a load behind some branches before where it's used, so that it
can execute in parallel with other loads.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 20 Dec 2021 17:53:06 +0000 (12:53 -0500)]
bcachefs: Make sure bch2_bucket_alloc_new_fs() obeys buckets_nouse
This fixes the filesystem migrate tool.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 14 Dec 2021 19:24:04 +0000 (14:24 -0500)]
bcachefs: Kill some obsolete sysfs code
fs internal/alloc_debug doesn't show anything bcachefs fs usage shows.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 20 Dec 2021 00:01:41 +0000 (19:01 -0500)]
bcachefs: Don't call bch2_bkey_transform() unnecessarily
If the packed format isn't changing, there's no need to call
bch2_bkey_transform().
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 19 Dec 2021 23:59:22 +0000 (18:59 -0500)]
bcachefs: Kill bch2_sort_repack_merge()
The main function of bch2_sort_repack_merge() was to call .key_normalize
on every key, which drops stale (cached) pointers - it hasn't actually
merged extents in quite some time.
But bch2_gc_gens() now works on individual keys - we used to gc old gens
by rewriting entire btree nodes. With that gone, there's no need for
internal btree code to be calling .key_normalize anymore.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 17 Dec 2021 01:36:26 +0000 (20:36 -0500)]
bcachefs: Split out CONFIG_BCACHEFS_DEBUG_TRANSACTIONS
This puts the btree_transactions sysfs/debugfs file behind a separate
config option - it's highly useful, but not cheap enough to enable
permenantly.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Thu, 16 Dec 2021 01:35:45 +0000 (20:35 -0500)]
bcachefs: Fix an assertion in bch2_truncate()
We recently added an assertion that when we truncate a file to 0,
i_blocks should also go to 0 - but that's not necessarily true if we're
doing an emergency shutdown, lots of invariants no longer hold true in
that case.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 14 Dec 2021 21:05:47 +0000 (16:05 -0500)]
bcachefs: Fix debug build in userspace
This fixes some compiler warnings that only trigger in userspace - dead
code, a maybe uninitialed variable, a maybe null ptr passed to printk.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 14 Dec 2021 05:08:06 +0000 (00:08 -0500)]
bcachefs: Add missing bch2_trans_iter_exit() call
This fixes a bug where the filesystem goes read only when reading from
debugfs.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 11 Dec 2021 02:24:36 +0000 (21:24 -0500)]
bcachefs: Improve alloc_mem_to_key()
This moves some common code into alloc_mem_to_key(), which translates
from the in-memory format for a bucket to the btree key format.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 11 Dec 2021 01:58:44 +0000 (20:58 -0500)]
bcachefs: bch2_alloc_write()
This adds a new helper that much like the one we have for inode updates,
that allocates the packed alloc key, packs it and calls
bch2_trans_update.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Kent Overstreet [Fri, 10 Dec 2021 20:41:38 +0000 (15:41 -0500)]
bcachefs: Add more time_stats
This adds more latency/event measurements and breaks some apart into
more events. Journal writes are broken apart into flush writes and
noflush writes, btree compactions are broken out from btree splits,
btree mergers are added, as well as btree_interior_updates - foreground
and total.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Fri, 10 Dec 2021 19:03:42 +0000 (14:03 -0500)]
bcachefs: Print out OPT_SECTORS options in bytes
This matches the conversion the parsing code does.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Thu, 9 Dec 2021 20:21:26 +0000 (15:21 -0500)]
bcachefs: Fix null ptr deref in fsck_inode_rm()
bch2_btree_delete_range() can split compressed extents, thus needs to
pass in a disk reservation when we're operating on extents btrees.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Thu, 9 Dec 2021 19:19:18 +0000 (14:19 -0500)]
bcachefs: Don't erasure code cached ptrs
It doesn't make much sense to be erasure coding cached pointers, we
should be erasure coding one of the dirty pointers in an extent. This
patch makes sure we're passing BCH_WRITE_CACHED when we expect the new
pointer to be a cached pointer, and tweaks the write path to not
allocate from a stripe when BCH_WRITE_CACHED is set - and fixes an
assertion we were hitting in the ec path where when adding the stripe to
an extent and deleting the other pointers the pointer to the stripe
didn't exist (because dropping all dirty pointers from an extent turns
it into a KEY_TYPE_error key).
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 04:07:33 +0000 (23:07 -0500)]
bcachefs: Split out struct gc_stripe from struct stripe
We have two radix trees of stripes - one that mirrors some information
from the stripes btree in normal operation, and another that GC uses to
recalculate block usage counts.
The normal one is now only used for finding partially empty stripes in
order to reuse them - the normal stripes radix tree and the GC stripes
radix tree are used significantly differently, so this patch splits them
into separate types.
In an upcoming patch we'll be replacing c->stripes with a btree that
indexes stripes by the order we want to reuse them.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 8 Dec 2021 18:31:36 +0000 (13:31 -0500)]
bcachefs: Convert a BUG_ON() to a warning
A user reported hitting this assertion, and we can't reproduce it yet,
but it shouldn't be fatal - so convert it to a warning.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 05:30:49 +0000 (00:30 -0500)]
bcachefs: bch2_trans_update() is now __must_check
With snapshots, bch2_trans_update() has to check if we need a whitout,
which can cause a transaction restart, so this is important now.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 03:03:07 +0000 (22:03 -0500)]
bcachefs: Make __bch2_journal_debug_to_text() more readable
Switch to one line of output per pr_buf() call - longer lines but quite
a bit more readable.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 02:53:13 +0000 (21:53 -0500)]
bcachefs: Specify filesystem options
We've got three types of options now - filesystem, device and inode, and
a given option may belong to more than one of those types.
This patch changes the options to specify explicitly when they're a
filesystem option - in the future we'll probably be adding more device
options.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 02:52:09 +0000 (21:52 -0500)]
bcachefs: Fix copygc sectors_to_move calculation
With erasure coding, copygc's count of sectors to move was off, which
matters for the debug statement it prints out when it's not able to move
all the data it tried to.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 5 Dec 2021 01:07:19 +0000 (20:07 -0500)]
bcachefs: Convert journal sysfs params to regular options
This converts journal_write_delay, journal_flush_disabled, and
journal_reclaim_delay to normal filesystems options, and also adds them
to the superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 1 Dec 2021 08:47:54 +0000 (03:47 -0500)]
bcachefs: Fix btree_path leaks in bch2_trans_update()
bch2_trans_update() had some dodgy gets() and puts() - this fixes a few
leaks.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 29 Nov 2021 21:38:27 +0000 (16:38 -0500)]
bcachefs: Erasure coding fixes
When we added the stripe and stripe_redundancy fields to alloc keys, we
neglected to add them to the functions that convert back and forth with
the in-memory types.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 28 Nov 2021 20:13:54 +0000 (15:13 -0500)]
bcachefs: Handle replica marking fsck errors locally
This simplifies the code quite a bit and eliminates an inconsistency - a
given bkey doesn't necessarily translate to a single replicas entry for
disk space accounting.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 28 Nov 2021 19:31:19 +0000 (14:31 -0500)]
bcachefs: Push c->mark_lock usage down to where it is needed
This changes the bch2_mark_key() and related paths to take mark lock
where it is needed, instead of taking it in the upper transaction commit
path - by pushing down locking we'll be able to handle fsck errors
locally instead of requiring a separate check in the btree_gc code for
replicas being marked.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 28 Nov 2021 19:08:58 +0000 (14:08 -0500)]
bcachefs: Kill bch2_replicas_delta_list_marked()
This changes bch2_trans_fs_usage_apply() to handle failure (replicas
entry missing) by reverting the changes it made - meaning we can make
the main transaction commit path a bit slimmer, and perhaps also
simplify some locking in upcoming patches.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 29 Nov 2021 21:36:50 +0000 (16:36 -0500)]
bcachefs: Fix reflink path for snapshots
make_extent_indirect() was missing the
BTREE_UPDATE_INTERNAL_SNAPSHOT_NODE - it's updating the extent in the
original snapshot, not the curret one.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 28 Nov 2021 18:42:05 +0000 (13:42 -0500)]
bcachefs: Convert bucket_alloc_ret to negative error codes
Start a new header, errcode.h, for bcachefs-private error codes - more
error codes will be converted later.
This patch just converts bucket_alloc_ret so that they can be mixed with
standard error codes and passed as ERR_PTR errors - the ec.c code was
doing this already, but incorrectly.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 23 Nov 2021 23:21:09 +0000 (18:21 -0500)]
bcachefs: Fix page state after fallocate
This tweaks the fallocate code to also update the page cache to reflect
the new on disk reservations, giving us better i_sectors consistency.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 23 Nov 2021 23:17:04 +0000 (18:17 -0500)]
bcachefs: Fix page state when reading into !PageUptodate pages
This patch adds code to read page state before writing to pages that
aren't uptodate, which corrects i_sectors being tempororarily too large
and means we may not need to get a disk reservation.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
# Conflicts:
# fs/bcachefs/fs-io.c
Kent Overstreet [Wed, 24 Nov 2021 01:00:34 +0000 (20:00 -0500)]
bcachefs: Kill PAGE_SECTOR_SHIFT
Replace it with the new, standard PAGE_SECTORS_SHIFT
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 24 Nov 2021 00:00:23 +0000 (19:00 -0500)]
bcachefs: Apply workaround for too many btree iters to read path
Reading from cached data, which calls bch2_bucket_io_time_reset(), is
leading to transaction iterator overflows - this standardizes the
workaround.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 27 Oct 2021 17:05:56 +0000 (13:05 -0400)]
bcachefs: Fix quota support for snapshots
Quota support was disabled when snapshots were released, because of some
tricky interactions with snpashots. We're sidestepping that for now -
we're simply disabling quota accounting on snapshot subvolumes.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 23 Nov 2021 22:05:56 +0000 (17:05 -0500)]
bcachefs: SECTOR_DIRTY_RESERVED
This fixes another i_sectors accounting bug - we need to differentiate
between dirty writes that overwrite a reservation and dirty writes to
unallocated space - dirty writes to unallocated space increase
i_sectors, dirty writes over a reservation do not.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 22 Nov 2021 17:47:20 +0000 (12:47 -0500)]
bcachefs: Fix i_sectors_leak in bch2_truncate_page
When bch2_truncate_page() discards dirty sectors in the page cache, we
need to account for that - we don't need to account for allocated
sectors because that'll be done by the bch2_fpunch() call when it
updates the btree.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 22 Nov 2021 03:34:26 +0000 (22:34 -0500)]
bcachefs: Fix an i_sectors accounting bug
We weren't checking for errors before calling i_sectors_acct()
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 21 Nov 2021 21:15:48 +0000 (16:15 -0500)]
bcachefs: Fix BCH_FS_ERROR flag handling
We were setting BCH_FS_ERROR on startup if the superblock was marked as
containing errors, which is not what we wanted - BCH_FS_ERROR indicates
whether errors have been found, so that after a successful fsck we're
able to clear the error bit in the superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 21 Nov 2021 03:59:25 +0000 (22:59 -0500)]
bcachefs: Add a bit of missing repair code
This adds repair code to drop very stale pointers.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 15 Nov 2021 22:30:11 +0000 (17:30 -0500)]
bcachefs: Fix error reporting from bch2_journal_flush_seq
- bch2_journal_halt() was unconditionally overwriting j->err_seq, the
sequence number that we failed to write
- journal_write_done was updating seq_ondisk and flushed_seq_ondisk even
for writes that errored, which broke the way bch2_journal_flush_seq_async()
locklessly checked for completions.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Wed, 27 Oct 2021 16:51:12 +0000 (12:51 -0400)]
bcachefs: Run insert triggers before overwrite triggers
Currently, btree triggers are run in natural key order, which presents a
problem for fallocate in INSERT_RANGE mode: since we're moving existing
extents to higher offsets, the trigger for deleting the old extent runs
before the trigger that adds the new extent, potentially leading to
indirect extents being deleted that shouldn't be when the delete causes
the refcount to hit 0.
This changes the order we run triggers so that for a givin btree, we run
all insert triggers before overwrite triggers, nicely sidestepping this
issue.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 15 Nov 2021 20:02:13 +0000 (15:02 -0500)]
bcachefs: Disk space accounting fix on brand-new fs
The filesystem initialization path first marks superblock and journal
buckets non transactionally, since the btree isn't functional yet. That
path was updating the per-journal-buf percpu counters via
bch2_dev_usage_update(), and updating the wrong set of counters so those
updates didn't get written out until journal entry 4.
The relevant code is going to get significantly rewritten in the future
as we transition away from the in memory bucket array, so this just
hacks around it for now.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Mon, 15 Nov 2021 20:03:06 +0000 (15:03 -0500)]
bcachefs: Also log device name in userspace
Change log messages in userspace to be closer to what they are in kernel
space, and include the device name - it's also useful in userspace.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sun, 14 Nov 2021 00:49:14 +0000 (19:49 -0500)]
bcachefs: Update export_operations for snapshots
When support for snapshots was merged, export operations weren't
updated yet. This patch adds new filehandle types for bcachefs that
include the subvolume ID and updates export operations for subvolumes -
and also .get_parent, support for which was added just prior to
snapshots.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Tue, 26 Oct 2021 21:35:58 +0000 (17:35 -0400)]
bcachefs: Refactor journal replay code
This consolidates duplicated code in journal replay - it's only a few
flags that are different for replaying alloc keys.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 13 Nov 2021 22:57:52 +0000 (17:57 -0500)]
bcachefs: Fix missing field initialization
When unpacking v1 inodes, we were failing to initialize the journal_seq
field, leading to a BUG_ON() when fsync tries to flush a garbage journal
sequence number.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 13 Nov 2021 22:53:55 +0000 (17:53 -0500)]
bcachefs: Convert journal BUG_ON() to a warning
It's definitely indicative of a bug if we request to flush a journal
sequence number that hasn't happened yet, but it's more useful if we
warn and print out the relevant sequence numbers instead of just dying.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
Kent Overstreet [Sat, 13 Nov 2021 22:44:13 +0000 (17:44 -0500)]
bcachefs: Improve bch2_reflink_p_to_text()
.to_text methods generally ought to print all the value fields.
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>