maple_tree: optimize mas_wr_append(), also improve duplicating VMAs
authorPeng Zhang <zhangpeng.00@bytedance.com>
Wed, 28 Jun 2023 07:36:56 +0000 (15:36 +0800)
committerAndrew Morton <akpm@linux-foundation.org>
Fri, 18 Aug 2023 17:12:05 +0000 (10:12 -0700)
commit23e9dde0b246d47e4a1942ea50bf7fef63e2d41a
treee2b20abd11e18a30cc7aa69853865943bada465a
parentc38d9ff2cc6777f7dc5c02fcb4842402ab5f362a
maple_tree: optimize mas_wr_append(), also improve duplicating VMAs

When the new range can be completely covered by the original last range
without touching the boundaries on both sides, two new entries can be
appended to the end as a fast path. We update the original last pivot at
the end, and the newly appended two entries will not be accessed before
this, so it is also safe in RCU mode.

This is useful for sequential insertion, which is what we do in
dup_mmap(). Enabling BENCH_FORK in test_maple_tree and just running
bench_forking() gives the following time-consuming numbers:

before:               after:
17,874.83 msec        15,738.38 msec

It shows about a 12% performance improvement for duplicating VMAs.

Link: https://lkml.kernel.org/r/20230628073657.75314-4-zhangpeng.00@bytedance.com
Signed-off-by: Peng Zhang <zhangpeng.00@bytedance.com>
Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
lib/maple_tree.c