mm: page_alloc: batch vmstat updates in expand()

expand() currently updates vmstat for every subpage.  This is unnecessary,
since they're all of the same zone and migratetype.

Count added pages locally, then do a single vmstat update.

Link: https://lkml.kernel.org/r/20240327190111.GC7597@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 489dd74d2232a1cc97b4475b2709d7bb4d1c94ae..382d1c98f8e5284906fc7b80dab4f6d4a7a79d22 100644 (file)
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1335,6 +1335,7 @@ static inline void expand(struct zone *zone, struct page *page,
        int low, int high, int migratetype)
 {
        unsigned long size = 1 << high;
+       unsigned long nr_added = 0;
 
        while (high > low) {
                high--;
@@ -1350,9 +1351,11 @@ static inline void expand(struct zone *zone, struct page *page,
                if (set_page_guard(zone, &page[size], high))
                        continue;
 
-               add_to_free_list(&page[size], zone, high, migratetype, false);
+               __add_to_free_list(&page[size], zone, high, migratetype, false);
                set_buddy_order(&page[size], high);
+               nr_added += size;
        }
+       account_freepages(zone, nr_added, migratetype);
 }
 
 static void check_new_page_bad(struct page *page)