mm: compaction: consider the number of scanning compound pages in isolate fail path

commit b717d6b93b54 ("mm: compaction: include compound page count for
scanning in pageblock isolation") added compound page statistics for
scanning in pageblock isolation, to make sure the number of scanned pages
is always larger than the number of isolated pages when isolating
mirgratable or free pageblock.

However, when failing to isolate the pages when scanning the migratable or
free pageblocks, the isolation failure path did not consider the scanning
statistics of the compound pages, which result in showing the incorrect
number of scanned pages in tracepoints or in vmstats which will confuse
people about the page scanning pressure in memory compaction.

Thus we should take into account the number of scanning pages when failing
to isolate the compound pages to make the statistics accurate.

Link: https://lkml.kernel.org/r/73d6250a90707649cc010731aedc27f946d722ed.1678962352.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: William Lam <william.lam@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

diff --git a/mm/compaction.c b/mm/compaction.c
index 709136556b9e1f845fb2aa172ba6489064584a93..0bddf869bbebaebde40435d7ced2a6f456ef51bb 100644 (file)
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -586,6 +586,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
                        if (likely(order <= MAX_ORDER)) {
                                blockpfn += (1UL << order) - 1;
                                cursor += (1UL << order) - 1;
+                               nr_scanned += (1UL << order) - 1;
                        }
                        goto isolate_fail;
                }
@@ -904,6 +905,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
                                if (ret == -EBUSY)
                                        ret = 0;
                                low_pfn += compound_nr(page) - 1;
+                               nr_scanned += compound_nr(page) - 1;
                                goto isolate_fail;
                        }
 
@@ -938,8 +940,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
                         * a valid page order. Consider only values in the
                         * valid order range to prevent low_pfn overflow.
                         */
-                       if (freepage_order > 0 && freepage_order <= MAX_ORDER)
+                       if (freepage_order > 0 && freepage_order <= MAX_ORDER) {
                                low_pfn += (1UL << freepage_order) - 1;
+                               nr_scanned += (1UL << freepage_order) - 1;
+                       }
                        continue;
                }
 
@@ -954,8 +958,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
                if (PageCompound(page) && !cc->alloc_contig) {
                        const unsigned int order = compound_order(page);
 
-                       if (likely(order <= MAX_ORDER))
+                       if (likely(order <= MAX_ORDER)) {
                                low_pfn += (1UL << order) - 1;
+                               nr_scanned += (1UL << order) - 1;
+                       }
                        goto isolate_fail;
                }
 
@@ -1077,6 +1083,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
                         */
                        if (unlikely(PageCompound(page) && !cc->alloc_contig)) {
                                low_pfn += compound_nr(page) - 1;
+                               nr_scanned += compound_nr(page) - 1;
                                SetPageLRU(page);
                                goto isolate_fail_put;
                        }