Merge tag 'pull-qapi-2023-04-28' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2023-04-28

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmRLlpsSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTTjwP/iezbnBFv55HxaRFisEpBom7skXakV1r
# 0KTyYNo0BatS01yVlX7IOh37ZVmoBHrpbs0SLb2V5/uVALQBvmlnhUCCQ+BJwhJF
# JON6yjZRKUikVKn8CGj0ql9KDF6bC/T1ByFEotGc7SBMCx+FWuGXnmVzwOKOK2O3
# XAHG1P04wlxuyHMt3V7qqbc5IyzZIEHLbEitDhQM/6quQmwx4NzFR+JwvzUXCok1
# F7cEt0cKbE3JZHSXWIVTOuseeZR8vnk3giKBgFOetqhVorGusiTPMQwpDPvvgKGn
# fkg2l5SZQz9hSDfuzCKpJSQDqO8ji+62fHci9+PPpG10raoMiwCiuhP9yYECYekM
# 7BcAY7fItrUB+x+vjm8JRY15JxOT8ouHLfyknNLOwow8QwtsiNn35VlRJU7O+k+D
# gk9qvwl1RoLUSXzOo6Ye1NvzF8oMZ8o8f0vmYC+W3sdg89U9QnRrpCaX8YCytLnd
# rEXJiL26mg54yXrWShIzLvC33//sz7AvH6IjwrRt0ZQu7ANRJsNmYY9S1THAJBKW
# b9hY3Aosw8Advt0RUFkYr/PndZp/gxDpbtOCIkKZkti6XRkY89ExPk6zVmeo8miw
# DeEo1kRtzKYDC7ZU0yp73MZDSmUt17L1AfdgS0QuCQCfPHsa8LwtZoQmRIihT/Bo
# GZTLIZPiVC79
# =mkwX
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 28 Apr 2023 10:49:15 AM BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2023-04-28' of https://repo.or.cz/qemu/armbru:
  docs/devel/qapi-code-gen: Describe some doc markup pitfalls
  qapi storage-daemon/qapi: Fix documentation section structure
  qapi: Format since information the conventional way: (since X.Y)
  qapi: Fix misspelled section tags in doc comments
  qapi: Replace ad hoc "since" documentation by member documentation
  qapi: Fix argument documentation markup
  qga/qapi-schema: Fix member documentation markup
  qapi: Fix unintended definition lists in documentation
  qapi: Fix bullet list markup in documentation
  qapi: Delete largely misleading "Stability Considerations"
  qapi: Tidy up examples
  qapi: @foo should be used to reference, not ``foo``
  qapi/block-core: Clean up after removal of dirty bitmap @status
  qapi: Fix up references to long gone error classes
  qapi: Fix misspelled references
  qga/qapi-schema: Fix a misspelled reference
  qga/qapi-schema: Tidy up documentation of guest-fsfreeze-status

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

Merge tag 'pull-request-2023-04-28' of https://gitlab.com/thuth/qemu into staging

* Prevent reentrant DMA accesses by default
* Only compile hw/rdma code when necessary
* Fix a potential locking issue in the vhost-user-test
* Offer more registers in GDB for s390x TCG

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmRLlDkRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbU98RAApVbehXIXMF4lm0B4/pTRlP7CEjbKvNdQ
# Ca/xuqiCctjisJWQMLf8WDdymzaA02ycSYo0X9diO4uisKog4kN4uYje3VTbSJnx
# 7a0MlOpUqP6AR3hZYy5y/wv1M7Yjfhnj7yTUsAo/mQYoZmvqRviNwJ6ekdIU9J4W
# V2EfY+R/ebNami9VvuHTqudcZxmmR+4S5m5JipeHvjFE0DJucES0S8sN3W+iSCxG
# ydqG02CUoZtoorkJZccy3FPyeTVHgvcNqJbhAA5pFBgo30L+mx0llqztzLnWnLCe
# GWvpW2lZ2jC5MOg8oEpbiR807eB3ITpzCu9SX0IUOxMUHpGtl49sx1WNFrrlgD4Z
# b+mZSwJx4grwkU8iD/nF0apQoqEIAH6VSyRAIq1yLAsq1H4/K4c8M+bZ/jAD+gw+
# +DujZIPJ6Gcu3BeqWgOcOY1rRol3ksmRYlU8GrHTR/zIU+q3h7Gk3aH4+Ore1qnE
# hI/YzyyZlQgFMvENqakPJIS9IH+h41gCltwE8YxqJyUWIBfRs7iBMzdxQuUW+c2b
# n6OPYqeDlQt4hEpq4mUtZ+aJV1ey6kyk5ESLTj8QaJiZtcT1+lmvZX2ug2oeyrJm
# ys2458ulKZu0rGjXWWXNFB8u3MM9xN/PLqeD/wKShyLzkit5Nfx4azw5LPbmjrbj
# Twmf1HGeUiY=
# =v84A
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 28 Apr 2023 10:39:05 AM BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [undefined]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [undefined]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2023-04-28' of https://gitlab.com/thuth/qemu:
  apic: disable reentrancy detection for apic-msi
  raven: disable reentrancy detection for iomem
  bcm2835_property: disable reentrancy detection for iomem
  lsi53c895a: disable reentrancy detection for script RAM
  hw: replace most qemu_bh_new calls with qemu_bh_new_guarded
  checkpatch: add qemu_bh_new/aio_bh_new checks
  async: Add an optional reentrancy guard to the BH API
  memory: prevent dma-reentracy issues
  tests: vhost-user-test: release mutex on protocol violation
  hw/rdma: VMW_PVRDMA should depend on VMXNET3_PCI
  hw/rdma: Compile target-independent parts of the rdma code only once
  hw/rdma: Remove unused macros PG_DIR_SZ and PG_TBL_SZ
  s390x/gdb: Split s390-virt.xml

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

docs/devel/qapi-code-gen: Describe some doc markup pitfalls

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230427095346.1238913-1-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

qapi storage-daemon/qapi: Fix documentation section structure

In the QEMU QMP Reference Manual, subsection "Block core (VM
unrelated)" is empty.  Its contents is at the end of subsection
"Background jobs" instead.  That's because qapi/job.json is included
first from qapi/block-core.json, which makes qapi/job.json's
documentation go between qapi/block-core.json's subsection heading and
contents.

In the QEMU Storage Daemon QMP Reference Manual, section "Block
Devices" contains nothing but an empty subsection "Block core (VM
unrelated)".  The latter's contents is at the end section "Socket data
types", along with subsection "Block device exports".  Subsection
"Background jobs" is at the end of section "Cryptography".  All this
is because storage-daemon/qapi/qapi-schema.json includes modules in a
confused order.

Fix both as follows.

Turn subsection "Background jobs" into a section.

Move it before section "Block devices" in the QEMU QMP Reference
Manual, by including qapi/jobs.json right before qapi/block.json.

Reorder include directives in storage-daemon/qapi/qapi-schema.json to
match the order in qapi/qapi-schema.json, so that the QEMU Storage
Daemon QMP Reference Manual's section structure the QEMU QMP Reference
Manual's.

In the QEMU QMP Reference Manual, qapi/cryptodev.json's documentation
is at the end of section "Virtio devices".  That's because it lacks a
section heading, and therefore gets squashed into whatever section
happens to precede it.

Add section heading so it's in section "Cryptography devices".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: zhenwei pi <pizhenwei@bytedance.com>
Message-Id: <20230425064223.820979-17-armbru@redhat.com>

qapi: Format since information the conventional way: (since X.Y)

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-16-armbru@redhat.com>

qapi: Fix misspelled section tags in doc comments

Section tags are case sensitive and end with a colon.  Screwing up
either gets them interpreted as ordinary paragraph.  Fix a few.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-15-armbru@redhat.com>

qapi: Replace ad hoc "since" documentation by member documentation

MemoryDeviceInfoKind, NetClientDriver, and GuestPanicAction mention
some members only in ad hoc since documentation.  The generated
documentation shows these members as "Not documented".

Replace by formal member documentation.

Add actual documentation text for the GuestPanicAction members, to
match existing member documentation there.  For the others, merely
move existing "since" information.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-14-armbru@redhat.com>

qapi: Fix argument documentation markup

Member / argument documentation of BlockdevAmendOptionsQcow2,
job-resume, and RDMA_GID_STATUS_CHANGED is parsed as ordinary text due
to missing colon or space before the colon.  The generated
documentation shows these members / arguments as "Not documented".

The fix is obvious: add missing colons, delete extra spaces.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-13-armbru@redhat.com>

qga/qapi-schema: Fix member documentation markup

GuestDiskStatsInfo's member documentation is parsed as ordinary text
due to missing colons.  The generated documentation shows these
members as "Not documented".

The fix is obvious: add the missing colons.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-12-armbru@redhat.com>

qapi: Fix unintended definition lists in documentation

rST parses something like

    first line
        second line

as a definition list item, where "first line" is the term being
defined by "second line".

This bites us in a couple of places.  Here's one:

    # @bps_max: total throughput limit during bursts,
    #                     in bytes (Since 1.7)

scripts/qapi/parser.py parses this into an "argument section" with
name "bps_max" and text

    total throughput limit during bursts,
      in bytes (Since 1.7)

docs/sphinx/qapidoc.py duly passes the text to the rST parser, which
parses it as another definition list.  Comes out as nested
definitions: term "bps_max: int (optional)" defined as term "total
throughput limit during bursts," defined as "in bytes (Since 1.7)".

rST truly is the Perl of ASCII-based markups.

Fix by deleting the extra indentation.

Fixes: 26ec4e53f2bf (qapi: Fix indent level on doc comments in json files)
Fixes: c0ac533b6f97 (qapi: Stop using whitespace for alignment in comments)
Fixes: 81ad2964e938 (net/vmnet: add vmnet backends to qapi/net)
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230425064223.820979-11-armbru@redhat.com>

qapi: Fix bullet list markup in documentation

Peter Maydell's commit 100cc4fe0f08 explains:

    rST insists on a blank line before and after a bulleted list [...]
    Add some extra blank lines in the doc comments so they're
    acceptable rST input.

It missed one in qapi/trace.json.

Paolo Bonzini later added another instance in qapi/stats.json,
providing further, if unintended, evidence for his quip that rST is
the Perl of ASCII-based markups.

Both are parsed as ordinary paragraph, resulting in garbled output.

John Snow missed the need for a blank line when converting
docs/devel/qapi-code-gen.txt to rST.

Add the blank lines we need to get the bullet lists recognized as
such.

Kevin Wolf and Lukas Straub added two more, but indented.  Sphinx
recognizes them as (indented) bullet lists.  The indentation looks
slightly off.

Insert a blank line and delete the extra indentation.

Fixes: 100cc4fe0f0827f8da1a5c05f9c65e2aaa40e03d (qapi: Add blank lines before bulleted lists)
Fixes: 467ef823d83e (qmp: add filtering of statistics by target vCPU)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-10-armbru@redhat.com>
[Fix of docs/devel/qapi-code-gen.rst squashed, commit message adjusted]

qapi: Delete largely misleading "Stability Considerations"

Documentation section "Stability Considerations" dates back to the
early days of QMP (commit 82a56f0d83d (Monitor: Introduce the
qmp-commands.hx file)).  It became largely misleading years ago.
Delete it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-9-armbru@redhat.com>

qapi: Tidy up examples

A few examples neglect to prefix QMP input with '->'.  Fix that.

Two examples have extra space after '<-'.  Delete it.

A few examples neglect to show output.  Provide some.  The example
output for query-vcpu-dirty-limit could use further improvement.  Add
a TODO comment.

Use "Examples:" instead of "Example:" where multiple examples are
given.

One example section numbers its two examples.  Not done elsewhere;
drop.

Another example section separates them with "or".  Likewise.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-8-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>

qapi: @foo should be used to reference, not ``foo``

Documentation suggests @foo is merely shorthand for ``foo``.  It's
not, it carries additional meaning: it's a reference to a QAPI schema
name.

Reword the documentation to spell that out.

Fix up the few ``foo`` that should be @foo.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-7-armbru@redhat.com>

qapi/block-core: Clean up after removal of dirty bitmap @status

Commit 81cbfd50886 (block: remove dirty bitmaps 'status' field)
removed deprecated BlockDirtyInfo member @status.  It neglected to
remove references to its enumeration values from the documentation of
its replacements.  Do that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20230425064223.820979-6-armbru@redhat.com>

qapi: Fix up references to long gone error classes

Commit de253f14912e88f4 (qmp: switch to the new error format on the
wire) removed most error classes.  Several later commits mistakenly
mentioned them in documentation.  Replace them by the actual error
class there.

Fixes: 44e3e053af56 (qmp: add interface blockdev-snapshot-delete-internal-sync)
Fixes: f323bc9e8b3b (qmp: add interface blockdev-snapshot-internal-sync)
Fixes: ba1c048a8f9c (qapi: Introduce add-fd, remove-fd, query-fdsets)
Fixes: ed61fc10e8c8 (QAPI: add command for live block commit, 'block-commit')
Fixes: e4c8f004c55d (qapi: convert sendkey)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-5-armbru@redhat.com>

qapi: Fix misspelled references

query-cpu-definitions returns a list of CpuDefinitionInfo, but
documentation claims CpuDefInfo, which doesn't exist.

query-migrate-capabilities returns a list of
MigrationCapabilityStatus, but documentation claims
MigrationCapabilitiesStatus, which doesn't exist.

balloon and query-balloon can fail with KVMMissingCap, but
documentation claims KvmMissingCap, which doesn't exist.

Fix the documentation.

Fixes: e4e31c6324af (qapi: add query-cpu-definitions command (v2))
Fixes: bbf6da32b5bd (Add migration capabilities)
Fixes: d72f326431e2 (qapi: Convert balloon)
Fixes: 96637bcdf9e0 (qapi: Convert query-balloon)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230425064223.820979-4-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>

qga/qapi-schema: Fix a misspelled reference

Code returns a list of GuestNetworkInterface, documentation claims
GuestNetworkInfo, which doesn't exist.  Fix the documentation.

Fixes: 3424fc9f16a1 (qemu-ga: add guest-network-get-interfaces command)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-3-armbru@redhat.com>

qga/qapi-schema: Tidy up documentation of guest-fsfreeze-status

Delete "error state indicates", because it doesn't make sense.
I suspect it was an accident.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230425064223.820979-2-armbru@redhat.com>

apic: disable reentrancy detection for apic-msi

As the code is designed for re-entrant calls to apic-msi, mark apic-msi
as reentrancy-safe.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20230427211013.2994127-9-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

raven: disable reentrancy detection for iomem

As the code is designed for re-entrant calls from raven_io_ops to
pci-conf, mark raven_io_ops as reentrancy-safe.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20230427211013.2994127-8-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

bcm2835_property: disable reentrancy detection for iomem

As the code is designed for re-entrant calls from bcm2835_property to
bcm2835_mbox and back into bcm2835_property, mark iomem as
reentrancy-safe.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230427211013.2994127-7-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

lsi53c895a: disable reentrancy detection for script RAM

As the code is designed to use the memory APIs to access the script ram,
disable reentrancy checks for the pseudo-RAM ram_io MemoryRegion.

In the future, ram_io may be converted from an IO to a proper RAM MemoryRegion.

Reported-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20230427211013.2994127-6-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

hw: replace most qemu_bh_new calls with qemu_bh_new_guarded

This protects devices from bh->mmio reentrancy issues.

Thanks: Thomas Huth <thuth@redhat.com> for diagnosing OS X test failure.
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230427211013.2994127-5-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

checkpatch: add qemu_bh_new/aio_bh_new checks

Advise authors to use the _guarded versions of the APIs, instead.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20230427211013.2994127-4-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

async: Add an optional reentrancy guard to the BH API

Devices can pass their MemoryReentrancyGuard (from their DeviceState),
when creating new BHes. Then, the async API will toggle the guard
before/after calling the BH call-back. This prevents bh->mmio reentrancy
issues.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20230427211013.2994127-3-alxndr@bu.edu>
[thuth: Fix "line over 90 characters" checkpatch.pl error]
Signed-off-by: Thomas Huth <thuth@redhat.com>

memory: prevent dma-reentracy issues

Add a flag to the DeviceState, when a device is engaged in PIO/MMIO/DMA.
This flag is set/checked prior to calling a device's MemoryRegion
handlers, and set when device code initiates DMA.  The purpose of this
flag is to prevent two types of DMA-based reentrancy issues:

1.) mmio -> dma -> mmio case
2.) bh -> dma write -> mmio case

These issues have led to problems such as stack-exhaustion and
use-after-frees.

Summary of the problem from Peter Maydell:
https://lore.kernel.org/qemu-devel/CAFEAcA_23vc7hE3iaM-JVA6W38LK4hJoWae5KcknhPRD5fPBZA@mail.gmail.com

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/62
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/540
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/541
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/556
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/557
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/827
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1282
Resolves: CVE-2023-0330

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230427211013.2994127-2-alxndr@bu.edu>
[thuth: Replace warn_report() with warn_report_once()]
Signed-off-by: Thomas Huth <thuth@redhat.com>

Merge tag 'migration-20230427-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request (20230427 edition)

Hi

Everything that has been reviewed:
- stat64_set() by paolo
- atomic_counters series fully reviewed (juan)
- move capabilities to options.c fully reviewed (juan)
- fix the channels_ready semaphore (juan)
- multifd flush optimization reviewed (juan)

Please, apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmRKi3YACgkQ9IfvGFhy
# 1yPtERAAgkBt4g3tpn90q+QeFLBPyo0KGxX4UiAmh8aiGRAJzmSJfjaQG3RgK0vM
# Iyat60Tx6lpHta7xRE+pDF7tXmoi0631B5FJjfuyTIFGWdrOpjGHihuxWdnIjK0H
# ZhR3LfQvHqybMfiyZN/7T5/qICB0zZ+erIeoaurRU3RVrO8yRPM+VBwsYr1RW1S4
# DxUeXEJF09tat7TgiQYp2w5cMPcQ2AvzgViR2r7yc0WjY2yUscyMjuPbHPqAsYWB
# UkbNuH7M/quvHqGvgxGYEQ+a6wyBzKlissjxWe8KTuSLNML2RVR6IrpuEPe6XyGe
# FWi+XRB5114EMy7YjjgTWzcLMC9EmhpzKTgJi0NEtL+cGRb9ZaSaSLE5yO7GLF9b
# efhCXT9jIef9/3Hi+Kr0juJ3nX6c7pINTAsx7wxKU0phz5kYx37IqIioMlVDh1ew
# ECpH+8Syrp/zYytc/HIpyOwIR9VdZrRi6jr2yqcc9lLLcuNUzl+E+9mAxWzO/gef
# +cYPnQZC0rjIzJSBZjq7TXYFNblD6HreONGL8UdFN0B7Js5mPu2PU3449jYNSOxi
# 1aj/eHNXEcqKcVZ3GDGgtus3ARvFsbEba0UZYIVboKp5i96f8O9YqKvRdBjAjpq1
# IDT/fvI2Fj1mjKi8KuAq8fK5utm+g4NR9WkQX0d661dQNZ3XqVc=
# =NE9v
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 27 Apr 2023 03:49:26 PM BST
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [undefined]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20230427-pull-request' of https://gitlab.com/juan.quintela/qemu:
  migration: Make dirty_bytes_last_sync atomic
  migration: Make dirty_pages_rate atomic
  stat64: Add stat64_set() operation
  multifd: Only flush once each full round of memory
  multifd: Protect multifd_send_sync_main() calls
  multifd: Create property multifd-flush-after-each-section
  migration: Move migration_properties to options.c
  migration: Create migrate_block_bitmap_mapping() function
  migration: Create migrate_tls_hostname() function
  migration: Create migrate_tls_authz() function
  migration: Create migrate_tls_creds() function
  migration: Remove MigrationState from block_cleanup_parameters()
  migration: Move block_cleanup_parameters() to options.c
  migration: Move migrate_set_block_incremental() to options.c
  migration: Create migrate_downtime_limit() function
  migration: Make all functions check have the same format
  migration: Create migrate_params_init() function
  multifd: Fix the number of channels ready

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

tests: vhost-user-test: release mutex on protocol violation

chr_read() is printing an error message and returning with s->data_mutex taken.
This can potentially cause a hang.  Reported by Coverity.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230427125423.103536-1-pbonzini@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

hw/rdma: VMW_PVRDMA should depend on VMXNET3_PCI

The "pvrdma" device is only usable in conjunction with the "vmxnet3"
NIC - see the check for TYPE_VMXNET3 in pvrdma_realize().

By adding this dependency, the amount of total files that have to
be compiled for a configuration with all targets decreases by 64
files (!), since the rdma code is marked as target specific and thus
got recompiled for all targets that enable PCI so far.

Message-Id: <20230419111337.651673-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

hw/rdma: Compile target-independent parts of the rdma code only once

Some files of the rdma code do not depend on any target specific
macros. Compile these only once to save some time during the build.

Message-Id: <20230419114937.667221-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

hw/rdma: Remove unused macros PG_DIR_SZ and PG_TBL_SZ

They have apparently never been used.

Message-Id: <20230419103018.627115-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

s390x/gdb: Split s390-virt.xml

Both TCG and KVM emulate ckc, cputm, last_break and prefix, and it's
quite useful to have them during debugging. Right now they are grouped
together with KVM-only pp, pfault_token, pfault_select and
pfault_compare in s390-virt.xml, and are not available when debugging
TCG-emulated code.

Move KVM-only registers into the new s390-virt-kvm.xml file. Advertise
s390-virt.xml always, and the new s390-virt-kvm.xml only for KVM.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230314101813.174874-1-iii@linux.ibm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

Merge tag 'pull-testing-docs-270423-1' of https://gitlab.com/stsquad/qemu into staging

Testing and documentation updates:

  - bump avocado to 101.0
  - use snapshots for tuxrun baseline tests
  - add sbda-ref test to avocado
  - avoid spurious re-configure in gitlab
  - better description of blockdev options
  - drop FreeBSD 12 from Cirrus CI
  - fix up the ast2[56]00 tests to be more stable
  - improve coverage of ppc64 tests in tuxrun baselines
  - limit plugin tests to just the generic multiarch binaries

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmRKgI0ACgkQ+9DbCVqe
# KkQG9Qf/fvSaiNvkttmb0OFDf3+Qz1uQ33YzLZxETCwI1cSqpGZbssQUrTJZWgpu
# c7FHzdOnTem3Q3PsBY9UN5oqm5IraXEu+nZRO+QyCwHEZzdL9DBfJs46La4BkWG6
# 9vXbNtXLUPd6qJy9ntcZzRShrYy0x0KeszDq7371LS/fng+zMtaIhm4ck0fVWKnj
# htrZEN6nn+CqEnvOc06ICmxiysUVLGRScWKgAHCS9ORGyOtZsj3vWafBoIC6hwzi
# oM3kebhFsVOdbGdL0ZiBdHZqUGAEq3gr+3CpX/48bQ0pYnuYMX8iHk1FhqEK7Adk
# H9ZLnpYUXVyt70rJUjImPyIHpdyq0A==
# =fCif
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 27 Apr 2023 03:02:53 PM BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-testing-docs-270423-1' of https://gitlab.com/stsquad/qemu:
  docs/style: call out the use of GUARD macros
  docs/devel: mention the spacing requirement for QOM
  docs/devel: make a statement about includes
  docs/system: remove excessive punctuation from guest-loader docs
  qemu-options.hx: Update descriptions of memory options for NUMA node
  tests/tcg: limit the scope of the plugin tests
  tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
  avocado_qemu/__init__.py: factor out the qemu-img finding
  MAINTAINERS: Cover tests/avocado/machine_aspeed.py
  tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
  tests/avocado: Make ssh_command_output_contains() globally available
  .gitlab-ci.d/cirrus: Drop the CI job for compiling with FreeBSD 12
  qemu-options: finesse the recommendations around -blockdev
  scripts/device-crash-test: Add a parameter to run with TCG only
  gitlab-ci: Avoid to re-run "configure" in the device-crash-test jobs
  tests/avocado: Add set of boot tests on SBSA-ref
  tests/avocado: use the new snapshots for testing
  tests/requirements.txt: bump up avocado-framework version to 101.0

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

migration: Make dirty_bytes_last_sync atomic

As we set its value, it needs to be operated with atomics.
We rename it from remaining to better reflect its meaning.

Statistics always return the real reamaining bytes.  This was used to
store how much pages where dirty on the previous generation, so we can
calculate the expected downtime as: dirty_bytes_last_sync /
current_bandwith.

If we use the actual remaining bytes, we would see a very small value
at the end of the iteration.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---

I am open to use ram_bytes_remaining() in its only use and be more
"optimistic" about the downtime.

Don't use __nocheck() functions.
Use stat64_get() now that it exists.

migration: Make dirty_pages_rate atomic

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
---

Don't use __nocheck() variants
Use stat64_get()

stat64: Add stat64_set() operation

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Juan Quintela <quintela@redhat.com>

multifd: Only flush once each full round of memory

We need to add a new flag to mean to flush at that point.
Notice that we still flush at the end of setup and at the end of
complete stages.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
---

Add missing qemu_fflush(), now it passes all tests always.
In the previous version, the check that changes the default value to
false got lost in some rebase.  Get it back.

multifd: Protect multifd_send_sync_main() calls

We only need to do that on the ram_save_iterate() call on sending and
on destination when we get a RAM_SAVE_FLAG_EOS.

In setup() and complete() we need to synch in both new and old cases,
so don't add a check there.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
---

Remove the wrappers that we take out on patch 5.

multifd: Create property multifd-flush-after-each-section

We used to flush all channels at the end of each RAM section
sent.  That is not needed, so preparing to only flush after a full
iteration through all the RAM.

Default value of the property is false.  But we return "true" in
migrate_multifd_flush_after_each_section() until we implement the code
in following patches.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
---

Rename each-iteration to after-each-section
Rename multifd-sync-after-each-section to
       multifd-flush-after-each-section
Move to machine-8.0 (peter)

migration: Move migration_properties to options.c

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Create migrate_block_bitmap_mapping() function

Notice that we changed the test of ->has_block_bitmap_mapping
for the test that block_bitmap_mapping is not NULL.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---

Make it return const (vladimir)

migration: Create migrate_tls_hostname() function

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---

Moved the type to const char * (vladimir)

migration: Create migrate_tls_authz() function

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---

Moved the type to const char * (vladimir)

migration: Create migrate_tls_creds() function

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---

Moved the type to const char * (vladimir)

migration: Remove MigrationState from block_cleanup_parameters()

This makes the function more regular with everything else.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Move block_cleanup_parameters() to options.c

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Move migrate_set_block_incremental() to options.c

Once there, make it more regular and remove the need for
MigrationState parameter.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Create migrate_downtime_limit() function

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Make all functions check have the same format

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Create migrate_params_init() function

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

multifd: Fix the number of channels ready

We don't wait in the sem when we are doing a sync_main.  Make it wait
there.  To make things clearer, we mark the channel ready at the
begining of the thread loop.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>

docs/style: call out the use of GUARD macros

There use makes our code safer so we should mention them.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <20230424092249.58552-19-alex.bennee@linaro.org>

docs/devel: mention the spacing requirement for QOM

We have a more complete document on QOM but we should at least mention
the style requirements in the style guide.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20230424092249.58552-18-alex.bennee@linaro.org>

docs/devel: make a statement about includes

While not explicitly disallowing header macro abuse (because that
would make us hypocrites) lets at least address some things to think
about.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230424092249.58552-17-alex.bennee@linaro.org>

docs/system: remove excessive punctuation from guest-loader docs

A possessive its needs no ' whereas the contraction of it is does.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230424092249.58552-16-alex.bennee@linaro.org>

qemu-options.hx: Update descriptions of memory options for NUMA node

This commit adds the following description:
1. `memdev` option is recommended over `mem` option (see [1,2])
2. users must specify memory for all NUMA nodes (see [2])

This commit also separates descriptions for `mem` and `memdev` into two
paragraphs. The old doc describes legacy `mem` option first, and it was
a bit confusing.

Related documentation:
[1] https://wiki.qemu.org/ChangeLog/5.1#Incompatible_changes
[2] https://www.qemu.org/docs/master/about/removed-features.html

Signed-off-by: Yohei Kojima <y-koj@outlook.jp>
Message-Id: <TYZPR06MB5418D6B0175A49E8E76988439D8E9@TYZPR06MB5418.apcprd06.prod.outlook.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[AJB: fix documentation in commit message]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-15-alex.bennee@linaro.org>

tests/tcg: limit the scope of the plugin tests

Running every plugin with every test is getting excessive as well as
not really improving coverage that much. Restrict the plugin tests to
just the MULTIARCH_TESTS which are shared between most architecture
for both system and user-mode. For those that aren't we need to squash
MULTIARCH_TESTS so we don't add them when they are not part of the
TESTS global.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230424092249.58552-14-alex.bennee@linaro.org>

tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the
boot_linux.py test-case due to which the code coverage for ppc
decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/87sfdpqcy4.fsf@linaro.org/ it
was mentioned that the baseline test for ppc64 could be modified
to make up this 2% code coverage. This patch attempts to achieve
this 2% code coverage by adding various device command line
arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py
test-case.

The code coverage report with boot_linux.py, without it and finally
with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
  lines......: 13.8% (58006 of 420997 lines)
  functions..: 20.7% (7675 of 36993 functions)
  branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
  lines......: 11.9% (50174 of 420997 lines)
  functions..: 18.8% (6947 of 36993 functions)
  branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
  lines......: 13.8% (58287 of 420997 lines)
  functions..: 20.7% (7640 of 36993 functions)
  branches...: 8.4% (20223 of 240611 branches)

Rebased on Alex Benee's testing/next branch:
https://gitlab.com/stsquad/qemu/-/tree/testing/next

Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424041830.1275636-1-kconsul@linux.vnet.ibm.com>
Message-Id: <20230424092249.58552-13-alex.bennee@linaro.org>

avocado_qemu/__init__.py: factor out the qemu-img finding

Factor out the code that finds the qemu-img binary in the
QemuSystemTest class and create a new get_qemu_img() function
with it. This function will get called also from the new code
in tuxrun_baselines.py avocado test-case.

Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
Suggested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230421042322.684093-2-kconsul@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-12-alex.bennee@linaro.org>

MAINTAINERS: Cover tests/avocado/machine_aspeed.py

tests/avocado/machine_aspeed.py should belong to the ASPEED section
in the maintainers file. Improve the wildcards here a little bit,
so that it is covered, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20230421110345.1294131-4-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-11-alex.bennee@linaro.org>

tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests

test_arm_ast2500_evb_sdk and test_arm_ast2600_evb_sdk are currently
failing. The problem is that they are trying to look for the login
prompt that does not have a newline at the end - but the logic in
_console_interaction() only handles full lines. It used to work by
accident in the past since there were sometimes kernel (warning and
error) messages popping up that finally provided a newline character
in the output, but since the tests have been changed to run with the
"quiet" kernel parameter, this is not working anymore.

To make this work reliably, we must not look for the "login:" prompt,
but have to use some text ending with a newline instead. And in the
ast2600 test, switch to ssh instead of trying to log into the serial
console - this works much more reliable and also has the benefit of
excercising the network interface here a little bit, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20230421110345.1294131-3-thuth@redhat.com>
[AJB: remove stray debug log]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-10-alex.bennee@linaro.org>

tests/avocado: Make ssh_command_output_contains() globally available

This function will be useful in other tests, too, so move it to the
core LinuxSSHMixIn class.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20230421110345.1294131-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-9-alex.bennee@linaro.org>

.gitlab-ci.d/cirrus: Drop the CI job for compiling with FreeBSD 12

FreeBSD 13.0 has been released in April 2021:

 https://www.freebsd.org/releases/13.0R/announce/

According to QEMU's support policy, we stop supporting the previous
major release two years after the the new major release has been
published. So we can stop testing FreeBSD 12 in our CI now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230418160225.529172-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20230424092249.58552-8-alex.bennee@linaro.org>

qemu-options: finesse the recommendations around -blockdev

We are a bit premature in recommending -blockdev/-device as the best
way to configure block devices. It seems there are times the more
human friendly -drive still makes sense especially when -snapshot is
involved.

Improve the language to hopefully make things clearer.

Suggested-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230424092249.58552-7-alex.bennee@linaro.org>

scripts/device-crash-test: Add a parameter to run with TCG only

We're currently facing the problem that the device-crash-test script
runs twice as long in the CI when a runner supports KVM - which sometimes
results in a timeout of the CI job. To get a more deterministic runtime
here, add an option to the script that allows to run it with TCG only.

Reported-by: Eldon Stegall <eldon-qemu@eldondev.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230414145845.456145-3-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-6-alex.bennee@linaro.org>

gitlab-ci: Avoid to re-run "configure" in the device-crash-test jobs

After "make check-venv" had been added to these jobs, they started
to re-run "configure" each time since our logic in the makefile
thinks that some files are out of date here. Avoid it with the same
trick that we are using in buildtest-template.yml already by disabling
the up-to-date check via NINJA=":".

Fixes: 1d8cf47e5b ("tests: run 'device-crash-test' from tests/venv")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230414145845.456145-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-5-alex.bennee@linaro.org>

tests/avocado: Add set of boot tests on SBSA-ref

This change adds set of boot tests on SBSA-ref machine:

1. boot firmware up to the EDK2 banner
2. boot Alpine Linux

Prebuilt flash volumes are included, built using upstream documentation.

To unify tests for AArch64/virt and AArch64/sbsa-ref we boot
the same Alpine Linux image on both.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230323082813.971535-1-marcin.juszkiewicz@linaro.org>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
Message-Id: <20230328171426.14258-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-4-alex.bennee@linaro.org>

tests/avocado: use the new snapshots for testing

The tuxboot images now have a stable snapshot URL so we can enable the
checksums and remove the avocado warnings. We will have to update as
old snapshots retire but that won't be too frequent.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230424092249.58552-3-alex.bennee@linaro.org>

tests/requirements.txt: bump up avocado-framework version to 101.0

Avocado version 101.0 has a fix to re-compute the checksum
of an asset file if the algorithm used in the *-CHECKSUM
file isn't the same as the one being passed to it by the
avocado user (i.e. the avocado_qemu python module).
In the earlier avocado versions this fix wasn't there due
to which if the checksum wouldn't match the earlier
checksum (calculated by a different algorithm), the avocado
code would start downloading a fresh image from the internet
URL thus making the test-cases take longer to execute.

Bump up the avocado-framework version to 101.0.

Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com>
Message-Id: <20230424092249.58552-2-alex.bennee@linaro.org>

Merge tag 'migration-20230426-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request (take2)

With respect to the last PULL request:
- fix compilation on hosts without userfaultd.

Please, apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmRKMYYACgkQ9IfvGFhy
# 1yNoIBAAu4nyERXe7LuRcG02mXQ7N8NtKJXxlhvFGivmlZEzPPkI/SKSXVmzP47E
# uIq1+N+3couYlIof6mY3sjVDwv9Rk7AaGUNUsGv5ygLbXEd/3/qBnqcGV9pUBr3e
# Ls07l2bdb6QPSUGEfu3YATrYzEvWQOAA6AHEBJeqplPlIprgUM3qTlX3bSXrBb26
# J7+p+6A3xdH2koCwmBcWt7cO6IzM7t8pazOS+rJepFri/ZD9aVNr7E3QjXgMUeCs
# 4htBrFaCEXzb80n8O7yPCk90ei0PvGiYBdxI2MGCAy96MN0I5rfMLC/DDFQ+uMH2
# wMVpB5S5YiKrCkoP/ftiYZsSmfRMmQD9aVS6YHR8yt82kEbz616lkXIpvC5Zeads
# J9zTmFuI6fdaF/2PEDzFUACz3YDMtlqQZqKy+e9Ca8gVeoX+ZL4e5yQGNe5nx4BN
# wMr7tPjZenXUpDhuDMcw3/APqaPPZIPVKwYwSgiEL3mqVpU67pre5F4phERCVVVu
# GP/h9yhRa+dkBVNtkqIkK48vQvr7XXisukwSCWmFDA9kfDyDolN6IDaIjv5B7TdL
# DOAW+oz+O4QLVHQhbiQN9WGIc2OenhpKOJ9Rd6cUph6s9tBppaJTrTRRBTYNwPLv
# 8X5h/e7Q1Ee8xGWI1RhAsmD54/e8bu8zxczgf9iMIT8GskLoPco=
# =Ye0/
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 27 Apr 2023 09:25:42 AM BST
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [undefined]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20230426-pull-request' of https://gitlab.com/juan.quintela/qemu:
  vmstate-static-checker: Recognize "num" field
  migration/vmstate-dump: Dump array size too as "num"
  migration: Allow postcopy_ram_supported_by_host() to report err
  migration: Move qmp_migrate_set_parameters() to options.c
  migration: Move migrate_use_tls() to options.c
  MAINTAINERS: Add Leonardo and Peter as reviewers
  migration: Disable postcopy + multifd migration

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

vmstate-static-checker: Recognize "num" field

Recognize this field for VMS_ARRAY typed vmsd fields, then we can do proper
size matching with previous patch.

Note that this is compatible with old -dump-vmstate output, because when
"num" is not there we'll still use the old "size" only.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

migration/vmstate-dump: Dump array size too as "num"

For VMS_ARRAY typed vmsd fields, also dump the number of entries in the
array in -vmstate-dump.

Without such information, vmstate static checker can report false negatives
of incompatible vmsd on VMS_ARRAY typed fields, when the src/dst do not
have the same type of array defined.  It's because in the checker we only
check against size of fields within a VMSD field.

One example: e1000e used to have a field defined as a boolean array with 5
entries, then removed it and replaced it with UNUSED (in 31e3f318c8b535):

-        VMSTATE_BOOL_ARRAY(core.eitr_intr_pending, E1000EState,
-                           E1000E_MSIX_VEC_NUM),
+        VMSTATE_UNUSED(E1000E_MSIX_VEC_NUM),

It's a legal replacement but vmstate static checker is not happy with it,
because it checks only against the "size" field between the two
fields (here one is BOOL_ARRAY, the other is UNUSED):

For BOOL_ARRAY:

      {
        "field": "core.eitr_intr_pending",
        "version_id": 0,
        "field_exists": false,
        "size": 1
      },

For UNUSED:

      {
        "field": "unused",
        "version_id": 0,
        "field_exists": false,
        "size": 5
      },

It's not the script to blame because there's just not enough information
dumped to show the total size of the entry for an array.  Add it.

Note that this will not break old vmstate checker because the field will
just be ignored.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

migration: Allow postcopy_ram_supported_by_host() to report err

Instead of print it to STDERR, bring the error upwards so that it can be
reported via QMP responses.

E.g.:

{ "execute": "migrate-set-capabilities" ,
  "arguments": { "capabilities":
  [ { "capability": "postcopy-ram", "state": true } ] } }

{ "error":
  { "class": "GenericError",
    "desc": "Postcopy is not supported: Host backend files need to be TMPFS
    or HUGETLBFS only" } }

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

migration: Move qmp_migrate_set_parameters() to options.c

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

migration: Move migrate_use_tls() to options.c

Once there, rename it to migrate_tls() and make it return bool for
consistency.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---

Fix typos found by fabiano

MAINTAINERS: Add Leonardo and Peter as reviewers

Now that David has stepped down with Migration maintainership,
Leonardo and Peter has volunteer to review the migration patches.
This way they got CC'd on every migration patch.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Acked-by: Leonardo Bras <leobras@redhat.com>

migration: Disable postcopy + multifd migration

Since the introduction of multifd, it's possible to perform a multifd
migration and finish it using postcopy.

A bug introduced by yank (fixed on cfc3bcf373) was previously preventing
a successful use of this migration scenario, and now thing should be
working on most scenarios.

But since there is not enough testing/support nor any reported users for
this scenario, we should disable this combination before it may cause any
problems for users.

Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Leonardo Bras <leobras@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

Merge tag 'pull-qapi-2023-04-26' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2023-04-26

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmRIvOkSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTX3MQAIqrKQbOzQ81/cDZ7aeLOroDoGYf1Cs0
# 0NiEVlyoblWNzL3HraGgXiNRTP+zaG/TcFKza1nz8qjdkxWxBjdbfF5Lm6mQf5Zo
# tcHUjksmnUlPkLYSOyEjfY9SNvcS6g7djE/NF5lbJtzYGZScZpLarELR7oUvrcXR
# AEiw8N5FZXp+j6cTeWvrLzSqz9qBsFJUCGcGER0T/Mt5MlUwDexE1xe7g8oD5l+b
# s0jeQr1PTZm5k6ehajQtgbHvAkgH8xVTKqbB/U5iz4VhYriH+IPEOtfCFt6/1soz
# pVkYikJpazCvQMjqnWu9dE1onthgSsEIOV29kFU0Kr8ATZuJBQMuLVp4hSsbKANj
# BUVyL2/fUsIp7gd+KikXUOjKYajxek6Q2YLAPpL+1pBCTql/PBQ7td8CECdiv/9e
# Xh50q+BGvyEiyoyf4EEpaLXUZog605WHEaODj9uPtNHJP9x6Rqt93FUsdWUtt/k9
# hJ8RSKy8njr0vxGoJkj89m2XfCwtuX3VQ5IXvv/If4U5Y4+JhcLtiqW+Njh8fAM4
# ZwIrlUYG7inLUKFVcQ3sEGpaj611i5ixIxctUvEiggZX+fPeSFKYUr+Rq8WXM8gv
# suLXz7VF6H4Sw30lCvdQ4LSungbzlYAtQYpmdEQGoM8iasIi4PoDf0cTYBbMYHDX
# +pZvWC50cVtf
# =wLx6
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 26 Apr 2023 06:55:53 AM BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2023-04-26' of https://repo.or.cz/qemu/armbru:
  qapi: allow unions to contain further unions
  qapi: Improve specificity of type/member descriptions
  qapi: support updating expected test output via make
  qapi: Require boxed for conditional command and event arguments
  qapi: Fix code generated for optional conditional struct member
  tests/qapi-schema: Cover optional conditional struct member
  tests/qapi-schema: Clean up positive test for conditionals
  tests/qapi-schema: Rename a few conditionals
  tests/qapi-schema: Improve union discriminator coverage
  qapi: Fix to reject 'data': 'mumble' in struct
  qapi: Fix error message when type name or array is expected
  qapi: Simplify code a bit after previous commits
  qapi: Improve error message for unexpected array types
  qapi: Split up check_type()
  qapi: Clean up after removal of simple unions
  qapi/schema: Use super()
  qapi: Fix error message format regression

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Protect BlockBackend.queued_requests with its own lock
- Switch to AIO_WAIT_WHILE_UNLOCKED() where possible
- AioContext removal: LinuxAioState/LuringState/ThreadPool
- Add more coroutine_fn annotations, use bdrv/blk_co_*
- Fix crash when execute hmp_commit

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmRH0b0RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Y0yw/6A/vzA4TGgFUP3WIvH/sQri4/V3gyR+PT
# u3hOQUCYZ99nioTpKV91TSuUPuU/Mdspy/0NKM+K92yIXqxa9172A2zLOsGOu21l
# qKpse+nBf1zqEgB8YzUHyCBdetPz916C/f9RS26SNUCW85GCHYGHA3u7nKvWLMyV
# oKIoTlA8QOglOuEKlRoYh7hCFm7ET51NOSEftm8GsYbsW/I2Vzl8a1SHN1lHufjd
# We3+898zUrmFqNMp6Rjdhn+yZmmoGzoZqV4YQi83z7xjiv+Ms4VHVVW7X8d20xRX
# 5BLFiLHAuZ/1d26HyVhgBUr7KHyf94odocz8BylWKXGl5SXMCZun1Td1vgVKlGK+
# GRxzB2cWGWqzC2UmqSTc0Z0aIWbXukKwvcX76uBKsQZ+kB2A7jFobxHiaoQEDJ8B
# WRNEMH2+CqCAu9rsrNRinnJKhT2nXcr9F9YfwRIlagdAePGWin+EUW8huf14dDBm
# Z2Y34aKW4RQibF8xirMHeRBbOLmcq2VpKLKwNfBHUDgZB8iuD7bLn4n9nwWXMG1w
# zgNsTybkv46vLPamTpEaUoNTHfuRDTAuE7Z7lkcc7jF41Z0V1DC/DCCWcL/0LvhP
# GIxFdkYug3hetdF2U/OZhUoEfxvkqcuBnrr55LFzqheKEllQpPwPpt7UF0aH8bg3
# i/YpjHsf3xU=
# =mpYX
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 25 Apr 2023 02:12:29 PM BST
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (25 commits)
  block/monitor: Fix crash when executing HMP commit
  vmdk: make vmdk_is_cid_valid a coroutine_fn
  qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK
  tests: mark more coroutine_fns
  qemu-pr-helper: mark more coroutine_fns
  9pfs: mark more coroutine_fns
  nbd: mark more coroutine_fns, do not use co_wrappers
  mirror: make mirror_flush a coroutine_fn, do not use co_wrappers
  blkdebug: add missing coroutine_fn annotation
  vvfat: mark various functions as coroutine_fn
  thread-pool: avoid passing the pool parameter every time
  thread-pool: use ThreadPool from the running thread
  io_uring: use LuringState from the running thread
  linux-aio: use LinuxAioState from the running thread
  block: add missing coroutine_fn to bdrv_sum_allocated_file_size()
  include/block: fixup typos
  monitor: convert monitor_cleanup() to AIO_WAIT_WHILE_UNLOCKED()
  hmp: convert handle_hmp_command() to AIO_WAIT_WHILE_UNLOCKED()
  block: convert bdrv_drain_all_begin() to AIO_WAIT_WHILE_UNLOCKED()
  block: convert bdrv_graph_wrlock() to AIO_WAIT_WHILE_UNLOCKED()
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

qapi: allow unions to contain further unions

This extends the QAPI schema validation to permit unions inside unions,
provided the checks for clashing fields pass.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230420102619.348173-4-berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

qapi: Improve specificity of type/member descriptions

Error messages describe object members, enumeration values, features,
and variants like ROLE 'NAME', where ROLE is "member", "value",
"feature", or "branch", respectively.  When the member is defined in
another type, e.g. inherited from a base type, we add "of type
'TYPE'".  Example: test case struct-base-clash-deep reports a member
of type 'Sub' clashing with a member of its base type 'Base' as

    struct-base-clash-deep.json: In struct 'Sub':
    struct-base-clash-deep.json:10: member 'name' collides with member 'name' of type 'Base'

Members of implicitly defined types need special treatment.  We don't
want to add "of type 'TYPE'" for them, because their named are made up
and mean nothing to the user.  Instead, we describe members of an
implicitly defined base type as "base member 'NAME'", and command and
event parameters as "parameter 'NAME'".  Example: test case
union-bad-base reports member of a variant's type clashing with a
member of its implicitly defined base type as

    union-bad-base.json: In union 'TestUnion':
    union-bad-base.json:8: member 'string' of type 'TestTypeA' collides with base member 'string'

The next commit will permit unions as variant types.  "base member
'NAME' would then be ambigious: is it the union's base, or is it the
union's variant's base?  One of its test cases would report a clash
between two such bases as "base member 'type' collides with base
member 'type'".  Confusing.

Refine the special treatment: add "of TYPE" even for implicitly
defined types, but massage TYPE and ROLE so they make sense for the
user.

Message-Id: <20230420102619.348173-3-berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

qapi: support updating expected test output via make

It is possible to pass --update to tests/qapi-schema/test-qapi.py
to make it update the output files on error. This is inconvenient
to achieve though when test-qapi.py is run indirectly by make/meson.

Instead simply allow for an env variable to be set:

 $ QAPI_TEST_UPDATE= make check-qapi-schema

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230420102619.348173-2-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

block/monitor: Fix crash when executing HMP commit

hmp_commit() calls blk_is_available() from a non-coroutine context (and
in the main loop). blk_is_available() is a co_wrapper_mixed_bdrv_rdlock
function, and in the non-coroutine context it calls AIO_WAIT_WHILE(),
which crashes if the aio_context lock is not taken before.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1615
Signed-off-by: Wang Liang <wangliangzz@inspur.com>
Message-Id: <20230424103902.45265-1-wangliangzz@126.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

vmdk: make vmdk_is_cid_valid a coroutine_fn

Functions that can do I/O are prime candidates for being coroutine_fns.  Make the
change for the one that is itself called only from coroutine_fns.  Unfortunately
vmdk does not use a coroutine_fn for the bulk of the open (like qcow2 does) so
vmdk_read_cid cannot have the same treatment.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-10-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK

Functions that can do I/O (including calling bdrv_is_allocated
and bdrv_block_status functions) are prime candidates for being
coroutine_fns.  Make the change for those that are themselves called
only from coroutine_fns.  Also annotate that they are called with the
graph rdlock taken, thus allowing them to call bdrv_co_*() functions
for I/O.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-9-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

tests: mark more coroutine_fns

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-8-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

qemu-pr-helper: mark more coroutine_fns

do_sgio can suspend via the coroutine function thread_pool_submit_co, so it
has to be coroutine_fn as well---and the same is true of all its direct and
indirect callers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-7-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

9pfs: mark more coroutine_fns

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-6-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

nbd: mark more coroutine_fns, do not use co_wrappers

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

mirror: make mirror_flush a coroutine_fn, do not use co_wrappers

mirror_flush calls a mixed function blk_flush but it is only called
from mirror_run; so call the coroutine version and make mirror_flush
a coroutine_fn too.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-4-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

blkdebug: add missing coroutine_fn annotation

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-3-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

vvfat: mark various functions as coroutine_fn

Functions that can do I/O are prime candidates for being coroutine_fns.  Make the
change for those that are themselves called only from coroutine_fns.

In addition, coroutine_fns should do I/O using bdrv_co_*() functions, for
which it is required to hold the BlockDriverState graph lock.  So also nnotate
functions on the I/O path with TSA attributes, making it possible to
switch them to use bdrv_co_*() functions.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230309084456.304669-2-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

thread-pool: avoid passing the pool parameter every time

thread_pool_submit_aio() is always called on a pool taken from
qemu_get_current_aio_context(), and that is the only intended
use: each pool runs only in the same thread that is submitting
work to it, it can't run anywhere else.

Therefore simplify the thread_pool_submit* API and remove the
ThreadPool function parameter.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20230203131731.851116-5-eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

thread-pool: use ThreadPool from the running thread

Use qemu_get_current_aio_context() where possible, since we always
submit work to the current thread anyways.

We want to also be sure that the thread submitting the work is
the same as the one processing the pool, to avoid adding
synchronization to the pool list.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20230203131731.851116-4-eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

io_uring: use LuringState from the running thread

Remove usage of aio_context_acquire by always submitting asynchronous
AIO to the current thread's LuringState.

In order to prevent mistakes from the caller side, avoid passing LuringState
in luring_io_{plug/unplug} and luring_co_submit, and document the functions
to make clear that they work in the current thread's AioContext.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20230203131731.851116-3-eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

linux-aio: use LinuxAioState from the running thread

Remove usage of aio_context_acquire by always submitting asynchronous
AIO to the current thread's LinuxAioState.

In order to prevent mistakes from the caller side, avoid passing LinuxAioState
in laio_io_{plug/unplug} and laio_co_submit, and document the functions
to make clear that they work in the current thread's AioContext.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20230203131731.851116-2-eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

block: add missing coroutine_fn to bdrv_sum_allocated_file_size()

Not a coroutine_fn, you say?

  static int64_t bdrv_sum_allocated_file_size(BlockDriverState *bs)
  {
      BdrvChild *child;
      int64_t child_size, sum = 0;

      QLIST_FOREACH(child, &bs->children, next) {
          if (child->role & (BDRV_CHILD_DATA | BDRV_CHILD_METADATA |
                             BDRV_CHILD_FILTERED))
          {
              child_size = bdrv_co_get_allocated_file_size(child->bs);
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Well what do we have here?!

I rest my case, your honor.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230308211435.346375-1-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

include/block: fixup typos

Fixup a few minor typos

Signed-off-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Message-Id: <20230313003744.55476-1-wilfred.mallawa@opensource.wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

monitor: convert monitor_cleanup() to AIO_WAIT_WHILE_UNLOCKED()

monitor_cleanup() is called from the main loop thread. Calling
AIO_WAIT_WHILE(qemu_get_aio_context(), ...) from the main loop thread is
equivalent to AIO_WAIT_WHILE_UNLOCKED(NULL, ...) because neither unlocks
the AioContext and the latter's assertion that we're in the main loop
succeeds.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230309190855.414275-7-stefanha@redhat.com>
Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>