Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200601a' into staging

Migration/virtio/hmp pull 2020-06-01

A mixed pull with:
  - RDMA migration fix (CID 1428762)
  - HMP qom-get addition and qom-set cleanup
  - a virtiofsd fix
  - COLO fixes

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 01 Jun 2020 19:37:15 BST
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20200601a:
  migration/migration.c: Fix hang in ram_save_host_page
  migration/colo.c: Move colo_notify_compares_event to the right place
  migration/colo.c: Relaunch failover even if there was an error
  migration/colo.c: Flush ram cache only after receiving device state
  migration/colo.c: Use cpu_synchronize_all_states()
  migration/colo.c: Use event instead of semaphore
  migration/vmstate: Remove unnecessary MemoryRegion forward declaration
  virtiofsd: remove symlink fallbacks
  hmp: Simplify qom-set
  hmp: Implement qom-get HMP command
  migration/rdma: cleanup rdma context before g_free to avoid memleaks
  migration/rdma: fix potential nullptr access in rdma_start_incoming_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/bkoppelmann2/tags/pull-tricore-20200601' into staging

Remove ctx->env ptr, add TriCore gdb stub

# gpg: Signature made Mon 01 Jun 2020 16:06:35 BST
# gpg:                using RSA key 6E636A7E83F2DD0CFA6E6E370AD2C6396B69CA14
# gpg:                issuer "kbastian@mail.uni-paderborn.de"
# gpg: Good signature from "Bastian Koppelmann <kbastian@mail.uni-paderborn.de>" [full]
# Primary key fingerprint: 6E63 6A7E 83F2 DD0C FA6E  6E37 0AD2 C639 6B69 CA14

* remotes/bkoppelmann2/tags/pull-tricore-20200601:
  target/tricore: Implement gdbstub
  target/tricore: Implement tricore_cpu_get_phys_page_debug
  target/tricore: Raise EXCP_DEBUG in gen_goto_tb() for singlestep
  target/tricore: Move translate feature check to ctx
  target/tricore: Don't save pc in generate_qemu_excp

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

migration/migration.c: Fix hang in ram_save_host_page

migration_rate_limit will erroneously ratelimit a shutdown socket,
which causes the migration thread to hang in ram_save_host_page
if the socket is shutdown.

Fix this by explicitly testing if the socket has errors or was
shutdown in migration_rate_limit.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <e79085bbe2d46dfa007dd41820194d5e2d4fcd80.1590007004.git.lukasstraub2@web.de>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/colo.c: Move colo_notify_compares_event to the right place

If the secondary has to failover during checkpointing, it still is
in the old state (i.e. different state than primary). Thus we can't
expose the primary state until after the checkpoint is sent.

This fixes sporadic connection reset of client connections during
failover.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <d4555dd5146a54518c4d9d4efd996b7c745c6687.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/colo.c: Relaunch failover even if there was an error

If vmstate_loading is true, secondary_vm_do_failover will set failover
status to FAILOVER_STATUS_RELAUNCH and return success without initiating
failover. However, if there is an error during the vmstate_loading
section, failover isn't relaunched. Instead we then wait for
failover on colo_incoming_sem.

Fix this by relaunching failover even if there was an error. Also,
to make this work properly, set vmstate_loading to false when
returning during the vmstate_loading section.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <f60b0a8e2fadaaec792e04819dfc46951842d6ba.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/colo.c: Flush ram cache only after receiving device state

If we suceed in receiving ram state, but fail receiving the device
state, there will be a mismatch between the two.

Fix this by flushing the ram cache only after the vmstate has been
received.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <3289d007d494cb0e2f05b1cf4ae6a78d300fede3.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/colo.c: Use cpu_synchronize_all_states()

cpu_synchronize_all_pre_loadvm() marks all vcpus as dirty, so the
registers are loaded from CPUState before we continue running
the vm. However if we failover during checkpoint, CPUState is not
initialized and the registers are loaded with garbage. This causes
guest hangs and crashes.

Fix this by using cpu_synchronize_all_states(), which initializes
CPUState from the current cpu registers additionally to marking
the vcpus as dirty.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <9675031ce557b73ebd10e7bd20ebbf57f30b177c.1589193382.git.lukasstraub2@web.de>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/colo.c: Use event instead of semaphore

If multiple packets miscompare in a short timeframe, the semaphore
value will be increased multiple times. This causes multiple
checkpoints even if one would be sufficient.

Fix this by using a event instead of a semaphore for triggering
checkpoints. Now, checkpoint requests will be ignored until the
checkpoint event is sent to colo-compare (which releases the
miscompared packets).

Benchmark results (iperf3):
Client-to-server tcp:
without patch: ~66 Mbit/s
with patch: ~61 Mbit/s
Server-to-client tcp:
without patch: ~702 Kbit/s
with patch: ~16 Mbit/s

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Message-Id: <fd601ba1beb524aada54ba66e87ebfc12cf4574b.1589193382.git.lukasstraub2@web.de>
Reviewed-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/vmstate: Remove unnecessary MemoryRegion forward declaration

"migration/vmstate.h" only uses pointer to MemoryRegion, which
is already forward declared in "qemu/typedefs.h".

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200530165512.15225-1-f4bug@amsat.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: remove symlink fallbacks

Path lookup in the kernel has special rules for looking up magic symlinks
under /proc.  If a filesystem operation is instructed to follow symlinks
(e.g. via AT_SYMLINK_FOLLOW or lack of AT_SYMLINK_NOFOLLOW), and the final
component is such a proc symlink, then the target of the magic symlink is
used for the operation, even if the target itself is a symlink.  I.e. path
lookup is always terminated after following a final magic symlink.

I was erronously assuming that in the above case the target symlink would
also be followed, and so workarounds were added for a couple of operations
to handle the symlink case.  Since the symlink can be handled simply by
following the proc symlink, these workardouds are not needed.

Also remove the "norace" option, which disabled the workarounds.

Commit bdfd66788349 ("virtiofsd: Fix xattr operations") already dealt with
the same issue for xattr operations.

Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Message-Id: <20200514140736.20561-1-mszeredi@redhat.com>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

hmp: Simplify qom-set

Simplify qom_set by making it use qmp_qom_set and the JSON parser.

(qemu) qom-get /machine smm
"auto"
(qemu) qom-set /machine smm "auto"

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20200520151108.160598-3-dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  With 's'->'S' type change suggested by Paolo and Markus

hmp: Implement qom-get HMP command

This started off as Andreas Färber's implementation from
March 2015, but after feedback from Paolo and Markus it morphed into
using the json output which handles structs reasonably.

Use with qom-list to find the members of an object.

(qemu) qom-get /backend/console[0]/device/vga.rom[0] size
65536
(qemu) qom-get /machine smm
"auto"
(qemu) qom-get /machine rtc-time
{
    "tm_year": 120,
    "tm_sec": 51,
    "tm_hour": 9,
    "tm_min": 50,
    "tm_mon": 4,
    "tm_mday": 20
}
(qemu) qom-get /machine frob
Error: Property '.frob' not found

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20200520151108.160598-2-dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/rdma: cleanup rdma context before g_free to avoid memleaks

When error happen in initializing 'rdma_return_path', we should cleanup rdma context
before g_free(rdma) to avoid some memleaks. This patch fix that.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Message-Id: <20200508100755.7875-3-pannengyuan@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/rdma: fix potential nullptr access in rdma_start_incoming_migration

'rdma' is NULL when taking the first error branch in rdma_start_incoming_migration.
And it will cause a null pointer access in label 'err'. Fix that.

Fixes: 59c59c67ee6b0327ae932deb303caa47919aeb1e
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Message-Id: <20200508100755.7875-2-pannengyuan@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Note this is CID 1428762

target/tricore: Implement gdbstub

Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20200529072148.284037-6-kbastian@mail.uni-paderborn.de>

target/tricore: Implement tricore_cpu_get_phys_page_debug

this also removes tricore_cpu_get_phys_page_attrs_debug() as it was a
temporary fix from b190f477e29c7cd03a8fee49c96d27f160e3f5b0.

Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20200529072148.284037-5-kbastian@mail.uni-paderborn.de>

target/tricore: Raise EXCP_DEBUG in gen_goto_tb() for singlestep

this is needed for remote gdb connections.

Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20200529072148.284037-4-kbastian@mail.uni-paderborn.de>

target/tricore: Move translate feature check to ctx

this allows us to remove the references to env from ctx. This also fixes
a segfault that was due to the unititalized ctx->env ptr.

Reported-by: Andreas Konopik <andreas.konopik@fau.de>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20200529072148.284037-3-kbastian@mail.uni-paderborn.de>

target/tricore: Don't save pc in generate_qemu_excp

EXCP_DEBUG is the only user. If we encounter a jump in tricore-gdb it's
target was overwritten by generate_qemu_excp() and we would never leave.

Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20200529072148.284037-2-kbastian@mail.uni-paderborn.de>

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-june-01-2020' into staging

MIPS queue for June 1st, 2020

# gpg: Signature made Mon 01 Jun 2020 12:29:25 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [full]
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-june-01-2020:
  hw/mips: fuloong2e: Set preferred page size to 16KB
  target/mips: Support variable page size
  target/mips: Add more CP0 register for save/restore
  hw/mips: Add CPU IRQ3 delivery for KVM
  configure: Add KVM target support for MIPS64
  tests/Makefile: Fix description of "make check"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/mips: fuloong2e: Set preferred page size to 16KB

Loongson processor prefers 16KB page size in system emulator, so let's
define mc->minimum_page_bits to 14.

Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1586337380-25217-2-git-send-email-chenhc@lemote.com>

target/mips: Support variable page size

Traditionally, MIPS use 4KB page size, but Loongson prefer 16KB page
size in system emulator. So, let's define TARGET_PAGE_BITS_VARY and
TARGET_PAGE_BITS_MIN to support variable page size.

Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1586337380-25217-1-git-send-email-chenhc@lemote.com>

target/mips: Add more CP0 register for save/restore

Add more CP0 register for save/restore, including: EBase, XContext,
PageGrain, PWBase, PWSize, PWField, PWCtl, Config*, KScratch1~KScratch6.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-6-git-send-email-chenhc@lemote.com>

hw/mips: Add CPU IRQ3 delivery for KVM

Currently, KVM/MIPS only deliver I/O interrupt via IP2, this patch add
IP3 delivery as well, because Loongson-3 based machine use both IRQ2
(CPU's IP2) and IRQ3 (CPU's IP3).

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-4-git-send-email-chenhc@lemote.com>

configure: Add KVM target support for MIPS64

Preparing for Loongson-3 virtualization, add KVM target support for
MIPS64 in configure script.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588501221-1205-2-git-send-email-chenhc@lemote.com>

tests/Makefile: Fix description of "make check"

The description of "make check" is out-of-date, so fix it by adding
block and softfloat.

Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <1588674291-6486-1-git-send-email-chenhc@lemote.com>

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200531' into staging

Python queue:

* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Python3, style)
* vm-test can set QEMU_LOCAL=1 to use locally built binaries
* refactored BootLinuxBase & LinuxKernelTest acceptance classes

https://gitlab.com/philmd/qemu/pipelines/151323210
https://travis-ci.org/github/philmd/qemu/builds/693157969

# gpg: Signature made Sun 31 May 2020 17:37:35 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200531: (25 commits)
  tests/acceptance: refactor boot_linux to allow code reuse
  tests/acceptance: refactor boot_linux_console test to allow code reuse
  tests/acceptance: allow console interaction with specific VMs
  tests/acceptance/migration.py: Wait for both sides
  tests/migration/guestperf: Use Python 3 interpreter
  tests/vm: allow wait_ssh() to specify command
  tests/vm: Add ability to select QEMU from current build
  tests/vm: Pass --debug through for vm-boot-ssh
  python/qemu/qtest: Check before accessing _qtest
  python/qemu/qmp: assert sockfile is not None
  python/qemu/qmp: use True/False for non/blocking modes
  python/qemu: Adjust traceback typing
  python/qemu: fix socket.makefile() typing
  python/qemu: remove Python2 style super() calls
  python/qemu: delint; add flake8 config
  python/qemu: delint and add pylintrc
  python/qemu/machine: remove logging configuration
  python/qemu/machine: add kill() method
  python: remove more instances of sys.version_info
  scripts/qmp: Fix shebang and imports
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-05-30' into staging

- Replace hw_error() with qemu_log_mask() in the m68k coldfire machine code

# gpg: Signature made Sat 30 May 2020 08:44:41 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "huth@tuxfamily.org"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-05-30:
  hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask()
  hw/m68k/mcf5206: Reduce m5206_mbar_read/write() offset arg to 16-bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

tests/acceptance: refactor boot_linux to allow code reuse

This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073593167.20809.17582679291556188984.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/acceptance: refactor boot_linux_console test to allow code reuse

This patch splits code in BootLinuxConsole class into two different
classes to allow reusing it by record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073588490.20809.13942096070255577558.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/acceptance: allow console interaction with specific VMs

Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073587933.20809.5122618715976660635.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/acceptance/migration.py: Wait for both sides

When the source finishes migration the destination will still be
receiving the data sent by the source, so it might not have quite
finished yet, so won't quite have reached 'completed'.
This lead to occasional asserts in the next few checks.

After the source has finished, check the destination as well.
(We can't just switch to checking the destination, because it doesn't
give a status until it has started receiving the migration).

Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528112404.121972-1-dgilbert@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/migration/guestperf: Use Python 3 interpreter

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-7-philmd@redhat.com>

tests/vm: allow wait_ssh() to specify command

This allows for waiting for completion of arbitrary commands.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-7-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/vm: Add ability to select QEMU from current build

Added a new special variable QEMU_LOCAL=1, which
will indicate to take the QEMU binary from the current
build.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-6-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/vm: Pass --debug through for vm-boot-ssh

This helps debug issues that occur during the boot sequence.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-5-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu/qtest: Check before accessing _qtest

It can be None; so add assertions or exceptions where appropriate to
guard the access accordingly.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-30-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu/qmp: assert sockfile is not None

In truth, if you don't do this, you'll just get a TypeError
exception. Now, you'll get an AssertionError.

Is this tangibly better? No.
Does mypy complain less? Yes.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-21-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu/qmp: use True/False for non/blocking modes

The type system doesn't want integers.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-15-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu: Adjust traceback typing

mypy considers it incorrect to use `bool` to statically return false,
because it will assume that it could conceivably return True, and gives
different analysis in that case. Use a None return to achieve the same
effect, but make mypy happy.

Note: Pylint considers function signatures as code that might trip the
duplicate-code checker. I'd rather not disable this as it does not
trigger often in practice, so I'm disabling it as a one-off and filed a
change request; see https://github.com/PyCQA/pylint/issues/3619

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-14-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu: fix socket.makefile() typing

Note:

A bug in typeshed (https://github.com/python/typeshed/issues/3977)
misinterprets the type of makefile(). Work around this by explicitly
stating that we are opening a text-mode file.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-13-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu: remove Python2 style super() calls

Use the Python3 style instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-12-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu: delint; add flake8 config

Mostly, ignore the "no bare except" rule, because flake8 is not
contextual and cannot determine if we re-raise. Pylint can, though, so
always prefer pylint for that.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-5-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu: delint and add pylintrc

Bring our these files up to speed with pylint 2.5.0.
Add a pylintrc file to formalize which pylint subset
we are targeting.

The similarity ignore is there to suppress similarity
reports across imports, which for typing constants,
are going to trigger this report erroneously.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-4-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu/machine: remove logging configuration

Python 3.5 and above do not print a warning when logging is not
configured. As a library, it's best practice to leave logging
configuration to the client executable.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-22-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python/qemu/machine: add kill() method

Add method to hard-kill vm, without any quit commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <20200217150246.29180-19-vsementsov@virtuozzo.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

python: remove more instances of sys.version_info

We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514035230.25756-1-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

scripts/qmp: Fix shebang and imports

There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20200528222129.23826-2-jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

scripts/modules/module_block: Use Python 3 interpreter & add pseudo-main

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-6-philmd@redhat.com>

scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-5-philmd@redhat.com>

scripts/qmp: Use Python 3 interpreter

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

scripts/qemu-gdb: Use Python 3 interpreter

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

scripts/qemugdb: Remove shebang header

These scripts are loaded as plugin by GDB (and they don't
have any __main__ entry point). Remove the shebang header.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-2-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask()

hw_error() calls exit(). This a bit overkill when we can log
the accesses as unimplemented or guest error.

When fuzzing the devices, we don't want the whole process to
exit. Replace some hw_error() calls by qemu_log_mask().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <huth@tuxfamily.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>

hw/m68k/mcf5206: Reduce m5206_mbar_read/write() offset arg to 16-bit

All calls to m5206_mbar_read/m5206_mbar_write are used with
'offset = hwaddr & 0x3ff', so we are sure the offset fits
in 16-bit.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
  iotests: Add test 291 to for qemu-img bitmap coverage
  qemu-img: Add convert --bitmaps option
  qemu-img: Factor out code for merging bitmaps
  qcow2: Expose bitmaps' size during measure
  iotests: Fix test 178
  migration: forbid bitmap migration by generated node-name
  migration: add_bitmaps_to_list: check disk name once
  iotests: 194: test also migration of dirty bitmap
  migration: fix bitmaps pre-blockdev migration with mirror job
  block/dirty-bitmap: add bdrv_has_named_bitmaps helper
  migration: refactor init_dirty_bitmap_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1' into staging

Testing and one plugin fix:

  - support alternates for genisoimage to test/vm
  - add clang++ to clang tests
  - fix record/replay smoke test
  - enable more softfloat tests
  - better detection of hung gdb
  - upgrade aarch64 tcg test x-compile to gcc-10
  - fix plugin cpu_index clash vs threads

# gpg: Signature made Wed 27 May 2020 14:29:20 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1:
  tests/tcg: add new threadcount test
  linux-user: properly "unrealize" vCPU object
  cpus-common: ensure auto-assigned cpu_indexes don't clash
  tests/docker: use a gcc-10 based image for arm64 tests
  tests/docker: add debian11 base image
  tests/tcg: better detect confused gdb which can't connect
  tests/fp: split and audit the conversion tests
  tests/fp: enable extf80_le_quite tests
  tests/tcg: fix invocation of the memory record/replay tests
  travis.yml: Use clang++ in the Clang tests
  tests/vm: pass --genisoimage to basevm script
  configure: add alternate binary for genisoimage

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

iotests: Add test 291 to for qemu-img bitmap coverage

Add a new test covering the 'qemu-img bitmap' subcommand, as well as
'qemu-img convert --bitmaps', both added in recent patches.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521192137.1120211-6-eblake@redhat.com>

qemu-img: Add convert --bitmaps option

Make it easier to copy all the persistent bitmaps of (the top layer
of) a source image along with its guest-visible contents, by adding a
boolean flag for use with qemu-img convert.  This is basically
shorthand, as the same effect could be accomplished with a series of
'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source'
commands, or by their corresponding QMP commands.

Note that this command will fail in the same scenarios where 'qemu-img
measure' omits a 'bitmaps size:' line, namely, when either the source
or the destination lacks persistent bitmap support altogether.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893

While touching this, clean up a couple coding issues spotted in the
same function: an extra blank line, and merging back-to-back 'if
(!skip_create)' blocks.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

qemu-img: Factor out code for merging bitmaps

The next patch will add another client that wants to merge dirty
bitmaps; it will be easier to refactor the code to construct the QAPI
struct correctly into a helper function.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-4-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

qcow2: Expose bitmaps' size during measure

It's useful to know how much space can be occupied by qcow2 persistent
bitmaps, even though such metadata is unrelated to the guest-visible
data.  Report this value as an additional QMP field, present when
measuring an existing image and output format that both support
bitmaps.  Update iotest 178 and 190 to updated output, as well as new
coverage in 190 demonstrating non-zero values made possible with the
recently-added qemu-img bitmap command (see 3b51ab4b).

The new 'bitmaps size:' field is displayed automatically as part of
'qemu-img measure' any time it is present in QMP (that is, any time
both the source image being measured and destination format support
bitmaps, even if the measurement is 0 because there are no bitmaps
present).  If the field is absent, it means that no bitmaps can be
copied (source, destination, or both lack bitmaps, including when
measuring based on size rather than on a source image).  This behavior
is compatible with an upcoming patch adding 'qemu-img convert
--bitmaps': that command will fail in the same situations where this
patch omits the field.

The addition of a new field demonstrates why we should always
zero-initialize qapi C structs; while the qcow2 driver still fully
populates all fields, the raw and crypto drivers had to be tweaked to
avoid uninitialized data.

Consideration was also given towards having a 'qemu-img measure
--bitmaps' which errors out when bitmaps are not possible, and
otherwise sums the bitmaps into the existing allocation totals rather
than displaying as a separate field, as a potential convenience
factor.  But this was ultimately decided to be more complexity than
necessary when the QMP interface was sufficient enough with bitmaps
remaining a separate field.

See also: https://bugzilla.redhat.com/1779904

Reported-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

iotests: Fix test 178

A recent change to qemu-img changed expected error message output, but
178 takes long enough to execute that it does not get run by 'make
check' or './check -g quick'.

Fixes: 43d589b074
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521192137.1120211-2-eblake@redhat.com>

migration: forbid bitmap migration by generated node-name

It actually never worked with libvirt, as auto-generated names are
different on source and destination.

It's unsafe and useless to migrate by auto-generated node-names, so
let's forbid it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-7-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>

migration: add_bitmaps_to_list: check disk name once

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-6-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>

iotests: 194: test also migration of dirty bitmap

Test that dirty bitmap migration works when we deal with mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-5-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

migration: fix bitmaps pre-blockdev migration with mirror job

Important thing for bitmap migration is to select destination block
node to obtain the migrated bitmap.

Prepatch, on source we use bdrv_get_device_or_node_name() to identify
the node, and on target we do bdrv_lookup_bs.
bdrv_get_device_or_node_name() returns blk name only for direct
children of blk. So, bitmaps of direct children of blks are migrated by
blk name and others - by node name.

Old libvirt is unprepared to bitmap migration by node-name,
node-names are mostly auto-generated. So actually only migration by blk
name works for it.

Newer libvirt will use new interface (which will be added soon) to
specify node-mapping for bitmaps migration explicitly. Still, let's
improve the current behavior a bit.

Now, consider classic libvirt migrations assisted by mirror block job:
mirror block job inserts filter, so our source is not a direct child of
blk, and bitmaps are migrated by node-names. And this just doesn't work
with auto-generated node names.

Let's fix it by using blk-name even if some implicit filters are
inserted.

Note2: we, of course, can't skip filters and use blk name to migrate
bitmaps in filtered node by blk name for this blk if these filters have
named bitmaps which should be migrated.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652424
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521220648.3255-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: comment typo fix, shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>

block/dirty-bitmap: add bdrv_has_named_bitmaps helper

To be used for bitmap migration in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-3-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

migration: refactor init_dirty_bitmap_migration

Split out handling one bs, it is needed for the following commit, which
will handle BlockBackends separately.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-2-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20200528-pull-request' into staging

hw/dispaly/sm501: bugfixes, add sanity checks.
hw/display: use tracepoints, misc cleanups.

# gpg: Signature made Thu 28 May 2020 13:35:32 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20200528-pull-request: (21 commits)
  sm501: Remove obsolete changelog and todo comment
  sm501: Optimize small overlapping blits
  sm501: Replace hand written implementation with pixman where possible
  sm501: Clean up local variables in sm501_2d_operation
  sm501: Use BIT(x) macro to shorten constant
  sm501: Shorten long variable names in sm501_2d_operation
  sm501: Convert printf + abort to qemu_log_mask
  hw/display/pxa2xx_lcd: Replace printf() call by qemu_log_mask()
  hw/display/omap_dss: Replace fprintf() call by qemu_log_mask(LOG_UNIMP)
  hw/display/exynos4210_fimd: Use qemu_log_mask(GUEST_ERROR)
  hw/display/vmware_vga: Let the PCI device own its I/O MemoryRegion
  hw/display/vmware_vga: Replace printf() calls by qemu_log_mask(ERROR)
  hw/display/xlnx_dp: Replace disabled DPRINTF() by error_report()
  hw/display/dpcd: Convert debug printf()s to trace events
  hw/display/dpcd: Fix memory region size
  hw/display/cirrus_vga: Convert debug printf() to trace event
  hw/display/cirrus_vga: Use qemu_log_mask(ERROR) instead of debug printf
  hw/display/cirrus_vga: Use qemu_log_mask(UNIMP) instead of debug printf
  hw/display/cirrus_vga: Convert debug printf() to trace event
  hw/display/cg3: Convert debug printf()s to trace events
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-05-28' into staging

Fixes and improvements for the gitlab-CI:
- Build with other images instead of the broken Debian containers
- Fix building with the latest version of Clang (at least wrt. to
  the gitlab-CI pipeline)
- Add Philippe, Alex and Wainer to the Gitlab-CI section in MAINTAINERS

# gpg: Signature made Thu 28 May 2020 10:16:15 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-05-28:
  gitlab-ci: Determine the number of jobs dynamically
  gitlab-ci: Do not use the standard container images from gitlab
  gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder
  GitLab CI: avoid calling before_scripts on unintended jobs
  gitlab-ci: Remove flex/bison packages
  MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section
  linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/alistair/tags/pull-register-api-20200527' into staging

A single patch to avoid clashes with the regiser field macros.

# gpg: Signature made Wed 27 May 2020 19:24:07 BST
# gpg:                using RSA key F6C4AC46D4934868D3B8CE8F21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>" [full]
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-register-api-20200527:
  hw/registerfields: Prefix local variables with underscore in macros

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2020-05-27' into staging

Error reporting patches for 2020-05-27

# gpg: Signature made Wed 27 May 2020 06:48:02 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2020-05-27:
  i386: Fix x86_cpu_load_model() error API violation
  arm/sabrelite: Consistently use &error_fatal in sabrelite_init()
  mips/boston: Plug memory leak in boston_mach_init()
  mips/boston: Fix boston_mach_init() error handling
  mips/malta: Fix create_cps() error handling
  error: Use error_reportf_err() where appropriate
  tests/migration: Tighten error checking
  s390x/cpumodel: Fix harmless misuse of visit_check_struct()
  xen: Fix and improve handling of device_add usb-host errors
  nvdimm: Plug memory leak in uuid property setter

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

sm501: Remove obsolete changelog and todo comment

Also update copyright year for latest changes

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 1392cad2ad1315a5a50409970e0af061821462e6.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Optimize small overlapping blits

AmigaOS tends to do a lot of small blits (even 1 pixel). Avoid malloc
overhead by keeping around a buffer for this and only alloc when
blitting larger areas.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: 7946852258d528497e85f465327fc90b5c3b59fb.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Replace hand written implementation with pixman where possible

Besides being faster this should also prevent malicious guests to
abuse 2D engine to overwrite data or cause a crash.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: 58666389b6cae256e4e972a32c05cf8aa51bffc0.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Clean up local variables in sm501_2d_operation

Make variables local to the block they are used in to make it clearer
which operation they are needed for.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: ae59f8138afe7f6a5a4a82539d0f61496a906b06.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Use BIT(x) macro to shorten constant

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 124bf5de8d7cf503b32b377d0445029a76bfbd49.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Shorten long variable names in sm501_2d_operation

This increases readability and cleans up some confusing naming.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: b9b67b94c46e945252a73c77dfd117132c63c4fb.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

sm501: Convert printf + abort to qemu_log_mask

Some places already use qemu_log_mask() to log unimplemented features
or errors but some others have printf() then abort(). Convert these to
qemu_log_mask() and avoid aborting to prevent guests to easily cause
denial of service.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 305af87f59d81e92f2aaff09eb8a3603b8baa322.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/pxa2xx_lcd: Replace printf() call by qemu_log_mask()

Replace printf() calls by qemu_log_mask(UNIMP), which is
disabled by default. This avoid flooding the terminal when
fuzzing the device.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-15-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/omap_dss: Replace fprintf() call by qemu_log_mask(LOG_UNIMP)

Replace fprintf() call by qemu_log_mask(LOG_UNIMP), which is
disabled by default. This avoid flooding the terminal when
fuzzing the device.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-14-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/exynos4210_fimd: Use qemu_log_mask(GUEST_ERROR)

Replace DPRINT_ERROR() by qemu_log_mask(GUEST_ERROR).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-13-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/vmware_vga: Let the PCI device own its I/O MemoryRegion

To avoid the orphan I/O memory region being added in the /unattached
QOM container, register the PCI device as its owner.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20200526062252.19852-12-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/vmware_vga: Replace printf() calls by qemu_log_mask(ERROR)

Avoid flooding stdio by converting printf() calls to
qemu_log_mask(GUEST_ERROR), which are disabled by default.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-11-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/xlnx_dp: Replace disabled DPRINTF() by error_report()

DPRINTF() calls are disabled by default, so when unexpected
data is used, the whole process abort without information.

Display a bit of information with error_report() before crashing.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-10-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/dpcd: Convert debug printf()s to trace events

Convert DPRINTF() to trace events and remove ifdef'ry.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-9-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/dpcd: Fix memory region size

The memory region size is 512K.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-8-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/cirrus_vga: Convert debug printf() to trace event

Convert the final bit of DEBUG_BITBLT to a tracepoint.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-7-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/cirrus_vga: Use qemu_log_mask(ERROR) instead of debug printf

Replace some debug printf() calls by qemu_log_mask(LOG_GUEST_ERROR).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-6-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/cirrus_vga: Use qemu_log_mask(UNIMP) instead of debug printf

Replace some debug printf() calls by qemu_log_mask(LOG_UNIMP),
and add a new one in cirrus_linear_bitblt_read().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-5-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/cirrus_vga: Convert debug printf() to trace event

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-4-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/cg3: Convert debug printf()s to trace events

Convert DPRINTF() to trace events and remove ifdef'ry.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-3-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

hw/display/edid: Add missing 'qdev-properties.h' header

When trying to consume the DEFINE_EDID_PROPERTIES() macro
by including "hw/display/edid.h", we get this build failure:

  include/hw/display/edid.h:24:5: error: implicit declaration of
  function ‘DEFINE_PROP_UINT32’ [-Werror=implicit-function-declaration]
     24 |     DEFINE_PROP_UINT32("xres", _state, _edid_info.prefx, 0),    \
        |     ^~~~~~~~~~~~~~~~~~

Headers should be self-contained, and one shouldn't have to
dig to find the missing headers.
In this case "hw/qdev-properties.h" is missing. Add it.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20200526062252.19852-2-f4bug@amsat.org
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

gitlab-ci: Determine the number of jobs dynamically

Some people might want to run the gitlab CI pipelines in an environment
where multiple CPUs are available to the runners, so let's rather get
the number for "-j" from the "nproc" program (increased by 1 to compensate
for jobs that wait for I/O) instead of hard-coding it.

Message-Id: <20200525131823.715-7-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

gitlab-ci: Do not use the standard container images from gitlab

Currently all pipelines of the gitlab CI are failing, except for the
"build-user" pipeline. There is an issue with the default container
image (likely Debian stable) where they imported something bad in one
of the system headers:

 /usr/include/linux/swab.h: In function '__swab':
 /builds/huth/qemu/include/qemu/bitops.h:20:34: error: "sizeof" is not
  defined, evaluates to 0 [-Werror=undef]
 #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)

We could maybe work-around this issue or wait for the default containers
to get fixed, but considering that we use Ubuntu (and thus Debian-style)
CI in Travis already to a very large extent, we should consider to use
some RPM-based distros in our gitlab CI instead. Thus let's change the
failing pipelines to use Fedora and CentOS (and also one Ubuntu 19.10,
since 20.04 is broken, too) now.

Message-Id: <20200525131823.715-6-thuth@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder

We have a dedicated folder for the gitlab-ci - so there is no need
to clutter the top directory with these .yml files.

Message-Id: <20200525131823.715-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

GitLab CI: avoid calling before_scripts on unintended jobs

At this point it seems that all jobs depend on those steps, with
maybe the EDK2 jobs as exceptions.

The jobs that will be added later will not want those scripts to be
run, so let's move these steps to the appropriate jobs, while
still trying to avoid repetition.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200525131823.715-4-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[thuth: Rebased to current master branch, use separate template]
Signed-off-by: Thomas Huth <thuth@redhat.com>

gitlab-ci: Remove flex/bison packages

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200515163029.12917-4-philmd@redhat.com>
Message-Id: <20200525131823.715-3-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section

Initially, I was the only one who was using Gitlab while most developers
had their git trees still on other systems, but that has changed nowadays.
There is now much more interest in the Gitlab-CI today, so it would be
good to have more than only one maintainer / reviewer for the gitlab-ci.yml
file. Alex, Wainer and Philippe kindly offered their help here, so let's
add them to the corresponding section in the MAINTAINERS file now.

Message-Id: <20200210155115.9371-1-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

Newer clangs rightly spot that you can never exceed the full address
space of 64 bit hosts with:

  linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
  long' > 18446744073709551615 is always false
  [-Werror,-Wtautological-type-limit-compare]
  4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
  4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
  4687 1 error generated.

So lets limit the check to 32 bit hosts only.

Fixes: ee94743034bf
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200525131823.715-8-thuth@redhat.com>
[thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
Signed-off-by: Thomas Huth <thuth@redhat.com>