Merge remote-tracking branch 'remotes/philmd-gitlab/tags/renesas-20201027' into staging

Renesas patches (SH4 and RX)

- Fix few warnings (Thomas Huth)
- Fix typos (Lichang Zhao, Chetan Pant)

CI jobs results:
. https://cirrus-ci.com/build/6368903343374336
. https://gitlab.com/philmd/qemu/-/pipelines/207919103
. https://travis-ci.org/github/philmd/qemu/builds/739133105

# gpg: Signature made Mon 26 Oct 2020 23:27:39 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/renesas-20201027:
  target/rx: Fix Lesser GPL version number
  target/rx: Fix some comment spelling errors
  target/sh4: fix some comment spelling errors
  target/sh4: Update coding style to make checkpatch.pl happy
  hw/timer/sh_timer: Remove superfluous "break" statements
  hw/timer/sh_timer: Silence warnings about missing fallthrough statements
  hw/timer/sh_timer: Coding style clean-up
  elf: Add EM_RX definition

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-5.2-pull-request' into staging

Update syscall numbers to 5.9-rc7
Fixes for prctl(), accept4() and xtensa

# gpg: Signature made Mon 26 Oct 2020 22:02:56 GMT
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-5.2-pull-request:
  target/xtensa: enable all coprocessors for linux-user
  linux-user: correct errno returned from accept4() syscall
  linux-user: remove _sysctl
  linux-user: update syscall.tbl to Linux 5.9-rc7
  linux-user: update mips/syscall-args-o32.c.inc to Linux 5.9-rc7
  linux-user: update syscall_nr.h to Linux 5.9-rc7
  linux-user: Support f_flags in statfs64 when available.
  Fix stack smashing when handling PR_GET_PDEATHSIG

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Revert series: virtiofsd: Announce submounts to the guest

This reverts the following commits due to their basis on a bogus
linux kernel header update:

c93a656f7b65 ("tests/acceptance: Add virtiofs_submounts.py")
45ced7ca2f27 ("tests/acceptance/boot_linux: Accept SSH pubkey")
08dce386e77e ("virtiofsd: Announce sub-mount points")
eba8b096c17c ("virtiofsd: Store every lo_inode's parent_dev")
ede24b6be798 ("virtiofsd: Add fuse_reply_attr_with_flags()")
e2577435d343 ("virtiofsd: Add attr_flags to fuse_entry_param")
2f10415abfc5 ("virtiofsd: Announce FUSE_ATTR_FLAGS")
97d741cc96dd ("linux/fuse.h: Pull in from Linux")

Cc: Max Reitz <mreitz@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 160385090886.20017.13382256442750027666.stgit@gimli.home
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/led-api-20201026' into staging

API to model LED.

CI jobs results:
. https://cirrus-ci.com/build/4879251751043072
. https://gitlab.com/philmd/qemu/-/pipelines/207661784
. https://travis-ci.org/github/philmd/qemu/builds/738958191
. https://app.shippable.com/github/philmd/qemu/runs/891/summary/console

# gpg: Signature made Mon 26 Oct 2020 22:03:59 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/led-api-20201026:
  hw/arm/tosa: Replace fprintf() calls by LED devices
  hw/misc/mps2-scc: Use the LED device
  hw/misc/mps2-fpgaio: Use the LED device
  hw/arm/aspeed: Add the 3 front LEDs drived by the PCA9552 #1
  hw/misc/led: Emit a trace event when LED intensity has changed
  hw/misc/led: Allow connecting from GPIO output
  hw/misc/led: Add a LED device

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20201026' into staging

Acceptance testing patches

- More ARM tests
- Documentation update

CI jobs results:
. https://cirrus-ci.com/build/5754555684093952
. https://gitlab.com/philmd/qemu/-/pipelines/207807648

# gpg: Signature made Mon 26 Oct 2020 19:06:52 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20201026:
  docs/devel/testing.rst: Update outdated Avocado URLs
  tests/acceptance: Allow running Orange Pi test using cached artifacts
  tests/acceptance: Add a 'virt_kvm' test using the GICv3
  tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-virtiofs-20201026' into staging

virtiofsd pull 2020-10-26

Misono
   Set default log level to info
   Explicit build option for virtiofsd

Me
   xattr name mapping

Stefan
  Alternative chroot sandbox method

Max
  Submount mechanism

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 26 Oct 2020 18:41:36 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert-gitlab/tags/pull-virtiofs-20201026:
  tests/acceptance: Add virtiofs_submounts.py
  tests/acceptance/boot_linux: Accept SSH pubkey
  virtiofsd: Announce sub-mount points
  virtiofsd: Store every lo_inode's parent_dev
  virtiofsd: Add fuse_reply_attr_with_flags()
  virtiofsd: Add attr_flags to fuse_entry_param
  virtiofsd: Announce FUSE_ATTR_FLAGS
  linux/fuse.h: Pull in from Linux
  tools/virtiofsd: xattr name mappings: Simple 'map'
  tools/virtiofsd: xattr name mapping examples
  tools/virtiofsd: xattr name mappings: Map server xattr names
  tools/virtiofsd: xattr name mappings: Map client xattr names
  tools/virtiofsd: xattr name mappings: Add option
  virtiofsd: add container-friendly -o sandbox=chroot option
  virtiofsd: passthrough_ll: set FUSE_LOG_INFO as default log_level
  configure: add option for virtiofsd

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/tracing-pull-request' into staging

Pull request

v2:
 * Fix Anthony Perard's email address [Philippe]

# gpg: Signature made Mon 26 Oct 2020 17:04:57 GMT
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/tracing-pull-request:
  Add execute bit back to scripts/tracetool.py
  trace/simple: Enable tracing on startup only if the user specifies a trace option

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20201026a' into staging

migration pull: 2020-10-26

Another go at Peter's postcopy fixes

Cleanups from Bihong Yu and Peter Maydell.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 26 Oct 2020 16:17:03 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20201026a:
  migration-test: Only hide error if !QTEST_LOG
  migration/postcopy: Release fd before going into 'postcopy-pause'
  migration: Sync requested pages after postcopy recovery
  migration: Maintain postcopy faulted addresses
  migration: Introduce migrate_send_rp_message_req_pages()
  migration: Pass incoming state into qemu_ufd_copy_ioctl()
  migration: using trace_ to replace DPRINTF
  migration: Delete redundant spaces
  migration: Open brace '{' following function declarations go on the next line
  migration: Do not initialise statics and globals to 0 or NULL
  migration: Add braces {} for if statement
  migration: Open brace '{' following struct go on the same line
  migration: Add spaces around operator
  migration: Don't use '#' flag of printf format
  migration: Do not use C99 // comments
  migration: Drop unused VMSTATE_FLOAT64 support

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/rx: Fix Lesser GPL version number

There is no "version 2" of the "Lesser" General Public License.
It is either "GPL version 2.0" or "Lesser GPL version 2.1".
This patch replaces all occurrences of "Lesser GPL version 2" with
"Lesser GPL version 2.1" in comment section.

Signed-off-by: Chetan Pant <chetan4windows@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201023123840.19988-1-chetan4windows@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

target/rx: Fix some comment spelling errors

There are many spelling errors in the comments of target/rx.
Use spellcheck to check the spelling errors, then fix them.

Signed-off-by: zhaolichang <zhaolichang@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Philippe Mathieu-Daude<f4bug@amsat.org>
Message-Id: <20201009064449.2336-5-zhaolichang@huawei.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

target/sh4: fix some comment spelling errors

There are many spelling errors in the comments of target/sh4.
Use spellcheck to check the spelling errors, then fix them.

Signed-off-by: zhaolichang <zhaolichang@huawei.com>
Reviewed-by: David Edmondson <david.edmondson@oracle.com>
Reviewed-by: Philippe Mathieu-Daude<f4bug@amsat.org>
Message-Id: <20201009064449.2336-10-zhaolichang@huawei.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

target/sh4: Update coding style to make checkpatch.pl happy

Avoid checkpatch.pl warnings in the next commit.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

hw/timer/sh_timer: Remove superfluous "break" statements

hw_error() is marked as QEMU_NORETURN, so the "break" statements
after this function are just dead code.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20201020153935.54315-4-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

hw/timer/sh_timer: Silence warnings about missing fallthrough statements

When compiling with -Werror=implicit-fallthrough, gcc complains about
missing fallthrough annotations in this file. Looking at the code,
the fallthrough is very likely intended here, so add some comments
to silence the compiler warnings.

Fixes: cd1a3f6840e ("Stand-alone TMU emulation code")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20201020153935.54315-3-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

hw/timer/sh_timer: Coding style clean-up

Replace TAB characters with spaces, put code after case-statement on
separate lines and add some curly braces in related lines to keep
checkpatch.pl happy.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20201020153935.54315-2-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

elf: Add EM_RX definition

RX's ELF machine is not defined in "elf.h".
Added it.

Signed-off-by: Yoshinori Sato <ysato@users.sourceforge.jp>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200814131438.28406-1-ysato@users.sourceforge.jp>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-10-26' into staging

bitmaps patches for 2020-10-26

- fix infloop on large bitmap granularity
- silence compiler warning

# gpg: Signature made Mon 26 Oct 2020 11:56:54 GMT
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-10-26:
  migration/block-dirty-bitmap: fix uninitialized variable warning
  migration/block-dirty-bitmap: fix larger granularity bitmaps

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

tests/acceptance: Add virtiofs_submounts.py

This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.

(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)

Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter.  (The on-image kernel can be used by providing an
empty string via vmlinuz=.)

So, invoking the test can be done as follows:
$ avocado run \
    tests/acceptance/virtiofs_submounts.py \
    -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage

This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them.  (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-9-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tests/acceptance/boot_linux: Accept SSH pubkey

Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-8-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: WIllian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: Announce sub-mount points

Whenever we encounter a directory with an st_dev that differs from that
of its parent, we set the FUSE_ATTR_SUBMOUNT flag so the guest can
create a submount for it.

Make this behavior optional, so submounts are only announced to the
guest with the announce_submounts option.  Some users may prefer the
current behavior, so that the guest learns nothing about the host mount
structure.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-7-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Manual merge

virtiofsd: Store every lo_inode's parent_dev

We want to detect mount points in the shared tree.  We report them to
the guest by setting the FUSE_ATTR_SUBMOUNT flag in fuse_attr.flags, but
because the FUSE client will create a submount for every directory that
has this flag set, we must do this only for the actual mount points.

We can detect mount points by comparing a directory's st_dev with its
parent's st_dev.  To be able to do so, we need to store the parent's
st_dev in the lo_inode object.

Note that mount points need not necessarily be directories; a single
file can be a mount point as well.  However, for the sake of simplicity
let us ignore any non-directory mount points for now.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-6-mreitz@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: Add fuse_reply_attr_with_flags()

The plain fuse_reply_attr() function does not allow setting
fuse_attr.flags, so add this new function that does.

Make fuse_reply_attr() a wrapper around it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-5-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: Add attr_flags to fuse_entry_param

fuse_entry_param is converted to fuse_attr on the line (by
fill_entry()), so it should have a member that mirrors fuse_attr.flags.

fill_entry() should then copy this fuse_entry_param.attr_flags to
fuse_attr.flags.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-4-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: Announce FUSE_ATTR_FLAGS

The fuse_attr.flags field is currently just initialized to 0, which is
valid.  Thus, there is no reason not to always announce FUSE_ATTR_FLAGS
(when the kernel supports it).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-3-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

linux/fuse.h: Pull in from Linux

Update the linux/fuse.h standard header from the kernel development tree
that implements FUSE submounts.

This adds the fuse_attr.flags field, the FUSE_ATTR_FLAGS INIT flag, and
the FUSE_ATTR_SUBMOUNT flag for fuse_attr.flags.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-2-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tools/virtiofsd: xattr name mappings: Simple 'map'

The mapping rule system implemented in the last few patches is
extremely flexible, but not easy to use.  Add a simple
'map' type as a sprinkling of sugar to make it easy.

e.g.

  -o xattrmap=":map::user.virtiofs.:"

would be sufficient to prefix all xattr's
or

  -o xattrmap=":map:trusted.:user.virtiofs.:"

would just prefix 'trusted.' xattr's and leave
everything else alone.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-6-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tools/virtiofsd: xattr name mapping examples

Add a few examples of xattrmaps to the documentation.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201023165812.36028-5-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tools/virtiofsd: xattr name mappings: Map server xattr names

Map xattr names coming from the server, i.e. the host filesystem;
currently this is only from listxattr.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-4-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tools/virtiofsd: xattr name mappings: Map client xattr names

Map xattr names originating at the client; from get/set/remove xattr.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-3-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tools/virtiofsd: xattr name mappings: Add option

Add an option to define mappings of xattr names so that
the client and server filesystems see different views.
This can be used to have different SELinux mappings as
seen by the guest, to run the virtiofsd with less privileges
(e.g. in a case where it can't set trusted/system/security
xattrs but you want the guest to be able to), or to isolate
multiple users of the same name; e.g. trusted attributes
used by stacking overlayfs.

A mapping engine is used with 3 simple rules; the rules can
be combined to allow most useful mapping scenarios.
The ruleset is defined by -o xattrmap='rules...'.

This patch doesn't use the rule maps yet.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-2-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: add container-friendly -o sandbox=chroot option

virtiofsd cannot run in a container because CAP_SYS_ADMIN is required to
create namespaces.

Introduce a weaker sandbox mode that is sufficient in container
environments because the container runtime already sets up namespaces.
Use chroot to restrict path traversal to the shared directory.

virtiofsd loses the following:

1. Mount namespace. The process chroots to the shared directory but
   leaves the mounts in place. Seccomp rejects mount(2)/umount(2)
   syscalls.

2. Pid namespace. This should be fine because virtiofsd is the only
   process running in the container.

3. Network namespace. This should be fine because seccomp already
   rejects the connect(2) syscall, but an additional layer of security
   is lost. Container runtime-specific network security policies can be
   used drop network traffic (except for the vhost-user UNIX domain
   socket).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201008085534.16070-1-stefanha@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtiofsd: passthrough_ll: set FUSE_LOG_INFO as default log_level

Just noticed that although help message says default log level is INFO,
it is actually 0 (EMRGE) and no mesage will be shown when error occurs.
It's better to follow help message.

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Message-Id: <20201008110148.2757734-1-misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

configure: add option for virtiofsd

Currently it is unknown whether virtiofsd will be built at
configuration time. It will be automatically built when dependency
is met. Also, required libraries are not clear.

To make this clear, add configure option --{enable,disable}-virtiofsd.
The default is the same as current (enabled if available) like many
other options. When --enable-virtiofsd is given and dependency is not
met, we get:

  ERROR: Problem encountered: virtiofsd requires libcap-ng-devel and seccomp-devel
or
  ERROR: Problem encountered: virtiofsd needs tools and vhost-user support

In addition, configuration summary now includes virtiofsd entry:

  build virtiofs daemon: YES/NO

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Message-Id: <20201008103133.2722903-1-misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  Manual merge

docs/devel/testing.rst: Update outdated Avocado URLs

Avocado documentation referred returns 404 error.
Update the broken links.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201010080741.2932406-1-philmd@redhat.com>

tests/acceptance: Allow running Orange Pi test using cached artifacts

Unfortunately the Armbian 19.11.3 image has been removed from the
dl.armbian.com file server. Developers having the artifact cached
can still run the test. Allow them to, until we find a proper
solution to share binaries with the whole community.

This avoids (when file manually added to cache):

  BootLinuxConsole.test_arm_orangepi_bionic_19_11: CANCEL: Missing asset https://dl.armbian.com/orangepipc/archive/Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z (1.06 s)

Reported-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-Id: <20201023131808.3198005-3-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

tests/acceptance: Add a 'virt_kvm' test using the GICv3

The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.

Noticed while running:

 $ avocado --show=app run -t machine:virt tests/acceptance/
 ...
 (2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: Unexpected empty reply from server (1.82 s)

The job.log content is:

  L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel GICv2 emulation\n'

With this patch:

 $ avocado --show=app run -t device:gicv3 tests/acceptance/
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: PASS (55.10 s)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200929224857.1225107-1-philmd@redhat.com>

tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3

This test runs Trusted Firmware-A on the Raspberry Pi 3.
We deliberately stop the boot process when the EDK2 UEFI version
is displayed.

The binary is build on AppVeyor CI using Pete Batard repository [1].
ATF v2.1 binary are used (see [2]).

It is very simple and fast:

  $ avocado --show=app,console run -t atf tests/acceptance
  JOB ID     : 1e748d7c9e9011cf0af3250ddc8ebf2389d6204e
  JOB LOG    : avocado/job-results/job-2020-02-16T18.08-1e748d7/job.log
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf:
  console: NOTICE:  Booting Trusted Firmware
  console: NOTICE:  BL1: v2.1(release):v2.1
  console: NOTICE:  BL1: Built : 15:26:06, May 13 2019
  console: NOTICE:  rpi3: Detected: Raspberry Pi 3 Model B (1GB, Sony, UK) [0x00a02082]
  console: NOTICE:  BL1: Booting BL2
  console: ERROR:   rpi3_sdhost: timeout status 0x40
  console: NOTICE:  BL2: v2.1(release):v2.1
  console: NOTICE:  BL2: Built : 15:26:01, May 13 2019
  console: NOTICE:  BL1: Booting BL31
  console: NOTICE:  BL31: v2.1(release):v2.1
  console: NOTICE:  BL31: Built : 15:26:04, May 13 2019
  console: =UEFI firmware (version UEFI Firmware v1.15 built at 11:58:44 on Feb 14 2020)
  PASS (1.54 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 1.88 s

[1] https://github.com/pbatard/RPi3#summary
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/v2.1/docs/plat/rpi3.rst

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200217103442.30318-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sd-next-20201026' into staging

SD/MMC patches

- Fix a bug in CMD6/SWITCH_FUNCTION (Bin Meng)
- Minor housekeeping patches

CI jobs result:
. https://cirrus-ci.com/build/5461987880599552
. https://gitlab.com/philmd/qemu/-/pipelines/207532287
. https://travis-ci.org/github/philmd/qemu/builds/738901111
. https://app.shippable.com/github/philmd/qemu/runs/888/summary/console

# gpg: Signature made Mon 26 Oct 2020 11:02:22 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/sd-next-20201026:
  hw/sd/sdcard: Zero out function selection fields before being populated
  hw/sd/sdcard: Make iolen unsigned
  hw/sd/sdcard: Constify sd_crc*()'s message argument
  hw/sd/sdcard: Simplify cmd_valid_while_locked()
  hw/sd/sdcard: Update the SDState documentation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Add execute bit back to scripts/tracetool.py

Commit a81df1b68b65 ("libqemuutil, qapi, trace: convert to meson")
removed it without explanation and it is useful to be able to run a
script without having to figure out which interpreter to use.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200923103620.1980151-1-anthony.perard@citrix.com>

trace/simple: Enable tracing on startup only if the user specifies a trace option

Tracing can be enabled at the command line or via the
monitor. Command-line trace options are recorded during
trace_opt_parse(), but tracing is not enabled until the various
front-ends later call trace_init_file(). If the user passes a trace
option on the command-line, remember that and enable tracing during
trace_init_file().  Otherwise, trace_init_file() should record the
trace file specified by the frontend and avoid enabling traces
until the user requests them via the monitor.

This fixes 1b7157be3a8c4300fc8044d40f4b2e64a152a1b4 and also
db25d56c014aa1a96319c663e0a60346a223b31e, by allowing the user
to enable traces on the command line and also avoiding
unwanted trace-<pid> files when the user has not asked for them.

Fixes: 1b7157be3a8c4300fc8044d40f4b2e64a152a1b4
Signed-off-by: Josh DuBois <josh@joshdubois.com>
Message-id: 20200816174610.20253-1-josh@joshdubois.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

migration-test: Only hide error if !QTEST_LOG

The errors are very useful when debugging qtest failures, especially when
QTEST_LOG=1 is set.  Let's allow override MigrateStart.hide_stderr when
QTEST_LOG=1 is specified, because that means the user wants to be verbose.

Not very nice to introduce the first QTEST_LOG env access in migration-test.c,
however it should be handy.  Without this patch, I was hacking error_report()
when debugging such errors.  Let's make things easier.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-7-peterx@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/postcopy: Release fd before going into 'postcopy-pause'

Logically below race could trigger with the old code:

          test program                        migration thread
          ------------                        ----------------
       wait_until('postcopy-pause')
                                          postcopy_pause()
                                            set_state('postcopy-pause')
       do_postcopy_recover()
         arm s->to_dst_file with new fd
                                            release s->to_dst_file [1]

Here [1] could have released the just-installed recoverying channel.  Then the
migration could hang without really resuming.

Instead, it should be very safe to release the fd before setting the state into
'postcopy-pause', because there's no reason for any other thread to touch it
during 'postcopy-active'.

Dave reported a very rare postcopy recovery hang that the migration-test
program waited for the migration to complete in migrate_postcopy_complete().
We do suspect it's the same thing that we're gonna fix here.  Hard to tell.
However since we've noticed this, fix this irrelevant of the hang report.

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-6-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Sync requested pages after postcopy recovery

We synchronize the requested pages right after a postcopy recovery happens.
This helps to synchronize the prioritized pages on source so that the faulted
threads can be served faster.

Reported-by: Xiaohui Li <xiaohli@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-5-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Maintain postcopy faulted addresses

Maintain a list of faulted addresses on the destination host for which we're
waiting on.  This is implemented using a GTree rather than a real list to make
sure even there're plenty of vCPUs/threads that are faulting, the lookup will
still be fast with O(log(N)) (because we'll do that after placing each page).
It should bring a slight overhead, but ideally that shouldn't be a big problem
simply because in most cases the requested page list will be short.

Actually we did similar things for postcopy blocktime measurements.  This patch
didn't use that simply because:

  (1) blocktime measurement is towards vcpu threads only, but here we need to
      record all faulted addresses, including main thread and external
      thread (like, DPDK via vhost-user).

  (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we
      don't want to add that extra dependency on the kernel version since not
      necessary.  E.g., we don't need to know which thread faulted on which
      page, we also don't care about multiple threads faulting on the same
      page.  But we only care about what addresses are faulted so waiting for a
      page copying from src.

  (3) blocktime measurement is not enabled by default.  However we need this by
      default especially for postcopy recover.

Another thing to mention is that this patch introduced a new mutex to serialize
the receivedmap and the page_requested tree, however that serialization does
not cover other procedures like UFFDIO_COPY.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-4-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Introduce migrate_send_rp_message_req_pages()

This is another layer wrapper for sending a page request to the source VM.  The
new migrate_send_rp_message_req_pages() will be used elsewhere in coming
patches.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-3-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Pass incoming state into qemu_ufd_copy_ioctl()

It'll be used in follow up patches to access more fields out of it.  Meanwhile
fetch the userfaultfd inside the function.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-2-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: using trace_ to replace DPRINTF

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Message-Id: <1603179176-5360-1-git-send-email-yubihong@huawei.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Delete redundant spaces

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-9-git-send-email-yubihong@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Open brace '{' following function declarations go on the next line

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-8-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Do not initialise statics and globals to 0 or NULL

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-7-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Add braces {} for if statement

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-6-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Open brace '{' following struct go on the same line

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-5-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Add spaces around operator

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-4-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Don't use '#' flag of printf format

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-3-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Do not use C99 // comments

Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-2-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Drop unused VMSTATE_FLOAT64 support

Commit ef96e3ae9698d6 in January 2019 removed the last user of the
VMSTATE_FLOAT64* macros. These were used by targets which defined
their floating point register file as an array of 'float64'.

We used to try to maintain a stricter distinction between
'float64' (a type for holding an integer representing an IEEE float)
and 'uint64_t', including having a debug option for 'float64' being
a struct and supposedly mandatory macros for converting between
float64 and uint64_t. We no longer think that's a usefully
strong distinction to draw and we allow ourselves to freely
assume that float64 really is just a 64-bit integer type, so
for new targets we would simply recommend use of the uint64_t type
for a floating point register file. The float64 type remains
as a useful way of documenting in the type signature of helper
functions and the like that they expect to receive an IEEE float
from the TCG generated code rather than an arbitrary integer.

Since the VMSTATE_FLOAT64* macros have no remaining users and
we don't recommend new code uses them, delete them.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20201022120830.5938-1-peter.maydell@linaro.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* fix --disable-tcg builds (Claudio)
* Fixes for macOS --enable-modules build and OpenBSD curses/iconv detection (myself)
* Start preparing for meson 0.56 (myself)
* Move directory configuration to meson (myself)
* Start untangling qemu_init (myself)
* Windows fixes (Sunil)
* Remove -no-kbm (Thomas)

# gpg: Signature made Mon 26 Oct 2020 11:12:17 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
  machine: move SMP initialization from vl.c
  machine: move UP defaults to class_base_init
  machine: remove deprecated -machine enforce-config-section option
  win32: boot broken when bind & data dir are the same
  WHPX: Fix WHPX build break
  configure: move install_blobs from configure to meson
  configure: remove unused variable from config-host.mak
  configure: move directory options from config-host.mak to meson
  configure: allow configuring localedir
  Makefile: separate meson rerun from the rest of the ninja invocation
  Remove deprecated -no-kvm option
  replay: do not build if TCG is not available
  qtest: unbreak non-TCG builds in bios-tables-test
  hw/core/qdev-clock: add a reference on aliased clocks
  do not use colons in test names
  meson: rewrite curses/iconv test
  build: fix macOS --enable-modules build

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20201026' into staging

some s390x fixes

# gpg: Signature made Mon 26 Oct 2020 10:46:50 GMT
# gpg:                using RSA key C3D0D66DC3624FF6A8C018CEDECF6B93C6F02FAF
# gpg:                issuer "cohuck@redhat.com"
# gpg: Good signature from "Cornelia Huck <conny@cornelia-huck.de>" [unknown]
# gpg:                 aka "Cornelia Huck <huckc@linux.vnet.ibm.com>" [full]
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>" [full]
# gpg:                 aka "Cornelia Huck <cohuck@kernel.org>" [unknown]
# gpg:                 aka "Cornelia Huck <cohuck@redhat.com>" [unknown]
# Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0  18CE DECF 6B93 C6F0 2FAF

* remotes/cohuck/tags/s390x-20201026:
  s390x: pv: Fix diag318 PV fencing
  s390x: pv: Remove sclp boundary checks
  s390x/s390-virtio-ccw: Reset PCI devices during subsystem reset

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-10-26' into staging

* qtest fixes (e.g. memory leaks)
* Fix for Xen dummy cpu loop (which happened due to qtest accel rework)
* Introduction of the generic device fuzzer
* Run more check-acceptance tests in the gitlab-CI

# gpg: Signature made Mon 26 Oct 2020 09:34:04 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-10-26: (31 commits)
  tests/acceptance: Use .ppm extention for Portable PixMap files
  tests/acceptance: Remove unused import
  test/docker/dockerfiles: Add missing packages for acceptance tests
  tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
  test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
  tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
  scripts/oss-fuzz: ignore the generic-fuzz target
  scripts/oss-fuzz: use hardlinks instead of copying
  fuzz: register predefined generic-fuzz configs
  fuzz: add generic-fuzz configs for oss-fuzz
  fuzz: add an "opaque" to the FuzzTarget struct
  fuzz: Add instructions for using generic-fuzz
  scripts/oss-fuzz: Add crash trace minimization script
  scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace
  fuzz: add a crossover function to generic-fuzzer
  fuzz: add a DISABLE_PCI op to generic-fuzzer
  fuzz: Add support for custom crossover functions
  fuzz: Add fuzzer callbacks to DMA-read functions
  fuzz: Declare DMA Read callback function
  fuzz: Add DMA support to the generic-fuzzer
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/tosa: Replace fprintf() calls by LED devices

The recently added LED device reports LED status changes with
the 'led_set_intensity' trace event. It is less invasive than
the fprintf() calls. We need however to have a binary built
with tracing support.

Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200912134041.946260-8-f4bug@amsat.org>

hw/misc/mps2-scc: Use the LED device

Per the 'ARM MPS2 and MPS2+ FPGA Prototyping Boards Technical
Reference Manual' (100112_0200_07_en):

  2.1  Overview of the MPS2 and MPS2+ hardware

       The MPS2 and MPS2+ FPGA Prototyping Boards contain the
       following components and interfaces:

       * User switches and user LEDs:

         - Two green LEDs and two push buttons that connect to
           the FPGA.
         - Eight green LEDs and one 8-way dip switch that connect
           to the MCC.

Add the 8 LEDs connected to the MCC.

This replaces the 'mps2_scc_leds' trace events by the generic
'led_set_intensity' event.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200912134041.946260-7-f4bug@amsat.org>

hw/misc/mps2-fpgaio: Use the LED device

Per the 'ARM MPS2 and MPS2+ FPGA Prototyping Boards Technical
Reference Manual' (100112_0200_07_en):

  2.1  Overview of the MPS2 and MPS2+ hardware

       The MPS2 and MPS2+ FPGA Prototyping Boards contain the
       following components and interfaces:

       * User switches and user LEDs:

         - Two green LEDs and two push buttons that connect to
           the FPGA.
         - Eight green LEDs and one 8-way dip switch that connect
           to the MCC.

Add the 2 LEDs connected to the FPGA.

This replaces the 'mps2_fpgaio_leds' trace events by the generic
'led_set_intensity' event.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200912134041.946260-6-f4bug@amsat.org>

hw/arm/aspeed: Add the 3 front LEDs drived by the PCA9552 #1

The Witherspoon has 3 LEDs connected to a PCA9552. Add them.
The names and reset values are taken from:
https://github.com/open-power/witherspoon-xml/blob/master/witherspoon.xml

Example booting obmc-phosphor-image:

  $ qemu-system-arm -M witherspoon-bmc -trace led_change_intensity
  1592693373.997015:led_change_intensity LED desc:'front-fault-4' color:green intensity 0% -> 100%
  1592693373.997632:led_change_intensity LED desc:'front-power-3' color:green intensity 0% -> 100%
  1592693373.998239:led_change_intensity LED desc:'front-id-5' color:green intensity 0% -> 100%
  1592693500.291805:led_change_intensity LED desc:'front-power-3' color:green intensity 100% -> 0%
  1592693500.312041:led_change_intensity LED desc:'front-power-3' color:green intensity 0% -> 100%
  1592693500.821254:led_change_intensity LED desc:'front-power-3' color:green intensity 100% -> 0%
  1592693501.331517:led_change_intensity LED desc:'front-power-3' color:green intensity 0% -> 100%
  1592693501.841367:led_change_intensity LED desc:'front-power-3' color:green intensity 100% -> 0%
  1592693502.350839:led_change_intensity LED desc:'front-power-3' color:green intensity 0% -> 100%
  1592693502.861134:led_change_intensity LED desc:'front-power-3' color:green intensity 100% -> 0%
  1592693503.371090:led_change_intensity LED desc:'front-power-3' color:green intensity 0% -> 100%

We notice the front-power LED starts to blink at a ~2Hz rate.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20200912134041.946260-5-f4bug@amsat.org>

hw/misc/led: Emit a trace event when LED intensity has changed

Track the LED intensity, and emit a trace event when it changes.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200912134041.946260-4-f4bug@amsat.org>

hw/misc/led: Allow connecting from GPIO output

Some devices expose GPIO lines.

Add a GPIO qdev input to our LED device, so we can
connect a GPIO output using qdev_connect_gpio_out().

When used with GPIOs, the intensity can only be either
minium or maximum. This depends of the polarity of the
GPIO (which can be inverted).
Declare the GpioPolarity type to model the polarity.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200912134041.946260-3-f4bug@amsat.org>

hw/misc/led: Add a LED device

Add a LED device which can be connected to a GPIO output.
They can also be dimmed with PWM devices. For now we do
not implement the dimmed mode, but in preparation of a
future implementation, we start using the LED intensity.

LEDs are limited to a fixed set of colors.

Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200912134041.946260-2-f4bug@amsat.org>

migration/block-dirty-bitmap: fix uninitialized variable warning

A default value is provided for the variable 'bitmap_name' to avoid
a compiler warning.

The compiler showed the warning:
migration/block-dirty-bitmap.c:1090:13: warning: ‘bitmap_name’
may be used uninitialized in this function [-Wmaybe-uninitialized]
       g_strlcpy(s->bitmap_name, bitmap_name, sizeof(s->bitmap_name));
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Chen Qun <kuhn.chenqun@huawei.com>
Message-Id: <20201014114430.1898684-1-kuhn.chenqun@huawei.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: commit message grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>

migration/block-dirty-bitmap: fix larger granularity bitmaps

sectors_per_chunk is a 64 bit integer, but the calculation is done in 32
bits, leading to an overflow for coarse bitmap granularities.

If that results in the value 0, it leads to a hang where no progress is
made but send_bitmap_bits is constantly called with nr_sectors being 0.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Message-Id: <20201021144456.1072-1-s.reiter@proxmox.com>
Fixes: b35ebdf07 migration: add postcopy migration of dirty bitmaps
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: Use correct type for 8ULL, use () to avoid overflow]
Signed-off-by: Eric Blake <eblake@redhat.com>

Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20201023' into staging

A collection of RISC-V fixes for the next QEMU release.

This includes:
 - Improvements to logging output
 - Hypervisor instruction fixups
 - The ability to load a noMMU kernel
 - SiFive OTP support

# gpg: Signature made Fri 23 Oct 2020 16:13:57 BST
# gpg:                using RSA key F6C4AC46D4934868D3B8CE8F21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>" [full]
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-riscv-to-apply-20201023:
  hw/misc/sifive_u_otp: Add backend drive support
  hw/misc/sifive_u_otp: Add write function and write-once protection
  target/riscv: raise exception to HS-mode at get_physical_address
  hw/riscv: Load the kernel after the firmware
  hw/riscv: Add a riscv_is_32_bit() function
  hw/riscv: Return the end address of the loaded firmware
  hw/riscv: sifive_u: Allow specifying the CPU
  target/riscv: Fix implementation of HLVX.WU instruction
  target/riscv: Fix update of hstatus.GVA in riscv_cpu_do_interrupt
  target/riscv: Fix update of hstatus.SPVP
  hw/intc: Move sifive_plic.h to the include directory
  riscv: Convert interrupt logs to use qemu_log_mask()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

machine: move SMP initialization from vl.c

Initialize the object's values from the class when the object is
created, no need to have vl.c do it for us.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

machine: move UP defaults to class_base_init

Clean up vl.c, default min/max/default_cpus to uniprocessor
directly in the QOM class initialization code.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

machine: remove deprecated -machine enforce-config-section option

Deprecated since 3.1 and complicates the initialization sequence,
remove it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

win32: boot broken when bind & data dir are the same

With upstream commit#ea1edcd7da1a "vl: relocate paths to data
directories", the data dir logic was unified between POSIX &
Win32. That patch moved to using 'get_relocated_path()', to
find the data dir. There is a latent bug in get_relocated_path
which can cause it to spin indefinitely, when the bind dir is
the same as the passed in dir (in this case, it was the data
dir).

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Message-Id: <SN4PR2101MB08802BF242C429A15DDB32ACC01B0@SN4PR2101MB0880.namprd21.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

WHPX: Fix WHPX build break

With upstream commit#8a19980e3fc4, logic was introduced to only
allow WHPX build on x64. But, the logic checks for the cpu family
and not the cpu. On my fedora container build, the cpu family is
x86 and the cpu is x86_64. Fixing the build break by checking for
the cpu, instead of the cpu family.

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Message-Id: <SN4PR2101MB0880D706A85793DDFC411304C01D0@SN4PR2101MB0880.namprd21.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: move install_blobs from configure to meson

Move the conditions under which edk2 blobs are decompressed
and installed to pc-bios/meson.build.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: remove unused variable from config-host.mak

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: move directory options from config-host.mak to meson

Since installation is not part of Makefiles anymore, Make need not
know the directories anymore.  Meson already knows them through
built-in options, do everything using them instead of the config_host
dictionary.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: allow configuring localedir

Meson has a localedir option, so passing the path through that option
is the cleanest way when we move directories out of config-host.mak.
In preparation for doing that without changing semantics and without
special-casing localedir code, add a configure option.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Makefile: separate meson rerun from the rest of the ninja invocation

The rules to build Makefile.mtest are suffering from the "tunnel vision"
problem that is common with recursive makefiles.  Makefile.mtest depends
on build.ninja, but Make does not know when build.ninja needs to be
rebuilt before creating Makefile.mtest.

To fix this, separate the ninja invocation into the "regenerate build
files" phase and the QEMU build phase.  Sentinel files such as
meson-private/coredata.dat or build.ninja are used to figure out the
phases that haven't run yet; however, because those files' timestamps
are not guaranteed to be touched, the usual makefile stamp-file trick
is used on top.

Reported-by: Havard Skinnemoen <hskinnemoen@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

target/xtensa: enable all coprocessors for linux-user

Linux userspace always sees coprocessors as enabled. CPENABLE register
and coprocessor exceptions are used internally by the kernel to manage
lazy coprocessor context switch. None of it is needed for linux-user.
Always enable all coprocessors for user emulation.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200829104758.22337-1-jcmvbkbc@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

linux-user: correct errno returned from accept4() syscall

accept4() returned wrong errno, that did not match current linux

Signed-off-by: Matus Kysel <mkysel@tachyum.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20200930151616.3588165-1-mkysel@tachyum.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

linux-user: remove _sysctl

It has been removed from linux since

  61a47c1ad3a4 ("sysctl: Remove the sysctl system call")

It's a good news because it was not really supported by qemu.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200930003033.554124-5-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

linux-user: update syscall.tbl to Linux 5.9-rc7

Updated running scripts/update-syscalltbl.sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200930003033.554124-4-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

linux-user: update mips/syscall-args-o32.c.inc to Linux 5.9-rc7

Updated running scripts/update-mips-syscall-args.sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200930003033.554124-3-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

linux-user: update syscall_nr.h to Linux 5.9-rc7

Update gensyscalls.sh not to generate an empty line at the end of the file

And then automatically update syscall_nr.h running scripts/gensyscalls.sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200930003033.554124-2-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

v3:
 * Fix linker error on FreeBSD and other !CONFIG_LINUX host OSes [Peter]

# gpg: Signature made Fri 23 Oct 2020 16:21:19 BST
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/block-pull-request: (28 commits)
  iotests: add commit top->base cases to 274
  block/io: fix bdrv_is_allocated_above
  block/io: bdrv_common_block_status_above: support bs == base
  block/io: bdrv_common_block_status_above: support include_base
  block/io: fix bdrv_co_block_status_above
  block/export: add vhost-user-blk multi-queue support
  block/export: add iothread and fixed-iothread options
  block: move block exports to libblockdev
  qemu-storage-daemon: avoid compiling blockdev_ss twice
  util/vhost-user-server: use static library in meson.build
  util/vhost-user-server: move header to include/
  block/export: convert vhost-user-blk server to block export API
  block/export: report flush errors
  util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
  util/vhost-user-server: check EOF when reading payload
  util/vhost-user-server: fix memory leak in vu_message_read()
  util/vhost-user-server: drop unused DevicePanicNotifier
  block/export: consolidate request structs into VuBlockReq
  util/vhost-user-server: drop unnecessary watch deletion
  util/vhost-user-server: drop unnecessary QOM cast
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

tests/acceptance: Use .ppm extention for Portable PixMap files

The HMP 'screendump' command generates Portable PixMap files.
Make it obvious by using the .ppm file extention.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

tests/acceptance: Remove unused import

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-2-f4bug@amsat.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

test/docker/dockerfiles: Add missing packages for acceptance tests

Some of the "check-acceptance" tests are still skipped in the CI
since the docker images do not provide the necessary packages, e.g.
the netcat binary. Add them to get more test coverage.

Message-Id: <20201023073351.251332-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI

The tests are running in containers here, so it should be OK to
run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case.

Message-Id: <20201023073351.251332-4-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

test/acceptance: Remove the CONTINUOUS_INTEGRATION tags

We are not running the acceptance tests on Travis anymore, so these
checks can be removed now.

Message-Id: <20201023073351.251332-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive

The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
instead seems to be fine.

Message-Id: <20201023073351.251332-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

scripts/oss-fuzz: ignore the generic-fuzz target

generic-fuzz is not a standalone fuzzer - it requires some env variables
to be set. On oss-fuzz, we set these with some predefined
generic-fuzz-{...} targets, that are thin wrappers around generic-fuzz.
Do not make a link for the generic-fuzz from the oss-fuzz build, so
oss-fuzz does not treat it as a standalone fuzzer.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-18-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
[thuth: Reformatted one comment to stay within the 80 columns limit]
Signed-off-by: Thomas Huth <thuth@redhat.com>

scripts/oss-fuzz: use hardlinks instead of copying

Prior to this, fuzzers in the output oss-fuzz directory were exactly
the same executable, with a different name to do argv[0]-based
fuzz-target selection. This is a waste of space, especially since these
binaries can weigh many MB.

Instead of copying, use hard links, to cut down on wasted space. We need
to place the primary copy of the executable into DEST_DIR, since this is
a separate file-system on oss-fuzz. We should not place it directly into
$DEST_DIR, since oss-fuzz will treat it as an independent fuzzer and try
to run it for fuzzing. Instead, we create a DEST_DIR/bin directory to
store the primary copy.

Suggested-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-17-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

fuzz: register predefined generic-fuzz configs

We call get_generic_fuzz_configs, which fills an array with
predefined {name, args, objects} triples. For each of these, we add a
new FuzzTarget, that uses a small wrapper to set
QEMU_FUZZ_{ARGS,OBJECTS} to the corresponding predefined values.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-16-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

fuzz: add generic-fuzz configs for oss-fuzz

Predefine some generic-fuzz configs. For each of these, we will create a
separate FuzzTarget that can be selected through argv0 and, therefore,
fuzzed on oss-fuzz.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-15-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

fuzz: add an "opaque" to the FuzzTarget struct

It can be useful to register FuzzTargets that have nearly-identical
initialization handlers (e.g. for using the same fuzzing code, with
different configuration options). Add an opaque pointer to the
FuzzTarget struct, so that FuzzTargets can hold some data, useful for
storing target-specific configuration options, that can be read by the
get_init_cmdline function.

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-14-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

fuzz: Add instructions for using generic-fuzz

Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-13-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

scripts/oss-fuzz: Add crash trace minimization script

Once we find a crash, we can convert it into a QTest trace. Usually this
trace will contain many operations that are unneeded to reproduce the
crash. This script tries to minimize the crashing trace, by removing
operations and trimming QTest bufwrite(write addr len data...) commands.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-12-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>

scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace

The generic-fuzzer uses hooks to fulfill DMA requests just-in-time.
This means that if we try to use QTEST_LOG=1 to build a reproducer, the
DMA writes will be logged _after_ the in/out/read/write that triggered
the DMA read. To work work around this, the generic-fuzzer annotates
these just-in time DMA fulfilments with a tag that we can use to
discern them. This script simply iterates over a raw qtest
trace (including log messages, errors, timestamps etc), filters it and
re-orders it so that DMA fulfillments are placed directly _before_ the
qtest command that will cause the DMA access.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-11-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>