slirp: adapt a subset of QEMU vmstate code

Add vmstate serialization code adapted from QEMU.

Keep only the bits that are required for libslirp.

Introduce a IStream/OStream interface to replace QEMU QFile
abstraction.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190212162524.31504-2-marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

slirp: Mark pieces missing IPv6 support

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

slirp: fix big/little endian conversion in ident protocol

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
Based-on: <1551476756-25749-1-git-send-email-will@wbowling.info>

slirp: check sscanf result when emulating ident

When emulating ident in tcp_emu, if the strchr checks passed but the
sscanf check failed, two uninitialized variables would be copied and
sent in the reply, so move this code inside the if(sscanf()) clause.

Signed-off-by: William Bowling <will@wbowling.info>
Cc: qemu-stable@nongnu.org
Cc: secalert@redhat.com
Message-Id: <1551476756-25749-1-git-send-email-will@wbowling.info>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

slirp: check for ioctlsocket error and 0-length udp payload.

Sometimes sorecvfrom() is called from slirp.c because revents == G_IO_IN,
but there is 0 bytes available and recvfrom could be blocking indefinitely.
This is likely due to 0-length udp payload. This also adds an error
checking for ioctlsocket.

Signed-off-by: Vic Lee <llyzs.vic@gmail.com>
Message-Id: <20190301064809.3074-1-llyzs.vic@gmail.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

slirp: Fix build with gcc 9

Build fails with gcc 9:

  CC      slirp/ndp_table.o
slirp/ndp_table.c: In function ‘ndp_table_add’:
slirp/ndp_table.c:31:23: error: taking address of packed member of ‘struct ndpentry’ may result in an unaligned pointer value [-Werror=address-of-packed-member]
   31 |         if (in6_equal(&ndp_table->table[i].ip_addr, &ip_addr)) {
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
slirp/ndp_table.c: In function ‘ndp_table_search’:
slirp/ndp_table.c:75:23: error: taking address of packed member of ‘struct ndpentry’ may result in an unaligned pointer value [-Werror=address-of-packed-member]
   75 |         if (in6_equal(&ndp_table->table[i].ip_addr, &ip_addr)) {
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

The ndpentry structure isn't used to model on-the-wire data or anything
else that would care for the struct layout. It doesn't need to be packed
actually. Just drop SLIRP_PACKED.

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <155143315831.102868.17515265400523392682.stgit@bahia.lan>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine queue, 2019-03-06

* qdev: Hotplug handler chaining (David Hildenbrand)
* qdev: fix qbus_is_full() (Tony Krowiak)
* hostmem: fix crash when querying empty host-nodes property via
  QMP (Igor Mammedov)

# gpg: Signature made Wed 06 Mar 2019 18:39:29 GMT
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
  qdev: Provide qdev_get_bus_hotplug_handler()
  qdev: Let machine hotplug handler to override bus hotplug handler
  qdev: Let the hotplug_handler_unplug() caller delete the device
  hostmem: fix crash when querying empty host-nodes property via QMP
  qdev/core: fix qbus_is_full()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

qdev: Provide qdev_get_bus_hotplug_handler()

Let's use a wrapper instead of looking it up manually. This function can
than be reused when we explicitly want to have the bus hotplug handler
(e.g. when the bus hotplug handler was overwritten by the machine
hotplug handler).

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-4-david@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

qdev: Let machine hotplug handler to override bus hotplug handler

it will allow to return another hotplug handler than the default
one for a specific bus based device type. Which is needed to handle
non trivial plug/unplug sequences that need the access to resources
configured outside of bus where device is attached.

That will allow for returned hotplug handler to orchestrate wiring
in arbitrary order, by chaining other hotplug handlers when
it's needed.

PS:
It could be used for hybrid virtio-mem and virtio-pmem devices
where it will return machine as hotplug handler which will do
necessary wiring at machine level and then pass control down
the chain to bus specific hotplug handler.

Example of top level hotplug handler override and custom plug sequence:

  some_machine_get_hotplug_handler(machine){
      if (object_dynamic_cast(OBJECT(dev), TYPE_SOME_BUS_DEVICE)) {
          return HOTPLUG_HANDLER(machine);
      }
      return NULL;
  }

  some_machine_device_plug(hotplug_dev, dev) {
      if (object_dynamic_cast(OBJECT(dev), TYPE_SOME_BUS_DEVICE)) {
          /* do machine specific initialization */
          some_machine_init_special_device(dev)

          /* pass control to bus specific handler */
          hotplug_handler_plug(dev->parent_bus->hotplug_handler, dev)
      }
  }

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-3-david@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

qdev: Let the hotplug_handler_unplug() caller delete the device

When unplugging a device, at one point the device will be destroyed
via object_unparent(). This will, one the one hand, unrealize the
removed device hierarchy, and on the other hand, destroy/free the
device hierarchy.

When chaining hotplug handlers, we want to overwrite a bus hotplug
handler by the machine hotplug handler, to be able to perform
some part of the plug/unplug and to forward the calls to the bus hotplug
handler.

For now, the bus hotplug handler would trigger an object_unparent(), not
allowing us to perform some unplug action on a device after we forwarded
the call to the bus hotplug handler. The device would be gone at that
point.

machine_unplug_handler(dev)
    /* eventually do unplug stuff */
    bus_unplug_handler(dev)
    /* dev is gone, we can't do more unplug stuff */

So move the object_unparent() to the original caller of the unplug. For
now, keep the unrealize() at the original places of the
object_unparent(). For implicitly chained hotplug handlers (e.g. pc
code calling acpi hotplug handlers), the object_unparent() has to be
done by the outermost caller. So when calling hotplug_handler_unplug()
from inside an unplug handler, nothing is to be done.

hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
    machine_unplug_handler(dev) {
        /* eventually do unplug stuff */
        bus_unplug_handler(dev) -> calls unrealize(dev)
        /* we can do more unplug stuff but device already unrealized */
    }
object_unparent(dev)

In the long run, every unplug action should be factored out of the
unrealize() function into the unplug handler (especially for PCI). Then
we can get rid of the additonal unrealize() calls and object_unparent()
will properly unrealize the device hierarchy after the device has been
unplugged.

hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
    machine_unplug_handler(dev) {
        /* eventually do unplug stuff */
        bus_unplug_handler(dev) -> only unplugs, does not unrealize
        /* we can do more unplug stuff */
    }
object_unparent(dev) -> will unrealize

The original approach was suggested by Igor Mammedov for the PCI
part, but I extended it to all hotplug handlers. I consider this one
step into the right direction.

To summarize:
- object_unparent() on synchronous unplugs is done by common code
-- "Caller of hotplug_handler_unplug"
- object_unparent() on asynchronous unplugs ("unplug requests") has to
  be done manually
-- "Caller of hotplug_handler_unplug"

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-2-david@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

hostmem: fix crash when querying empty host-nodes property via QMP

QEMU will crashes with
 qapi/qobject-output-visitor.c:210: qobject_output_complete: Assertion `qov->root && ((&qov->stack)->slh_first == ((void *)0))' failed
when trying to get value of not set hostmem's "host-nodes"
property, HostMemoryBackend::host_nodes bitmap doesn't have
any bits set in it, which leads to find_first_bit() returning
MAX_NODES and consequently to an early return from
host_memory_backend_get_host_nodes() without calling visitor.

Fix it by calling visitor even if "host-nodes" property wasn't
set before exiting from property getter to return valid empty
list.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190214105733.25643-1-imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

qdev/core: fix qbus_is_full()

The qbus_is_full(BusState *bus) function (qdev_monitor.c) compares the max_index
value of the BusState structure with the max_dev value of the BusClass structure
to determine whether the maximum number of children has been reached for the
bus. The problem is, the max_index field of the BusState structure does not
necessarily reflect the number of devices that have been plugged into
the bus.

Whenever a child device is plugged into the bus, the bus's max_index value is
assigned to the child device and then incremented. If the child is subsequently
unplugged, the value of the max_index does not change and no longer reflects the
number of children.

When the bus's max_index value reaches the maximum number of devices
allowed for the bus (i.e., the max_dev field in the BusClass structure),
attempts to plug another device will be rejected claiming that the bus is
full -- even if the bus is actually empty.

To resolve the problem, a new 'num_children' field is being added to the
BusState structure to keep track of the number of children plugged into the
bus. It will be incremented when a child is plugged, and decremented when a
child is unplugged.

Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
Reviewed-by: Pierre Morel<pmorel@linux.ibm.com>
Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
Message-Id: <1545062250-7573-1-git-send-email-akrowiak@linux.ibm.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20190306a' into staging

Migation pull 2019-03-06

(This replaces the pull sent yesterday)

   a) 4 small fixes including the cancel problem
     that caused the ahci migration test to fail
     intermittently
   b) Yury's ignore-shared feature
   c) Juan's extra tests
   d) Wei Wang's free page hinting
   e) Some Colo fixes from Zhang Chen

Diff from yesterdays pull:
  1) A missing fix of mine (cleanup during exit)
  2) Changes from Eric/Markus on 'Create socket-address parameter'

# gpg: Signature made Wed 06 Mar 2019 11:39:53 GMT
# gpg:                using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20190306a: (22 commits)
  qapi/migration.json: Remove a variable that doesn't exist in example
  Migration/colo.c: Make COLO node running after failover
  Migration/colo.c: Fix double close bug when occur COLO failover
  virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
  migration/ram.c: add the free page optimization enable flag
  migration/ram.c: add a notifier chain for precopy
  migration: API to clear bits of guest free pages from the dirty bitmap
  migration: use bitmap_mutex in migration_bitmap_clear_dirty
  bitmap: bitmap_count_one_with_offset
  bitmap: fix bitmap_count_one
  tests: Add basic migration precopy tcp test
  migration: Create socket-address parameter
  tests: Add migration xbzrle test
  migration: Add capabilities validation
  tests/migration-test: Add a test for ignore-shared capability
  migration: Add an ability to ignore shared RAM blocks
  migration: Introduce ignore-shared capability
  exec: Change RAMBlockIterFunc definition
  migration/rdma: clang compilation fix
  migration: Cleanup during exit
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

trivial patches pull request (20190206)

- acpi: remove unused functions/variables
- tests: remove useless architecture checks
- some typo fixes and documentation update
- flash_cfi02: fix memory leak

# gpg: Signature made Wed 06 Mar 2019 11:05:12 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  thunk: fix of malloc to g_new
  hostmem-file: simplify ifdef-s in file_backend_memory_alloc()
  build: Correct explanation of unnest-vars example
  bswap: Fix accessors syntax in comment
  doc: fix typos for documents in tree
  block/pflash_cfi02: Fix memory leak and potential use-after-free
  hw/acpi: remove unnecessary variable acpi_table_builtin
  hw/acpi: remove unused function acpi_table_add_builtin()
  hw/i386/pc.c: remove unused function pc_acpi_init()
  tests: Remove (mostly) useless architecture checks

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

qapi/migration.json: Remove a variable that doesn't exist in example

Remove the "active" variable in example for query-colo-status.
It is a doc bug from commit f56c0065

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190303145021.2962-6-chen.zhang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Migration/colo.c: Make COLO node running after failover

Delay to close COLO for auto start VM after failover.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190303145021.2962-4-chen.zhang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Migration/colo.c: Fix double close bug when occur COLO failover

In migration_incoming_state_destroy(void) will check the mis->to_src_file
to double close the mis->to_src_file when occur COLO failover.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190303145021.2962-2-chen.zhang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

The new feature enables the virtio-balloon device to receive hints of
guest free pages from the free page vq.

A notifier is registered to the migration precopy notifier chain. The
notifier calls free_page_start after the migration thread syncs the dirty
bitmap, so that the free page optimization starts to clear bits of free
pages from the bitmap. It calls the free_page_stop before the migration
thread syncs the bitmap, which is the end of the current round of ram
save. The free_page_stop is also called to stop the optimization in the
case when there is an error occurred in the process of ram saving.

Note: balloon will report pages which were free at the time of this call.
As the reporting happens asynchronously, dirty bit logging must be
enabled before this free_page_start call is made. Guest reporting must be
disabled before the migration dirty bitmap is synchronized.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-8-git-send-email-wei.w.wang@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Dropped kernel header update, fixed up CMD_ID_* name change

migration/ram.c: add the free page optimization enable flag

This patch adds the free page optimization enable flag, and a function
to set this flag. When the free page optimization is enabled, not all
the pages are needed to be sent in the bulk stage.

Why using a new flag, instead of directly disabling ram_bulk_stage when
the optimization is running?
Thanks for Peter Xu's reminder that disabling ram_bulk_stage will affect
the use of compression. Please see save_page_use_compression. When
xbzrle and compression are used, if free page optimizaion causes the
ram_bulk_stage to be disabled, save_page_use_compression will return
false, which disables the use of compression. That is, if free page
optimization avoids the sending of half of the guest pages, the other
half of pages loses the benefits of compression in the meantime. Using a
new flag to let migration_bitmap_find_dirty skip the free pages in the
bulk stage will avoid the above issue.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-7-git-send-email-wei.w.wang@intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/ram.c: add a notifier chain for precopy

This patch adds a notifier chain for the memory precopy. This enables various
precopy optimizations to be invoked at specific places.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-6-git-send-email-wei.w.wang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: API to clear bits of guest free pages from the dirty bitmap

This patch adds an API to clear bits corresponding to guest free pages
from the dirty bitmap. Spilt the free page block if it crosses the QEMU
RAMBlock boundary.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-5-git-send-email-wei.w.wang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: use bitmap_mutex in migration_bitmap_clear_dirty

The bitmap mutex is used to synchronize threads to update the dirty
bitmap and the migration_dirty_pages counter. For example, the free
page optimization clears bits of free pages from the bitmap in an
iothread context. This patch makes migration_bitmap_clear_dirty update
the bitmap and counter under the mutex.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-4-git-send-email-wei.w.wang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

bitmap: bitmap_count_one_with_offset

Count the number of 1s in a bitmap starting from an offset.

Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1544516693-5395-3-git-send-email-wei.w.wang@intel.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

bitmap: fix bitmap_count_one

BITMAP_LAST_WORD_MASK(nbits) returns 0xffffffff when "nbits=0", which
makes bitmap_count_one fail to handle the "nbits=0" case. It appears to be
preferred to remain BITMAP_LAST_WORD_MASK identical to the kernel
implementation that it is ported from.

So this patch fixes bitmap_count_one to handle the nbits=0 case.

Inital Discussion Link:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg554316.html
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
CC: Juan Quintela <quintela@redhat.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-2-git-send-email-wei.w.wang@intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

tests: Add basic migration precopy tcp test

Not sharing code from precopy/unix because we have to read back the
tcp parameter.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190227105128.1655-4-quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert:  Fixup for clash with Yury's

migration: Create socket-address parameter

It will be used to store the uri parameters. We want this only for
tcp, so we don't set it for other uris.  We need it to know what port
is migration running.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Removed DummyStruct as suggested by Eric & Markus

--

tests: Add migration xbzrle test

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20190227105128.1655-2-quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Fixup for class with Yury's series

migration: Add capabilities validation

Currently we don't check which capabilities set in the source QEMU.
We just expect that the target QEMU has the same enabled capabilities.

Add explicit validation for capabilities to make sure that the target VM
has them too. This is enabled for only new capabilities to keep compatibily.

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-6-yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Manual merge

tests/migration-test: Add a test for ignore-shared capability

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-5-yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
  dgilbert: Disabled the test for now, not happy on aarch64

migration: Add an ability to ignore shared RAM blocks

If ignore-shared capability is set then skip shared RAMBlocks during the
RAM migration.
Also, move qemu_ram_foreach_migratable_block (and rename) to the
migration code, because it requires access to the migration capabilities.

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-4-yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration: Introduce ignore-shared capability

We want to use local migration to update QEMU for running guests.
In this case we don't need to migrate shared (file backed) RAM.
So, add a capability to ignore such blocks during live migration.

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-3-yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

exec: Change RAMBlockIterFunc definition

Currently, qemu_ram_foreach_* calls RAMBlockIterFunc with many
block-specific arguments. But often iter func needs RAMBlock*.
This refactoring is needed for fast access to RAMBlock flags from
qemu_ram_foreach_block's callback. The only way to achieve this now
is to call qemu_ram_block_from_host (which also enumerates blocks).

So, this patch reduces complexity of
qemu_ram_foreach_block() -> cb() -> qemu_ram_block_from_host()
from O(n^2) to O(n).

Fix RAMBlockIterFunc definition and add some functions to read
RAMBlock* fields witch were passed.

Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-2-yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/rdma: clang compilation fix

Configuring QEMU with:
        ../configure --cc=clang --enable-rdma

Leads to compilation error:

  CC      migration/rdma.o
  CC      migration/block.o
  qemu/migration/rdma.c:3615:58: error: taking address of packed member 'rkey' of class or structure
      'RDMARegisterResult' may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member]
                            (uintptr_t)host_addr, NULL, &reg_result->rkey,
                                                         ^~~~~~~~~~~~~~~~
Fix it by using a temp local variable.

Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20190304184923.24215-1-marcel.apfelbaum@gmail.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

migration: Cleanup during exit

Currently we cleanup the migration object as we exit main after the
main_loop finishes; however if there's a migration running things
get messy and we can end up with the migration thread still trying
to access freed structures.

We now take a ref to the object around the migration thread itself,
so the act of dropping the ref during exit doesn't cause us to lose
the state until the thread quits.

Cancelling the migration during migration also tries to get the thread
to quit.

We do this a bit earlier; so hopefully migration gets out of the way
before all the devices etc are freed.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20190227164900.16378-1-dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

migration/rdma: Fix qemu_rdma_cleanup null check

If the migration fails before the channel is open (e.g. a bad
address) we end up in the cleanup with rdma->channel==NULL.

Spotted by Coverity: CID 1398634
Fixes: fbbaacab2758cb3f32a0
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190214185351.5927-1-dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

migration: Fix cancel state

During a cancelled migration there's a race where the fd can
go into an error state before we get back around the migration loop
and migration_detect_error transitions from cancelling->failed.

Check for cancelled/cancelling and don't change the state.

Red Hat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1608649

Fixes: b23c2ade250
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190219195928.12289-1-dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>

thunk: fix of malloc to g_new

Note that since thunking occurs throughout the lifetime of the QEMU
instance, there is no matching 'free' to correct.

Signed-off-by: Aarushi Mehta <mehta.aaru20@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <5310bd5d152fa36c1828a7cbd19fc893739d1609.camel@gmail.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

hostmem-file: simplify ifdef-s in file_backend_memory_alloc()

cleanup file_backend_memory_alloc() by using one CONFIG_POSIX ifdef
instead of several ones within the function to make it simpler to follow.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Suggested-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <20190213123858.24620-1-imammedo@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20190214031004.32522-2-stefanha@redhat.com>
[lv: s/hostmem/hostmem-file/]
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

build: Correct explanation of unnest-vars example

Cc: Fam Zheng <fam@euphon.net>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190213130240.15492-1-armbru@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

bswap: Fix accessors syntax in comment

All accessors that have an endian infix DO have an underscore between
{size} and {endian}.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <155119086741.1037569.12734854713022304642.stgit@bahia.lan>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

doc: fix typos for documents in tree

Signed-off-by: Like Xu <like.xu@linux.intel.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1550640446-18788-1-git-send-email-like.xu@linux.intel.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

block/pflash_cfi02: Fix memory leak and potential use-after-free

Don't dynamically allocate the pflash's timer. But do use timer_del in
an unrealize function to make sure that the timer can't fire after the
pflash_t has been freed.

Signed-off-by: Stephen Checkoway <stephen.checkoway@oberlin.edu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <20190219153727.62279-1-stephen.checkoway@oberlin.edu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

hw/acpi: remove unnecessary variable acpi_table_builtin

acpi_table_builtin is now always false, it is not necessary to check it
again.

This patch just removes it.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190214084939.20640-4-richardw.yang@linux.intel.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

hw/acpi: remove unused function acpi_table_add_builtin()

Function acpi_table_add_builtin() is not used anymore.

Remove the definition and declaration.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190214084939.20640-3-richardw.yang@linux.intel.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

hw/i386/pc.c: remove unused function pc_acpi_init()

Function pc_acpi_init() is not used anymore.

Remove the definition and declaration.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190214084939.20640-2-richardw.yang@linux.intel.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

tests: Remove (mostly) useless architecture checks

These checks at the beginning of some of the tests are mostly useless:
We only run the tests on x86 anyway, and g_test_message() does not
print anything unless you call g_test_init() first.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Acked-by: Corey Minyard <cminyard@mvista.com>
Message-Id: <1551456970-463-1-git-send-email-thuth@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-mar-05-2019' into staging

MIPS queue for March 5th, 2019

# gpg: Signature made Tue 05 Mar 2019 16:06:34 GMT
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-mar-05-2019:
  target/mips: Add tests for integer add MSA instruction group
  tests/tcg: target/mips: Add tests for MSA pack instructions
  tests/tcg: target/mips: Add tests for MIPS64R6 int multiply instructions
  tests/tcg: target/mips: Add tests for MIPS64R6 shift instructions
  tests/tcg: target/mips: Add tests for MIPS64R6 bit count instructions
  tests/tcg: target/mips: Add tests for MIPS64R6 bit swap instructions
  tests/tcg: target/mips: Add tests for MIPS64R6 logic instructions
  tests/tcg: target/mips: Add wrappers for some MIPS64R6 instructions
  tests/tcg: target/mips: Extend functionality of MSA wrapper macros
  tests/tcg: target/mips: Fix test utilities for 128-bit tests
  tests/tcg: target/mips: Add test utilities for 64-bit tests
  tests/tcg: target/mips: Add test utilities for 32-bit tests
  tests/tcg: target/mips: Add wrappers for various MSA instructions
  disas: nanoMIPS: Add graphical description of pool organization
  disas: nanoMIPS: Correct comments to handlers of some DSP instructions

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-03-05' into staging

QAPI patches for 2019-03-05

# gpg: Signature made Tue 05 Mar 2019 16:47:17 GMT
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-03-05:
  qapi: Fix array first used in a different module
  tests/qapi-schema: Cover forward reference to sub-module
  tests: Rename UserDefNativeListUnion to UserDefListUnion
  qapi: Fix code generation for sub-modules in other directories
  qapi: Pass file name to QAPIGen constructor instead of methods
  tests/qapi-schema: Cover conditional arrays
  tests/qapi-schema: Make test-qapi.py print arrays

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190305' into staging

target-arm queue:
 * Fix PC test for LDM (exception return)
 * Implement ARMv8.0-SB
 * Implement ARMv8.0-PredInv
 * Implement ARMv8.4-CondM
 * Implement ARMv8.5-CondM
 * Implement ARMv8.5-FRINT
 * hw/arm/stellaris: Implement watchdog timer
 * virt: support more than 255GB of RAM

# gpg: Signature made Tue 05 Mar 2019 16:49:47 GMT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20190305: (22 commits)
  hw/arm/stellaris: Implement watchdog timer
  hw/arm/virt: Bump the 255GB initial RAM limit
  hw/arm/virt: Check the VCPU PA range in TCG mode
  hw/arm/virt: Implement kvm_type function for 4.0 machine
  hw/arm/virt: Dynamic memory map depending on RAM requirements
  vl: Set machine ram_size, maxram_size and ram_slots earlier
  kvm: add kvm_arm_get_max_vm_ipa_size
  hw/boards: Add a MachineState parameter to kvm_type callback
  hw/arm/virt: Split the memory map description
  hw/arm/virt: Rename highmem IO regions
  hw/arm/boot: introduce fdt_add_memory_node helper
  target/arm: Implement ARMv8.5-FRINT
  target/arm: Restructure handle_fp_1src_{single, double}
  target/arm: Implement ARMv8.5-CondM
  target/arm: Implement ARMv8.4-CondM
  target/arm: Rearrange disas_data_proc_reg
  target/arm: Add set/clear_pstate_bits, share gen_ss_advance
  target/arm: Split helper_msr_i_pstate into 3
  target/arm: Implement ARMv8.0-PredInv
  target/arm: Implement ARMv8.0-SB
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/mips: Add tests for integer add MSA instruction group

These are the regression tests for integer addition MSA instruction
- various flavors of instruction add (ADD, ADDS, HADD,...).

Signed-off-by: Mateja Marjanovic <mateja.marjanovic@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1551718283-4487-3-git-send-email-mateja.marjanovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MSA pack instructions

Add tests for MSA pack instructions. This includes following
instructions:

  * PCKEV.B - pack even (bytes)
  * PCKEV.H - pack even (halfwords)
  * PCKEV.W - pack even (words)
  * PCKEV.D - pack even (doublewords)
  * PCKOD.B - pack odd (bytes)
  * PCKOD.H - pack odd (halfwords)
  * PCKOD.W - pack odd (words)
  * PCKOD.D - pack odd (doublewords)
  * VSHF.B - data preserving shuffle (bytes)
  * VSHF.H - data preserving shuffle (halfwords)
  * VSHF.W - data preserving shuffle (words)
  * VSHF.D - data preserving shuffle (doublewords)

Each test consists of 80 test cases, so altogether there are 960
test cases.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-15-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MIPS64R6 int multiply instructions

Add tests for MIPS64R6 integer multiply instructions: MUL, MUH, MULU,
MUHU, DMUL, DMUH, DMULU, and DMUHU.

MUH and MUHU require 64 bit inputs in the form of 64-bit sign-extended
32-bit inputs.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-14-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MIPS64R6 shift instructions

Add tests for MIPS64R6 shift instructions: SLLV, SRLV, SRAV, DSLLV,
DSRLV, and DSRAV.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-13-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MIPS64R6 bit count instructions

Add tests for MIPS64R6 bit count instructions: CLO, CLZ, DCLO, and DCLZ.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-12-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MIPS64R6 bit swap instructions

Add tests for MIPS64R6 bit swap instructions: BITSWAP and DBITSWAP.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-11-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add tests for MIPS64R6 logic instructions

Add tests for MIPS64R6 logic instructions: AND, NOR, OR, and XOR.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-10-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add wrappers for some MIPS64R6 instructions

Add wrappers for some MIPS64R6 instructions.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-9-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Extend functionality of MSA wrapper macros

Add macros that will allow testing cases when one of the source
registers is identical to the destination register.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-8-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Fix test utilities for 128-bit tests

Add "static" and "const" modifiers where appropriate, and fix other
minor issues.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-7-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add test utilities for 64-bit tests

Add test utilities for 64-bit tests. Some of MIPS64R6 instructions
require 64-bit inputs to be 32-bit integers sign-extedned to 64 bits,
hence the need for sets of such inputs.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-6-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add test utilities for 32-bit tests

Add test utilities for 32-bit tests.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-5-git-send-email-aleksandar.markovic@rt-rk.com>

tests/tcg: target/mips: Add wrappers for various MSA instructions

Add wrappers for various MSA integer instructions.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-4-git-send-email-aleksandar.markovic@rt-rk.com>

disas: nanoMIPS: Add graphical description of pool organization

Add graphical description of nanoMIPS instruction pool organization.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-3-git-send-email-aleksandar.markovic@rt-rk.com>

disas: nanoMIPS: Correct comments to handlers of some DSP instructions

Correct comments to handlers of some DSP instructions.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <arikalo@wavecomp.com>
Message-Id: <1551800076-8104-2-git-send-email-aleksandar.markovic@rt-rk.com>

hw/arm/stellaris: Implement watchdog timer

Implement the watchdog timer for the stellaris boards.
This device is a close variant of the CMSDK APB watchdog
device, so we can model it by subclassing that device and
tweaking the behaviour of some of its registers.

Signed-off-by: Michel Heily <michelheily@gmail.com>
Reviewed-by: Peter Maydell <petser.maydell@linaro.org>
[PMM: rewrote commit message, fixed a few checkpatch nits,
 added comment giving the URL of the spec for the Stellaris
 variant of the watchdog device]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Bump the 255GB initial RAM limit

Now we have the extended memory map (high IO regions beyond the
scalable RAM) and dynamic IPA range support at KVM/ARM level
we can bump the legacy 255GB initial RAM limit. The actual maximum
RAM size now depends on the physical CPU and host kernel, in
accelerated mode. In TCG mode, it depends on the VCPU
AA64MMFR0.PARANGE.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-11-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Check the VCPU PA range in TCG mode

We are about to allow the memory map to grow beyond 1TB and
potentially overshoot the VCPU AA64MMFR0.PARANGE.

In aarch64 mode and when highmem is set, let's check the VCPU
PA range is sufficient to address the highest GPA of the memory
map.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-10-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Implement kvm_type function for 4.0 machine

This patch implements the machine class kvm_type() callback.
It returns the number of bits requested to implement the whole GPA
range including the RAM and IO regions located beyond.
The returned value is passed though the KVM_CREATE_VM ioctl and
this allows KVM to set the stage2 tables dynamically.

To compute the highest GPA used in the memory map, kvm_type()
must freeze the memory map by calling virt_set_memmap().

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-9-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Dynamic memory map depending on RAM requirements

Up to now the memory map has been static and the high IO region
base has always been 256GiB.

This patch modifies the virt_set_memmap() function, which freezes
the memory map, so that the high IO range base becomes floating,
located after the initial RAM and the device memory.

The function computes
- the base of the device memory,
- the size of the device memory,
- the high IO region base
- the highest GPA used in the memory map.

Entries of the high IO region are assigned a base address. The
device memory is initialized.

The highest GPA used in the memory map will be used at VM creation
to choose the requested IPA size.

Setting all the existing highmem IO regions beyond the RAM
allows to have a single contiguous RAM region (initial RAM and
possible hotpluggable device memory). That way we do not need
to do invasive changes in the EDK2 FW to support a dynamic
RAM base.

Still the user cannot request an initial RAM size greater than 255GB.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-8-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

vl: Set machine ram_size, maxram_size and ram_slots earlier

The machine RAM attributes will need to be analyzed during the
configure_accelerator() process. especially kvm_type() arm64
machine callback will use them to know how many IPA/GPA bits are
needed to model the whole RAM range. So let's assign those machine
state fields before calling configure_accelerator.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-7-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

kvm: add kvm_arm_get_max_vm_ipa_size

Add the kvm_arm_get_max_vm_ipa_size() helper that returns the
number of bits in the IPA address space supported by KVM.

This capability needs to be known to create the VM with a
specific IPA max size (kvm_type passed along KVM_CREATE_VM ioctl.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-6-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/boards: Add a MachineState parameter to kvm_type callback

On ARM, the kvm_type will be resolved by querying the KVMState.
Let's add the MachineState handle to the callback so that we
can retrieve the  KVMState handle. in kvm_init, when the callback
is called, the kvm_state variable is not yet set.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-5-eric.auger@redhat.com
[ppc parts]
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Split the memory map description

In the prospect to introduce an extended memory map supporting more
RAM, let's split the memory map array into two parts:

- the former a15memmap, renamed base_memmap, contains regions below
  and including the RAM. MemMapEntries initialized in this array
  have a static size and base address.
- extended_memmap, only initialized with entries located after the
  RAM. MemMapEntries initialized in this array only get their size
  initialized. Their base address is dynamically computed depending
  on the the top of the RAM, with same alignment as their size.

Eventually base_memmap entries are copied into the extended_memmap
array. Using two separate arrays however clarifies which entries
are statically allocated and those which are dynamically allocated.

This new split will allow to grow the RAM size without changing the
description of the high IO entries.

We introduce a new virt_set_memmap() helper function which
"freezes" the memory map. We call it in machvirt_init as
memory attributes of the machine are not yet set when
virt_instance_init() gets called.

The memory map is unchanged (the top of the initial RAM still is
256GiB). Then come the high IO regions with same layout as before.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-4-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/virt: Rename highmem IO regions

In preparation for a split of the memory map into a static
part and a dynamic part floating after the RAM, let's rename the
regions located after the RAM

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-id: 20190304101339.25970-3-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

hw/arm/boot: introduce fdt_add_memory_node helper

We introduce an helper to create a memory node.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190304101339.25970-2-eric.auger@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Implement ARMv8.5-FRINT

Tested-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-11-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Restructure handle_fp_1src_{single, double}

This will allow sharing code that adjusts rmode beyond
the existing users.

Tested-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-10-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Implement ARMv8.5-CondM

Tested-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-9-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Implement ARMv8.4-CondM

Tested-by: Laurent Desnogues <laurent.desnogues@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-8-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
[PMM: fixed up block comment style]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Rearrange disas_data_proc_reg

This decoding more closely matches the ARMv8.4 Table C4-6,
Encoding table for Data Processing - Register Group.

In particular, op2 == 0 is now more than just Add/sub (with carry).

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-7-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Add set/clear_pstate_bits, share gen_ss_advance

We do not need an out-of-line helper for manipulating bits in pstate.
While changing things, share the implementation of gen_ss_advance.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-6-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Split helper_msr_i_pstate into 3

The EL0+UMA check is unique to DAIF.  While SPSel had avoided the
check by nature of already checking EL >= 1, the other post v8.0
extensions to MSR (imm) allow EL0 and do not require UMA.  Avoid
the unconditional write to pc and use raise_exception_ra to unwind.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-5-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Implement ARMv8.0-PredInv

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-4-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Implement ARMv8.0-SB

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-3-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Split out arm_sctlr

Minimize the number of places that will need updating when
the virtual host extensions are added.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190301200501.16533-2-richard.henderson@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

target/arm: Fix PC test for LDM (exception return)

Found by inspection: Rn is the base register against which the
load began; I is the register within the mask being processed.
The exception return should of course be processed from the loaded PC.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20190301202921.21209-1-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

qapi: Fix array first used in a different module

We generally put implicitly defined types in whatever module triggered
their definition.  This is wrong for array types, as the included test
case demonstrates.  Let's have a closer look at it.

Type 'Status' is defined sub-sub-module.json.  Array type ['Status']
occurs in main module qapi-schema-test.json and in
include/sub-module.json.  The main module's use is first, so the array
type gets put into the main module.

The generated C headers define StatusList in qapi-types.h.  But
include/qapi-types-sub-module.h uses it without including
qapi-types.h.  Oops.

To fix that, put the array type into its element type's module.

Now StatusList gets generated into qapi-types-sub-module.h, which all
its users include.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

tests/qapi-schema: Cover forward reference to sub-module

The forward reference from the main module to the sub-module works
fine, except for an issue visible in qapi-schema-test.out: the array
type wrapped around the forward reference ends up in the main module,
not the sub-module.  The next commit will explain why that's bad, and
fix it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

tests: Rename UserDefNativeListUnion to UserDefListUnion

The lists in UserDefNativeListUnion aren't "native", they're lists of
built-in types.  The next commit will add a list of a user-defined
type.  Drop "Native", and adjust the tests using the type.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

qapi: Fix code generation for sub-modules in other directories

The #include directives to pull in sub-modules use file names relative
to the main module.  Works only when all modules are in the same
directory, or the main module's output directory is in the compiler's
include path.  Use relative file names instead.

The dummy variable we generate to avoid empty .o files has an invalid
name for sub-modules in other directories.  Fix that.

Both messed up in commit 252dc3105fc "qapi: Generate separate .h, .c
for each module".  Escaped testing because tests/qapi-schema-test.json
doesn't cover sub-modules in other directories, only
tests/qapi-schema/include-relpath.json does, and we generate and
compile C code only for the former, not the latter.  Fold the latter
into the former.  This would have caught the mistakes fixed in this
commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

qapi: Pass file name to QAPIGen constructor instead of methods

Not much of an improvement now, but the next commit will profit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-4-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

tests/qapi-schema: Cover conditional arrays

Commit 967c885108f neglected to cover arrays of conditional types.  Do
that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

tests/qapi-schema: Make test-qapi.py print arrays

The next few commits mess with array types, and having the changes
exposed in output of test-qapi.py will be useful.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Rationale added to commit message]

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Tue 05 Mar 2019 07:06:28 GMT
# gpg:                using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
  tests: Add a test for qemu self announcements
  hmp: Add hmp_announce_self
  qmp: Add announce-self command
  virtio-net: Allow qemu_announce_self to trigger virtio announcements
  net: Add a network device specific self-announcement ability
  migration: Switch to using announce timer
  virtio-net: Switch to using announce timer
  migration: Add announce parameters
  net: Introduce announce timer
  net: netmap: improve netmap_receive_iov()
  net: netmap: simplify netmap_receive()
  net: netmap: small improvements netmap_send()
  net/colo-compare.c: Remove duplicated code

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

tests: Add a test for qemu self announcements

We now expose qemu_announce_self through QMP and HMP.  Add a test
with some very basic packet validation (make sure we get a RARP).

Signed-off-by: Vlad Yasevich <vyasevic@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>

hmp: Add hmp_announce_self

Add an HMP command to trigger self annocements.
Unlike the QMP command (which takes a set of parameters), the HMP
command reuses the set of parameters used for migration.

Signend-off-by: Vladislav Yasevich <vyasevic@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>

qmp: Add announce-self command

Add a qmp command that can trigger guest announcements.

It uses its own announce-timer instance, and parameters
passed to it explicitly in the command.

Like most qmp commands, it's in the main thread/bql, so
there's no racing with any outstanding timer.

Based on work of Germano Veit Michel <germano@redhat.com> and
                 Vladislav Yasevich <vyasevic@redhat.com>

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>

virtio-net: Allow qemu_announce_self to trigger virtio announcements

Expose the virtio-net self announcement capability and allow
qemu_announce_self() to call it.

These announces are caused by something external (i.e. the
announce-self command); they won't trigger if the migration
counter is triggering announces at the same time.

Signed-off-by: Vladislav Yasevich <vyasevic@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>

net: Add a network device specific self-announcement ability

Some network devices have a capability to do self announcements
(ex: virtio-net).  Add infrastructure that would allow devices
to expose this ability.

Signed-off-by: Vladislav Yasevich <vyasevic@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>

migration: Switch to using announce timer

Switch the announcements to using the new announce timer.
Move the code that does it to announce.c rather than savevm
because it really has nothing to do with the actual migration.

Migration starts the announce from bh's and so they're all
in the main thread/bql, and so there's never any racing with
the timers themselves.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>