From: Leonardo Garcia Date: Tue, 18 Jan 2022 11:56:30 +0000 (+0100) Subject: docs: Rename ppc-spapr-hotplug.txt to ppc-spapr-hotplug.rst. X-Git-Url: http://git.maquefel.me/?a=commitdiff_plain;h=55ff468f7816ff40e4058153127c9d19ffd36261;p=qemu.git docs: Rename ppc-spapr-hotplug.txt to ppc-spapr-hotplug.rst. Signed-off-by: Leonardo Garcia Reviewed-by: Daniel Henrique Barboza Message-Id: <1f5860217273f272fddadc68b5d205b4090f6b04.1641995058.git.lagarcia@br.ibm.com> Signed-off-by: Cédric Le Goater --- diff --git a/docs/specs/ppc-spapr-hotplug.rst b/docs/specs/ppc-spapr-hotplug.rst new file mode 100644 index 0000000000..f84dc55ad9 --- /dev/null +++ b/docs/specs/ppc-spapr-hotplug.rst @@ -0,0 +1,510 @@ +============================= +sPAPR Dynamic Reconfiguration +============================= + +sPAPR or pSeries guests make use of a facility called dynamic reconfiguration +to handle hot plugging of dynamic "physical" resources like PCI cards, or +"logical"/para-virtual resources like memory, CPUs, and "physical" +host-bridges, which are generally managed by the host/hypervisor and provided +to guests as virtualized resources. The specifics of dynamic reconfiguration +are documented extensively in section 13 of the Linux on Power Architecture +Reference document ([LoPAR]_). This document provides a summary of that +information as it applies to the implementation within QEMU. + +Dynamic-reconfiguration Connectors +================================== + +To manage hot plug/unplug of these resources, a firmware abstraction known as +a Dynamic Resource Connector (DRC) is used to assign a particular dynamic +resource to the guest, and provide an interface for the guest to manage +configuration/removal of the resource associated with it. + +Device tree description of DRCs +=============================== + +A set of four Open Firmware device tree array properties are used to describe +the name/index/power-domain/type of each DRC allocated to a guest at +boot time. There may be multiple sets of these arrays, rooted at different +paths in the device tree depending on the type of resource the DRCs manage. + +In some cases, the DRCs themselves may be provided by a dynamic resource, +such as the DRCs managing PCI slots on a hot plugged PHB. In this case the +arrays would be fetched as part of the device tree retrieval interfaces +for hot plugged resources described under :ref:`guest-host-interface`. + +The array properties are described below. Each entry/element in an array +describes the DRC identified by the element in the corresponding position +of ``ibm,drc-indexes``: + +``ibm,drc-names`` +----------------- + + First 4-bytes: big-endian (BE) encoded integer denoting the number of entries. + + Each entry: a NULL-terminated ```` string encoded as a byte array. + + ```` values for logical/virtual resources are defined in the Linux on + Power Architecture Reference ([LoPAR]_) section 13.5.2.4, and basically + consist of the type of the resource followed by a space and a numerical + value that's unique across resources of that type. + + ```` values for "physical" resources such as PCI or VIO devices are + defined as being "location codes", which are the "location labels" of each + encapsulating device, starting from the chassis down to the individual slot + for the device, concatenated by a hyphen. This provides a mapping of + resources to a physical location in a chassis for debugging purposes. For + QEMU, this mapping is less important, so we assign a location code that + conforms to naming specifications, but is simply a location label for the + slot by itself to simplify the implementation. The naming convention for + location labels is documented in detail in the [LoPAR]_ section 12.3.1.5, + and in our case amounts to using ``C`` for PCI/VIO device slots, where + ```` is unique across all PCI/VIO device slots. + +``ibm,drc-indexes`` +------------------- + + First 4-bytes: BE-encoded integer denoting the number of entries. + + Each 4-byte entry: BE-encoded ```` integer that is unique across all + DRCs in the machine. + + ```` is arbitrary, but in the case of QEMU we try to maintain the + convention used to assign them to pSeries guests on pHyp (the hypervisor + portion of PowerVM): + + ``bit[31:28]``: integer encoding of ````, where ```` is: + + ``1`` for CPU resource. + + ``2`` for PHB resource. + + ``3`` for VIO resource. + + ``4`` for PCI resource. + + ``8`` for memory resource. + + ``bit[27:0]``: integer encoding of ````, where ```` is unique + across all resources of specified type. + +``ibm,drc-power-domains`` +------------------------- + + First 4-bytes: BE-encoded integer denoting the number of entries. + + Each 4-byte entry: 32-bit, BE-encoded ```` integer that specifies the + power domain the resource will be assigned to. In the case of QEMU we + associated all resources with a "live insertion" domain, where the power is + assumed to be managed automatically. The integer value for this domain is a + special value of ``-1``. + + +``ibm,drc-types`` +----------------- + + First 4-bytes: BE-encoded integer denoting the number of entries. + + Each entry: a NULL-terminated ```` string encoded as a byte array. + ```` is assigned as follows: + + "CPU" for a CPU. + + "PHB" for a physical host-bridge. + + "SLOT" for a VIO slot. + + "28" for a PCI slot. + + "MEM" for memory resource. + +.. _guest-host-interface: + +Guest->Host interface to manage dynamic resources +================================================= + +Each DRC is given a globally unique DRC index, and resources associated with a +particular DRC are configured/managed by the guest via a number of RTAS calls +which reference individual DRCs based on the DRC index. This can be considered +the guest->host interface. + +``rtas-set-power-level`` +------------------------ + +Set the power level for a specified power domain. + + ``arg[0]``: integer identifying power domain. + + ``arg[1]``: new power level for the domain, ``0-100``. + + ``output[0]``: status, ``0`` on success. + + ``output[1]``: power level after command. + +``rtas-get-power-level`` +------------------------ + +Get the power level for a specified power domain. + + ``arg[0]``: integer identifying power domain. + + ``output[0]``: status, ``0`` on success. + + ``output[1]``: current power level. + +``rtas-set-indicator`` +---------------------- + +Set the state of an indicator or sensor. + + ``arg[0]``: integer identifying sensor/indicator type. + + ``arg[1]``: index of sensor, for DR-related sensors this is generally the DRC + index. + + ``arg[2]``: desired sensor value. + + ``output[0]``: status, ``0`` on success. + +For the purpose of this document we focus on the indicator/sensor types +associated with a DRC. The types are: + +* ``9001``: ``isolation-state``, controls/indicates whether a device has been + made accessible to a guest. Supported sensor values: + + ``0``: ``isolate``, device is made inaccessible by guest OS. + + ``1``: ``unisolate``, device is made available to guest OS. + +* ``9002``: ``dr-indicator``, controls "visual" indicator associated with + device. Supported sensor values: + + ``0``: ``inactive``, resource may be safely removed. + + ``1``: ``active``, resource is in use and cannot be safely removed. + + ``2``: ``identify``, used to visually identify slot for interactive hot plug. + + ``3``: ``action``, in most cases, used in the same manner as identify. + +* ``9003``: ``allocation-state``, generally only used for "logical" DR resources + to request the allocation/deallocation of a resource prior to acquiring it via + ``isolation-state->unisolate``, or after releasing it via + ``isolation-state->isolate``, respectively. For "physical" DR (like PCI + hot plug/unplug) the pre-allocation of the resource is implied and this sensor + is unused. Supported sensor values: + + ``0``: ``unusable``, tell firmware/system the resource can be + unallocated/reclaimed and added back to the system resource pool. + + ``1``: ``usable``, request the resource be allocated/reserved for use by + guest OS. + + ``2``: ``exchange``, used to allocate a spare resource to use for fail-over + in certain situations. Unused in QEMU. + + ``3``: ``recover``, used to reclaim a previously allocated resource that's + not currently allocated to the guest OS. Unused in QEMU. + +``rtas-get-sensor-state:`` +-------------------------- + +Used to read an indicator or sensor value. + + ``arg[0]``: integer identifying sensor/indicator type. + + ``arg[1]``: index of sensor, for DR-related sensors this is generally the DRC + index + + ``output[0]``: status, 0 on success + +For DR-related operations, the only noteworthy sensor is ``dr-entity-sense``, +which has a type value of ``9003``, as ``allocation-state`` does in the case of +``rtas-set-indicator``. The semantics/encodings of the sensor values are +distinct however. + +Supported sensor values for ``dr-entity-sense`` (``9003``) sensor: + + ``0``: empty. + + For physical resources: DRC/slot is empty. + + For logical resources: unused. + + ``1``: present. + + For physical resources: DRC/slot is populated with a device/resource. + + For logical resources: resource has been allocated to the DRC. + + ``2``: unusable. + + For physical resources: unused. + + For logical resources: DRC has no resource allocated to it. + + ``3``: exchange. + + For physical resources: unused. + + For logical resources: resource available for exchange (see + ``allocation-state`` sensor semantics above). + + ``4``: recovery. + + For physical resources: unused. + + For logical resources: resource available for recovery (see + ``allocation-state`` sensor semantics above). + +``rtas-ibm-configure-connector`` +-------------------------------- + +Used to fetch an OpenFirmware device tree description of the resource associated +with a particular DRC. + + ``arg[0]``: guest physical address of 4096-byte work area buffer. + + ``arg[1]``: 0, or address of additional 4096-byte work area buffer; only + non-zero if a prior RTAS response indicated a need for additional memory. + + ``output[0]``: status: + + ``0``: completed transmittal of device tree node. + + ``1``: instruct guest to prepare for next device tree sibling node. + + ``2``: instruct guest to prepare for next device tree child node. + + ``3``: instruct guest to prepare for next device tree property. + + ``4``: instruct guest to ascend to parent device tree node. + + ``5``: instruct guest to provide additional work-area buffer via ``arg[1]``. + + ``990x``: instruct guest that operation took too long and to try again + later. + +The DRC index is encoded in the first 4-bytes of the first work area buffer. +Work area (``wa``) layout, using 4-byte offsets: + + ``wa[0]``: DRC index of the DRC to fetch device tree nodes from. + + ``wa[1]``: ``0`` (hard-coded). + + ``wa[2]``: + + For next-sibling/next-child response: + + ``wa`` offset of null-terminated string denoting the new node's name. + + For next-property response: + + ``wa`` offset of null-terminated string denoting new property's name. + + ``wa[3]``: for next-property response (unused otherwise): + + Byte-length of new property's value. + + ``wa[4]``: for next-property response (unused otherwise): + + New property's value, encoded as an OFDT-compatible byte array. + +Hot plug/unplug events +====================== + +For most DR operations, the hypervisor will issue host->guest add/remove events +using the EPOW/check-exception notification framework, where the host issues a +check-exception interrupt, then provides an RTAS event log via an +rtas-check-exception call issued by the guest in response. This framework is +documented by PAPR+ v2.7, and already use in by QEMU for generating powerdown +requests via EPOW events. + +For DR, this framework has been extended to include hotplug events, which were +previously unneeded due to direct manipulation of DR-related guest userspace +tools by host-level management such as an HMC. This level of management is not +applicable to KVM on Power, hence the reason for extending the notification +framework to support hotplug events. + +The format for these EPOW-signalled events is described below under +:ref:`hot-plug-unplug-event-structure`. Note that these events are not formally +part of the PAPR+ specification, and have been superseded by a newer format, +also described below under :ref:`hot-plug-unplug-event-structure`, and so are +now deemed a "legacy" format. The formats are similar, but the "modern" format +contains additional fields/flags, which are denoted for the purposes of this +documentation with ``#ifdef GUEST_SUPPORTS_MODERN`` guards. + +QEMU should assume support only for "legacy" fields/flags unless the guest +advertises support for the "modern" format via +``ibm,client-architecture-support`` hcall by setting byte 5, bit 6 of it's +``ibm,architecture-vec-5`` option vector structure (as described by [LoPAR]_, +section B.5.2.3). As with "legacy" format events, "modern" format events are +surfaced to the guest via check-exception RTAS calls, but use a dedicated event +source to signal the guest. This event source is advertised to the guest by the +addition of a ``hot-plug-events`` node under ``/event-sources`` node of the +guest's device tree using the standard format described in [LoPAR]_, +section B.5.12.2. + +.. _hot-plug-unplug-event-structure: + +Hot plug/unplug event structure +=============================== + +The hot plug specific payload in QEMU is implemented as follows (with all values +encoded in big-endian format): + +.. code-block:: c + + struct rtas_event_log_v6_hp { + #define SECTION_ID_HOTPLUG 0x4850 /* HP */ + struct section_header { + uint16_t section_id; /* set to SECTION_ID_HOTPLUG */ + uint16_t section_length; /* sizeof(rtas_event_log_v6_hp), + * plus the length of the DRC name + * if a DRC name identifier is + * specified for hotplug_identifier + */ + uint8_t section_version; /* version 1 */ + uint8_t section_subtype; /* unused */ + uint16_t creator_component_id; /* unused */ + } hdr; + #define RTAS_LOG_V6_HP_TYPE_CPU 1 + #define RTAS_LOG_V6_HP_TYPE_MEMORY 2 + #define RTAS_LOG_V6_HP_TYPE_SLOT 3 + #define RTAS_LOG_V6_HP_TYPE_PHB 4 + #define RTAS_LOG_V6_HP_TYPE_PCI 5 + uint8_t hotplug_type; /* type of resource/device */ + #define RTAS_LOG_V6_HP_ACTION_ADD 1 + #define RTAS_LOG_V6_HP_ACTION_REMOVE 2 + uint8_t hotplug_action; /* action (add/remove) */ + #define RTAS_LOG_V6_HP_ID_DRC_NAME 1 + #define RTAS_LOG_V6_HP_ID_DRC_INDEX 2 + #define RTAS_LOG_V6_HP_ID_DRC_COUNT 3 + #ifdef GUEST_SUPPORTS_MODERN + #define RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED 4 + #endif + uint8_t hotplug_identifier; /* type of the resource identifier, + * which serves as the discriminator + * for the 'drc' union field below + */ + #ifdef GUEST_SUPPORTS_MODERN + uint8_t capabilities; /* capability flags, currently unused + * by QEMU + */ + #else + uint8_t reserved; + #endif + union { + uint32_t index; /* DRC index of resource to take action + * on + */ + uint32_t count; /* number of DR resources to take + * action on (guest chooses which) + */ + #ifdef GUEST_SUPPORTS_MODERN + struct { + uint32_t count; /* number of DR resources to take + * action on + */ + uint32_t index; /* DRC index of first resource to take + * action on. guest will take action + * on DRC index through + * DRC index in + * sequential order + */ + } count_indexed; + #endif + char name[1]; /* string representing the name of the + * DRC to take action on + */ + } drc; + } QEMU_PACKED; + +``ibm,lrdr-capacity`` +===================== + +``ibm,lrdr-capacity`` is a property in the /rtas device tree node that +identifies the dynamic reconfiguration capabilities of the guest. It consists +of a triple consisting of ````, ```` and ````. + + ````, encoded in BE format represents the maximum address in bytes and + hence the maximum memory that can be allocated to the guest. + + ````, encoded in BE format represents the size increments in which + memory can be hot-plugged to the guest. + + ````, a BE-encoded integer, represents the maximum number of + processors that the guest can have. + +``pseries`` guests use this property to note the maximum allowed CPUs for the +guest. + +``ibm,dynamic-reconfiguration-memory`` +====================================== + +``ibm,dynamic-reconfiguration-memory`` is a device tree node that represents +dynamically reconfigurable logical memory blocks (LMB). This node is generated +only when the guest advertises the support for it via +``ibm,client-architecture-support`` call. Memory that is not dynamically +reconfigurable is represented by ``/memory`` nodes. The properties of this node +that are of interest to the sPAPR memory hotplug implementation in QEMU are +described here. + +``ibm,lmb-size`` +---------------- + +This 64-bit integer defines the size of each dynamically reconfigurable LMB. + +``ibm,associativity-lookup-arrays`` +----------------------------------- + +This property defines a lookup array in which the NUMA associativity +information for each LMB can be found. It is a property encoded array +that begins with an integer M, the number of associativity lists followed +by an integer N, the number of entries per associativity list and terminated +by M associativity lists each of length N integers. + +This property provides the same information as given by ``ibm,associativity`` +property in a ``/memory`` node. Each assigned LMB has an index value between +0 and M-1 which is used as an index into this table to select which +associativity list to use for the LMB. This index value for each LMB is defined +in ``ibm,dynamic-memory`` property. + +``ibm,dynamic-memory`` +---------------------- + +This property describes the dynamically reconfigurable memory. It is a +property encoded array that has an integer N, the number of LMBs followed +by N LMB list entries. + +Each LMB list entry consists of the following elements: + +- Logical address of the start of the LMB encoded as a 64-bit integer. This + corresponds to ``reg`` property in ``/memory`` node. +- DRC index of the LMB that corresponds to ``ibm,my-drc-index`` property + in a ``/memory`` node. +- Four bytes reserved for expansion. +- Associativity list index for the LMB that is used as an index into + ``ibm,associativity-lookup-arrays`` property described earlier. This is used + to retrieve the right associativity list to be used for this LMB. +- A 32-bit flags word. The bit at bit position ``0x00000008`` defines whether + the LMB is assigned to the partition as of boot time. + +``ibm,dynamic-memory-v2`` +------------------------- + +This property describes the dynamically reconfigurable memory. This is +an alternate and newer way to describe dynamically reconfigurable memory. +It is a property encoded array that has an integer N (the number of +LMB set entries) followed by N LMB set entries. There is an LMB set entry +for each sequential group of LMBs that share common attributes. + +Each LMB set entry consists of the following elements: + +- Number of sequential LMBs in the entry represented by a 32-bit integer. +- Logical address of the first LMB in the set encoded as a 64-bit integer. +- DRC index of the first LMB in the set. +- Associativity list index that is used as an index into + ``ibm,associativity-lookup-arrays`` property described earlier. This + is used to retrieve the right associativity list to be used for all + the LMBs in this set. +- A 32-bit flags word that applies to all the LMBs in the set. diff --git a/docs/specs/ppc-spapr-hotplug.txt b/docs/specs/ppc-spapr-hotplug.txt deleted file mode 100644 index f84dc55ad9..0000000000 --- a/docs/specs/ppc-spapr-hotplug.txt +++ /dev/null @@ -1,510 +0,0 @@ -============================= -sPAPR Dynamic Reconfiguration -============================= - -sPAPR or pSeries guests make use of a facility called dynamic reconfiguration -to handle hot plugging of dynamic "physical" resources like PCI cards, or -"logical"/para-virtual resources like memory, CPUs, and "physical" -host-bridges, which are generally managed by the host/hypervisor and provided -to guests as virtualized resources. The specifics of dynamic reconfiguration -are documented extensively in section 13 of the Linux on Power Architecture -Reference document ([LoPAR]_). This document provides a summary of that -information as it applies to the implementation within QEMU. - -Dynamic-reconfiguration Connectors -================================== - -To manage hot plug/unplug of these resources, a firmware abstraction known as -a Dynamic Resource Connector (DRC) is used to assign a particular dynamic -resource to the guest, and provide an interface for the guest to manage -configuration/removal of the resource associated with it. - -Device tree description of DRCs -=============================== - -A set of four Open Firmware device tree array properties are used to describe -the name/index/power-domain/type of each DRC allocated to a guest at -boot time. There may be multiple sets of these arrays, rooted at different -paths in the device tree depending on the type of resource the DRCs manage. - -In some cases, the DRCs themselves may be provided by a dynamic resource, -such as the DRCs managing PCI slots on a hot plugged PHB. In this case the -arrays would be fetched as part of the device tree retrieval interfaces -for hot plugged resources described under :ref:`guest-host-interface`. - -The array properties are described below. Each entry/element in an array -describes the DRC identified by the element in the corresponding position -of ``ibm,drc-indexes``: - -``ibm,drc-names`` ------------------ - - First 4-bytes: big-endian (BE) encoded integer denoting the number of entries. - - Each entry: a NULL-terminated ```` string encoded as a byte array. - - ```` values for logical/virtual resources are defined in the Linux on - Power Architecture Reference ([LoPAR]_) section 13.5.2.4, and basically - consist of the type of the resource followed by a space and a numerical - value that's unique across resources of that type. - - ```` values for "physical" resources such as PCI or VIO devices are - defined as being "location codes", which are the "location labels" of each - encapsulating device, starting from the chassis down to the individual slot - for the device, concatenated by a hyphen. This provides a mapping of - resources to a physical location in a chassis for debugging purposes. For - QEMU, this mapping is less important, so we assign a location code that - conforms to naming specifications, but is simply a location label for the - slot by itself to simplify the implementation. The naming convention for - location labels is documented in detail in the [LoPAR]_ section 12.3.1.5, - and in our case amounts to using ``C`` for PCI/VIO device slots, where - ```` is unique across all PCI/VIO device slots. - -``ibm,drc-indexes`` -------------------- - - First 4-bytes: BE-encoded integer denoting the number of entries. - - Each 4-byte entry: BE-encoded ```` integer that is unique across all - DRCs in the machine. - - ```` is arbitrary, but in the case of QEMU we try to maintain the - convention used to assign them to pSeries guests on pHyp (the hypervisor - portion of PowerVM): - - ``bit[31:28]``: integer encoding of ````, where ```` is: - - ``1`` for CPU resource. - - ``2`` for PHB resource. - - ``3`` for VIO resource. - - ``4`` for PCI resource. - - ``8`` for memory resource. - - ``bit[27:0]``: integer encoding of ````, where ```` is unique - across all resources of specified type. - -``ibm,drc-power-domains`` -------------------------- - - First 4-bytes: BE-encoded integer denoting the number of entries. - - Each 4-byte entry: 32-bit, BE-encoded ```` integer that specifies the - power domain the resource will be assigned to. In the case of QEMU we - associated all resources with a "live insertion" domain, where the power is - assumed to be managed automatically. The integer value for this domain is a - special value of ``-1``. - - -``ibm,drc-types`` ------------------ - - First 4-bytes: BE-encoded integer denoting the number of entries. - - Each entry: a NULL-terminated ```` string encoded as a byte array. - ```` is assigned as follows: - - "CPU" for a CPU. - - "PHB" for a physical host-bridge. - - "SLOT" for a VIO slot. - - "28" for a PCI slot. - - "MEM" for memory resource. - -.. _guest-host-interface: - -Guest->Host interface to manage dynamic resources -================================================= - -Each DRC is given a globally unique DRC index, and resources associated with a -particular DRC are configured/managed by the guest via a number of RTAS calls -which reference individual DRCs based on the DRC index. This can be considered -the guest->host interface. - -``rtas-set-power-level`` ------------------------- - -Set the power level for a specified power domain. - - ``arg[0]``: integer identifying power domain. - - ``arg[1]``: new power level for the domain, ``0-100``. - - ``output[0]``: status, ``0`` on success. - - ``output[1]``: power level after command. - -``rtas-get-power-level`` ------------------------- - -Get the power level for a specified power domain. - - ``arg[0]``: integer identifying power domain. - - ``output[0]``: status, ``0`` on success. - - ``output[1]``: current power level. - -``rtas-set-indicator`` ----------------------- - -Set the state of an indicator or sensor. - - ``arg[0]``: integer identifying sensor/indicator type. - - ``arg[1]``: index of sensor, for DR-related sensors this is generally the DRC - index. - - ``arg[2]``: desired sensor value. - - ``output[0]``: status, ``0`` on success. - -For the purpose of this document we focus on the indicator/sensor types -associated with a DRC. The types are: - -* ``9001``: ``isolation-state``, controls/indicates whether a device has been - made accessible to a guest. Supported sensor values: - - ``0``: ``isolate``, device is made inaccessible by guest OS. - - ``1``: ``unisolate``, device is made available to guest OS. - -* ``9002``: ``dr-indicator``, controls "visual" indicator associated with - device. Supported sensor values: - - ``0``: ``inactive``, resource may be safely removed. - - ``1``: ``active``, resource is in use and cannot be safely removed. - - ``2``: ``identify``, used to visually identify slot for interactive hot plug. - - ``3``: ``action``, in most cases, used in the same manner as identify. - -* ``9003``: ``allocation-state``, generally only used for "logical" DR resources - to request the allocation/deallocation of a resource prior to acquiring it via - ``isolation-state->unisolate``, or after releasing it via - ``isolation-state->isolate``, respectively. For "physical" DR (like PCI - hot plug/unplug) the pre-allocation of the resource is implied and this sensor - is unused. Supported sensor values: - - ``0``: ``unusable``, tell firmware/system the resource can be - unallocated/reclaimed and added back to the system resource pool. - - ``1``: ``usable``, request the resource be allocated/reserved for use by - guest OS. - - ``2``: ``exchange``, used to allocate a spare resource to use for fail-over - in certain situations. Unused in QEMU. - - ``3``: ``recover``, used to reclaim a previously allocated resource that's - not currently allocated to the guest OS. Unused in QEMU. - -``rtas-get-sensor-state:`` --------------------------- - -Used to read an indicator or sensor value. - - ``arg[0]``: integer identifying sensor/indicator type. - - ``arg[1]``: index of sensor, for DR-related sensors this is generally the DRC - index - - ``output[0]``: status, 0 on success - -For DR-related operations, the only noteworthy sensor is ``dr-entity-sense``, -which has a type value of ``9003``, as ``allocation-state`` does in the case of -``rtas-set-indicator``. The semantics/encodings of the sensor values are -distinct however. - -Supported sensor values for ``dr-entity-sense`` (``9003``) sensor: - - ``0``: empty. - - For physical resources: DRC/slot is empty. - - For logical resources: unused. - - ``1``: present. - - For physical resources: DRC/slot is populated with a device/resource. - - For logical resources: resource has been allocated to the DRC. - - ``2``: unusable. - - For physical resources: unused. - - For logical resources: DRC has no resource allocated to it. - - ``3``: exchange. - - For physical resources: unused. - - For logical resources: resource available for exchange (see - ``allocation-state`` sensor semantics above). - - ``4``: recovery. - - For physical resources: unused. - - For logical resources: resource available for recovery (see - ``allocation-state`` sensor semantics above). - -``rtas-ibm-configure-connector`` --------------------------------- - -Used to fetch an OpenFirmware device tree description of the resource associated -with a particular DRC. - - ``arg[0]``: guest physical address of 4096-byte work area buffer. - - ``arg[1]``: 0, or address of additional 4096-byte work area buffer; only - non-zero if a prior RTAS response indicated a need for additional memory. - - ``output[0]``: status: - - ``0``: completed transmittal of device tree node. - - ``1``: instruct guest to prepare for next device tree sibling node. - - ``2``: instruct guest to prepare for next device tree child node. - - ``3``: instruct guest to prepare for next device tree property. - - ``4``: instruct guest to ascend to parent device tree node. - - ``5``: instruct guest to provide additional work-area buffer via ``arg[1]``. - - ``990x``: instruct guest that operation took too long and to try again - later. - -The DRC index is encoded in the first 4-bytes of the first work area buffer. -Work area (``wa``) layout, using 4-byte offsets: - - ``wa[0]``: DRC index of the DRC to fetch device tree nodes from. - - ``wa[1]``: ``0`` (hard-coded). - - ``wa[2]``: - - For next-sibling/next-child response: - - ``wa`` offset of null-terminated string denoting the new node's name. - - For next-property response: - - ``wa`` offset of null-terminated string denoting new property's name. - - ``wa[3]``: for next-property response (unused otherwise): - - Byte-length of new property's value. - - ``wa[4]``: for next-property response (unused otherwise): - - New property's value, encoded as an OFDT-compatible byte array. - -Hot plug/unplug events -====================== - -For most DR operations, the hypervisor will issue host->guest add/remove events -using the EPOW/check-exception notification framework, where the host issues a -check-exception interrupt, then provides an RTAS event log via an -rtas-check-exception call issued by the guest in response. This framework is -documented by PAPR+ v2.7, and already use in by QEMU for generating powerdown -requests via EPOW events. - -For DR, this framework has been extended to include hotplug events, which were -previously unneeded due to direct manipulation of DR-related guest userspace -tools by host-level management such as an HMC. This level of management is not -applicable to KVM on Power, hence the reason for extending the notification -framework to support hotplug events. - -The format for these EPOW-signalled events is described below under -:ref:`hot-plug-unplug-event-structure`. Note that these events are not formally -part of the PAPR+ specification, and have been superseded by a newer format, -also described below under :ref:`hot-plug-unplug-event-structure`, and so are -now deemed a "legacy" format. The formats are similar, but the "modern" format -contains additional fields/flags, which are denoted for the purposes of this -documentation with ``#ifdef GUEST_SUPPORTS_MODERN`` guards. - -QEMU should assume support only for "legacy" fields/flags unless the guest -advertises support for the "modern" format via -``ibm,client-architecture-support`` hcall by setting byte 5, bit 6 of it's -``ibm,architecture-vec-5`` option vector structure (as described by [LoPAR]_, -section B.5.2.3). As with "legacy" format events, "modern" format events are -surfaced to the guest via check-exception RTAS calls, but use a dedicated event -source to signal the guest. This event source is advertised to the guest by the -addition of a ``hot-plug-events`` node under ``/event-sources`` node of the -guest's device tree using the standard format described in [LoPAR]_, -section B.5.12.2. - -.. _hot-plug-unplug-event-structure: - -Hot plug/unplug event structure -=============================== - -The hot plug specific payload in QEMU is implemented as follows (with all values -encoded in big-endian format): - -.. code-block:: c - - struct rtas_event_log_v6_hp { - #define SECTION_ID_HOTPLUG 0x4850 /* HP */ - struct section_header { - uint16_t section_id; /* set to SECTION_ID_HOTPLUG */ - uint16_t section_length; /* sizeof(rtas_event_log_v6_hp), - * plus the length of the DRC name - * if a DRC name identifier is - * specified for hotplug_identifier - */ - uint8_t section_version; /* version 1 */ - uint8_t section_subtype; /* unused */ - uint16_t creator_component_id; /* unused */ - } hdr; - #define RTAS_LOG_V6_HP_TYPE_CPU 1 - #define RTAS_LOG_V6_HP_TYPE_MEMORY 2 - #define RTAS_LOG_V6_HP_TYPE_SLOT 3 - #define RTAS_LOG_V6_HP_TYPE_PHB 4 - #define RTAS_LOG_V6_HP_TYPE_PCI 5 - uint8_t hotplug_type; /* type of resource/device */ - #define RTAS_LOG_V6_HP_ACTION_ADD 1 - #define RTAS_LOG_V6_HP_ACTION_REMOVE 2 - uint8_t hotplug_action; /* action (add/remove) */ - #define RTAS_LOG_V6_HP_ID_DRC_NAME 1 - #define RTAS_LOG_V6_HP_ID_DRC_INDEX 2 - #define RTAS_LOG_V6_HP_ID_DRC_COUNT 3 - #ifdef GUEST_SUPPORTS_MODERN - #define RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED 4 - #endif - uint8_t hotplug_identifier; /* type of the resource identifier, - * which serves as the discriminator - * for the 'drc' union field below - */ - #ifdef GUEST_SUPPORTS_MODERN - uint8_t capabilities; /* capability flags, currently unused - * by QEMU - */ - #else - uint8_t reserved; - #endif - union { - uint32_t index; /* DRC index of resource to take action - * on - */ - uint32_t count; /* number of DR resources to take - * action on (guest chooses which) - */ - #ifdef GUEST_SUPPORTS_MODERN - struct { - uint32_t count; /* number of DR resources to take - * action on - */ - uint32_t index; /* DRC index of first resource to take - * action on. guest will take action - * on DRC index through - * DRC index in - * sequential order - */ - } count_indexed; - #endif - char name[1]; /* string representing the name of the - * DRC to take action on - */ - } drc; - } QEMU_PACKED; - -``ibm,lrdr-capacity`` -===================== - -``ibm,lrdr-capacity`` is a property in the /rtas device tree node that -identifies the dynamic reconfiguration capabilities of the guest. It consists -of a triple consisting of ````, ```` and ````. - - ````, encoded in BE format represents the maximum address in bytes and - hence the maximum memory that can be allocated to the guest. - - ````, encoded in BE format represents the size increments in which - memory can be hot-plugged to the guest. - - ````, a BE-encoded integer, represents the maximum number of - processors that the guest can have. - -``pseries`` guests use this property to note the maximum allowed CPUs for the -guest. - -``ibm,dynamic-reconfiguration-memory`` -====================================== - -``ibm,dynamic-reconfiguration-memory`` is a device tree node that represents -dynamically reconfigurable logical memory blocks (LMB). This node is generated -only when the guest advertises the support for it via -``ibm,client-architecture-support`` call. Memory that is not dynamically -reconfigurable is represented by ``/memory`` nodes. The properties of this node -that are of interest to the sPAPR memory hotplug implementation in QEMU are -described here. - -``ibm,lmb-size`` ----------------- - -This 64-bit integer defines the size of each dynamically reconfigurable LMB. - -``ibm,associativity-lookup-arrays`` ------------------------------------ - -This property defines a lookup array in which the NUMA associativity -information for each LMB can be found. It is a property encoded array -that begins with an integer M, the number of associativity lists followed -by an integer N, the number of entries per associativity list and terminated -by M associativity lists each of length N integers. - -This property provides the same information as given by ``ibm,associativity`` -property in a ``/memory`` node. Each assigned LMB has an index value between -0 and M-1 which is used as an index into this table to select which -associativity list to use for the LMB. This index value for each LMB is defined -in ``ibm,dynamic-memory`` property. - -``ibm,dynamic-memory`` ----------------------- - -This property describes the dynamically reconfigurable memory. It is a -property encoded array that has an integer N, the number of LMBs followed -by N LMB list entries. - -Each LMB list entry consists of the following elements: - -- Logical address of the start of the LMB encoded as a 64-bit integer. This - corresponds to ``reg`` property in ``/memory`` node. -- DRC index of the LMB that corresponds to ``ibm,my-drc-index`` property - in a ``/memory`` node. -- Four bytes reserved for expansion. -- Associativity list index for the LMB that is used as an index into - ``ibm,associativity-lookup-arrays`` property described earlier. This is used - to retrieve the right associativity list to be used for this LMB. -- A 32-bit flags word. The bit at bit position ``0x00000008`` defines whether - the LMB is assigned to the partition as of boot time. - -``ibm,dynamic-memory-v2`` -------------------------- - -This property describes the dynamically reconfigurable memory. This is -an alternate and newer way to describe dynamically reconfigurable memory. -It is a property encoded array that has an integer N (the number of -LMB set entries) followed by N LMB set entries. There is an LMB set entry -for each sequential group of LMBs that share common attributes. - -Each LMB set entry consists of the following elements: - -- Number of sequential LMBs in the entry represented by a 32-bit integer. -- Logical address of the first LMB in the set encoded as a 64-bit integer. -- DRC index of the first LMB in the set. -- Associativity list index that is used as an index into - ``ibm,associativity-lookup-arrays`` property described earlier. This - is used to retrieve the right associativity list to be used for all - the LMBs in this set. -- A 32-bit flags word that applies to all the LMBs in the set.