powerpc/iommu: Do not do platform domain attach atctions after probe

POWER throws a splat at boot, it looks like the DMA ops were probably
changed while a driver was attached. Something is still weird about how
power sequences its bootup. Previously this was hidden since the core
iommu code did nothing during probe, now it calls
spapr_tce_platform_iommu_attach_dev().

Make spapr_tce_platform_iommu_attach_dev() do nothing on the probe time
call like it did before.

  WARNING: CPU: 0 PID: 8 at arch/powerpc/kernel/iommu.c:407 __iommu_free+0x1e4/0x1f0
  Modules linked in: sd_mod t10_pi crc64_rocksoft crc64 sg ibmvfc mlx5_core(+) scsi_transport_fc ibmveth mlxfw psample dm_multipath dm_mirror dm_region_hash dm_log dm_mod fuse
  CPU: 0 PID: 8 Comm: kworker/0:0 Not tainted 6.6.0-rc3-next-20230929-auto #1
  Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1030.30 (NH1030_062) hv:phyp pSeries
  Workqueue: events work_for_cpu_fn
  NIP:  c00000000005f6d4 LR: c00000000005f6d0 CTR: 00000000005ca81c
  REGS: c000000003a27890 TRAP: 0700   Not tainted (6.6.0-rc3-next-20230929-auto)
  MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 48000824  XER: 00000008
  CFAR: c00000000020f738 IRQMASK: 0
  GPR00: c00000000005f6d0 c000000003a27b30 c000000001481800 000000000000017
  GPR04: 00000000ffff7fff c000000003a27950 c000000003a27948 0000000000000027
  GPR08: c000000c18c07c10 0000000000000001 0000000000000027 c000000002ac8a08
  GPR12: 0000000000000000 c000000002ff0000 c00000000019cc88 c000000003042300
  GPR16: 0000000000000000 0000000000000000 0000000000000000 c000000003071ab0
  GPR20: c00000000349f80d c000000003215440 c000000003215480 61c8864680b583eb
  GPR24: 0000000000000000 000000007fffffff 0800000020000000 0000000000000010
  GPR28: 0000000000020000 0000800000020000 c00000000c5dc800 c00000000c5dc880
  NIP [c00000000005f6d4] __iommu_free+0x1e4/0x1f0
  LR [c00000000005f6d0] __iommu_free+0x1e0/0x1f0
  Call Trace:
  [c000000003a27b30] [c00000000005f6d0] __iommu_free+0x1e0/0x1f0 (unreliable)
  [c000000003a27bc0] [c00000000005f848] iommu_free+0x28/0x70
  [c000000003a27bf0] [c000000000061518] iommu_free_coherent+0x68/0xa0
  [c000000003a27c20] [c00000000005e8d4] dma_iommu_free_coherent+0x24/0x40
  [c000000003a27c40] [c00000000024698c] dma_free_attrs+0x10c/0x140
  [c000000003a27c90] [c008000000dcb8d4] mlx5_cmd_cleanup+0x5c/0x90 [mlx5_core]
  [c000000003a27cc0] [c008000000dc45a0] mlx5_mdev_uninit+0xc8/0x100 [mlx5_core]
  [c000000003a27d00] [c008000000dc4ac4] probe_one+0x3ec/0x530 [mlx5_core]
  [c000000003a27d90] [c0000000008c5edc] local_pci_probe+0x6c/0x110
  [c000000003a27e10] [c000000000189c98] work_for_cpu_fn+0x38/0x60
  [c000000003a27e40] [c00000000018d1d0] process_scheduled_works+0x230/0x4f0
  [c000000003a27f10] [c00000000018ff14] worker_thread+0x1e4/0x500
  [c000000003a27f90] [c00000000019cdb8] kthread+0x138/0x140
  [c000000003a27fe0] [c00000000000df98] start_kernel_thread+0x14/0x18
  Code: 481b004d 60000000 e89e0028 3c62ffe0 3863dd20 481b0039 60000000 e89e0038 3c62ffe0 3863dd38 481b0025 60000000 <0fe00000> 4bffff20 60000000 3c4c0142
  ---[ end trace 0000000000000000 ]---
  iommu_free: invalid entry
      entry     = 0x8000000203d0
      dma_addr  = 0x8000000203d0000
      Table     = 0xc00000000c5dc800
      bus#      = 0x1
      size      = 0x20000
      startOff  = 0x800000000000
      index     = 0x70200016

Fixes: 2ad56efa80db ("powerpc/iommu: Setup a default domain and remove set_platform_dma_ops")
Reported-by: Tasmiya Nalatwad <tasmiya@linux.vnet.ibm.com>
Link: https://lore.kernel.org/r/d06cee81-c47f-9d62-dfc6-4c77b60058db@linux.vnet.ibm.com
Tested-by: Tasmiya Nalatwad <tasmiya@linux.vnet.ibm.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/0-v1-2b52423411b9+164fc-iommu_ppc_defdomain_jgg@nvidia.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>

diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index d6ad3fde85a21207628f88a4ee35b6d4ac173923..bf19932147517bf6cfe0eab7e47f06404abce79c 100644 (file)
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -1280,13 +1280,19 @@ struct iommu_table_group_ops spapr_tce_table_group_ops = {
 /*
  * A simple iommu_ops to allow less cruft in generic VFIO code.
  */
-static int spapr_tce_platform_iommu_attach_dev(struct iommu_domain *dom,
-                                              struct device *dev)
+static int
+spapr_tce_platform_iommu_attach_dev(struct iommu_domain *platform_domain,
+                                   struct device *dev)
 {
+       struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
        struct iommu_group *grp = iommu_group_get(dev);
        struct iommu_table_group *table_group;
        int ret = -EINVAL;
 
+       /* At first attach the ownership is already set */
+       if (!domain)
+               return 0;
+
        if (!grp)
                return -ENODEV;