IB/ipoib: Improve latency in ipoib/cm connection formation

Currently IPoIB connected mode queries the device to get the pkey table
entry during connection formation. This will increase the time taken to
form the connection, especially when limited pkeys are in use.  This gets
worse when multiple connection attempts are done in parallel.

Since ipoib interfaces are locked to a single pkey, use the pkey index
that was determined at link up time instead of searching for anything.

This improved the latency from 500ms to 1ms on an internal setup.

Link: https://lore.kernel.org/r/1618338965-16717-1-git-send-email-manjunath.b.patil@oracle.com
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Manjunath Patil <manjunath.b.patil@oracle.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index d5d592bdab3585cd7e99a7eb00a3e22063ea2d50..9dbc85a6b702cc2bf9d5a37377aaf975508f0203 100644 (file)
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -1122,12 +1122,8 @@ static int ipoib_cm_modify_tx_init(struct net_device *dev,
        struct ipoib_dev_priv *priv = ipoib_priv(dev);
        struct ib_qp_attr qp_attr;
        int qp_attr_mask, ret;
-       ret = ib_find_pkey(priv->ca, priv->port, priv->pkey, &qp_attr.pkey_index);
-       if (ret) {
-               ipoib_warn(priv, "pkey 0x%x not found: %d\n", priv->pkey, ret);
-               return ret;
-       }
 
+       qp_attr.pkey_index = priv->pkey_index;
        qp_attr.qp_state = IB_QPS_INIT;
        qp_attr.qp_access_flags = IB_ACCESS_LOCAL_WRITE;
        qp_attr.port_num = priv->port;