Add ability to specify a network interface used to resolve XDP hints
kfuncs when loading program through bpftool.
Usage:
  bpftool prog load [...] xdpmeta_dev <ifname>
Writing just 'dev <ifname>' instead of 'xdpmeta_dev' is a very probable
mistake that results in not very descriptive errors,
so 'bpftool prog load [...] dev <ifname>' syntax becomes deprecated,
followed by 'bpftool map create [...] dev <ifname>' for consistency.
Now, to offload program, execute:
  bpftool prog load [...] offload_dev <ifname>
To offload map:
  bpftool map create [...] offload_dev <ifname>
'dev <ifname>' still performs offloading in the commands above, but now
triggers a warning and is excluded from bash completion.
'xdpmeta_dev' and 'offload_dev' are mutually exclusive options, because
'xdpmeta_dev' basically makes a program device-bound without loading it
onto the said device. For now, offloaded programs cannot use XDP hints [0],
but if this changes, using 'offload_dev <ifname>' should cover this case.
  [0] https://lore.kernel.org/bpf/
a5a636cc-5b03-686f-4be0-
000383b05cfc@linux.dev
Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <quentin@isovalent.com>
Link: https://lore.kernel.org/bpf/20230517160103.1088185-1-larysa.zaremba@intel.com
 
 |      **bpftool** **map** { **show** | **list** }   [*MAP*]
 |      **bpftool** **map create**     *FILE* **type** *TYPE* **key** *KEY_SIZE* **value** *VALUE_SIZE* \
 |              **entries** *MAX_ENTRIES* **name** *NAME* [**flags** *FLAGS*] [**inner_map** *MAP*] \
-|              [**dev** *NAME*]
+|              [**offload_dev** *NAME*]
 |      **bpftool** **map dump**       *MAP*
 |      **bpftool** **map update**     *MAP* [**key** *DATA*] [**value** *VALUE*] [*UPDATE_FLAGS*]
 |      **bpftool** **map lookup**     *MAP* [**key** *DATA*]
                  maps. On such kernels bpftool will automatically emit this
                  information as well.
 
-       **bpftool map create** *FILE* **type** *TYPE* **key** *KEY_SIZE* **value** *VALUE_SIZE*  **entries** *MAX_ENTRIES* **name** *NAME* [**flags** *FLAGS*] [**inner_map** *MAP*] [**dev** *NAME*]
+       **bpftool map create** *FILE* **type** *TYPE* **key** *KEY_SIZE* **value** *VALUE_SIZE*  **entries** *MAX_ENTRIES* **name** *NAME* [**flags** *FLAGS*] [**inner_map** *MAP*] [**offload_dev** *NAME*]
                  Create a new map with given parameters and pin it to *bpffs*
                  as *FILE*.
 
                  kernel needs it to collect metadata related to the inner maps
                  that the new map will work with.
 
-                 Keyword **dev** expects a network interface name, and is used
-                 to request hardware offload for the map.
+                 Keyword **offload_dev** expects a network interface name,
+                 and is used to request hardware offload for the map.
 
        **bpftool map dump**    *MAP*
                  Dump all entries in a given *MAP*.  In case of **name**,
 
 |      **bpftool** **prog dump xlated** *PROG* [{ **file** *FILE* | [**opcodes**] [**linum**] [**visual**] }]
 |      **bpftool** **prog dump jited**  *PROG* [{ **file** *FILE* | [**opcodes**] [**linum**] }]
 |      **bpftool** **prog pin** *PROG* *FILE*
-|      **bpftool** **prog** { **load** | **loadall** } *OBJ* *PATH* [**type** *TYPE*] [**map** {**idx** *IDX* | **name** *NAME*} *MAP*] [**dev** *NAME*] [**pinmaps** *MAP_DIR*] [**autoattach**]
+|      **bpftool** **prog** { **load** | **loadall** } *OBJ* *PATH* [**type** *TYPE*] [**map** { **idx** *IDX* | **name** *NAME* } *MAP*] [{ **offload_dev** | **xdpmeta_dev** } *NAME*] [**pinmaps** *MAP_DIR*] [**autoattach**]
 |      **bpftool** **prog attach** *PROG* *ATTACH_TYPE* [*MAP*]
 |      **bpftool** **prog detach** *PROG* *ATTACH_TYPE* [*MAP*]
 |      **bpftool** **prog tracelog**
                  contain a dot character ('.'), which is reserved for future
                  extensions of *bpffs*.
 
-       **bpftool prog { load | loadall }** *OBJ* *PATH* [**type** *TYPE*] [**map** {**idx** *IDX* | **name** *NAME*} *MAP*] [**dev** *NAME*] [**pinmaps** *MAP_DIR*] [**autoattach**]
+       **bpftool prog { load | loadall }** *OBJ* *PATH* [**type** *TYPE*] [**map** { **idx** *IDX* | **name** *NAME* } *MAP*] [{ **offload_dev** | **xdpmeta_dev** } *NAME*] [**pinmaps** *MAP_DIR*] [**autoattach**]
                  Load bpf program(s) from binary *OBJ* and pin as *PATH*.
                  **bpftool prog load** pins only the first program from the
                  *OBJ* as *PATH*. **bpftool prog loadall** pins all programs
                  to be replaced in the ELF file counting from 0, while *NAME*
                  allows to replace a map by name.  *MAP* specifies the map to
                  use, referring to it by **id** or through a **pinned** file.
-                 If **dev** *NAME* is specified program will be loaded onto
-                 given networking device (offload).
+                 If **offload_dev** *NAME* is specified program will be loaded
+                 onto given networking device (offload).
+                 If **xdpmeta_dev** *NAME* is specified program will become
+                 device-bound without offloading, this facilitates access
+                 to XDP metadata.
                  Optional **pinmaps** argument can be provided to pin all
                  maps under *MAP_DIR* directory.
 
 
             _bpftool_get_prog_tags
             return 0
             ;;
-        dev)
+        dev|offload_dev|xdpmeta_dev)
             _sysfs_get_netdevs
             return 0
             ;;
                             ;;
                         *)
                             COMPREPLY=( $( compgen -W "map" -- "$cur" ) )
-                            _bpftool_once_attr 'type dev pinmaps autoattach'
+                            _bpftool_once_attr 'type pinmaps autoattach'
+                            _bpftool_one_of_list 'offload_dev xdpmeta_dev'
                             return 0
                             ;;
                     esac
                             esac
                             ;;
                         *)
-                            _bpftool_once_attr 'type key value entries name flags dev'
+                            _bpftool_once_attr 'type key value entries name flags offload_dev'
                             if _bpftool_search_list 'array_of_maps' 'hash_of_maps'; then
                                 _bpftool_once_attr 'inner_map'
                             fi
 
                                          "flags"))
                                goto exit;
                } else if (is_prefix(*argv, "dev")) {
+                       p_info("Warning: 'bpftool map create [...] dev <ifname>' syntax is deprecated.\n"
+                              "Going further, please use 'offload_dev <ifname>' to request hardware offload for the map.");
+                       goto offload_dev;
+               } else if (is_prefix(*argv, "offload_dev")) {
+offload_dev:
                        NEXT_ARG();
 
                        if (attr.map_ifindex) {
                "Usage: %1$s %2$s { show | list }   [MAP]\n"
                "       %1$s %2$s create     FILE type TYPE key KEY_SIZE value VALUE_SIZE \\\n"
                "                                  entries MAX_ENTRIES name NAME [flags FLAGS] \\\n"
-               "                                  [inner_map MAP] [dev NAME]\n"
+               "                                  [inner_map MAP] [offload_dev NAME]\n"
                "       %1$s %2$s dump       MAP\n"
                "       %1$s %2$s update     MAP [key DATA] [value VALUE] [UPDATE_FLAGS]\n"
                "       %1$s %2$s lookup     MAP [key DATA]\n"
 
        struct bpf_program *prog = NULL, *pos;
        unsigned int old_map_fds = 0;
        const char *pinmaps = NULL;
+       __u32 xdpmeta_ifindex = 0;
+       __u32 offload_ifindex = 0;
        bool auto_attach = false;
        struct bpf_object *obj;
        struct bpf_map *map;
        const char *pinfile;
        unsigned int i, j;
-       __u32 ifindex = 0;
        const char *file;
        int idx, err;
 
                        map_replace[old_map_fds].fd = fd;
                        old_map_fds++;
                } else if (is_prefix(*argv, "dev")) {
+                       p_info("Warning: 'bpftool prog load [...] dev <ifname>' syntax is deprecated.\n"
+                              "Going further, please use 'offload_dev <ifname>' to offload program to device.\n"
+                              "For applications using XDP hints only, use 'xdpmeta_dev <ifname>'.");
+                       goto offload_dev;
+               } else if (is_prefix(*argv, "offload_dev")) {
+offload_dev:
                        NEXT_ARG();
 
-                       if (ifindex) {
-                               p_err("offload device already specified");
+                       if (offload_ifindex) {
+                               p_err("offload_dev already specified");
+                               goto err_free_reuse_maps;
+                       } else if (xdpmeta_ifindex) {
+                               p_err("xdpmeta_dev and offload_dev are mutually exclusive");
+                               goto err_free_reuse_maps;
+                       }
+                       if (!REQ_ARGS(1))
+                               goto err_free_reuse_maps;
+
+                       offload_ifindex = if_nametoindex(*argv);
+                       if (!offload_ifindex) {
+                               p_err("unrecognized netdevice '%s': %s",
+                                     *argv, strerror(errno));
+                               goto err_free_reuse_maps;
+                       }
+                       NEXT_ARG();
+               } else if (is_prefix(*argv, "xdpmeta_dev")) {
+                       NEXT_ARG();
+
+                       if (xdpmeta_ifindex) {
+                               p_err("xdpmeta_dev already specified");
+                               goto err_free_reuse_maps;
+                       } else if (offload_ifindex) {
+                               p_err("xdpmeta_dev and offload_dev are mutually exclusive");
                                goto err_free_reuse_maps;
                        }
                        if (!REQ_ARGS(1))
                                goto err_free_reuse_maps;
 
-                       ifindex = if_nametoindex(*argv);
-                       if (!ifindex) {
+                       xdpmeta_ifindex = if_nametoindex(*argv);
+                       if (!xdpmeta_ifindex) {
                                p_err("unrecognized netdevice '%s': %s",
                                      *argv, strerror(errno));
                                goto err_free_reuse_maps;
                                goto err_close_obj;
                }
 
-               bpf_program__set_ifindex(pos, ifindex);
+               if (prog_type == BPF_PROG_TYPE_XDP && xdpmeta_ifindex) {
+                       bpf_program__set_flags(pos, BPF_F_XDP_DEV_BOUND_ONLY);
+                       bpf_program__set_ifindex(pos, xdpmeta_ifindex);
+               } else {
+                       bpf_program__set_ifindex(pos, offload_ifindex);
+               }
                if (bpf_program__type(pos) != prog_type)
                        bpf_program__set_type(pos, prog_type);
                bpf_program__set_expected_attach_type(pos, expected_attach_type);
        idx = 0;
        bpf_object__for_each_map(map, obj) {
                if (bpf_map__type(map) != BPF_MAP_TYPE_PERF_EVENT_ARRAY)
-                       bpf_map__set_ifindex(map, ifindex);
+                       bpf_map__set_ifindex(map, offload_ifindex);
 
                if (j < old_map_fds && idx == map_replace[j].idx) {
                        err = bpf_map__reuse_fd(map, map_replace[j++].fd);
                "       %1$s %2$s dump jited  PROG [{ file FILE | [opcodes] [linum] }]\n"
                "       %1$s %2$s pin   PROG FILE\n"
                "       %1$s %2$s { load | loadall } OBJ  PATH \\\n"
-               "                         [type TYPE] [dev NAME] \\\n"
+               "                         [type TYPE] [{ offload_dev | xdpmeta_dev } NAME] \\\n"
                "                         [map { idx IDX | name NAME } MAP]\\\n"
                "                         [pinmaps MAP_DIR]\n"
                "                         [autoattach]\n"