526互联

Running DPDK Forwarding Applications With Pktgen-DPDK

发布时间 2023-11-20 18:15:35作者: ImreW

As part of the evaluation stage of our bachelor thesis, we set up a testbed for running forwarding applications in DPDK and with Pktgen-DPDK as the traffic generator. In this blog, we aim to cover

作为学士论文评估阶段的一部分，我们建立了一个测试平台，用于在 DPDK 中运行转发应用程序，并使用 Pktgen-DPDK 作为流量生成器。 在这篇博客中，我们的目标是涵盖

  • how to run Pktgen-DPDK using Lua scripts in some complex scenarios
  • 如何在一些复杂场景下使用Lua脚本运行Pktgen-DPDK
  • how to run L2fwd, L3fwd (with exact match and longest prefix match) and L3fwd power DPDK applications.
  • 如何运行 L2fwd、L3fwd（具有精确匹配和最长前缀匹配）和 L3fwd Power DPDK 应用程序

We did not find a lot of resources for setting up these tests and with this blog, we aim to bridge the gap.

我们没有找到很多资源来设置这些测试，通过这个博客，我们的目标是弥合差距。

DPDK and Pktgen-DPDK

The Data Plane Development Kit is an open-source software project managed by the Linux Foundation. It provides a set of data plane libraries and network interface controller polling-mode drivers running in userspace. This way, the NIC is directly accessible by the DPDK application. The advantage of using DPDK is that you can utilize the link speed completely in the case of high-speed links whereas, in the standard packet processing, the link may be underutilized. Thus, high performance can be achieved with DPDK.

数据平面开发套件是一个由 Linux 基金会管理的开源软件项目。 它提供了一组在用户空间中运行的数据平面库和网络接口控制器轮询模式驱动程序。 这样，DPDK 应用程序就可以直接访问 NIC。 使用DPDK的优点是，在高速链路的情况下可以完全利用链路速度，而在标准数据包处理中，链路可能没有得到充分利用。 因此，使用 DPDK 可以实现高性能。

Pktgen-DPDK is a software-based traffic generator powered by DPDK. Traffic generators are often used to simulate various situations and test the performance of the application.

Pktgen-DPDK 是一款由 DPDK 提供支持的基于软件的流量生成器。 流量生成器通常用于模拟各种情况并测试应用程序的性能。

Test Setup

All the tests were done with two systems, one as the System Under Test which is a bare metal server, and the other one as a generator which is a VM. The SUT has two NUMA sockets where each socket has two 14-core Intel Xeon Gold 5120 2.20 GHz processors, which has four Intel I350 10 Gigabit NICs and four Ethernet Controller X710/X557-AT 10GBASE-T NICs. The generator has two 12-core Intel Xeon Gold 5120 2.20 GHz processors, 94GB memory, and one Intel 82540EM Gigabit NIC and Ethernet Controller X710/X557-AT 10GBASE-T NICs. Each system runs Ubuntu 20.04 LTS.

所有测试均使用两个系统完成，一个作为被测系统，即裸机服务器，另一个作为生成器，即虚拟机。 SUT 有两个 NUMA 插槽，每个插槽有两个 14 核 Intel Xeon Gold 5120 2.20 GHz 处理器，其中有四个 Intel I350 10 Gigabit NIC 和四个以太网控制器 X710/X557-AT 10GBASE-T NIC。 该生成器具有 2 个 12 核 Intel Xeon Gold 5120 2.20 GHz 处理器、94GB 内存和 1 个 Intel 82540EM 千兆网卡和以太网控制器 X710/X557-AT 10GBASE-T 网卡。 每个系统都运行 Ubuntu 20.04 LTS。

Installation of both DPDK and Pktgen-DPDK was done by compiling from source. igb_uio kernel driver was used and 2048 huge pages were allotted. Here is a useful tutorial on installing and setting up DPDK and Pktgen-DPDK on your system. Tapped interfaces can be used in case your system does not have enough NICs. Here’s a diagrammatic representation of the setup.

DPDK 和 Pktgen-DPDK 的安装都是通过源代码编译完成的。 使用igb_uio内核驱动程序并分配了2048个大页面。 这是有关在系统上安装和设置 DPDK 和 Pktgen-DPDK 的有用教程。 如果您的系统没有足够的 NIC，则可以使用分接接口。 这是设置的示意图。

Lua Scripts with Pktgen-DPDK

We used Lua scripts to automate the process of setting parameters and configuring flows in Pktgen-DPDK. Pktgen-DPDK can be run on the generator using the command shown below. It is run in promiscuous mode with the first four cores (0,1,2,3), and 4 memory channels. The Lua script for each application is mentioned using the -f option. The port mapping to cores is done as:

我们使用 Lua 脚本来自动化 Pktgen-DPDK 中设置参数和配置流程的过程。 Pktgen-DPDK 可以使用下面所示的命令在生成器上运行。 它以混杂模式运行，前四个核心 (0,1,2,3) 和 4 个内存通道。 使用 -f 选项提及每个应用程序的 Lua 脚本。 端口映射到核心的方式如下：

  • core 1 handles port 0, 1 RX/TX
  • 核心 1 处理端口 0、1 RX/TX
  • core 2 handles port 2, 3 RX/TX.
  • 核心 2 处理端口 2、3 RX/TX

Note: The core 0 is used for keyboard, timer, and screen output and hence shouldn’t be used for port mapping. pktgen-dpdk with Lua script should be run from the pktgen directory only, otherwise, it gives an error.

注意：核心 0 用于键盘、定时器和屏幕输出，因此不应用于端口映射。 带有 Lua 脚本的 pktgen-dpdk 只能从 pktgen 目录运行，否则会出错。

sudo ./app/x86_64-native-linux-gcc/pktgen -c 0xf -n 4 — -P -m 1.[0–1] -m 2.[2–3] -f /path/to/script.lua

Situation 1: Flow for a Fixed Time

We were required to run tests where the packets were sent for a fixed time period. Since pktgen by default doesn’t support timers, we used os functions for the same.

我们需要运行测试，其中数据包在固定的时间段内发送。 由于 pktgen 默认情况下不支持计时器，因此我们使用 os 函数来实现同样的功能。

-- time.lua
local port = "0";
local send_for_secs = 120;
pktgen.start(port);
local start_time = os.time();

while os.difftime(os.time(), start_time) < send_for_secs do
    sleep(1);
end

pktgen.stop(port);

where send_for_secs is the time for which we want to send.

其中 send_for_secs 是我们要发送的时间。

Note: sleep(send_for_secs) was not used because it would block Pktgen’s output refreshing. So continuous check after a second and checking until the difference reaches the send_for_secs is done here.

注意：未使用 sleep(send_for_secs)，因为它会阻止 Pktgen 的输出刷新。 因此，在一秒钟后连续检查并检查直到差异达到 send_for_secs 为止。

Situation 2: Different Packet Size and Rate

We also wanted to test with different packet sizes and rates (rate in pktgen-dpdk is to be given as a percentage of the line rate). Since changing the packet size or rate in the Lua script after every run is tedious, we ask for user input for the packet size and rate at runtime using the script below.

我们还想使用不同的数据包大小和速率进行测试（pktgen-dpdk 中的速率以线路速率的百分比给出）。 由于每次运行后更改 Lua 脚本中的数据包大小或速率非常繁琐，因此我们使用下面的脚本要求用户在运行时输入数据包大小和速率。

-- size-rate.lua
local pktSize = tonumber(pktgen.input("Enter the size of packets to send: "));
if pktSize == nil then
    pktSize = 64;
end
pktgen.set(port, "size", pktSize);

local rate = tonumber(pktgen.input("Enter the sending rate: "));
if rate == nil then
    rate = 100;
end
pktgen.set(port, "rate", rate);

Forwarding Applications in DPDK

We mention the steps to run the forwarding application and a corresponding Lua script to set relevant parameters. The command to run Pktgen-DPDK with the Lua script has been given in the previous section.

我们提到了运行转发应用程序的步骤以及相应的Lua脚本来设置相关参数。 使用Lua脚本运行Pktgen-DPDK的命令已在上一节中给出。

l2fwd

Layer 2 forwarding (switching) does not need IP addresses to work. Thus, we set only the MAC address in the packets sent from pktgen-DPDK.

二层转发（交换）不需要IP地址即可工作。 因此，我们只设置从 pktgen-DPDK 发送的数据包中的 MAC 地址。

  • Start the l2fwd application on the SUT: The l2fwd example can be run with the following command on the SUT. This command starts l2fwd on four cores and with four ports. The SUT has two NUMA sockets but we use only one. The even-numbered cores are on socket 0 and we use the first four cores on it, i.e, cores 0,2,4,6. Thus the core mask is given as 0x55. The port mask is 0xf as we use four ports.
  • 在 SUT 上启动 l2fwd 应用程序：可以在 SUT 上使用以下命令运行 l2fwd 示例。 此命令在四个核心和四个端口上启动 l2fwd。 SUT 有两个 NUMA 插槽，但我们只使用一个。 偶数核心位于插槽 0 上，我们使用其上的前四个核心，即核心 0、2、4、6。 因此，核心掩码为 0x55。 由于我们使用四个端口，因此端口掩码为 0xf。
sudo ./${RTE_SDK}/examples/l2fwd/${RTE_TARGET}/app/l2fwd -c 0x55 -n 4 -- -p 0xf
  • Start pktgen-dpdk on the generator with Lua script: Get the MAC address of the ports by starting the l2fwd application on the SUT. Set them as the destination MAC address for the ports 0 and 2 in pktgen through the Lua script shown below.
  • 使用 Lua 脚本在生成器上启动 pktgen-dpdk：通过在 SUT 上启动 l2fwd 应用程序来获取端口的 MAC 地址。 通过如下所示的 Lua 脚本将它们设置为 pktgen 中端口 0 和 2 的目标 MAC 地址。
-- l2fwd.lua 
local port0 = "0";
local port2 = "2";

local port0_mac = "3C:FD:FE:7C:BC:E0";
local port2_mac = "3C:FD:FE:7C:BC:E2";

pktgen.set_mac(port0, "dst", port0_mac);
pktgen.set_mac(port2, "dst", port2_mac);

-- code for taking user input for packet size and rate
-- setting the packet size & rate

pktgen.start(port0);
pktgen.start(port2);

-- code to check time to run

pktgen.stop(port0);
pktgen.stop(port2);

l3fwd

A layer 3 forwarder matches the destination IP of an incoming packet to the entries stored in its forwarding table and updates the layer 2 header before sending the packet to the next-hop.

第 3 层转发器将传入数据包的目标 IP 与其转发表中存储的条目进行匹配，并在将数据包发送到下一跳之前更新第 2 层标头。

  • Longest Prefix Match: For the LPM method, an LPM object is used to emulate the forwarding stage for IPv4 packets. The LPM object is used as the routing table to identify the next hop for each input packet at runtime. The example can be run with the following command on the SUT.
  • 最长前缀匹配：对于 LPM 方法，LPM 对象用于模拟 IPv4 数据包的转发阶段。 LPM 对象用作路由表，用于在运行时识别每个输入数据包的下一跳。 可以在 SUT 上使用以下命令运行该示例。
sudo ./${RTE_SDK}/examples/l3fwd/${RTE_TARGET}/app/l3fwd -c 0x55 -n 4 — -P -p 0xf — config=(0,0,0),(1,0,2),(2,0,4),(3,0,6)

The IP addresses obtained from the source code of l3fwd are set as shown below.

从l3fwd源码中获取的IP地址设置如下。

-- l3fwd-lpm.lua
local port0 = "0";
local port1 = "1";
local port2 = "2";
local port3 = "3";

pktgen.set_ipaddr(port0, "dst", "198.18.1.1");
pktgen.set_ipaddr(port0, "src", "198.18.0.1");
pktgen.set_ipaddr(port1, "dst", "198.18.0.1");
pktgen.set_ipaddr(port1, "src", "198.18.1.1");

pktgen.set_ipaddr(port2, "dst", "198.18.3.1");
pktgen.set_ipaddr(port2, "src", "198.18.2.1");
pktgen.set_ipaddr(port3, "dst", "198.18.2.1");
pktgen.set_ipaddr(port3, "src", "198.18.3.1");

-- code for taking input of packet size, rate from the user
-- setting the packet size & rate

pktgen.start(port0);
pktgen.start(port2);

-- code to check time to run

pktgen.stop(port0);
pktgen.stop(port2);
  • Exact Match: In the exact match lookup method, the forwarding is based on a hash for which a hash object is used to emulate the flow classification stage. The example can be run with the following command on the SUT. The -E parameter selects the exact match method.
  • 精确匹配：在精确匹配查找方法中，转发基于哈希，使用哈希对象来模拟流分类阶段。 可以在 SUT 上使用以下命令运行该示例。 -E参数选择精确匹配方式。
sudo ./${RTE_SDK}/examples/l3fwd/${RTE_TARGET}/app/l3fwd -c 0x55 -n 4 -- -P -E --parse-ptype -p 0xf --config=(0,0,0),(1,0,2),(2,0,4),(3,0,6)

From the code for the l3fwd application, the exact match works only with TCP packets and thus we set the protocol to TCP in the packets generated from pktgen. The exact match is based on the combination of IP addresses and port numbers, as seen in the routing table in the code. We set the IP addresses and destination and source ports as shown below obtained from the source code.

从 l3fwd 应用程序的代码来看，精确匹配仅适用于 TCP 数据包，因此我们在 pktgen 生成的数据包中将协议设置为 TCP。 精确匹配基于 IP 地址和端口号的组合，如代码中的路由表所示。 我们设置从源代码中获得的 IP 地址以及目标和源端口，如下所示。

-- l3fwd-em.lua
local port0 = "0";
local port1 = "1";
local port2 = "2";
local port3 = "3";

pktgen.set_ipaddr(port0, "dst", "201.0.0.0");
pktgen.set_ipaddr(port0, "src", "200.20.0.1");
pktgen.set_ipaddr(port1, "dst", "101.0.0.0");
pktgen.set_ipaddr(port1, "src", "100.10.0.1");

pktgen.set_ipaddr(port2, "dst", "211.0.0.0");
pktgen.set_ipaddr(port2, "src", "200.40.0.1");
pktgen.set_ipaddr(port3, "dst", "111.0.0.0");
pktgen.set_ipaddr(port3, "src", "100.30.0.1");

pktgen.set(port0, "sport", 12);
pktgen.set(port0, "dport", 102);
pktgen.set(port1, "sport", 11);
pktgen.set(port1, "dport", 101);

pktgen.set(port2, "sport", 12);
pktgen.set(port2, "dport", 102);
pktgen.set(port3, "sport", 11);
pktgen.set(port3, "dport", 101);

-- code for taking input of packet size, rate from the user
-- setting the packet size & rate

pktgen.set_proto("all", "tcp");

pktgen.start(port0);
pktgen.start(port2);

-- code to check time to run

pktgen.stop(port0);
pktgen.stop(port2);

l3fwd-power

The L3 Forwarding with Power Management application is an example of power-aware packet processing using the DPDK. The example can be run with the following command on the SUT.

带电源管理的 L3 转发应用程序是使用 DPDK 进行电源感知数据包处理的示例。 可以在 SUT 上使用以下命令运行该示例。

sudo ./${RTE_SDK}/examples/l3fwd-power/${RTE_TARGET}/app/l3fwd-power -c 0x55 -n 4 -- -P -p 0xf --config=(0,0,0),(1,0,2),(2,0,4),(3,0,6)

The flow configured on pktgen with IP addresses can be done as below. The IP addresses are obtained from the source code of l3fwd-power. It runs the longest prefix match by default.

在 pktgen 上使用 IP 地址配置的流程可以如下完成。 IP地址是从l3fwd-power的源代码中获取的。 它默认运行最长的前缀匹配。

-- l3fwd-power.lua
local port0 = "0";
local port1 = "1";
local port2 = "2";
local port3 = "3";

pktgen.set_ipaddr(port0, "dst", "1.1.1.1");
pktgen.set_ipaddr(port0, "src", "2.1.1.1");
pktgen.set_ipaddr(port1, "dst", "2.1.1.1");
pktgen.set_ipaddr(port1, "src", "1.1.1.1");

pktgen.set_ipaddr(port2, "dst", "4.1.1.1");
pktgen.set_ipaddr(port2, "src", "3.1.1.1");
pktgen.set_ipaddr(port3, "dst", "3.1.1.1");
pktgen.set_ipaddr(port3, "src", "4.1.1.1");

-- code for taking input of packet size, rate from the user
-- setting the packet size & rate

pktgen.start(port0);
pktgen.start(port2);

-- code to check time to run

pktgen.stop(port0);
pktgen.stop(port2);

Conclusions

In this article, we looked at running various forwarding applications of DPDK with Pktgen-DPDKas the traffic generator with four ports. The main challenge we faced while trying to run these applications was a lack of proper documentation in setting up various cases with pktgen and making sure the traffic forwards on the application and reaches back to the other port. If you do not configure the IP/MAC address properly, it is possible that the packets do not reach the other port but come back on the same port.

在本文中，我们研究了使用具有四个端口的流量生成器 Pktgen-DPDKa 来运行 DPDK 的各种转发应用程序。 我们在尝试运行这些应用程序时面临的主要挑战是缺乏适当的文档来设置 pktgen 的各种情况并确保流量在应用程序上转发并返回到另一个端口。 如果未正确配置 IP/MAC 地址，数据包可能无法到达其他端口，而是返回同一端口。

Both DPDK and Pktgen-DPDK require a bit of work to set up and understand how the applications work, and undoubtedly there is a good learning curve. The DPDK documentation provides enough information about running the example applications and so we have not covered that in detail.

DPDK 和 Pktgen-DPDK 都需要做一些工作来设置和了解应用程序的工作原理，毫无疑问，有一个很好的学习曲线。 DPDK 文档提供了有关运行示例应用程序的足够信息，因此我们没有详细介绍。

The complete Lua scripts can be found here.

完整的 Lua 脚本可以在这里找到。

References