This document illustrates an example of running the UDP40G-IP demo using two different test environments to transfer UDP data. The first test environment employs a single FPGA board to transfer UDP data with a PC that runs a test application for transferring UDP data over 40G Ethernet. The performance result in this environment is constrained by the resources of the PC. On the other hand, the best performance for transferring UDP data using UDP40G-IP can be achieved by utilizing two FPGA boards in the second test environment, where they transfer data to each other.

The document covers three sections, which include setting up the 40G Ethernet card on the PC to achieve optimal performance for transferring data via 40G Ethernet in section 2. In section 3, the console example and test results are presented when operating under the first test environment, involving FPGA and PC. Lastly, section 4 shows the console example when operating the second test environment, involving FPGA and FPGA. Each topic is described in further detail as follows.

2 PC Setup

This section provides an example setting of the 40G Ethernet adapter on a PC to enable Jumbo frame and boost network performance on Windows10 OS.

2.1 Setting the IP address of the 40G Ethernet adapter

To set the IP address of the Ethernet adapter, follow the steps below.

Figure 2‑1 Setting IP address of the Ethernet adapter on PC

1) Open the Local Area Connection Properties of the 40G Ethernet connection, as shown in the left window of Figure 2‑1.

2) Select “TCP/IPv4” and click on Properties.

3) Set IP address = 192.168.40.25 and the Subnet mask = 255.255.255.0, as shown in the right window of Figure 2‑1.

2.2 Enabling Jumbo frame of the Ethernet adapter

To enable Jumbo Frame of the Ethernet adapter, follow the steps below.

Figure 2‑2 Set frame size = jumbo frame

1) On the Local Area Connection Properties window, click on “Configure”, as shown in the left window of Figure 2‑2.

2) On the Advanced Tab, select “Jumbo Packet” and set the Value to “9014 Bytes” for Jumbo Frame support, as shown in the right window of Figure 2‑2.

Note: Setting “Disabled” to disabling Jumbo frame may reduce the performance, so it is advisable to enable it.

2.3 Disabling Flow Control and Interrupt Moderation of the Ethernet adapter

To disable the flow control and interrupt moderation of the Ethernet adapter, follow the steps below.

Figure 2‑3 Disable Flow Control and Interrupt Moderation

1) Select “Flow Control” and set the value to “Disabled”, as shown in the left window of Figure 2‑3.

2) Select “Interrupt Moderation” and set the value to “Disabled”, as shown in the right window of Figure 2‑3.

3) Click the “OK” button to save and exit all the setting windows.

2.4 Setting Number of RSS Queues and RSS Base Processor Number

If the Ethernet adapter card comprises multiple Ethernet connections, configuring the number of RSS Queues and RSS Base Processor Number for each connection is necessary. Ensure that the values are not overlapped, as detailed below.

Figure 2‑4 Set Number of RSS Queues and RSS Base Processor Number

1) Config the Maximum Number of RSS Queues to a value greater than 4; for instance, set it to 8 queues.

2) Ensure that the RSS Base Processor Number assigned to each port does not overlap. For instance, if there are two Ethernet ports and the Maximum Number of RSS Queues for both ports is set to 8, assign RSS Base Processor Number of 0 and 8 to each Ethernet port, respectively.

3) Save and exit all setting windows by clicking the “OK” button.

2.5 Checking Power options Plan

To check the Power Options Plan, follow the steps below.

Figure 2‑5 Power options

1) Open Control Panel and select Power Options, as shown in the left window of Figure 2‑5.

2) Change the setting to “High Performance”, as shown in the right window of Figure 2‑5.

2.6 Disabling Windows Firewall

To disable the Windows Firewall, follow the steps below.

Figure 2‑6 Firewall setting

1) Open Control Panel and select Windows Firewall.

2) Click on “Turn Windows Firewall on or off”.

3) Select Turn off Firewall for both Private and Public network settings.

4) Confirm the setting by clicking the OK button.

3 Test result when using FPGA and PC

3.1 Display UDPIP parameters

Choose option ‘0’ to display the UDP/IP parameters. The console will show either seven parameters in Client/Server mode or eight parameters in Fixed-MAC mode.

Figure 3‑1 Display current parameter result

Here are the details of the parameters.

1) Mode : Set the initialization mode for UDP40G-IP to be Server, Client, or Fixed-MAC. To initialize the IP in Client mode to run with the PC, input ‘0’.

2) FPGA MAC address : Set the 48-bit hex value to be the MAC address of the FPGA. The default value is 0x000102030405.

3) FPGA IP : Set FPGA IP address. The default value is 192.168.40.42.

Note: This value is used as an FPGA IP address for the test application on PC.

4) FPGA port number : Set the port number of FPGA. The default value is 4000.

Note: This value is used as an FPGA port parameter for the test application on PC.

5) Target MAC address (displayed when running Fixed-MAC mode only): Set the 48-bit hex value to be the MAC address of the target device. The default value is 0x554433221100.

6) Target IP : Set the IP address of the target device (40G Ethernet connection on PC). The default value is 192.168.40.25.

7) Target port number (Target->FPGA) : Set the port number of the target device for receiving data from it. The default value is 61000.

Note: This value is used as PC port parameter for the test application on PC in transmit mode.

8) Target port number (FPGA->Target) : Set the port number of the target device for sending data to it. The default value is 60000.

Note: This value is used as PC port parameter for the test application on PC in receive mode.

To change any of these parameters, the user can set them using the menu option [1].

3.2 Reset UDPIP Parameters

Choose option ‘1’ from the menu to reset the IP and modify IP parameters. This menu allows user to change IP settings or reset the UDP40G-IP. Upon selection of this option, the current parameters are displayed on the console. Press ‘x’ to keep the same parameters, or press any other key to modify them. Once the parameters are confirmed, the UDP40G-IP is reset and the initialization process begins.

This menu contains seven or eight parameters that must be set. Each parameter is validated before being loaded into the UDP40G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in section 3.1 (Display UDPIP parameter), and their valid ranges are given below.

1) Mode : Input ‘0’ to initialize the IP as Client mode.

Note: When the PC and FPGA are connected to different networks, which cannot communicate via the ARP process, the UDP40G-IP must be run in Fixed-MAC mode to manually set the MAC address via the console.

2) FPGA MAC address : Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.

3) FPGA IP address : Input four decimal digits separated by “.”, where the valid range of each decimal digit is 0-255.

4) FPGA port number : The valid range is 0-65535.

5) Target MAC address (displayed only when running Fixed-MAC mode) : Input 12 digits of hex value, and add “0x” as a prefix to input as a hex value.

6) Target IP address : Similar to FPGA IP address, this value is a set of four decimal digits.

7) Target port number (Target->FPGA) : The valid range is 0-65535.

8) Target port number (FPGA->Target) : The valid range is 0-65535.

After finishing assigning the parameters, the final values are displayed on the console. Then, the reset signal is sent to the IP, and it initializes using the new parameters. Finally, “IP initialization complete” is displayed on the console once the initialization process is completed, as shown in Figure 3‑2.

Figure 3‑2 Change IP parameter result for Client/Fixed-MAC mode

3.3 Send Data Test

Choose option ‘2’ from the menu to send data from an FPGA to a PC. The test application, “udpdatatest”, is called on the PC with specified parameters via Command Prompt for receiving data. On the other hand, the user inputs the test parameters for sending data on the FPGA console. The steps to run the test are described below.

1) On the FPGA console, the user must input two parameters to initiate the send data test.

a) Transfer size: The unit of transfer size is byte. The valid range is 0x20 - 0xFFFF_FFFF_FFE0. The input must be aligned to 32. The input is a decimal unit inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b) Packet size: The unit of packet size is byte. The valid range is 32 – 8960. The input must be aligned to 32. The input is decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

Note: If packet byte size is greater than 1472, the packet output from UDP40G-IP is a jumbo frame. In this case, the PC and network devices must support jumbo frame.

2) If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Press any key to start data sending ...” is displayed to wait until user enters any key(s).

3) On the Command Prompt, input the test parameters following the recommended values, except for the last two parameters, which can be adjusted for performance testing. The “upddatatest” in Send mode requires five parameters and two optional parameters.

>> udpdatatest [Dir] [FPGAIP] [FPGAPort] [PCPort] [ByteLen] <Pattern> <Timeout>

a) Dir : Set to ‘r’ to receive test data from the FPGA

b) FPGA IP : Set to the same value as the FPGA IP address

c) FPGA port : Set to the same value as the FPGA port number

d) PC port : Set to the same value as the target port number (FPGA->Target)

e) ByteLen : Set to the same value as “Input transfer size” of step 1)

Optional parameters

f) Pattern : ‘1’- enable data verification, ‘0’-disable data verification.

The default value is ‘1’ if no input is provided.

g) Timeout : Timeout in msec unit. Valid values range from 50 to 65536.

It is recommended to set it to 100 for 40G Ethernet speed.

The default value is 100 if no input is provided.

4) After running the test application on the Command Prompt of the PC, a summary of parameter settings is displayed, and the application waits for received data from the FPGA.

5) On the FPGA console, user enters any key(s) to start sending data. While data is being transferred, the current size of the transferred data is displayed on both the FPGA console (transmit data size) and the Command Prompt (receive data size) every second.

6) Once all data has been sent, “Send data complete” is displayed on the FPGA console. On the PC, the test application completes under one of two conditions: first, when the total number of received data equals the set value (no data loss), and second, when no new data is received until a timeout occurs. If a timeout occurs, an error message is displayed on the Command Prompt. If the test application enables data verification, the position of the first error is also displayed. Finally, the total number of transferred data and the performance are displayed on the FPGA console (transmit performance) and the Command Prompt (receive performance).

Figure 3‑3 and Figure 3‑4 illustrate examples of the send data test when using jumbo frame size with a large transfer size. The left window depicts the FPGA console, while the right window displays the Command Prompt on PC. In both cases, the FPGA sends data at maximum speed of 40G Ethernet, which is 4966 MB/s. However, the PC is unable to receive and process all data at this speed, leading to an error message indicating data loss in the Command Prompt.

When disabling data verification on ‘udpdatatest’, the amount of lost data is lower compared to enabling data verification. Specifically, the amount of received data is approximately 70% of total sent data (24.18GB of 34.36GB) when data verification is disabled. On the other hand, when data verification is enabled, the amount of received data decreases to approximately 55% (18.87GB of 34.36GB).

Figure 3‑3 Send data test using large transfer size when the PC disables data verification

Figure 3‑4 Send data test using large transfer size when the PC enables data verification

Figure 3‑5 illustrates an example of a send data test without data loss, with a small transfer size set. However, the application has overhead time for receiving and processing data, and the timer resolution on the PC is constrained. As a result, although the FPGA console still indicates the same transmit performance, at 4900 MB/s, the Command Prompt reflects a lower performance, at 400 MB/s.

Figure 3‑5 Send data test using small transfer size when the PC enables data verification

If the input is invalid, the FPGA console displays “Out-of-range input”. Subsequently, the operation is cancelled, as shown in Figure 3‑6.

Figure 3‑6 Error from invalid input

3.4 Receive Data Test

Choose option ‘3’ from the menu to receive data sent by a PC to an FPGA. The test application, “udpdatatest”, is called on the PC with specified parameters via Command Prompt for receiving data. On the other hand, the user inputs the test parameters for receiving and verifying data on the FPGA console. The steps to run the test are described below.

1) On the FPGA console, the user must input two parameters to initiate the receive data test.

b) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.

2) If all inputs are valid, the recommended parameters to run test application on the PC will be displayed. Next, “Wait data from Target ...” is displayed, indicating that the system is waiting to receive data from the PC.

3) On the Command Prompt, input the test parameters following the recommended values, except for the last parameter, which can be adjusted for performance testing. The “udpdatatest” in Receive mode requires five parameters and one optional parameter.

>> udpdatatest [Dir] [FPGAIP] [FPGAPort] [PCPort] [ByteLen] <Pattern>

i) Dir : Set to ‘t’ to send test data from the PC

ii) FPGA IP : Set to the same value as the FPGA IP address

iii) FPGA port : Set to the same value as the FPGA port number

iv) PC port : Set to the same value as the target port number (Target->FPGA)

v) ByteLen : Set to the same value as “Input transfer size” of step 1)

Optional parameters

vi) Pattern : ‘1’- Incremental pattern, ‘0’- dummy pattern.

The default value is ‘1’ if no input is provided.

4) After running the test application on the Command Prompt of the PC, a summary of parameter settings is displayed, and data transmission to the FPGA begins. While data is being transferred, the current size of the transferred data is displayed on both the FPGA console (receive data size) and the Command Prompt (transmit data size) every second.

5) Once all data has been received, “Receive data completed” is displayed on the FPGA console after the FPGA receives all data. However, the FPGA has implemented a timer to trigger a timeout condition to terminate the data reception process if no new data is received until the timeout is reached. Upon completion of the data transfer, the total number of transferred data and the performance are displayed on the FPGA console (receive performance) and the Command Prompt (transmit performance).

Figure 3‑7 and Figure 3‑8 illustrate examples of the receive data test when the PC sends dummy data and incremental data. The left window depicts the FPGA console, while the right window displays the Command Prompt on PC. The results show that the transfer performance on the PC when disabling data pattern generation in Figure 3‑7 achieves 4300 MB/s, which is higher than the performance when enabling data pattern generation, achieving 3000 MB/s. The performance is constrained by the PC resources. On the FPGA console, there is no data loss in both test cases, ensuring that the FPGA can receive and process all data completely.

Figure 3‑7 Receive data test when data verification is disabled

Figure 3‑8 Receive data test when data verification is enabled

When disabling data pattern generation on the PC Command Prompt, it is important to disable data verification on the FPGA console. If not, it will result in an error message regarding data verification, as illustrated in Figure 3‑9.

Figure 3‑9 Receive data test when data verification is failed

3.5 Full-duplex Test

Choose option ‘4’ from the menu to perform a full duplex test on the FPGA and PC to transfer data in both directions simultaneously. The user inputs test parameters on both the FPGA console and the PC Command Prompt. The “udpdatatest” applications are called twice on the PC with specified parameters via Command Prompt, one for sending data and another for receiving data. Similarly, the test parameters for sending and receiving data are inputted on the FPGA console. The steps to run the test are described below.

1) On the FPGA console, the user must input two parameters to initiate full duplex test.

c) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.

2) If all inputs are valid, the recommended parameters to run test application for sending and receiving data on the PC will be displayed. Next, “Press any key to start data transfer ...” is displayed to wait until user enters any key(s) before initiating the data sending process.

3) Open two Command Prompts on the PC, and input the test parameters following the recommended values. Refer to step 3) of the preceding sections (section 3.3 for Receive mode and section 3.4 for Send mode) for details on the parameters.

4) On the FPGA console, user enters any key(s) to start sending data. While data is being transferred, the current size of the transferred data in both directions is displayed on both the FPGA console and the Command Prompt every second.

5) Upon completion of the data transfer, “Transfer data complete” is displayed on the FPGA console. Finally, the total number of transferred data and the performance in both transfer directions are displayed on both the FPGA console and the Command Prompt.

While Figure 3‑10 illustrates the results obtained with a large transfer size to attain high performance, but data is lost, Figure 3‑11 displays the results with a small transfer size, ensuring no data loss in the full duplex test. The left window depicts the FPGA console, while the right window displays the Command Prompt on PC. The upper-right window pertains to the receive function, while the lower-right window corresponds to the transmit function.

Using a single timer to measure the operation time of both the transmit and receive functions in the FPGA leas to a lower performance compared to the results observed in half-duplex test, as illustrated in section 3.3 and section 3.4. This limitation arises from the higher resource consumption required by the PC when executing two applications simultaneously. To demonstrate the performance capabilities of the FPGA, an FPGA-FPGA test environment is utilized, providing further details in the subsequent section.

Figure 3‑10 Full-duplex test using large transfer size (data loss)

Figure 3‑11 Full-duplex test using small transfer size (no data loss)

4 Test result when using Two FPGAs

4.1 Display UDPIP parameter

Choose option ‘0’ to display the UDP/IP parameters. The console will show either seven parameters in Client/Server mode or eight parameters in Fixed-MAC mode.

Figure 4‑1 Display current parameter result

Here are the details of the parameters.

1) Mode : Set the initialization mode for UDP40G-IP to be Server, Client, or Fixed-MAC. Input ‘0’ for Client, ‘1’ for Server, or ‘2’ for Fixed-MAC.

2) FPGA MAC address : Set the 48-bit hex value to be MAC address of FPGA. The default value is 0x000102030405 for Client/Fixed-MAC mode or 0x001122334455 for Server mode.

3) FPGA IP : Set FPGA IP address. The default value is 192.168.40.42 for Client/Fixed-MAC mode or 192.168.40.25 for Server mode.

4) FPGA port number : Set the port number of FPGA. The default value is 4000 for Client/Fixed-MAC mode or 60000 for Server mode.

5) Target MAC address (displayed when running Fixed-MAC mode only): Set the 48-bit hex value to be the MAC address of the target device. The default value is 0x554433221100.

6) Target IP : Set the IP address of the target device. The default value is 192.168.40.25 for Client/Fixed-MAC mode or 192.168.40.42 for Server mode.

7) Target port number (Target->FPGA) : Set the port number of the target device for receiving data from it. The default value is 61000 for Client/Fixed-MAC mode or 4000 for Server mode.

8) Target port number (FPGA->Target) : Set the port number of the target device for sending data to it. The Default value is 60000 for Client/Fixed-MAC mode or 4000 for Server mode.

To change any of these parameters, the user can set them using the menu option [1].

Note: Ensure that the FPGA and Target parameters of the two FPGA boards are cross-set. Specifically, the Target parameters of the first board must match the FPGA parameters of the second board, and vice versa.

4.2 Reset UDPIP parameters

This menu contains seven or eight parameters that must be set. Each parameter is validated before being loaded into the UDP40G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in section 4.1 (Display UDPIP parameter), and their valid ranges are given below.

Note:

i) Ensure that one of the following modes is set on both FPGAs: Server <–> Client, Client <–> Fixed-MAC, or Fixed-MAC <–> Fixed-MAC.

ii) When operating in Server <–> Client mode, if parameters on the Server need to be reset, the Client FPGA must also be reset. Additionally, the Server must be reset before the Client to ensure it waits until the ARP request is sent by the Client.

iii) The Target parameters of the first board must match the FPGA parameters of the second board, and vice versa.

iv) The Target port number for Target->FPGA and FPGA->Target must be the same value, which is equal to the FPGA port number of the other board.

1) Mode : Input ‘0’ (Client), ‘1’ (Server), or ‘2’ (Fixed-MAC) to determine FPGA initialization mode.

2) FPGA MAC address : Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.

3) FPGA IP address : Input four decimal digits separated by “.”, where the valid range of each decimal digit is 0-255.

4) FPGA port number : The valid range is 0-65535.

5) Target MAC address (displayed only when running Fixed-MAC mode) : Input 12 digits of hex value, and add “0x” as a prefix to input as hex value.

6) Target IP address : Similar to FPGA IP address, this value is a set of four decimal digits.

7) Target port number (Target->FPGA) : The valid range is 0-65535.

8) Target port number (FPGA->Target) : The valid range is 0-65535.

Figure 4‑2 Change IP parameter result for Server/Client mode

Figure 4‑3 Change IP parameter result for Fixed-MAC mode

4.3 Send and Receive Data Test (Half-duplex test)

Upon using two FPGA boards for data transfer in a single direction, choose option ‘2’ to initiate the send data test on the first board, and select option ‘3’ to initiate the receive data test on the other board. The user inputs the test parameters for each FPGA board by following the steps outlined below.

1) On the FPGA#1 console (Receive data test), the user must input two parameters.

b) Data verification mode: Set to ‘1’ to enable data verification to verify received data.

2) If all inputs are valid, “Wait data from Target ...” is displayed, indicating that the system is waiting to receive data from the other FPGA.

3) On the FPGA#2 console (Send data test), the user must input two parameters.

Note: If packet byte size is greater than 1472, the packet output from UDP40G-IP is a jumbo frame. If two FPGA boards connect through other network devices, ensure that the network devices support jumbo frame.

4) If all inputs are valid, “Press any key to start data sending ...” is displayed, awaiting the user’s input.

5) On the FPGA#2 console, the user inputs key(s) to start the data sending process. During data transmission, both FPGA consoles continuously display the current size of the transferred data every second.

6) Upon the completion of data transmission, “Send data complete” is displayed on the FPGA#2 console. Meanwhile, on the FPGA#1 console, “Receive data completed” appears once data reception concludes. Finally, both consoles display the total number of transferred data and the performance achieved.

Figure 4‑4 and Figure 4‑5 illustrate the results of half-duplex data transfer in an FPGA-FPGA environment utilizing jumbo frame and non-jumbo frame sizes, respectively. The performance achieved with jumbo frame size surpasses that of non-jumbo frame size by 4%. When employing FPGA-FPGA communication, a data transfer rate of 4900 MB/sec is attained.

In case of invalid inputs, the console displays “Out-of-range input”, and the operation is subsequently cancelled, as illustrated in Figure 3‑6 (similar to the FPGA<->PC test).

Figure 4‑4 Send/receive data test by using jumbo frame

Figure 4‑5 Send/receive data test by using non-jumbo frame

4.4 Full-duplex Test

Choose option ‘4’ to transfer data simultaneously in both directions using the same port number on both FPGAs. The user inputs the test parameters for FPGA board by following the steps outlined below.

1) On the FPGA#1 console, the user must input three parameters to initiate full duplex test.

c) Data verification mode: Set to ‘1’ to enable data verification to verify data sent from the FPGA#2.

2) If all inputs are valid, the console will display one of two messages based on the initialization mode.

a) For Client and Fixed-MAC modes, “Press any key to start data transferring ...” is displayed, prompting the user to input any key to start the data sending process.

b) For Server mode, “Wait data from Target …” is displayed, indicating that the system is awaiting data from the target before initiating the data sending process.

3) On the FPGA#2 console, the user must input three parameters to initiate the full duplex test, similar to step 1). Ensure that the input values of transfer size and data verification mode match the values set in step 1).

4) If all inputs are valid, the console will display one of two messages based on the initialization mode, similar to step 2).

5) On the FPGA consoles initialized by the Client and Fixed-MAC modes, the user enters any key(s) to start sending data. While data is being transferred, the current size of the transferred data in both directions is displayed on the FPGA consoles every second.

6) Upon completion of the data transfer, “Transfer data complete” is displayed on the FPGA consoles. Finally, the total number of transferred data and the performance in both transfer directions are displayed on both FPGA consoles.

Figure 4‑6 and Figure 4‑7 illustrate the results of full-duplex data transfer in an FPGA-FPGA environment utilizing jumbo frame and non-jumbo frame sizes, respectively. The performance results of full-duplex transfer match the results of half-duplex transfer, as illustrated in the preceding section.

Figure 4‑6 Full-duplex test when using jumbo frame

Figure 4‑7 Full-duplex test when using non-jumbo frame

5 Revision History

Revision	Date	Description
2.00	13-Mar-24	Remove board setup
1.00	28-Oct-19	Initial version release