TOE100G-IP with CPU Demo Instruction

 

1      Overview. 2

2      Test PC Setup. 3

2.1      Setting the IP address of the NIC. 3

2.2      Enabling Jumbo Frame of the NIC. 4

2.3      Disabling Flow Control and Interrupt Moderation of the NIC  5

2.4      Checking Power options Plan. 6

3      Test result when using FPGA and PC. 7

3.1      Display TCPIP parameters. 7

3.2      Reset TCPIP parameters. 8

3.3      Send Data Test 10

3.4      Receive Data Test 15

3.5      Full duplex Test 19

4      Test result when using two FPGAs. 21

4.1      Display TCPIP parameters. 21

4.2      Reset TCPIP parameters. 22

4.3      Send Data Test (Server to Client) 25

4.4      Receive Data Test (Client to Server) 27

4.5      Full duplex Test 29

5      Revision History. 31

 

 

1       Overview

 

This document illustrates an example of running the TOE100G-IP demo using two different test environments to transfer TCP data. The first test environment employs a single FPGA board to transfer TCP data with a PC that runs a test application for transferring TCP data over 100G Ethernet. The performance result in this environment is constrained by the resources of the PC. On the other hand, the best performance for transferring TCP data using TOE100G-IP can be achieved by utilizing two FPGA boards in the second test environment, where they transfer data to each other.

 

To optimize TCP data transfer performance, the test conducted in this document utilized the TOE100G-IP design with a buffer size of 1MB for both transmit and receive buffers. Other configurations may not attain the same level of transfer performance as this particular configuration. For additional information on buffer size settings for TOE100G-IP, please refer to the TOE100G-IP datasheet, which is available on our website.

 

The document covers three topics, which include setting up the 100G Ethernet card on the PC to achieve optimal performance for transferring data via 100G Ethernet in topic 2. In topic 3, the console example and test results are presented when operating under the first test environment, involving FPGA and PC. Lastly, topic 4 shows the console example when operating the second test environment, involving FPGA and FPGA. Each topic is described in further detail as follows.

 

 

2       Test PC Setup

 

This section provides an example setting of the 100G standard NIC on a PC to enable Jumbo frame and boost network performance on Windows 10 OS.

 

2.1      Setting the IP address of the NIC

To set the IP address of the NIC, follow the steps below.

 

 

Figure 21 Setting IP address of the NIC on PC

 

 

1)  Open the Local Area Connection Properties of the 100G Ethernet connection, as shown in the left window of Figure 2‑1.

2)  Select “TCP/IPv4” and click on Properties.

3)  Set the IP address = 192.168.100.25 and the Subnet mask = 255.255.255.0, as shown in the right window of Figure 2‑1.

 

 

2.2      Enabling Jumbo Frame of the NIC

To enable Jumbo Frame of the NIC, follow the steps below.

 

 

Figure 22 Set frame size = Jumbo frame

 

 

1)  On the Local Area Connection Properties window, click on “Configure”, as shown in the left window of Figure 2‑2.

2)  On the Advanced Tab, select “Jumbo Packet” and set the Value to “9014 Bytes” for Jumbo Frame support, as shown in the right window of Figure 2‑2.

Note: Setting “Disabled” to disabling Jumbo frame may reduce the performance, so it is advisable to enable it.

 

 

2.3      Disabling Flow Control and Interrupt Moderation of the NIC

To disable the flow control and interrupt moderation of the NIC, follow the steps below.

 

 

Figure 23 Disable Flow Control and Interrupt Moderation

 

 

1)  Select “Flow Control” and set the value to “Disabled”, as shown in the left window of Figure 2‑3.

2)  Select “Interrupt Moderation” and set the value to “Disabled”, as shown in the right window of Figure 2‑3.

3)  Click the “OK” button to save and exit all the setting windows.

 

 

2.4      Checking Power options Plan

To check the Power Options Plan, follow the steps below.

 

 

Figure 24 Power options

 

 

1)  Open Control Panel and select Power Options, as shown in the left window of Figure 2‑4.

2)  Change the setting to “High Performance”, as shown in the right window of Figure 2‑4.

 

 

3       Test result when using FPGA and PC

 

3.1      Display TCPIP parameters

 

Choose option ‘0’ to display the TCP/IP parameters. The console will show either eight parameters in Client/Server mode or nine parameters in Fixed MAC mode.

 

 

Figure 31 Display current parameter result

 

 

Here are the details of the parameters.

 

1)  Window Update Gap: This sets the threshold value for transmitting a window update packet. The valid value range is 0x000 – 0x3FF (0-1023) and the threshold value unit is 1Kbyte. The default value is 16.

2)  Reverse Packet: This flag enables the IP to send the retransmitted packet when the IP waits for a long time for the Window update packet returned from the target. The default value is ENABLE.

3)  Mode: This parameter sets the mode to TOE100G-IP to initialize in Server, Client, or Fixed MAC. To initialize the IP in Client mode to run with the PC, input ‘0’.

4)  FPGA MAC address: This parameter sets the 48-bit hex value to be the MAC address of the FPGA. The default value is 0x000102030405.

5)  FPGA IP: This parameter sets the IP address of the FPGA. The default value is 192.168.100.42.

Note: This value is used as a Server IP address parameter for the test application on PC.

6)  FPGA port number: This parameter sets the port number of the FPGA. The default value is 60000.

Note: This value is used as a Server port parameter for the test application on PC.

7)  Target MAC address (displayed when running Fixed MAC mode only): This parameter sets the 48-bit hex value to be the MAC address of the target device. The default value is 0x554433221100.

8)  Target IP: This parameter sets the IP address of the target device (100G Ethernet on PC). The default value is 192.168.100.25.

9)  Target port number: This parameter sets the port number of the target device to transfer TCP payload data. The default value is 60001.

To change any of these parameters, the user can set them using the menu option [1].

 

 

3.2      Reset TCPIP parameters

 

Choose option ‘1’ from the menu to reset the IP and modify IP parameters. This menu allows user to change IP settings or reset the TOE100G-IP. Upon selection of this option, the current parameters are displayed on the console. Press ‘x’ to keep the same parameters, or press any other key to modify them. Once the parameters are confirmed, the TOE100G-IP is reset and the initialization process begins.

 

This menu contains eight or nine parameters that must be set. Each parameter is validated before being loaded into the TOE100G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in topic 3.1(Display TCPIP parameter), and their valid ranges are given below.

 

1)  Mode: Input ‘0’ to initialize the IP as client mode.

Note: When the PC and FPGA are connected to different networks, which cannot communicate via the ARP process, the TOE100G-IP must be run in Fixed MAC mode to manually set the MAC address via the console.

2)  Reverse Packet: Set ‘0’ to disable or ‘1’ to enable this feature.

3)  Window Update Gap: Set threshold value for transmitting a window update packet. The valid range is 0x000 – 0x3FF (0-1023), and the unit size of the threshold value is 1Kbyte. The default value is 16.

4)  FPGA MAC address: Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.

5)  FPGA IP address: Input four decimal digits separated by “.”, where the valid range for each digit is 0-255.

6)  FPGA port number: The valid range is 0-65535.

7)  Target MAC address (displayed only when running Fixed MAC mode): Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.

8)  Target IP address: Similar to FPGA IP address, this value is a set of four decimal digits.

9)  Target port number: The valid range is 0-65535.

 

After finishing assigning the parameters, the final values are displayed on the console. Then, the reset signal is sent to the IP, and it initializes using the new parameters. Finally, “IP initialization complete” is displayed on the console once the initialization process is complete, as shown in Figure 3‑2.

 

 

Figure 32 Change IP parameter in Client mode result

 

 

3.3      Send Data Test

 

Choose option ‘2’ from the menu to send data from an FPGA to a PC. The test application, “tcpdatatest.exe”, are called on the PC with specified parameters via Command prompt for receiving data. On the other hand, the user inputs the test parameters for sending data on the FPGA console. The steps to run the test are described below.

 

1)  On the FPGA console, the user must input three parameters to initiate the send data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

Note: If packet size is greater than 1408, the packet output from TOE100G-IP is a jumbo frame. In this case, the PC must support jumbo frame.

c)   Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.

3)  On the Command prompt, input test parameters following the recommended value except for the last two parameters which can be adjusted for performance testing. There are seven parameters for “tcpdatatest”.

>> tcpdatatest <mode> <dir> <server IP> <server port> <bytelen> <pattern> <buffer>

a)  Mode: Input ‘c’ to run the PC as a client.

b)  Dir: Input ‘r’ to run the PC for receiving and verifying test data from the FPGA

c)   Server IP: Input the same value as the IP address of the FPGA

d)  Server port: Input the same value as the port number of the FPGA

e)  Bytelen: Input the same value as “Input transfer size” from step 1a)

f)    Pattern: Input ‘1’ to verify data from the FPGA or ‘0’ to not verify data

g)  Buffer: Input a number between 1-5 (64KB – 1MB) as a buffer size of the application.

4)  After running the test application on the PC, the port is created and the current amount of transferred data is displayed on the console (transmitted data) and Command prompt (received data) every second.

5)  The FPGA closes the connection. Then “Send data complete” message is displayed on the console. Finally, total transfer size and performance are displayed on the console (transmit performance) and Command prompt (receive performance).

 

Figure 3‑3 shows an example of the send data test with simple settings, which involves sending packets with non-jumbo frame size, enabling data verification on the test application, and utilizing the minimum buffer size of the test application (64Kbyte). The left window displays the FPGA console operating as Server, while the right window displays the Command prompt on the PC operating as Client.

 

 

Figure 33 Send data test with normal settings

 

 

 

Figure 34 Send data test with performance-aimed settings

 

 

To improve test performance beyond the example shown in Figure 3‑4, several settings must be adjusted. On the PC, the “tcpdatatest” application must disable data verification and utilize a large buffer of 512Kbyte (Buffer=4) to reduce CPU usage and enhance the TCP stack operation. Additionally, on the TOE100G-IP side, a maximum packet size which requires Jumbo frame support on the 100G Ethernet network is used to further increase performance. The result is much improved from these settings.

 

 

Figure 35 Send data test with performance-aimed settings and adjusted packet size

 

 

The optimal packet size for achieving the best performance in the test environment may vary depending on the specific testing conditions. In Figure 3‑5, for instance, a packet size of 4800 bytes was found to yield the best results. Nonetheless, it is important to note that this parameter may differ in other testing scenarios.

 

Despite the maximum value of the “Buffer” parameter being 5 or 1 MB on the PC’s test environment, it has been found that setting the buffer size to 1 MB can lead to significant performance drops. Figure 3‑6 shows an example of the console used to set up the test application on the PC, where some of the TCP packets sent from the FPGA are discarded. The console labeled “PC Command Prompt#1” is used to monitor the network status.

 

 

Figure 36 Send data test with settings that causes the packets to be discarded

 

 

In this test, the test application is configured to disable the data verification and uses a buffer size of 1MB, while the TOE100G-IP’s packet size is set to 4800 bytes. The results of this configuration show a very low speed of TCP data transfer, lower than the simple setting test displayed in Figure 3‑3. Investigation using the “netstat” command through the lower Command Prompt (PC Command Prompt#1) reveals that some of the TCP packets are being discarded by the Ethernet card.

 

When the test application’s buffer size is set to 1MB, TOE100G-IP can send a large amount of data in a very short time, which can overload the network traffic that the 100G Ethernet card can handle. Alternatively, the immediate buffer between the Ethernet card and CPU may be insufficient to support the large amount of data. In some settings, the Test PC may not discard the TCP packets, resulting in very high performance in TCP data transfer over 100G Ethernet.

 

If the input is invalid, “Out-of-range input” or “Invalid input” is displayed. After that, the operation is cancelled, as shown in Figure 3‑7 - Figure 3‑8.

 

 

Figure 37 Error from invalid input

 

 

 

Figure 38 Error from out-of-range input

 

 

3.4      Receive Data Test

 

Choose option ‘3’ from the menu to receive data sent by a PC to an FPGA. The test application, “tcpdatatest.exe”, are called on the PC with specified parameters via Command prompt for sending data. On the other hand, the user inputs the test parameters for receiving and verifying data on the FPGA console. The steps to run the test are described below.

 

1)  On the FPGA console, the user must input three parameters to initiate the receive data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.

c)   Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.

3)  On the Command prompt, input test parameters following the recommended value except the last two parameters which can be adjusted for performance testing. There are seven parameters for “tcpdatatest”.

>> tcpdatatest <mode> <dir> <server IP> <server port> <bytelen> <pattern> <buffer>

a)  Mode: Input ‘c’ to run the PC as a client.

b)  Dir: Input ‘t’ to run the PC for sending test data to the FPGA

c)   Server IP: Input the same value as the IP address of the FPGA

d)  Server port: Input the same value as the port number of the FPGA

e)  Bytelen: Input the same value as “Input transfer size” from step 1a)

f)    Pattern: Input the same value as “Input data verification mode” from step 1b). Select ‘0’ to send dummy data or ‘1’ to send incremental data.

g)  Buffer: Input a number between 1-5 (64KB – 1MB) as a buffer size of the application.

4)  After running the test application on the PC, a port is created, and the current amount of transferred data is displayed on the console (received data) and the Command prompt (transmitted data) every second.

5)  Once the PC finishes sending the data and closing the connection, the FPGA console displays “Connection closed” and “Received data completed”. Finally, the total transfer size and performance are displayed on the FPGA console (receive performance) and the Command prompt (transmit performance).

 

Figure 3‑9 shows an example of a receive data test with a simple setting of “tcpdatatest” application on a PC. The application activates incremental data generation and uses a buffer size of 64 Kbytes (Buffer=1). The left window displays FPGA console (Client), while the right window displays the Command prompt on PC (Server).

 

 

Figure 39 Receive data test with normal settings

 

 

To achieve maximum performance in the receive data test, the incremental data generation is turned off, and dummy data is transmitted, resulting in the best-case scenario for the test application settings, as shown in Figure 3‑10. Unlike the send data test, adjusting the buffer size has no effect on the receive performance.

 

The results of the receive data test show higher performance compared to the send data test, indicating that the TCP stack on the PC is better suited for transmitting data than receiving it.

 

 

Figure 310 Receive data test with the setting of disabling data verification

 

 

Figure 3‑11 provides an error example resulting from data verification failure. The error is caused by a mismatch between the verification mode values of the FPGA and the PC. The FPGA enables data verification, while the “tcpdatatest” application is configured to send dummy data. The error message is displayed on the FPGA console.

 

 

Figure 311 Receive data test when data verification is failed

 

 

3.5      Full duplex Test

 

Choose option ‘4’ from the menu to perform a full duplex test on the FPGA and PC to transfer data in both directions simultaneously. The user inputs test parameters on the FPGA console and the PC Command prompt. The “tcp_client_txrx_xg” application is called on the PC to send and receive data using the same port number. The sequence to run the test is outlined below.

 

1)  On the FPGA console, the user must input four parameters to initiate the full duplex test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

Note: If packet size is greater than 1408, the packet output from TOE100G-IP is a jumbo frame. In this case, the PC must support jumbo frame.

c)   Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.

d)  Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.

3)  On the Command prompt, input test parameters following the recommended value except the last two parameters which can be adjusted for performance testing. There are five parameters for “tcp_client_txrx_xg”.

>> tcp_client_txrx_xg <server IP> <server port> <bytelen> <pattern> <buffer>

a)  Server IP: Input the same value as the IP address of the FPGA

b)  Server port: Input the same value as the port number of the FPGA

c)   ByteLen: Input the same value as “Input transfer size” from step 1a).

d)  Pattern: Input the same value as “Input data verification mode” from step 1c).

·     ‘0’ to send dummy data and do nothing with the received data.

·     ‘1’ to send incremental data and verify the received data.

e)  Buffer: Input a number between 1-5 (64KB – 1MB) as a buffer size of the application.

4)  After running the test application, the port is created, and the current amount of transferred data is displayed on FPGA console and Command prompt every second.

5)  The PC (run as Client) closes the connection after transferring the data in both directions. Finally, the total transfer size and performance are displayed on the FPGA console and Command prompt.

 

There are a two-second time gap after step 5) for user to stop the operation by pressing any keys on both consoles. Otherwise, it repeats step 4) – 5) in a forever loop.

 

Examples of running full-duplex tests with and without data verification are illustrated in Figure 3‑12 and Figure 3‑13, respectively. In both cases, the “tcp_client_txrx_xg” application is configured to use a buffer size of 512 Kbyte to optimized transmission performance from FPGA to the PC. Disabling the data verification yield better overall performance than enabling it. The FPGA console operates as a Client on the left window, while the Command prompt on the PC functions as a Server on the right window.

 

 

Figure 312 Full duplex test with enabling data verification

 

 

 

Figure 313 Full duplex test with disabling data verification

 

 

4       Test result when using two FPGAs

 

4.1      Display TCPIP parameters

 

Choose option ‘0’ to display the TCP/IP parameters. The console will show either eight parameters in Client/Server mode or nine parameters in Fixed MAC mode.

 

 

Figure 41 Display current parameter result

 

 

Here are the details of the parameters.

 

1)  Window Update Gap: This sets the threshold value for transmitting a window update packet. The valid value range is 0x000 – 0x3FF (0-1023) and the threshold value unit is 1Kbyte. The default value is 16.

2)  Reverse Packet: This flag enables the IP to send the retransmitted packet when the IP waits for a long time for the Window update packet returned from the target. The default value is ENABLE.

3)  Mode: This parameter sets the mode to TOE100G-IP to initial in Server, Client, or Fixed MAC. Input ‘0’ for Client, ‘1’ for Server, or ‘2’ for Fixed MAC.

4)  FPGA MAC address: This parameter sets the 48-bit hex value to be the MAC address of the FPGA. The default value is 0x000102030405 (Client/Fixed MAC mode) or 0x001122334455 (Server mode).

5)  FPGA IP: This parameter sets the IP address of the FPGA. The default value is 192.168.100.42 (Client/Fixed mode) or 192.168.100.25 (Server mode).

6)  FPGA port number: This parameter sets the port number of the FPGA. The default value is 60000 (Client/Fixed MAC mode) or 60001 (Server mode).

7)  Target MAC address (displayed when running Fixed MAC mode only): This parameter sets the 48-bit hex value to be the MAC address of the target device. The default value is 0x554433221100.

8)  Target IP: This parameter sets the IP address of the target device. The default value is 192.168.100.25 (Client/Fixed MAC mode) or 192.168.100.42 (Server mode).

9)  Target port number: This parameter sets the port number of the destination device to transfer TCP payload data. The default value is 60001 (Client/Fixed MAC mode) or 60000 (Server mode).

To change any of these parameters, the user can set them using the menu option [1] (Reset TCPIP parameters).

 

 

4.2      Reset TCPIP parameters

 

Choose option ‘1’ from the menu to reset the IP and modify IP parameters. This menu allows user to change IP settings or reset the TOE100G-IP. Upon selection of this option, the current parameters are displayed on the console. Press ‘x’ to keep the same parameters, or press any other key to modify them. Once the parameters are confirmed, the TOE100G-IP is reset and the initialization process begins.

 

This menu contains eight or nine parameters that must be set. Each parameter is validated before being loaded into the TOE100G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in topic 4.1 (Display TCPIP parameters), and their valid ranges are given below.

 

Note:

1.  Please ensure that one of the following modes is set on both FPGAs: Server – Client, Client – Fixed MAC, or Fixed MAC – Fixed MAC.

2.  When operating in Server – Client mode, if the parameters on the Server need to be reset, the Client FPGA must also be reset. Additionally, the Server must be reset before the Client to ensure that it waits until the ARP request is sent by the Client.

3.  It is essential to match the parameters of both FPGAs as listed below.

a.   The Target IP address of board#1 = The FPGA IP address of board#2

b.   The FPGA IP address of board#1 = The Target IP address of board#2

c.   The Target port number of board#1 = The FPGA port number of board#2

d.   The FPGA port number of board#1 = The Target port number of board#2

 

1)   Mode: Input ‘0’ (Client), ‘1’ (Server), or ‘2’ (Fixed MAC) to determine FPGA initialization mode.

2)   Reverse Packet: Set ‘0’ to disable or ‘1’ to enable this feature.

3)   Window Update Gap: Set threshold value for transmitting a window update packet. The valid range is 0x000 – 0x3FF (0-1023), and the unit size of the threshold value is 1Kbyte. The default value is 16.

4)   FPGA MAC address: Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.

5)   FPGA IP address: Input four decimal digits separated by “.”, where the valid range for each digit is 0-255.

6)   FPGA port number: The valid range is 0-65535.

7)   Target MAC address (displayed when running Fixed MAC mode only): Input 12 digits of hex value, and add “0x” as a prefix to input as a hex value.

8)   Target IP address: Similar to FPGA IP address, this value is a set of four decimal digits.

9)   Target port number: The valid range is 0-65535.

 

After finishing assigning the parameters, the final values are displayed on the console. Then, the reset signal is sent to the IP, and it initializes using the new parameters. Finally, “IP initialization complete” is displayed on the console once the initialization process is complete, as shown in Figure 4‑2 and Figure 4‑3.

 

 

Figure 42 Change IP parameter result for Server and Client mode

 

 

 

Figure 43 Change IP parameter result for Fixed MAC mode

 

 

4.3      Send Data Test (Server to Client)

 

To transmit data from the Server FPGA to the Client FPGA, choose option ‘2’ to initiate the send data test on the Server FPGA, and select option ‘3’ to initiate the receive data test on the Client FPGA. The user can input test parameters on the consoles. The test sequence is outlined below.

 

Note: The modes configured in the Send data test, Receive data test, and Full duplex test, for communication between two FPGAs determine how the connection is opened and closed. These modes can be configured as active mode (Client) or passive mode (Server). However, this setting is not related to the initialization mode, which is used to configure the FPGAs as a Client, Server, or Fixed-MAC mode. It is important to set the two FPGAs to different connection modes, with one being set as the Client and the other as the Server.

 

1)  On the Server console, the user must input three parameters to initiate the send data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

c)   Mode: This is the connection mode. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request. If any inputs are invalid, the console will display “Out-of-range input” or “Invalid input”, and the operation will be cancelled, similar to the test with PC (as shown in Figure 3‑7 - Figure 3‑8).

3)  On the Client console, three parameters must be inputted to initiate the receive data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).

b)  Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Server FPGA.

c)   Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).

4)  Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size on both consoles every second. After sending all data, “Send data complete” is displayed on the Server console.

5)  The Server FPGA then closes the connection, and the total transfer size and performance are displayed on both consoles.

 

In Figure 4‑4, the Server console is displayed on the left window, while the right window displays the Client console. This example demonstrates the performance of the send data test using a non-jumbo frame size.

 

When a jumbo frame size is utilized as the packet size, the performance result reveals an improvement compared to using a normal packet size. In fact, it can achieve maximum performance over a 100G Ethernet, as demonstrated in Figure 4 5.

 

 

Figure 44 Send data test by using non-jumbo frame

 

 

 

Figure 45 Send data test by using jumbo frame

 

 

4.4      Receive Data Test (Client to Server)

 

This section describes the steps for transferring data from the Client FPGA to the Server FPGA. The Server FPGA must be initiated by selection option ‘3’ to run received data test before entering option ‘2’ to initiate the send data test at the Client FPGA. The test sequence is outlined below.

 

1)  On the Server console, three parameters must be inputted to initiate the receive data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Client FPGA.

c)   Mode: This is the connection mode. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request.

3)  On the Client console, three parameters must be inputted to initiate the send data test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

c)   Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).

4)  Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size on both consoles every second. After sending all data, “Send data complete” is displayed on the Client console.

5)  The Client FPGA then closes the connection, and the total transfer size and performance are displayed on both consoles.

 

The left window in Figure 4‑6 shows the Server console, and the right window displays the Client console. Similar to Figure 4‑5, the performance achieved over 100G Ethernet is also maximum when using Jumbo frame size.

 

 

Figure 46 Receive data test with data verification

 

 

4.5      Full duplex Test

 

Choose option ‘4’ to transfer data simultaneously in both directions and run a full duplex test on both the Server FPGA and the Client FPGA. The user can input test parameters on the consoles. The following sequence illustrates how to run the test.

 

1)  On the Server console, the user must input four parameters to initiate the full duplex test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

c)   Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Client FPGA.

d)  Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).

2)  If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request.

3)  On the Client console, four parameters must be inputted to initiate the full duplex test.

a)  Transfer size: The unit of transfer size is byte. The valid range is 0x40 - 0x3F_FFFF_FFC0. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).

b)  Packet size: The unit of packet size is byte. The valid range is 64 – 8960. The input must be aligned to 64. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.

c)   Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Server FPGA.

d)  Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).

4)  Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size of both transfer directions on both consoles every second.

5)  After transferring all data in both directions, the Client FPGA closes the connection. Finally, both consoles display the total transfer size and performance achieved.

 

There are a two-second time gap after step 5) for user to stop the operation by pressing any keys on both consoles. Otherwise, it repeats step 4) – 5) in a forever loop.

 

Figure 4‑7 illustrates a full-duplex test between two FPGAs. The left window shows the Server console, while the right window displays the Client console. With the use of the jumbo frame size as the packet size transferred between the two, the full-duplex test attains its maximum performance.

 

 

Figure 47 Full duplex test example

 

 

5       Revision History

 

Revision

Date

Description

1.2

24-Apr-23

Update performance results from utilizing large buffer size, supporting window scaling, and updating TCP test application

1.1

11-Mar-22

Add reverse packet feature

1.0

25-Feb-21

Initial version release