TOE1G-IP with CPU Demo Instruction
2.1 Setting the IP address of the NIC
2.2 Enabling Jumbo Frame and 1G Ethernet speed of the NIC
3 Test result when using FPGA and TestPC
4 Test result when using two FPGAs
4.3 Send Data Test (server to client)
4.4 Receive Data Test (Client to Server)
1 Overview
This document illustrates an example of running the TOE1G-IP demo using two different test environments to transfer TCP data. The first test environment employs a single FPGA board to transfer TCP data with a PC that runs a test application for transferring TCP data over 1G Ethernet. The performance result may be constrained by the resources of the PC in the specific test environment. On the other hand, the best performance for transferring TCP data using TOE1G-IP can be achieved by utilizing two FPGA boards in the second test environment, where they transfer data to each other.
The document covers three topics, which include setting up the 1G Ethernet connection on the PC to achieve optimal performance for transferring data via 1G Ethernet in section 2. In section 3, the console example and test results are presented when operating under the first test environment, involving FPGA and PC. Lastly, section 4 shows the console example when operating the second test environment, involving FPGA and FPGA. Each topic is described in further detail as follows.
2 PC Setup
Before running demo, please check the network setting on your PC. The example for setting 1G Ethernet connection is described as follows.
2.1 Setting the IP address of the NIC
To set the IP address of the NIC, follow the steps below.
Figure 2‑1 Setting IP address of the NIC on PC
1) Open Local Area Connection Properties of the 1G Ethernet connection, as shown in the left window of Figure 2‑1.
2) Select “TCP/IPv4” and click on Properties.
3) Set the IP address = 192.168.11.25 and the Subnet mask = 255.255.255.0, as shown in the right window of Figure 2‑1.
2.2 Enabling Jumbo Frame and 1G Ethernet speed of the NIC
Figure 2‑2 Jumbo frame setting
1) On Local Area Connection Properties window, click on “Configure”, as shown in the left windows of Figure 2‑2.
2) On Advanced Tab, select “Jumbo Packet” and set Value to “9014 Bytes” for Jumbo Frame support, as shown in the right windows of Figure 2‑2.
Note: Setting “Disabled” to disabling Jumbo frame may reduce the performance, so it is advisable to enable it.
3) On Link Speed Tab, set “1 Gbps Full Duplex” and then click OK button, as shown in Figure 2‑3.
Figure 2‑3 Set link speed = 1 Gbps
2.3 Power Option Setting
1) Open Control Panel and select Power Options as shown in the left window of Figure 2‑4.
2) Change setting to High Performance as shown in the right window of Figure 2‑4.
Figure 2‑4 Power options
3 Test result when using FPGA and TestPC
3.1 Display TCPIP parameters
Choose option ‘0’ to display the TCP/IP parameters. The console will show eight parameters.
Figure 3‑1 Display current parameter result
Here are the details of the parameters.
2) Reverse Packet: This flag enables the IP to send the retransmitted packet when the IP waits for a long time for the Window update packet returned from the target. The default value is ENABLE.
3) Mode: This parameter sets mode to TOE1G-IP to initialize in Server or Client. To initialize the IP in Client mode to run with the PC, input ‘0’.
4) FPGA MAC address: This parameter sets the 48-bit hex value to be the MAC address of the FPGA. The default value is 0x000102030405.
5) FPGA IP: This parameter sets the IP address of FPGA. The default value is 192.168.11.42.
Note: This value is used as a Server IP address parameter for the test application on PC.
6) FPGA port number: This parameter sets the port number of the FPGA. The default value is 60000.
Note: This value is used as a Server port parameter for the test application on PC.
7) Target IP: This parameter sets the IP address of the target device (1G Ethernet on PC). The default value is 192.168.11.25.
8) Target port number: This parameter sets the port number of the target device to transfer TCP payload data. The default value is 60001.
To change any of these parameters, the user can set them by using the menu option [1].
3.2 Reset TCPIP parameters
This menu contains eight parameters that must be set. Each parameter is validated before being loaded into the TOE1G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in section 3.1(Display TCPIP parameter), and their valid ranges are given below.
1) Mode: Input ‘0’ to initialize the IP as Client mode.
2) Reverse Packet: Set ‘0’ to disable or ‘1’ to enable this feature.
3) Window Update Gap: Set threshold value to transmit window update packet. The valid range is 0x000 – 0x3F (0-63), and the unit size of the threshold value is 1Kbyte. The default value is 0 (disable window update feature).
4) FPGA MAC address: Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.
5) FPGA IP address: Input four decimal digits separated by “.”, where the valid range for each digit is 0-255.
6) FPGA port number: The valid range is 0-65535.
7) Target IP address: Similar to FPGA IP address, this value is a set of four decimal digits.
8) Target port number: The valid range is 0-65535.
After finishing assigning the parameters, the final values are displayed on the console. Then, the reset signal is sent to the IP, and it initializes using the new parameters. Finally, “IP initialization complete” is displayed on the console once the initialization process is completed, as shown in Figure 3‑2.
Figure 3‑2 Change IP parameter result
3.3 Send Data Test
Choose option ‘2’ from the menu to send data from an FPGA to a PC. The test application, “tcpdatatest.exe”, is called on the PC with specified parameters via Command prompt for receiving data. On the other hand, the user inputs the test parameters for sending data on the FPGA console. The steps to run the test are described below.
1) On FPGA console, the user must input three parameters to initiate the send data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is decimal unit when inputting only digit number. User should add “0x” as a prefix for hexadecimal units.
b) Packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
Note: If packet size is greater than 1460, the packet output from TOE1G-IP is a jumbo frame. In this case, the PC must support jumbo frame.
c) Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the recommended parameters to run test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.
3) On the Command prompt, input test parameters following the recommended value. There are six parameters for “tcpdatatest”.
>> tcpdatatest <mode> <dir> <server IP> <server port> <bytelen> <pattern>
a) Mode: Input ‘c’ to run Test PC as a Client.
b) Dir: Input ‘r’ to run the PC for receiving and verifying test data from the FPGA
c) Server IP: Input the same value as the IP address of the FPGA
d) Server port: Input the same value as the port number of the FPGA
e) Bytelen: Input the same value as “Input transfer size” of step 1a)
f) Pattern: Input ‘1’ to verify data from FPGA or ‘0’ to not verify data
4) After running the test application on the PC, the port is created and the current amount of transferred data is displayed on the console (transmitted data) and Command prompt (received data) every second.
5) The FPGA closes the connection. Then “Send data complete” message is displayed on the console. Finally, total transfer size and performance are displayed on the console (transmit performance) and Command prompt (receive performance).
Figure 3‑3 shows an example of the send data test with simple settings, which involves sending packets with non-jumbo frame size, enabling data verification on test application. The left window displays the FPGA console operating as Server, while the right window displays the Command prompt on the PC operating as Client.
To improve test performance, a maximum packet size which requires Jumbo frame support is used, as shown in Figure 3‑4. The result improves from this setting.
Figure 3‑3 Send data test by using non-jumbo frame
Figure 3‑4 Send data test by using jumbo frame
If the input is invalid, “Out-of-range input” or “Invalid input” is displayed. After that, the operation is cancelled, as shown in Figure 3‑5 - Figure 3‑7.
Figure 3‑5 Error from invalid transfer size
Figure 3‑6 Error from invalid packet size
Figure 3‑7 Error from invalid mode
3.4 Receive Data Test
Choose option ‘3’ from the menu to receive data sent by a PC to an FPGA. The test application, “tcpdatatest.exe”, are called on the PC with specified parameters via Command prompt for sending data. On the other hand, the user inputs the test parameters for receiving and verifying data on the FPGA console. The steps to run the test are described below.
1) On the FPGA console, the user must input three parameters to initiate the receive data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
b) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.
c) Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.
3) On the Command prompt, input test parameters following the recommended value. There are six parameters for “tcpdatatest”.
>> tcpdatatest <mode> <dir> <server IP> <server port> <bytelen> <pattern>
a) Mode: Input ‘c’ to run the PC as a client.
b) Dir: Input ‘t’ to run the PC for sending test data to the FPGA
c) Server IP: Input the same value as the IP address of the FPGA
d) Server port: Input the same value as the port number of the FPGA
e) Bytelen: Input the same value as “Input transfer size” from step 1a)
f) Pattern: Input the same value as “Input data verification mode” from step 1b). Select ‘0’ to send dummy data or ‘1’ to send incremental data.
4) After running the test application on the PC, a port is created, and the current amount of transferred data is displayed on the console (received data) and the Command prompt (transmitted data) every second.
5) Once the PC finishes sending the data and closing the connection, the FPGA console displays “Connection closed” and “Received data completed”. Finally, the total transfer size and performance are displayed on the FPGA console (receive performance) and the Command prompt (transmit performance).
Figure 3‑8 Receive data test with normal settings
To achieve maximum performance in the receive data test for the specific test environment, the incremental data generation is turned off, and dummy data is transmitted, resulting in the best-case scenario for the test application settings, as shown in Figure 3‑9.
Figure 3‑9 Receive data test with the setting of disabling data verification
Figure 3‑10 provides an error example resulting from data verification failure. The error is caused by a mismatch between the verification mode values of the FPGA and the PC. The FPGA enables data verification, while the “tcpdatatest” application is configured to send dummy data. The error message is displayed on the FPGA console.
Figure 3‑10 Receive data test of data verification failure
3.5 Full duplex Test
Choose option ‘4’ from the menu to perform a full duplex test on the FPGA and PC to transfer data in both directions simultaneously. The user inputs test parameters on the FPGA console and the PC Command prompt. The “tcp_client_txrx_xg” application is called on the PC to send and receive data using the same port number. The sequence to run the test is outlined below.
1) On the FPGA console, the user must input four parameters to initiate the full duplex test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1-0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
b) Input packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
Note: If packet size is greater than 1460, the packet output from TOE1G-IP is a jumbo frame. In this case, the PC must support jumbo frame.
c) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the PC.
d) Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the recommended parameters to run the test application on the PC will be displayed. Next, “Wait Open connection …” will be displayed to wait until the application is run on PC.
3) On the Command prompt, input test parameters following the recommended value. There are four parameters for “tcp_client_txrx_xg”.
>> tcp_client_txrx_xg <server IP> <server port> <bytelen> <pattern>
a) Server IP: Input the same value as the IP address of the FPGA
b) Server port: Input the same value as the port number of the FPGA
c) ByteLen: Input the same value as “Input transfer size” from step 1a).
d) Pattern: Input the same value as “Input data verification mode” from step 1c).
· ‘0’ to send dummy data and do nothing with the received data.
· ‘1’ to send incremental data and verify the received data.
4) After running the test application, the port is created, and the current amount of transferred data is displayed on FPGA console and Command prompt every second.
5) The PC (run as Client) closes the connection after transferring the data in both directions. Finally, the total transfer size and performance are displayed on the FPGA console and Command prompt.
There are a two-second time gap after step 5) for user to stop the operation by pressing any keys on both consoles. Otherwise, it repeats step 4) – 5) in a forever loop.
Examples of running full-duplex tests with and without data verification are illustrated in Figure 3‑11 and Figure 3‑12, respectively. In specific test environment, disabling the data verification yield better overall performance than enabling it. The FPGA console operates as a Client on the left window, while the Command prompt on the PC functions as a Server on the right window.
Figure 3‑11 Full duplex test with enabling data verification
Figure 3‑12 Full duplex test with disabling data verification
4 Test result when using two FPGAs
4.1 Display TCPIP parameters
Choose option ‘0’ to display the TCP/IP parameters. The console will show eight parameters.
Figure 4‑1 Display current parameter result
Here are the details of the parameters.
1) Window Update Gap: This sets the threshold value for transmitting a window update packet. The valid value range is 0x00 – 0x3F (0-63) and the threshold value unit is 1Kbyte. The default value is 0 (disable window update feature).
2) Reverse Packet: This flag enables the IP to send the retransmitted packet when the IP waits for a long time for the Window update packet returned from the target. The default value is ENABLE.
3) Mode: This parameter sets the mode to TOE1G-IP to initial in Server or Client. Input ‘0’ for Client or ‘1’ for Server.
4) FPGA MAC address: This parameter sets the 48-bit hex value to be the MAC address of the FPGA. The default value is 0x000102030405 (Client mode) or 0x001122334455 (Server mode).
5) FPGA IP: This parameter sets the IP address of the FPGA. The default value is 192.168.11.42 (Client mode) or 192.168.11.25 (Server mode).
6) FPGA port number: This parameter sets the port number of the FPGA. The default value is 60000 (Client mode) or 60001 (Server mode).
7) Target IP: This parameter sets the IP address of the target device. The default value is 192.168.11.25 (Client mode) or 192.168.11.42 (Server mode).
8) Target port number: This parameter sets the port number of the target device to transfer TCP payload data. The default value is 60001 (Client MAC mode) or 60000 (Server mode).
To change any of these parameters, the user can set them using the menu option [1] (Reset TCPIP parameters).
4.2 Reset TCPIP parameters
Choose option ‘1’ from the menu to reset the IP and modify IP parameters. This menu allows user to change IP settings or reset the TOE1G-IP. Upon selection of this option, the current parameters are displayed on the console. Press ‘x’ to keep the same parameters, or press any other key to modify them. Once the parameters are confirmed, the TOE1G-IP is reset and the initialization process begins.
This menu contains eight parameters that must be set. Each parameter is validated before being loaded into the TOE1G-IP. If the input is invalid, the parameter remains unchanged. Once all parameters have been loaded, the IP is reset. The details of each parameter are described in section 4.1(Display TCPIP parameter), and their valid ranges are given below.
Note:
1. Please ensure that two FPGAs are configured in different initialization modes, Server and Client.
2. If the parameters on the Server need to be reset, the Client FPGA must also be reset. Additionally, the Server must be reset before the Client to ensure that it waits until the ARP request is sent by the Client.
3. It is essential to match the parameters of both FPGAs as listed below.
a. The Target IP address of board#1 = The FPGA IP address of board#2
b. The FPGA IP address of board#1 = The Target IP address of board#2
c. The Target port number of board#1 = The FPGA port number of board#2
d. The FPGA port number of board#1 = The Target port number of board#2
1) Mode: Input ‘0’ (Client) or ‘1’ (Server) to determine FPGA initialization mode.
2) Reverse Packet: Set ‘0’ to disable or ‘1’ to enable this feature.
3) Window Update Gap: Set threshold value for transmitting a window update packet. The valid range is 0x00 – 0x3F (0-63), and the unit size of the threshold value is 1Kbyte. The default value is 0 (disable window update feature).
4) FPGA MAC address: Input 12 digits of hex value, and add “0x” as a prefix to input it as a hex value.
5) FPGA IP address: Input four decimal digits separated by “.”, where the valid range for each digit is 0-255.
6) FPGA port number: The valid range is 0-65535.
7) Target IP address: Similar to FPGA IP address, this value is a set of four decimal digits.
8) Target port number: The valid range is 0-65535.
After finishing assigning the parameters, the final values are displayed on the console. Then, the reset signal is sent to the IP, and it initializes using the new parameters. Finally, “IP initialization complete” is displayed on the console once the initialization process is complete, as shown in Figure 4‑2.
Figure 4‑2 Change IP parameter result
4.3 Send Data Test (server to client)
To transmit data from the Server FPGA to the Client FPGA, choose option ‘2’ to initiate the send data test on the Server FPGA, and select option ‘3’ to initiate the receive data test on the Client FPGA. The user can input test parameters on the consoles. The test sequence is outlined below.
Note: The modes configured in the Send data test, Receive data test, and Full duplex test, for communication between two FPGAs determine how the connection is opened and closed. These modes can be configured as active mode (Client) or passive mode (Server). However, this setting is not related to the initialization mode, which is used to configure the FPGAs as a Client or Server mode. It is important to set the two FPGAs to different connection modes, with one being set as the Client and the other as the Server.
1) On the Server console, the user must input three parameters to initiate the send data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
b) Packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
c) Mode: This is the connection mode. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request. If any inputs are invalid, the console will display “Out-of-range input” or “Invalid input”, and the operation will be cancelled, similar to the test with PC (as shown in Figure 3‑5 - Figure 3‑7).
3) On the Client console, three parameters must be inputted to initiate the receive data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).
b) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Server FPGA.
c) Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).
4) Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size on both consoles every second. After sending all data, “Send data complete” is displayed on the Server console.
5) The Server FPGA then closes the connection, and the total transfer size and performance are displayed on both consoles.
In Figure 4‑3, the Server console is displayed on the left window, while the right window displays the Client console. This example demonstrates the performance of the send data test using a non-jumbo frame size.
When a jumbo frame size is utilized as the packet size, the performance result reveals an improvement compared to using a normal packet size. In fact, it can achieve maximum performance over a 1G Ethernet, as demonstrated in Figure 4‑4.
Figure 4‑3 Send data test by using non-jumbo frame
Figure 4‑4 Send data test by using jumbo frame
4.4 Receive Data Test (Client to Server)
This section describes the steps for transferring data from the Client FPGA to the Server FPGA. The Server FPGA must be initiated by selection option ‘3’ to run received data test before entering option ‘2’ to initiate the send data test at the Client FPGA. The test sequence is outlined below.
1) On the Server console, three parameters must be inputted to initiate the receive data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
b) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Client FPGA.
c) Mode: This is the connection mode. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request.
3) On the Client console, three parameters must be inputted to initiate the send data test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).
b) Packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
c) Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).
4) Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size on both consoles every second. After sending all data, “Send data complete” is displayed on the Client console.
5) The Client FPGA then closes the connection, and the total transfer size and performance are displayed on both consoles.
The left window in Figure 4‑5 shows the Server console, and the right window displays the Client console. Similar to Figure 4‑4, the performance achieved over 1G Ethernet is also maximum when using Jumbo frame size.
Figure 4‑5 Receive data test with data verification
4.5 Full duplex Test
Choose option ‘4’ to transfer data simultaneously in both directions and run a full duplex test on both the Server FPGA and the Client FPGA. The user can input test parameters on the consoles. The following sequence illustrates how to run the test.
1) On the Server console, the user must input four parameters to initiate the full duplex test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - 0xFFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
b) Packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
c) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Client FPGA.
d) Mode: This is the connection mode of the FPGA. Input ‘1’ to open connection by Server mode (Passive open).
2) If all inputs are valid, the console will display “Wait Open connection …” and wait for the Client FPGA sending a new connection request.
3) On the Client console, four parameters must be inputted to initiate the full duplex test.
a) Transfer size: The unit of transfer size is byte. The valid range is 1 - FFFF_FFFF. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units. This value must be equal to the transfer size specified on the Server FPGA at step 1a).
b) Packet size: The unit of packet size is byte. The valid range is 1 – 8960. The input is a decimal unit when inputting only digit number. The user should add “0x” as a prefix for hexadecimal units.
c) Data verification mode: Set to ‘0’ to disable data verification or ‘1’ to enable data verification to verify data sent from the Server FPGA.
d) Mode: This is the connection mode. Input ‘0’ to open connection by Client mode (Active open).
4) Once all inputs are valid, the operation begins. The port is created and the console displays the current transfer size of both transfer directions on both consoles every second.
5) After transferring all data in both directions, the Client FPGA closes the connection. Finally, both consoles display the total transfer size and performance achieved.
There are a two-second time gap after step 5) for user to stop the operation by pressing any keys on both consoles. Otherwise, it repeats step 4) – 5) in a forever loop.
Figure 4‑6 illustrates a full-duplex test between two FPGAs. The left window shows the Server console, while the right window displays the Client console. With the use of the jumbo frame size as the packet size transferred between the two, the full-duplex test attains its maximum performance.
Figure 4‑6 Full duplex test with data verification
5 Revision History
|
Revision |
Date |
Description |
|
2.1 |
10-Oct-23 |
Update the test result using ‘tcp_client_txrx_xg’ application. |
|
2.0 |
31-Jul-20 |
Remove hardware setup from the document |
|
1.0 |
2-Nov-18 |
Initial version release |