FPGA setup for TOE/UDP25G-IP with CPU Demo

Rev2.2　15-Jun-23

 

1       Overview.. 2

2       Test environment setup when using FPGA and PC.. 4

3       Test environment setup when using two FPGAs. 13

4       Revision History. 18

 

 

1       Overview

 

This document provides a guide on setting up an FPGA board and preparing the necessary test environment to run the TOE25G-IP/UDP25G-IP demo. By following these instructions, user can establish two test environments for transferring TCP/UDP payload data through a 25G Ethernet connection, utilizing the TOE25G-IP/UDP25G-IP, as illustrated in Figure 1‑1.

 

Figure 1‑1 Two test environments for running the demo

 

The first test environment utilizes a single FPGA board and a PC equipped with a 25G Ethernet card. The PC executes a test application responsible for data transfer, employing the TOE25G-IP/ UDP25G-IP functionalities implemented on the FPGA. For executing both half-duplex and full-duplex tests with TOE25G-IP, the “tcpdatatest” and “tcp_client_txrx_xg” applications are used, while the “udpdatatest” application is applied for UDP25G-IP testing. Additionally, the PC incorporates a NiosII terminal, serving as the user interface console.

 

The second test environment involves utilizing two FPGA boards, which can be different from another one. Both boards execute the TOE25G-IP/UDP25G-IP demo while configuring different initialization modes (Client, Server, or Fixed-MAC) to initiate data transfer.

 

The demo is implemented on multiple FPGA boards, each with different settings for RS-FEC features. For specific information of the RS-FEC settings one ach FPGA board, please refer to the table provided below.

 

Table 1‑1 The features of RS-FEC for TOE25G-IP/UDP25G-IP demo on each board

Board name

RS-FEC feature

TOE25G-IP demo

Stratix10 GX

Disabled

Stratix10 MX

Disabled

Agilex 7 F-series

Enabled

UDP25G-IP demo

Agilex 7 F-series

Disabled

 

 

2       Test environment setup when using FPGA and PC

 

Before running the demo using an FPGA and PC, please prepare the following.

a)    TOE25G-IP: “tcpdatatest.exe” and “tcp_client_txrx_40G.exe”

b)    UDP25G-IP: “udpdatatest.exe”

 

Note: Example hardware for running the demo is listed as follows.

[1] 25G Network Adapter: Nvidia MCX631102AC-ADAT

https://store.nvidia.com/en-us/networking/store/product/MCX631102AC-ADAT/NVIDIAMCX631102ACADATConnectX6LxENAdapterCard25GbECryptoEnabled/

[2] 25G Ethernet cable

i) QSFP28 Transceiver: AMQ28-SR4-M1

https://www.sfpcables.com/100gbase-sr4-qsfp28-transceiver-for-mmf-70-100-meters-mpo-mtp-4813

ii) SFP28 Transceiver: AZS85-S28-M1

https://www.sfpcables.com/25gb-s-sfp28-sr-transceiver-850nm-up-to-100m-2866

iii) MTO to 8 LC breakout cable: OM3-MTO-8LC-1M

https://www.sfpcables.com/mpo-to-8x-lc-multimode-om3-50-125-m-1-meter

[3] Test PC:

Motherboard:     Gigabyte Z590 AORUS MASTER (rev. 1.0)

CPU:                    Intel i7-11700K CPU 3.6 GHz

RAM:                   32 GB DDR4

OS:                      64-bit Windows10 OS

 

 

Figure 2‑1 TOE25G-IP/UDP25G-IP with CPU demo (FPGA<->PC) on Stratix10 GX


 

Figure 2‑2 TOE25G-IP/UDP25G-IP with CPU demo (FPGA<->PC) on Stratix10 MX

 

 

Figure 2‑3 TOE25G-IP/UDP25G-IP with CPU demo (FPGA<->PC) on Agilex 7 F-Series board

 

The steps for setting up a test environment using an FPGA board and a PC are described below.

 

1)    Turn off power switch and connect the power supply to the FPGA board.

2)    Connect a micro USB cable from the FPGA board to the PC for JTAG programming and JTAG UART.

 

Figure 2‑4 Power connection and microUSB connection

 

3)    Establish a connection between the FPGA board and the PC by connecting a 25G Ethernet cable. Insert a QSFP28 to 4xSFP28 cable between the FPGA board and the PC. Use SFP28 no.1 to connect to QSFP28, as shown in Figure 2‑5.

Note: If you use a Stratix10 MX or Agilex 7 F-series board with two QSFP28 connectors, use the right connector.

 

Figure 2‑5 25G Ethernet connection

 

4)    For Agilex 7 F-series board users, check SW1 located at the bottom side of the board. Ensure that the setting of bit[1]-[3] is OFF OFF OFF to configure the FPGA using JTAG only.

Figure 2‑6  SW1 setting on Agilex 7 F-series board

 

5)    Turn on the power switch on the FPGA board.

6)    Open Quartus Programmer and follow these steps to program the FPGA via USB-1.

i)     Click on “Hardware Setup…” and select USB-BlasterII[USB-1].

ii)    Click on “Auto Detect” and select the FPGA number.

iii)   Select the FPGA device icon.

iv)   Click on the “Change File” button, choose the SOF file in the pop-up window, and click “open” button.

v)    Check the “Program” option.

vi)   Click on the “Start” button to program the FPGA.

vii)  Wait until the Progress status reaches 100%.

 

Figure 2‑7 FPGA Programmer

 

7)    Open the NiosII command shell.

i)     Type “nios2-terminal” to launch the console.

 

Figure 2‑8 Run NiosII terminal

 

ii)    Enter ‘0’ to initiate TOE25G-IP/UDP25G-IP initialization in Client mode (which will send an ARP request to retrieve the PC MAC address).

iii)   The default parameter for the Client mode will be displayed on the console.

 

Figure 2‑9 Message after system boot-up

 

However, if there is an Ethernet connection problem and the status is linked down, an error message will be displayed instead of the welcome message, as shown in Figure 2‑10.

 

Figure 2‑10 Error message when ethernet connection link down

 

iv)   If the user wishes to skip parameter setting and use default parameters to start the system initialization, input ‘x’ as shown in Figure 2‑11. If any other keys are entered, the menu for changing parameter will appear, similar to the “Reset TCPIP/UDPIP parameters” menu. The examples of running the main menu of TOE25G-IP, and UDP25G-IP are described in “dg_toe25gip_cpu_instruction”

and “dg_udp25gip_instruction” documents, respectively.

Figure 2‑11 Initialization complete

 

Note: Transfer performance in the demo is limited by the PC performance. The best performance can be achieved when the test is run using FPGA-to-FPGA connection.

 

 

3       Test environment setup when using two FPGAs

 

Before running the test, please prepare following test environment.

Stratix10 GX (H-Tile), Stratix10 MX, and Agilex 7 F-series development board

https://www.fs.com/products/68017.html

 

Figure 3‑1 TOE25G-IP/UDP25G-IP with CPU demo (FPGA<->FPGA)

The steps for setting up a test environment using two FPGAs are described below.

 

To get started with the demo, follow steps 1) – 6) of topic 2 (Test environment setup when using FPGA and PC) to set up the FPGA board and QSFP28 connection. Once you have completed the configuration for two FPGA boards, a menu will be displayed on the console for selecting Client mode, Server mode, or Fixed MAC mode. Follow the detailed steps below to continue the demo.

 

Note: When connecting two FPGA boards to the same PC via USB cable simultaneously, the Quartus programmer will detect two USB-BlasterII hardware devices, namely USB-1 and USB-2. Choose the appropriate USB channel and start programming the configuration file to first board. Once the first board programming is completed, switch the USB channel and program the configuration file to the second board.

 

Figure 3‑2 Two USB-Blaster cables when connecting two FPGA boards to PC

 

1)    Open NiosII Command Shell.

i)     Run nios2-terminal --cable 1 command for FPGA#1

ii)    Run nios2-terminal –cable 2 command for FPGA#2

 

Figure 3‑3 Run NiosII terminal on two consoles

 

2)    Enter the input to initialize in Server/Client/Fixed-MAC mode. An example to initialize by Server-Client mode is below.

i)     Set ‘1’ on console of FPGA board#1 for running in Server mode.

ii)    Set ‘0’ on console of FPGA board#2 for running in Client mode.

iii)   The default parameters for the selected mode will be displayed on the console, as shown in Figure 3‑4.

 

Note: The rules for setting the initialization mode are below.

·      If the first board is initialized in Server mode, the other board must be initialized in Client mode.

·      If the first board is initialized in Fixed MAC mode, the other board can be run in Client mode or Fixed MAC mode.

 

Figure 3‑4 Input mode

 

3)    Input ‘x’ to use default parameters or use other keys to change parameters. The parameters of Server mode must be set before Client mode. The details are divided into two parts, running the TOE25G-IP demo and running the UDP25G-IP demo.

 

When running TOE25G-IP,

i)     Set parameters on the Server console (board#1 console).

ii)    Set parameters on the Client console (board#2 console) to start IP initialization by transferring ARP packet.

iii)   After finishing the initialization process, “IP initialization complete” and the main menu are displayed on the Server and Client consoles.

 

Figure 3‑5 Main menu of TOE25G-IP

 

 

When running UDP25G-IP,

i)     For Server mode (board#1 console), if user does not change the default parameters, input ‘x’ to skip parameter setting.

ii)    For Client mode, the user must change Target port number (Target->FPGA) to use same value as Target port number (FPGA->Target).

iii)   After finishing initialization process, “IP initialization complete” and the main menu will be displayed on the Server and Client consoles.

 

Figure 3‑6 Main menu of UDP25G-IP

 

 

4       Revision History

 

Revision

Date

Description

2.2

25-May-23

1) Change software full duplex test from “tcp_client_txrx_40G” to “tcp_client_txrx_xg”

2) Add RSFEC feature lists on each demo

2.1

15-Aug-22

Support TOE25G-IP on Agilex 7 F-series board

2.0

24-Jun-21

Support UDP25G-IP and Agilex 7 F-series board

1.1

4-Mar-21

Support S10MX board

1.0

8-Sep-20

Initial release