FPGA Setup for NVMe IP series demo
This document describes the process to configure FPGA and set up the environment to run the NVMe IP series demo on an FPGA development board. These demos enable direct access to an NVMe SSD through a PCIe adapter board. The user can control the test operation through Serial console.
1 Environment Requirement
To run the demo on the FPGA development board, the following environment must be prepared.
1) Supported FPGA Development board:
· PCIe Gen2: AC701, ZC706, VC707
· PCIe Gen3 (Hard IP): KCU105, KCU116, ZCU106, VCU118, ZCU111
· PCIe Gen3 (Soft IP): KCU105, KCU116, ZCU102, ZCU106, VCU118
· PCIe Gen4 (Hard IP): VCK190, Alveo-U50
· PCIe Gen4 (Soft IP): KCU116, ZCU102, ZCU106, VCU118
2) PC installing Xilinx programmer software (Vivado) and Serial console software such as HyperTerminal or TeraTerm
3) The PCIe adapter board provided by Design Gateway
https://dgway.com/ABseries_E.html
· AB17-M2FMC
· Customized AB18-PCIeX16 adapter board: To enable connection of the NVMe SSD to the Alveo-U50 card, the AB18 board must be customized by the user. More information of the modifications can be found in section 3 (Customized AB18 board).
4) Xilinx power adapter for FPGA board, except Alveo-U50
5) ATX power supply for the PCIe adapter board if required
6) NVMe SSD for connecting with the PCIe adapter board. Please ensure that the PCIe speed of the SSD model is compatible with the NVMe-IP demo requirement.
7) USB cable for JTAG programming and Serial console
7-series board : One micro USB cable and one mini USB cable
UltraScale/UltraScale+ board : One or two USB cables
Versal board : One USB type-C cable
Alveo board : Alveo programming cable with micro USB cable
7-series board
Figure 1‑1 NVMe IP series demo setup on VC707 (PCIe Gen2 by Hard IP)
Figure 1‑2 NVMe IP series demo setup on ZC706 (PCIe Gen2 by Hard IP)
Figure 1‑3 NVMe IP series demo setup on AC701 (PCIe Gen2 by Hard IP)
UltraScale board
Figure 1‑4 NVMe IP series demo setup on KCU105 with AB18 (PCIe Gen3 by Hard IP)
Figure 1‑5 NVMe IP series demo setup on KCU105 with AB17 (PCIe Gen3 by Hard IP)
UltraScale+ board
Figure 1‑6 NVMe IP series demo setup on KCU116 with AB18
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 1‑7 NVMe IP series demo setup on KCU116 with AB17
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 1‑8 NVMe IP series demo setup on VCU118 with AB18
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 1‑9 NVMe IP series demo setup on VCU118 with AB17
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Alveo card
Figure 1‑10 NVMe IP series demo setup on Alveo-U50 with customized AB18
(PCIe Gen4 by Hard IP)
Note: The modification details of customized AB18 for using with Alveo-U50 are described in section 3 (Customized AB18 board)
Zynq UltraScale+ board
Figure 1‑11 NVMe IP series demo setup on ZCU102 with AB17
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 1‑12 NVMe IP series demo setup on ZCU106 with AB18
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 1‑13 NVMe IP series demo setup on ZCU106 with AB17
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Figure 14 NVMe IP series demo set up on ZCU111 with AB17
(PCIe Gen3 by Hard IP or Gen4 by Soft IP)
Versal board
Figure 1‑15 NVMe IP series demo setup on VCK190 with AB18 (PCIe Gen4 by Hard IP)
2 Demo setup
1) Power off system.
2) Check DIP switch setting for JTAG configuration on Zynq and Versal board.
· For ZC706 board, set SW11=all OFF to configure PS from JTAG and set SW4[1:2]=[OFF ON] to connect JTAG with USB-to-JTAG interface, as shown in Figure 2‑1.
Figure 2‑1 SW11 setting to configure PS from JTAG on ZC706
· For ZCU106/ZCU102/ZCU111 board, set SW6=all ON to configure PS from JTAG, as shown in Figure 2‑2.
Figure 2‑2 SW6 setting to configure PS from JTAG on ZCU106
· For VCK190 board, set SW11=all OFF in order to not use micro SD card to boot, as shown in Figure 2‑3
Figure 2‑3 SW11 setting on VCK190
3) Setup and connect NVMe SSD to PCIe adapter board.
AB18-PCIeX16, AB16-PCIeXOVR, or customized AB18 (Alveo-U50 card)
i) Confirm that two mini jumpers are inserted at J5 connector on AB18.
ii) Connect ATX power supply to AB board.
iii) Connect PCIe connector on FPGA board/Alveo card to FPGA Side (A-side) and connect NVMe PCIe SSD to device side (B-Side) on AB board, as shown in Figure 2‑4.
Warning: Please confirm that the SSD is correctly inserted into the B-side of the AB18 (not the A-side) before powering on the system.
Figure 2‑4 Setup AB18-PCIeX16 connection
AB17-M2FMC
i) Connect M.2 NVMe SSD to Drive#1 M.2 connector on AB17-M2FMC.
ii) Connect AB17-M2FMC to HPC/HPC1 connector on KCU105(J22), ZCU106(J5), ZCU102(J4), or HSPC on VCU118 (J22), ZCU111 (J26), as shown in Figure 2‑5
Figure 2‑5 Setup AB17-M2FMC connection
4) Connect USB cables for JTAG programming and Serial console.
a) For 7-series board, connect micro USB cable for JTAG and mini USB cable for Serial console.
b) For UltraScale/UltraScale+ board
ZCU111 : Connect a micro USB cable for both JTAG and Serial console.
Others : Connect two micro USB cables for JTAG and Serial console.
Figure 2‑6 USB cable connection
c) For Alveo-U50, connect a Ribbon cable to attach the Alveo programming cable to Alveo-U50 board, and a micro USB cable for JTAG and Serial console, as shown in Figure 2‑7.
Note: More details about Alveo programming cable are described in the user guide.
Figure 2‑7 Alveo Programming cable
d) For VCK190, connect USB type-C for JTAG and Serial console.
Figure 2‑8 USB type-C cable connection
5) Power on FPGA development board, adapter board, and ATX power supply for AB18.
Figure 2‑9 Turn on power switch on adapter board
6) After connecting USB cables to PC, additional COM port is detected.
For Ultrascale/Ultrascale+ board/Alveo card, multiple COM ports are detected.
a) KCU105/KCU116/VCU118: Select Standard COM port
b) ZCU106/ZCU102/ZCU111/VCK190: Select the lowest number of the additional COM ports
c) Alveo-U50: Select number#2 of the additional COM ports
Figure 2‑10 Additional COM port when USB cable is plugged-in
On Serial console, set Baud rate=115,200, Data=8-bit, Non-Parity, and Stop = 1, as shown in Figure 2‑11.
Figure 2‑11 Serial console setting
7) When using AB17-M2FMC connection on KCU105 or VCU118 board, user must set VADJ on FMC connector as following step.
KCU105 board
Open Serial console to connect with Enhanced COM port (Buad rate=115,200 Data=8 bit Non-Parity Stop=1). The console shows System Controller menu, as shown in Figure 2‑12.
Figure 2‑12 Setting VADJ of FMC on KCU105
To set VADJ of FMC to 1.8V, the following step is recommended.
i) Input ‘4’ to select Adjust FMC Settings.
ii) Input ‘4’ to set FMC VADJ to 1.8V.
iii) Input ‘0’ to return to Main Menu.
iv) Input ‘2’ to get PMBUS Voltages.
v) Input ‘7’ to get VADJ1V8 Voltage. The output voltage of this menu must be equal to 1.8V to confirm that VADJ has been set completely.
For more details of System Controller, please check “UG917 KCU105 Board User Guide” in section “Appendix C: System Controller”.
https://www.xilinx.com/support/documentation/boards_and_kits/kcu105/ug917-kcu105-eval-bd.pdf
VCU118 board
Open SCUI.exe and run following step.
Figure 2‑13 Setting VADJ of FMC for VCU118
i) Select File->Change the System Controller Port.
ii) Select COM port number which is Enhanced COM Port.
iii) Click “OK” button to confirm the port.
iv) Select FMC tab -> Set VADJ tab -> Current tab. Click “Set VADJ to 1.8V” button.
v) Select Voltages tab and click “Get VADJ_1V8 Voltage” button. The output voltage must be equal to 1.8V to confirm that VADJ has been set completely.
System controller tool (SCUI.exe) can be downloaded from Xilinx website.
https://www.xilinx.com/products/boards-and-kits/vcu118.html#documentation
Direct link for SCUI.exe on Vivado2017.4.
8) Download and program configuration file and firmware to FPGA board.
AC701, VC707, KCU105, KCU116, VCU118, and Alveo-U50 card
Configure FPGA by using Vivado tool, as shown in Figure 2‑14.
Figure 2‑14 Program FPGA by Vivado
ZC706, ZCU106, ZCU102, VCK190, and ZCU111 board
Open Vivado TCL shell and run the bat file of the NVMeIPTest, as shown in Figure 2‑15.
Figure 2‑15 Command script to download demo file on Vivado TCL shell
9) Check LED status on FPGA board. The description of LED is as follows.
Note: There is no LED status on Alveo card.
Table 2‑1 LED Definition
|
GPIO LED |
ON |
OFF |
|
0 |
Normal operation |
Clock is not locked or reset button is pressed |
|
1/R |
System is busy |
Idle status |
|
2/C |
IP Error detect |
Normal operation |
|
3/L |
Data verification fail |
Normal operation |
10) After programming completely, LED[0] and LED[1] are ON during PCIe initialization process. Then, LED[1] changes to OFF after PCIe completes initialization process.
Figure 2‑16 LED status after configuration file is loaded and PCIe initialization is done
11) Main menu is displayed on Serial console. Now it is ready to receive command from the user.
Figure 2‑17 Main menu after IP finishes initialization
3 Customized AB18 board
This topic describes the process for modifying the AB18-PCIeX16 to connect with the Alveo card and NVMe SSD. Once completely modified, the customized AB18 board should only be connected to the Alveo card. Connecting the customized AB18 to other Xilinx development kit may result in board damage due to power supply collisions. The steps for modifying the AB18 are outlined below.
Figure 3‑1 JP6 and JP7 modification on AB18 for Alveo connection
1) Short pin 1, 2, and 3 of JP6 by soldering.
2) Short pin 1, 2, and 3 of JP7 by soldering.
Note: Step 1) and step 2) are necessary to supply power from the ATX supply that connects to the AB18 to both the FPGA side (Alveo card) and the device side (SSD card). The default mode only connects pin 2 and 3 to supply power to the device, as the FPGA board has its own power supply. However, the Alveo card does not have its own power supply, so it needs power from the AB18.
Figure 3‑2 J1 modification on AB18 for Alveo connection
3) Short pin B8 and B10 of J1 connector.
For further information on the AB18 board, refer to the following document.
https://dgway.com/products/IP/ABseries/AB18-PCIEx16-MAN-E.pdf
Caution: The AB18 modification is a customized process undertaken by the user. As a result, Design Gateway does not guarantee the outcome of the modification.
4 Revision History
|
Revision |
Date |
Description |
|
4.8 |
12-Jul-23 |
Correct Figure 1‑7 |
|
4.7 |
11-May-23 |
Support ZCU111 |
|
4.6 |
3-Feb-23 |
Update supported board lists for each PCIe IP model |
|
4.5 |
21-Jul-22 |
Support VCK190 |
|
4.4 |
16-Jun-22 |
Support raNVMe-IP and muNVMeIP |
|
4.3 |
13-Sep-21 |
Support NVMe-IP for Gen4 |
|
4.2 |
22-Dec-20 |
Support NVMeG4-IP |
|
4.1 |
27-Aug-20 |
Support KCU116 |
|
4.0 |
29-Jun-20 |
Remove instruction from the document and include NVMeG3-IP |
|
1.0 |
2-Jun-16 |
Initial version release |