Demo Instruction for muNVMe-IP
2.2.3 Mixed Write/Read command
1 Overview
This document provides instructions for running the muNVMe-IP demo on an FPGA development board. Prior to executing the test, please refer to the “dg_nvmeip_fpgasetup” document for setting the test environment. Once the demo configuration file has been downloaded to the FPGA, a welcome message will be displayed, as shown in Figure 1‑1.
Figure 1‑1 muNVMe-IP main menu
The welcome screen presents the default message, which includes the IP name and version number, confirming the successful completion of FPGA configuration. Subsequently, when a PCIe connection is established, the screen will show the PCIe speed and the number of PCIe lanes. Finally, the console will display the test menu. The menu offers five options for sending various commands to the muNVMe-IP.
The Write/Read command menu allows two users to simultaneously send Write/Read commands to an NVMe SSD. The other commands (Identify, SMART, Flush, and Shutdown) can be requested by as single user (User#0) without sending Write/Read command requests to the other user (User#1). Further details about each test menu will be covered in the following topic.
2 Test Menu
2.1 Identify Command
To send the Identify command to NVMe SSD via User#0 I/F, select option ‘0’.
Figure 2‑1 Test result when running Identify command
Once the operation is completed, the console will display the output of the Identify command, which includes three values.
1) SSD model number : This value is obtained by decoding the Identify controller data.
2) SSD capacity : This value is the signal output from the muNVMe-IP.
3) Data size per LBA : This value is the signal output from the muNVMe-IP. It can have two values - 512 bytes and 4 KB.
2.2 Write/Read Command
To initiate the Write/Read command, select option ‘1’. Each user’s parameters can be configured independently in the submenu. The console displays the submenu of User#0 and User#1, respectively. Within this submenu, users can select the command as Write, Read, or no operation. Additionally, users can set the maximum data rate for executing Write/Read commands. Consequently, the test system facilitates real-world scenario testing by enabling multiple commands while limiting the maximum speeds for specific users.
2.2.1 Write command
Figure 2‑2 Write command by 1-2 users at maximum speed
As shown in Figure 2‑2, the user must specify the five inputs for operating the Write command, which are described in detail as follows.
1) Command: Select the command type - Write, Read, or Disable (no operation).
2) Start Address: Specify the start address to write the SSD as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
3) Transfer Length: Specify the total transfer size as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
4) Test pattern: Used to select the test data pattern for writing to the SSD. The user can choose from five patterns, including 32-bit incremental, 32-bit decremental, all 0, all 1, and 32-bit LFSR counter.
5) Maximum Speed: Set the maximum speed as a percentage of 4,400 MB/s. Valid values range from 1 to 100. It is noted that 4400 MB/s is derived from multiplying the user clock frequency (275 MHz) by the data width (128-bit), exceeding the maximum throughput of 4-lane PCIe Gen3 speed.
Once all the inputs are valid, the operation begins. During the command execution, the console will display the current amount of write data for each active user every second to indicate that the system is still running. Upon completion of the operation, the console will display the total size, total time usage, and test speed of each active user.
Figure 2‑2 demonstrates the results obtained when running the Write command with two-user and a single user. The total performance achieved by the two users (1563 x 2 = 3126 MB/s) and the single user (3014 MB/s) are nearly identical.
Figure 2‑3 illustrates an example where the maximum speed is limited for User#1. While User#0 is set to 100% to assess the best Write performance, User#1 have their maximum speed set to 7% or 308 MB/s.
Figure 2‑3 Write command by 2-user with limited maximum speed
The result in Figure 2‑3 shows the higher performance of User#0 compared to User#1. Considering the previous result indicating a maximum Write performance of approximately 3126 MB/s for the SSD, it is expected that User#0’s performance should be limited to 3126 – 308 = 2818 MB/s, which closely matches the actual result.
Since User#0’s expected write performance (2818 MB/s) is around nine times the write speed of User#1 (308 MB/s), the transfer size of User#0 is set to be eight times that of User1. This ensures that the write operation of two channels is completed simultaneously. As a result, the speed of each user remains relatively constant throughout the entire duration of the operation.
Figure 2‑4 Example Test data of the 1st and 2nd 512-byte by using incremental/LFSR pattern
Test data in the SSD is divided into 512-byte units. For incremental, decremental, and LFSR patterns, each 512-byte data has a unique 64-bit header that consists of a 48-bit address (in 512-byte units) and a 16-bit zero value. The data following the 64-bit header is the test pattern selected by the user.
The left window of Figure 2‑4 shows an example when using a 32-bit incremental pattern while the right window shows an example when using a 32-bit LFSR pattern. The unique header is not included when running an all-0 or all-1 pattern.
When a user runs the Write/Read command with a 4KB LBA SSD, a message is displayed on the console to show the input limitation, which must be aligned to 8, as shown in Figure 2‑5. If the input is not aligned to 8, the console displays “Invalid input”, and the operation is cancelled.
Figure 2‑5 Error message when the input is unaligned for SSD with 4KB LBA unit
Figure 2‑6 shows an example when the input is out of the recommended range for each parameter. The console displays “Invalid input”, and then the operation is cancelled.
Figure 2‑6 Error message from the invalid input
2.2.2 Read Command
Figure 2‑7 Read command by 1-2 users
For operating the Read command, the user must specify the five inputs, which are described in detail as follows.
1) Command: Select the command type - Write, Read, or Disable (no operation).
2) Start Address: Specify the start address to read the SSD as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
3) Transfer Length: Specify the total transfer size as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
4) Test pattern: Used to select the test data pattern for reading and verifying data from the SSD. The test pattern must match the one used in the Write command menu. There are five available patterns: 32-bit incremental, 32-bit decremental, all 0, all 1, and 32-bit LFSR counter.
5) Maximum Speed: Set the maximum speed as a percentage of 4,400 MB/s. Valid values range from 1 to 100.
When valid inputs are provided, the test system reads data from the SSD. During the execution of the Read command, the console displays the current amount of read data every second to indicate that the system is still running. Upon completion of the operation, the console will display the total size, total time usage, and test speed of each active user.
Figure 2‑7 shows the comparable total read performance between two-user (1648 x 2 = 3296 MB/s) and a single user (2977 MB/s). However, in cases where inputs are invalid or not aligned to 8 (in relation to 4-KB LBA SSD connection), an “Invalid input” message is displayed.
Note: The read performance of most SSDs is enhanced when using larger buffer sizes. When executing a Read Command with two users, the total buffer size is twice that of a single user. Therefore, the read performance of two users might surpass that of a single user.
Figure 2‑8 Data verification has failed
In case of a failed data verification during Read command, an error message is displayed on the console, as shown in Figure 2‑8. The message “Verify fail in User#<i>” (where ‘i’ is the user index ranging from 0 to 1) is displayed with information about the first failure data, such as the error byte address, the expected value, and the read value.
To cancel the Read operation, the user can press any key(s). However, if the operation is not cancelled, it will continue running until it finishes. Once it has finished, the output performance is displayed on the console.
Though the operation is cancelled, the Read command continues running as a background process and may not finish in a proper sequence. Therefore, it is recommended to power off and then power on both the FPGA board and adapter board (if connected) after cancelling the operation.
2.2.3 Mixed Write/Read command
Figure 2‑9 Test result when running mixed Write/Read command
As shown in Figure 2‑9, the user must specify the five inputs for operating the Write and Read commands, which are described in detail as follows.
1) Command: Select the command type - Write, Read, or Disable (no operation).
2) Start Address: Specify the start address to write/read the SSD as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
3) Transfer Length: Specify the total transfer size as a 512-byte unit. The input is in decimal unit when the user inputs only digits. The user can add “0x” as a prefix for hexadecimal units. When the LBA unit of SSD is 4 KB, this input must be aligned to 8.
4) Test pattern: Used to select the test data pattern for writing to the SSD or verifying from the SSD. The user can choose from five patterns, including 32-bit incremental, 32-bit decremental, all 0, all 1, and 32-bit LFSR counter.
5) Maximum Speed: Set the maximum speed as a percentage of 4,400 MB/s. Valid values range from 1 to 100.
Once all the inputs are valid, the operation begins. During the command execution, the console will display the current amount of write data or read data for each active user every second to indicate that the system is still running. Upon completion of the operation, the console will display the total size, total time usage, and test speed of each active user. If an error is found from invalid input or failed data verification, an error message is displayed on the console.
Figure 2‑9 illustrates how SSDs respond to mixed Write-Read commands. On the left side, the write performance of User#0 matches the read performance, allowing simultaneous completion for both User#0 and User#1. In contrast, on the right side, the Write command shows better performance compared to the Read command, resulting in the completion of the Write command before the Read command finishes.
Figure 2‑10 Mixed Write-Read command with limited maximum Read speed
Figure 2‑10 showcases an example result involving two users: one for the Write command and the other for Read command, operating at both maximum and limited speeds. When operating at maximum speed, as shown on the left side, the performance of both Write and Read commands equal 2560 MB/s. In various applications, the Read command is commonly utilized for system monitoring at lower speeds, while the Write command is utilized at the highest speed to log data. Therefore, the example on the right side of Figure 2‑10 aims to evaluate the maximum achievable Write speed while simultaneously operating the Read command at a slower speed.
2.3 SMART Command
To send a SMART command to the NVMe SSD via User#0 I/F, select option ‘2’. Once the operation is completed, the console will display the SMART/Health Information output, as shown in Figure 2‑11. This information includes both the Health status and SMART log information.
Figure 2‑11 Test result when running SMART command
The Health status displays the remaining life of the SSD as a percentage, which is calculated from the Percentage Used value in the SMART log information. The SMART log information displays the following seven parameters.
1) Percentage used: The percentage of the SSD’s lifespan that has been consumed.
2) Temperature: The temperature of the SSD in degree Celsius.
3) Total Data Read: The total amount of data that has been read from the SSD, displayed in GB/TB units. Additionally, the raw data without decoding is displayed as a 32-digit hex number (128 bits). The unit size of raw data is 512,000 bytes.
4) Total Data Written: The total amount of data that has been written to the SSD, displayed in GB/TB units. Additionally, the raw data without decoding is displayed as a 32-digit hex number (128 bits). The unit size of raw data is 512,000 bytes.
5) Power On Cycles: The number of times the SSD has been powered on.
6) Power On Hours: The total amount of time in hours that the SSD has been powered on.
7) Unsafe Shutdowns: The number of times the SSD has experienced an unsafe shutdown.
2.4 Flush Command
To initiate a Flush command on the NVMe SSD, select option ‘3’ from the menu. The Flush command ensures that all modified data in the cache memory is written to Flash memory in the SSD.
Figure 2‑12 Test result when running Flush command
Once the Flush operation is completed, the consoled will display the message “Flush Command Complete”.
2.5 Shutdown Command
To send a Shutdown command to the NVMe SSD via User#0 I/F, select option ‘4’.
Figure 2‑13 Test result when running Shutdown command
A confirmation message will be displayed on the console, and the user will need to enter ‘y’ or ‘Y’ to proceed with the operation. Press any other key to cancel the operation.
Once the Shutdown operation is complete, “Shutdown command is complete” will be displayed as the final message. The console becomes inactive. To begin a new test operation, the user will need to power off and on the test system.
3 Revision History
|
Revision |
Date |
Description |
|
2.0 |
16-Nov-23 |
Add maximum speed limitation feature |
|
1.0 |
14-Jun-22 |
Initial version release |