TOE40G-IP Core Data Sheet
Low Latency 40G Ethernet Intel FPGA IP
Core Facts |
|
Provided with Core |
|
Documentation |
User Guide, Design Guide |
Design File Formats |
Encrypted HDL |
Instantiation Templates |
VHDL |
Reference Designs & Application Notes |
QuartusII Project, See Reference Design Manual |
Additional Items |
Demo on Arria10 GX board |
Support |
|
Support Provided by Design Gateway Co., Ltd. |
E-mail: ip-sales@design-gateway.com
URL: www.design-gateway.com
· TCP/IP stack implementation
· Support IPv4 protocol
· Support one session per one TOE40G IP (Multisession could be implemented by using multiple TOE40G IPs)
· Support both Server and Client mode (Passive/Active open and close)
· Support Jumbo frame
· Packet size for transmit must be aligned to 256-bit because Transmit data bus size is 256-bit
· Received data bus size is 256-bit, so total receive size must be aligned to 256-bit
· Transmit/Receive buffer size, adjustable for optimizing resource and performance
· Simple data interface by standard FIFO interface
· Simple control interface by single port RAM interface
· 256-bit FIFO interface with Ethernet MAC
· At least 200 MHz is recommended for clock input to TOE40G IP
· Reference design available on Arria10 GX development kit
· Not support data fragmentation feature
· Customized service for following features
· Unaligned 256-bit data transferring
· Buffer size extension by using Windows Scaling feature
· Network parameter assignment by other methods
Table 1: Example Implementation Statistics
Family |
Example Device |
Fmax (MHz) |
ALMs1 |
Registers1 |
Pin |
Block Memory bit2 |
Design Tools |
Arria 10 GX |
10AX115S2F45I1SG |
322 |
3,656 |
5,696 |
- |
1,179,648 |
QuartusII18.0 |
Notes:
1) Actual logic resource dependent on percentage of unrelated logic
2) Block memory resources are based on 64kB Tx data buffer size and 64kB Rx data buffer size.
Figure 1: TOE40G IP Application
40 Gb Ethernet is the communication channel which can transfer data at ultra high speed with remote controlling system. By using TCP/IP protocol for transferring via 40 Gb Ethernet, the system can transfer very big data at very high speed rate with reliability. TOE40G IP is the IP which is integrated to the system for transfering data via 40 Gb Ethernet without using CPU and external memory. So, the IP can fit with the application which needs to send or receive data at high-speed rate by using FPGA solution such as video data streaming and sensor monitoring system.
Figure 1 shows the example application of sensor monitoring system. The data from sensor is stored to the FIFO and forwarded to remote system via 40 Gb Ethernet by TOE40G IP. TOE40G IP is desigend to support full-duplex transfer in the same session, so Remote system can send the paramter for controlling the sensor monitoring system via 40 Gb Ethernet. In our website, we also have FTP server demo by using TOEXG IP. Please contact us for more details about FTP server demo.
TOE40G IP is designed for transferring data at the highest speed, so the latency time is much from internal pipeline register and buffer. For low-latency application such as FinTech, it is recommended to use our low-latency IP instead. Please see more details of low-latency IP from our website.
https://dgway.com/Lowlatency-IP_A_E.html
Figure 2: TOE40G IP Block Diagram
TOE40G IP core implements TCP/IP stack by hardwire logic and connects with EMAC and PCS/PMA (BASE-R) module through Adapter logic as the lower layer hardware. User interface of TOE40G IP consists of two interfaces, i.e. Register interface for control signals and FIFO interface for data signals.
Register interface has 4-bit address for accessing up to 16 registers, consisting of network parameters, command register and system parameters. The IP supports only one session, so the network parameters set by the user are fixed for assigning to TOE40G IP and the target device. After that, start IP initialization by de-asserting reset signal. Also, the reset process is necessary when some network parameters must be changed. The initialization process has two modes to get MAC address of the target device. After finishing the initialization process, the IP is ready for transferring data with the target device.
Following TCP/IP standard, the connection must be created by opening the port as the first step. The IP supports for both active open (the port opened by the IP) or passive open (the port opened by the target device). After that, data can be transferred from both sides. To send the data, the user sets total transfer size and packet size to the IP and then transfers the data via TxFIFO interface which is 256-bit data size. When the data is received from the target, the user reads the received data from the IP via RxFIFO interface. After finishing data transferring, the connection can be destroyed by using active close (the port closed by the IP) or passive close (the port closed by the target).
To meet the user system requirement which may be sensitive on the memory resource or the performance, the buffer size inside the IP can be assigned by the user. Tx data buffer and Rx data buffer can be adjusted for Tx path and Rx path respectively. Using the bigger buffer size may increase the transfer performance in each direction. More details of the hardware inside the IP are described in the next topic.
TOE40G IP core can be divided into three parts, i.e. control block, transmit block and receive block.
All parameters of the IP are set via register interface which has 4-bit address signals and 32-bit data signals. Timing diagram of register interface is similar to single-port RAM interface. The address is shared for both write and read directions. The description of each register is defined as shown in Error! Reference source not found..
Table 2 Register map Definition
RegAddr [3:0] |
Reg Name |
Dir |
Bit |
Description |
0000b |
RST |
Wr /Rd |
[0] |
Reset IP. 0: No reset, 1: Reset. Default value is 1. After all parameters are assigned, the user sets 0 to this register for loading parameters and starting system initialization. User must set this register to 1 and 0 respectively when some network paramters are changed. The network parameters controlled by RST register are SML, SMH, DIP, SIP, DPN, SPN and SRV register. |
0001b |
CMD |
Wr |
[1:0] |
User command. 00: Send data, 10: Open connection (active), 11: Close connection (active), 01: Undefined. The command operation begins after the user sets CMD register. Before setting this register to start new operation, user needs to confirm that the system is in Idle status by checking busy signal de-asserted to 0. Busy signal is read by bit[0] of CMD register. |
Rd |
[0] |
System busy flag. 0: Idle, 1: IP is busy. |
||
[3:1] |
Current operation. 000: Send data, 001: Idle, 010: Active open, 011: Active close, 100: Receive data, 101: Initialization, 110: Passive open, 111: Passive close. |
|||
0010b |
SML |
Wr /Rd |
[31:0] |
Define 32-bit lower MAC address (bit [31:0]) for this IP. To update this value, the IP must be reset by RST register. |
0011b |
SMH |
Wr /Rd |
[15:0] |
Define 16-bit upper MAC address (bit [47:32]) for this IP. To update this value, the IP must be reset by RST register. |
0100b |
DIP |
Wr /Rd |
[31:0] |
Define 32-bit target IP address. To update this value, the IP must be reset by RST register. |
0101b |
SIP |
Wr /Rd |
[31:0] |
Define 32-bit IP address for this IP. To update this value, the IP must be reset by RST register. |
0110b |
DPN |
Wr /Rd |
[15:0] |
Define 16-bit target port number. Unused when the port is opened in passive mode. To update this value, the IP must be reset by RST register. |
0111b |
SPN |
Wr /Rd |
[15:0] |
Define 16-bit port number for this IP. To update this value, the IP must be reset by RST register. |
1000b |
TDL |
Wr |
[31:0] |
Total Tx data length in byte unit, but the length must be aligned to 32-byte (data bus size). Valid range is 32-0xFFFFFFE0 (Bit[4:0] is ignored by the IP). User needs to set this register before setting CMD register = Send data (00b). The IP loads TDL register when CMD register is set. After the IP runs Send data command (Busy=1), the user can set the new value of TDL register for the next command. The user does not need to set TDL register again when the next command uses the same total data length. |
Rd |
[31:0] |
Remaining transfer length in byte unit which does not transmit. |
RegAddr [3:0] |
Reg Name |
Dir |
Bit |
Description |
1001b |
TMO |
Wr |
[31:0] |
Define timeout value for waiting Rx packet returned from the target. The counter is run under Clk input, so timer unit is equal to 1/Clk (5 ns when Clk is 200 MHz). TimerInt is asserted to 1 when the packet is not received in time. Please see more details of TimerInt from Read value of TMO[7:0] register. This value should be more than latency time in the network to generate timeout when packet is lost, not normal condition. |
Rd |
|
The details of timeout interrupt are shown in TMO[7:0]. Other bits are read for IP monitoring. [0]-Timeout from not receiving ARP reply packet After timeout, the IP resends ARP request until ARP reply is received. [1]-Timeout from not receiving SYN and ACK flag during active open operation After timeout, the IP resends SYN packet for 16 times and then sends FIN packet to close connection. [2]-Timeout from not receiving ACK flag during passive open operation After timeout, the IP resends SYN/ACK packet for 16 times and then sends FIN packet to close connection. [3]-Timeout from not receiving FIN and ACK flag during active close operation After the 1st timeout, the IP sends RST packet to close connection. [4]-Timeout from not receiving ACK flag during passive close operation After timeout, the IP resends FIN/ACK packet for 16 times and then sends RST packet to close connection. [5]-Timeout from not receiving ACK flag during data transmit operation After timeout, the IP resends the previous data packet. [6]-Timeout from Rx packet lost, Rx data FIFO full or wrong sequence number The IP generates duplicate ACK to request data retransmission. [7]-Timeout from too small receive window size when running Send data command and setting PSH[2] to 1. After timeout, the IP retransmits data packet, similar to TMO[5] recovery process. [21]-Lost flag when the sequence number of the received ACK packet is skipped. As a result, TimerInt is asserted and TMO[6] is equal to 1. [22]-FIN flag is detected during sending operation. [23]-Rx packet is ignored due to Rx data buffer full (fatal error). [27]-Rx packet lost detected [30]-RST flag is detected in Rx packet [31],[29:28],[26:24]-Internal test status |
||
1010b |
PKL |
Wr /Rd |
[15:0] |
TCP data length of one Tx packet in byte unit, but the length must be aligned to 32-byte. Valid from 32-16000. Default value is 1440 byte which is the maximum size of non-jumbo frame that is aligned to 32-byte. Bit[4:0] of this register is ignored by the IP. During running Send data command (Busy=1), the user must not set this register. Similar to TDL register, the user does not need to set PKL register again when the next command uses the same packet length. |
1011b |
PSH |
Wr /Rd |
[2:0] |
Sending mode when running Send data command. [0]-Disable to retransmit packet. 0: Generate the duplicated data packet for the last data packet in Send data command (default). 1: Disable the duplicated data packet. |
RegAddr [3:0] |
Reg Name |
Dir |
Bit |
Description |
1011b |
PSH |
Wr /Rd |
[2:0] |
[1]-PSH flag in the transmit packet. 0: PSH flag in TCP header of all transmitted packet is 0 (default). 1: PSH flag in TCP header of all transmitted packet is 1. [2]-Enable to retransmit data packet when Send data command is paused until timeout, caused by the receive window size smaller than the packet size. This flag is designed to solve the system hang problem from the window update packet lost. Data retransmission can activate the target device to regenerate the lost ACK packet. All following conditions must be met to start data retransmission. (1) PSH[2] is set to 1. (2) The current command is Send data and all data are not completely sent. (3) The receive window size is smaller than the packet size. (4) Timer set by TMO register is overflowed. 0: Disable the feature (default) 1: Enable the feature. |
1100b |
WIN |
Wr /Rd |
[5:0] |
Threshold value in 1Kbyte unit for sending window update packet. Default value is 0 (Not enable window update feature). The IP transmits the window update packet when the free space of the receive buffer is increased from the value in the latest transmitted packet more than the threshold value. For example, the user sets WIN=000001b (1 Kbyte) and the the window size of the latest transmitted packet is equal to 2 Kbyte. After that, the user reads 1 Kbyte data from the IP. Free space of the receive buffer is updated from 2 Kbyte to be 3 Kbyte. The IP detects that the increased window size is more than 1 Kbyte (3K 2K) which is the threshold value. As a result, the IP sends the window update packet to update the receive buffer size. |
1101b |
ETL |
Wr |
[31:0] |
Extended total Tx data length in byte unit. The size must be aligned to 32 byte. Bit[4:0] is ignored by the IP. User sets this register during Send data command operating (Busy=1) for extending total Tx data length. So, the data can be transmitted continuously without re-sending the new command to IP. The caution points to use this feature are as follows. 1) ETL register must be programmed when read value of TDL is not less than 128 Kbyte to be the safe gap that Busy is not de-asserted to 0 before setting ETL register. 2) The set value of ETL must be less than (0xFFFFFFE0 read value of TDL). For example, the user sets TDL = 3.5 Gbyte in the first send data command. After the IP completes 2 Gbyte data (remaining size = 1.5 Gbyte), the user sets ETL register = 1.5 Gbyte.The total length is equal to 5 Gbyte (3.5 Gbyte of TDL + 1.5 Gbyte of ETL). |
1110b |
SRV |
Wr/ Rd |
[0] |
0: Client mode (default). After RST register changes from 1 to 0, the IP sends ARP request to get Target MAC address from the ARP reply returned by the target device. IP busy is deasserted to 0 after receiving ARP reply. 1: Server mode. After RST register changes from 1 to 0, the IP waits ARP request from the Target to get Target MAC address. After receiving ARP request, the IP generates ARP reply and then de-asserts IP busy to 0. Note: In Server mode, when RST register changes from 1 to 0, the target device needs to resend ARP request for TOE40G IP completing the IP initialization. |
1111b |
VER |
Rd |
[31:0] |
IP version |
After RST register changes from 1 to 0, the initialization phase begins. There are two modes for running the initialization phase, set by SRV[0] register, i.e. Client mode or Server mode. The parameters from Reg module is read by TCP Stack and then set to Transmit block and Receive block for transferring the packet to complete the initialization process, following the mode. After that, the IP changes to data transferring phase.
To transfer data between TOE40G IP and the target device, it consists of three processes, i.e. opening the port, transferring data and closing the port. The IP supports to start the port opening or closing by sending SYN or FIN packet when the user sets CMD register = 10 (port opening) or 11 (port closing). Also, the port can be closed or opened by the target device as passive mode when TCP Stack detects SYN or FIN packet from Receive block. During port opening or closing process, TCP Stack asserts Busy flag to 1. After finishing transferring all packets and the port is completely opened or closed, Busy flag is de-asserted to 0. ConnOn signal can be monitored to confirm the port status that is completely opened or closed. The data can be transferred when ConnOn is asserted to 1 (the port is opened completely).
To send the data, the data from the user is stored in Tx data buffer and Tx packet buffer. After the network parameters are read to build TCP header by Packet Builder, Transmit block sends TCP packet including the data from the user to the target device via Ethernet MAC. When the target receives the data correctly, ACK packet is returned to Receive block. TCP Stack monitors the status of Transmit block and Receive block to confirm that the data is transferred successfully. If the data is lost, TCP Stack pauses the current data transmission and then start data retransmission process in Transmit block.
When the data is received by Receive block, TCP Stack checks the order of received data. If the data is in the correct order, normal ACK packet is generated by Transmit block. Otherwise, TCP Stack starts the data lost recovery process by controlling the Transmit block for generating duplicate ACKs to the target device.
Table 3 TxBuf/RxBufBitWidth Parameter description
Value of BitWidth |
Buffer Size |
TxBufBitWidth |
RxBufBitWidth |
9 |
16kByte |
Valid |
Valid |
10 |
32kByte |
Valid |
Valid |
11 |
64kByte |
Valid |
Valid |
This buffer size is set by TxBufBitWidth parameter of the IP. The valid value is 9-11 which is equal to the address size of 256-bit buffer, as shown in Table 3. The buffer size should be more than or equal to two times of Tx packet size, set by PKL register. This buffer stores the data from the user for preparing the transmit packet sent to the target device. Data is removed from the buffer when the target device confirms that the data is completely received. Consequently, when the buffer size is large, the IP can send many data to the target device without waiting ACK packet returned from the target to clear the buffer. If the ACK packet to clear the buffer is received and the user can fill the new data to the IP in time, the IP can send the new data to the target continuously without waiting the new data from the user. As a result, the system can achieve the best transmit performance on 40Gb Ethernet connection. Nevertheless, when the carrier and the networking interface have much latency time, all data in the Tx data buffer will be completely transferred before the ACK packet to flush the buffer is received and the new data is filled by the user. As a result, the transmit performance is reduced from the latency time.
This buffer size is 16 Kbyte to store at least one transmit packet. Data in Tx packet buffer is forwarded to EMAC.
TCP packet consists of the header and the data. Packet builder receives network parameters, set in Reg module, and then prepares TCP header. Also, IP and TCP checksum are calculated to be TCP header. After that, all TCP header is built, the header combining with the data from Tx packet buffer is transmitted to EMAC.
In the receive block, Rx data buffer is included to store the received data from the target device. The data is stored in the buffer when the header in the packet is matched to the expected value, set by the network parameters inside Reg module. Also, the IP and TCP checksum in the packet must be correct. Otherwise, the received packet is rejected. Using bigger size of Rx data buffer may increase the receive performance. Besides, TOE40G IP can support the packet re-ordering when only one packet is swapped. For example, the receive order is packet#1, #3, #2 and #4 (packet #2 is swapped with packet#3). If the packet order is switched more than one packet such as packet#1, #3, #4, and #2 (packet #3 and #4 are received before packet#2), TOE40G IP cannot reorder the data and detect as data lost condition. After that, the data recovery process is run by generating duplicated ACK packet.
This is temporary buffer to store all Rx packets from EMAC when the previous packet is not completely processed.
The header in Rx packet are verifed by this module to validate the packet. The packet is valid when the following conditions are met.
(1) Network parameters are matched to the value in Reg module, i.e. MAC address, IP address and Port number.
(2) The packet is ARP packet or TCP/IPv4 packet without data fragment flag.
(3) IP header length and TCP header length are valid (IP header length is equal to 20 bytes and TCP header length is equal to 20 60 bytes).
(4) IP checksum and TCP checksum are correct.
(5) The data pointer decoded by the sequence number is in valid range.
(6) The acknowledge number is in valid range.
This module is designed to remove the packet header and split only TCP data to store to Rx data buffer.
This buffer size is set by RxBufBitWidth parameter of the IP. The valid value is 9-11 for 16Kbyte 64Kbyte buffer size. Rx data buffer size is applied to be the window size of the transmit packet. When Rx data buffer is big enough, the target device can send many data to TOE40G IP without waiting ACK packet returned by the IP which may be delayed from the networking system. As a result, the bigger size of Rx data buffer may increase the receive performance.
The data is stored in the buffer until the user reads it. If the user does not read data out from the buffer for long time, the buffer is full and then the target device cannot send more data to the IP. So, it is recommended for the user logic to read the data from the IP when the data is ready. If the Rx data buffer is not full, the receive performance will not be dropped by the full window size.
The reference design uses 40G Ethernet MAC and PHY IP from Intel FPGA which can check for more details from following website.
https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_ll_40gbe.pdf
Descriptions of all parameters and I/O signals are provided in Table 4 and Table 5. The EMAC interface of the IP uses 256-bit FIFO standard as Show-ahead mode.
Table 4: Core Parameters
Name |
Value |
Description |
TxBufBitWidth |
9-11 |
Setting Tx Data buffer size. The value is the address bus size of this buffer. |
RxBufBitWidth |
9-11 |
Setting Rx Data buffer size. The value is the address bus size of this buffer. |
Table 5: Core I/O Signals
Signal |
Dir |
Description |
Common Interface Signal |
||
RstB |
In |
Reset IP core. Active Low. |
Clk |
In |
Clock input. Clock frequency should be at least 200 MHz to achieve the best performance. |
User Interface |
||
RegAddr[3:0] |
In |
Register address bus. In Write access, RegAddr is valid when RegWrEn=1. |
RegWrData[31:0] |
In |
Register write data bus. Valid when RegWrEn=1. |
RegWrEn |
In |
Register write enable. Valid at the same clock as RegAddr and RegWrData. |
RegRdData[31:0] |
Out |
Register read data bus. Valid in the next clock after RegAddr is valid. |
ConnOn |
Out |
Connection Status. 1: connection is opened, 0: connection is closed. |
TimerInt |
Out |
Timer interrupt. Assert to high for 1 clock cycle when timeout is detected. More details of Interrupt status are monitored from TMO[7:0] register. |
RegDataA1[31:0] |
Out |
32-bit read value of CMD register (RegAddr=0001b). Bit[0] is busy flag of TOE40G-IP. |
RegDataA8[31:0] |
Out |
32-bit read value of TDL register (RegAddr=1000b). |
RegDataA9[31:0] |
Out |
32-bit read value of TMO register (RegAddr=1001b). |
Tx Data Buffer Interface |
||
TCPTxFfFlush |
Out |
Tx data buffer within the IP is reset. Assert to 1 when the connection is closed or the IP is reset. |
TCPTxFfFull |
Out |
Asserted to 1 when Tx data buffer is full. User needs to stop writing data within 4 clock cycles after this flag is asserted to 1. |
TCPTxFfWrEn |
In |
Write enable to Tx data buffer. Asserted to 1 to write data to Tx data buffer. |
TCPTxFfWrData[255:0] |
In |
Write data to Tx data buffer. Valid when TCPTxFfWrEn=1. |
Rx Data Buffer Interface |
||
TCPRxFfFlush |
Out |
Rx data buffer within the IP is reset. Assert to 1 when the new connection is opened. |
TCPRxFfRdCnt[10:0] |
Out |
Data counter of Rx data buffer to show the number of received data in 256-bit unit. |
TCPRxFfLastRdCnt[4:0] |
Out |
Remaining byte of the last data in Rx data buffer when total received data in the buffer is not aligned to 32-byte unit. User cannot read the data until all 32-byte data is received. |
TCPRxFfRdEmpty |
Out |
Asserted to 1 when Rx data buffer is empty. User needs to stop reading data immediately when this signal is asserted to 1. |
TCPRxFfRdEn |
In |
Assert to 1 to read data from Rx data buffer. |
TCPRxFfRdData[255:0] |
Out |
Data output from Rx data buffer. Valid in the next clock cycle after TCPRxFfRdEn is asserted to 1. |
Signal |
Dir |
Description |
MAC FIFO Interface (Show-ahead mode) |
||
MacTxFfWrCnt[15:0] |
In |
Write data counter of MacTxFIFO in 256-bit. This signal is used to check full condition. If FIFO counter is less than 16-bit, please fill 1 to upper bit. |
MacTxFfWrData[255:0] |
Out |
256-bit write data to MacTxFIFO. Valid at the same clock as MacTxFfWrEn=1. |
MacTxFfWrEn |
Out |
Assert to 1 to write data to MacTxFIFO. This signal is asserted to 1 continuously to transmit one Tx packet. |
MacTxLastEmpty[4:0] |
Out |
Specify the number of bytes which are unused of the final word in the frame. This signal is valid when MacTxEnd=1 and MacTxFfWrEn=1. In IP core, this signal could be equal to two values, i.e. 0x0A (10 empty byte) and 0x04 (4 empty byte). |
MacTxSizeData[15:0] |
Out |
Total size of Tx packet in 256-bit unit. This value is stable during MacTxFfWrEn=1. Bit[15:11] is always equal to 0. |
MacTxEnd |
Out |
Assert to 1 to indicate the final data of each Tx packet. |
MacRxFfEmpty |
In |
Empty flag of MacRxFIFO. Assert to 1 when there is no data within MacRxFIFO. |
MacRxFfRdEn |
Out |
Assert to 1 to read data from MacRxFIFO. This signal could be asserted to 1 when MacRxFfEmpty=0. |
MacRxFfRdData[257:0] |
In |
Read data output from MacRxFIFO. This signal is valid at the same clock as MacRxFfRdEn=1. So, MacRxFIFO must be designed to run as Show-ahead mode. The definition of each received data is as follows. [255:0]: 256-bit received data from EMAC. [256]: Assert to 1 to indicate the final data in each Rx packet from EMAC. [257]: 0-normal packet, 1-error packet. This bit must be asserted to 1 at the same time as bit[256]=1 which indicates final data. |
The initialization process begins after user changes RST register from 1 to 0. TOE40G IP can run in two modes, set by SRV[0] register, i.e. Client mode (SRV[0]=0) and Server mode (SRV[0]=1). The details of each mode are shown in the following timing diagram.
Figure 3: IP Initialization in Client mode
As shown in Figure 3, in Client mode TOE40G IP sends ARP request and waits ARP reply returned from the target device. Target MAC address is extracted from ARP reply packet. After finishing, Busy signal (bit0 of RegDataA1) is de-asserted to 0.
Figure 4: IP Initialization in Server mode
As shown in Figure 4, after reset process in Server mode, TOE40G IP waits ARP request sent by the target device. After that, TOE40G IP returns ARP reply to the target. Target MAC address is extracted from ARP request packet. Finally, Busy signal is de-asserted to 0.
All control signals and the network parameters for the operation are set and monitored via Register interface. Timing diagaram of Register interface is similar to Single-port RAM which shares the address bus for write and read access. Read latency time of the read data from the address is one clock cycle. Register map is defined in Error! Reference source not found..
Figure 5: Register interface timing diagram
As shown in Figure 6, before the user sets CMD register to start the new command operation, Busy flag must be equal to 0 to confirm that IP is in Idle status. After CMD register is set, Busy flag is asserted to 1. Busy is de-asserted to 0 when the command is completed.
Figure 6: CMD register timing diagram
Figure 7: Tx FIFO interface timing diagram
(1) Before sending data, user needs to confirm that full flag (TCPTxFfFull) is not asserted to 1 and ConnOn must be equal to 1. After that, TCPTxFfWrEn can be asserted to 1 with valid value of TCPTxFfWrData.
(2) TCPTxFfWrEn must be de-asserted to 0 within 4 clock cycles to pause data sending after TCPTxFfFull is asserted to 1.
(3) After finishing transferring all data, the port can be closed by TOE40G IP (active) or the target device (passive). After the port is closed, the following situations are found.
a) ConnOn changes from 1 to 0.
b) TCPTxFfFlush is asserted to 1 to flush all data inside TxFIFO.
c) TCPTxFfFull is asserted to 1 to pause data sent by the user during closing the connection.
After the received data is stored in Rx data buffer, the user can read the data from Rx data buffer by using Rx FIFO interface. Empty flag is monitored to check data available status and then asserts read enable signal to read the data, similar to read interface of general FIFO, as shown in Figure 8.
Figure 8: Rx FIFO interface timing diagram by Empty flag
(1) TCPRxFfEmpty is monitored to check data available status. When data is ready (TCPRxFfEmpty=0), TCPRxFfRdEn can be asserted to 1 to read data from Rx data buffer.
(2) TCPRxFfRdData is valid in next clock cycle.
(3) Reading data must be immediately paused by de-asserting TCPRRxFfRdEn=0 when TCPRxFfEmpty=1.
(4) User must read all data from Rx data buffer before the connection is new created. All data in Rx data buffer is flushed and TCPRxFfFlush is asserted to 1 when the new connection is created. After finishing new connection created, ConnOn changes from 0 to 1.
(5) After finishing Flush operation, TCPRxFfEmpty is asserted to 1.
Figure 9: Rx FIFO interface timing diagram by using read counter
If user logic reads data as burst mode, TOE40G IP has read counter signal to show the total data stored in Rx FIFO interface as 256-bit unit. For example, in Figure 9 there are five data available in Rx data buffer. So, user can assert TCPRxFfRdEn to 1 for 5 clock cycles to read all data from Rx data buffer. The latency time from read enable (TCPRxFfRdEn) to read counter (TCPRxFfRdCnt) is 2 clock cycles.
EMAC interface of TOE40G-IP is designed to be FIFO in Show-ahead mode interface. To integrate TOE40G IP with 40G EMAC, adapter logic including Show-ahead FIFO is recommended as shown in Figure 10.
Figure 10: TOE40G IP and EMAC connection
Before transmitting packet, TOE40G IP monitors MacTxFfWrCnt to check if free space inside MacTxFIFO is enough to store one packet. After that, MacTxFfWrEn is asserted to 1 continuously until end of packet. MacTxFfWrData is valid at the same clock as MacTxFfWrEn=1. At the end of packet, MacTxEnd is asserted to 1 with the valid MacTxLastEmpty. MacTxSizeData is stable during transmitting each packet to show total data of the packet in 256-bit unit. The example of timing diagram when TOE40G IP sends packet and packet size = n is shown in Figure 11.
Figure 11: MacTxFIFO interface timing diagram
(1) TOE40G IP starts transmitting the packet by asserting MacTxFfWrEn to 1 with the valid data on MacTxFfWrData when free space of MacTxFf is enough for storing the packet (MacTxFfWrCnt <65535-n). The packet is transmitted continuously until the end of packet.
(2) During transmitting a packet, transmit packet size is valid on MacTxSizeData.
(3) MacTxEnd is asserted to 1 with the valid byte enable of the last data on MacTxLastEmpty signal when transmitting the last data of the packet.
Figure 12 shows the example of MacRxFIFO interface when Rx packet size is equal to n. MacRxFfEmpty is applied to monitor data status in MacRxFIFO. The interface uses Show-ahead FIFO characteristic, so MacRxFfRdData is valid at the same clock as MacRxFfRdEn=1.
Figure 12: MacRxFIFO interface timing diagram
(1) TOE40G IP starts reading the received packet when MacRxFfEmpty is de-asserted to 0. MacRxFfRdEn is asserted to 1 to read the data and MacRxFfRdData is valid in the same clock cycle which is Show-ahead FIFO characteristic. 256-bit received data is placed on bit[255:0].
(2) During reading the packet, if data is not ready to read (MacRxFfEmpty=1), MacRxFfRdEn is de-asserted to 0 to pause data reading in the same clock.
(3) Bit256 of MacRxFfRdData is asserted to 1 when the last data of the packet is transferred. Also, bit257 of MacRxFfRdData is valid to check error status in this clock. MacRxFfRdEn is de-asserted to 0 after receiving the last data for 8 clock cycles which is the time for IP post-processing the packet.
The example step to set register for transferring data in Client mode is shown as follows.
1) Set RST register=1 to reset the IP.
2) Set SML/SMH for MAC address, DIP/SIP for IP address and DPN/SPN for port number.
Note: DPN is optional setting when the port is opened by IP (Active open).
4) The new connection can be created by two modes.
a. Active open: Write CMD register = Open connection to create the connection (SYN packet is firstly sent from TOE40G IP). After that, wait until Busy flag is de-asserted to 0.
b. Passive open: Wait until ConnOn signal = 1 (the target device sends SYN packet to TOE40G IP firstly).
5) a. For data transmission, set TDL register (total transmit length) and PKL register (packet size). Next, set CMD register = Send Data to start data transmission. The user sends the data to TOE40G IP via TxFIFO interface before or after setting CMD register. When the command is finished, busy flag is de-asserted to 0. The user can set the new value to TDL/PKL register and then set CMD register = Send Data to start the next transmission.
b. For data reception, user monitors RxFIFO status and reads data until RxFIFO is empty.
6) Similar to creating the connection, the connection can be destroyed by two modes.
a. Active close: Set CMD register = Close connection to close the connection (FIN packet is firstly sent by TOE40G IP). After that, wait until Busy flag is de-asserted to 0.
b. Passive close: Wait until ConnOn signal = 0 (FIN packet is sent from the target to TOE40G IP firstly).
Comparing to Client mode which MAC address is decoded from ARP reply packet after TOE40G IP sends ARP request packet, Server mode decodes MAC address from ARP request packet. The process for transferring data is the same as Client mode. The example step of Server mode is shown as follows.
1) Set RST register=1 to reset the IP.
2) Set SML/SMH for MAC address, DIP/SIP for IP address and DPN/SPN for port number.
3) Set RST register=0 to start the IP initialization process by waiting ARP request packet to get Target MAC address. Next, the IP creates ARP reply packet returned to the target device. After finishing the initialization, busy signal is de-asserted to 0.
4) Remaining steps are similar to step 4 6 of Client mode
The TOE40G-IP Core functionality was verified by simulation and also proved on real board design by using Arria10 GX development board.
User must be familiar with HDL design methodology to integrate this IP into their design.
This product is available directly from Design Gateway Co., Ltd. Please contact Design Gateway Co., Ltd. For pricing and additional information about this product using the contact information on the front page of this datasheet.
Revision |
Date |
Description |
1.0 |
7-Dec-2018 |
New release |
1.1 |
2-Oct-2020 |
Update company info |