Core Facts
Provided with Core
Documentation	Reference design manual Demo instruction manual
Design File Formats	Encrypted File
Instantiation Templates	VHDL
Reference Designs & Application Notes	Vivado Project, See Reference design manual
Additional Items	Demo on KCU116 and VCK190
Support
Support Provided by Design Gateway Co., Ltd.

Design Gateway Co.,Ltd

E-mail: ip-sales@design-gateway.com

URL: design-gateway.com

Features

· Comprehensive TCP/IP offloading engine

· IPv4 protocol support without IP fragmentation

· Capability to handle 4 TCP sessions with the same target

· Jumbo frame support

· Configurable TCP buffer size: Up to 1 MB

· User data interface: Utilizes a 512-bit AXI4 stream interface

· Recommended clock frequency: A minimum of 220 MHz

· Ethernet MAC interface: Utilizes a 512-bit AXI4 stream interface

· Reference design available for using with KCU116 and VCK190

· Customized options for following features

· Additional buffer sizes

· Specified numbers of TCP sessions

· Support for the Ping command

Table 1: Example Implementation Statistics (UltraScale+)

Family	Example Device	Session	Buffer size (Tx and Rx)	Fmax (MHz)	CLB Regs	CLB LUTs	CLB¹	BRAM Tile	URAM	Design Tools
Kintex UltraScale+	XCKU5P-FFVB676-2E	1	64KB	330	22,273	20,478	4,049	45.5	-	Vivado2022.1
		1	1MB	330	22,296	20,728	4,203	15	64	Vivado2022.1
		4	64KB	330	33,199	33,853	6,511	137	-	Vivado2022.1
		4	256KB	330	33,228	33,726	6,391	15	64	Vivado2022.1

Table 2: Example Implementation Statistics (Versal)

Family	Example Device	Session	Buffer size (Tx and Rx)	Fmax (MHz)	CLB Regs	CLB LUTs	Slice¹	BRAM Tile	URAM	Design Tools
Versal AI Core	XCVC1902-VSVA2197-2MP-ES	1	64KB	330	22,226	21,044	4,340	44	-	Vivado2022.1
		1	1MB	330	22,245	20,781	4,329	15	61	Vivado2022.1
		4	64KB	330	33,385	35,373	7,196	131	-	Vivado2022.1
			256KB	330	33,411	33,926	6,720	15	64	Vivado2022.1
			1MB	330	33,446	34,041	7,144	15	244	Vivado2022.1

Notes: 1) Actual logic resource dependent on percentage of unrelated logic

Applications

The TOE100GADV IP core serves as a fully offloading engine designed for TCP/IP packet processing, eliminating the need for CPU and external memory usage. This IP allows for the seamless transmission of TCP/IP packets at the full 100G Ethernet line rate.

Figure 1 illustrates an example of TOE100GADV IP core integration to an FPGA Network Interface Card, serving as a high-performance TCP Accelerator engine for managing four concurrent TCP sessions.

Figure 1: TCP Accelerator System

Within the Host system, the host processor takes on the responsibility of selecting four performance-sensitive TCP sessions, handled by the TOE100GADV IP. Concurrently, the host processor processes the remaining Ethernet packets via Host2MAC module to support the remaining TCP sessions and other protocols that do not require high-performance handling. User data processed by TOE100GADV IP is directly transferred to the Host memory by DMA engine, allowing applications running on the host processor to access this data.

In contrast, other Ethernet packets transferred through the Host2MAC module are stored separately within the host memory, awaiting the host processor decoding before they can be utilized by the applications. The incorporation of the TOE100GADV IP core significantly enhances the performance of TCP data transfer within the Network interface application.

As illustrated in Figure 2, another application scenario for TOE100GADV IP revolves around the utilization of an FPGA card to implement NVMe/TCP Initiator system.

Figure 2: NVMe/TCP Initiator System

The NVMe/TCP protocol facilitates remote access to storage systems equipped with NVMe SSDs through network connectivity. It requires at least two TCP sessions for the distinct transfer of commands and data. The incorporation of the TOE100GADV IP core empowers the system to concurrently handle up to four TCP sessions. Consequently, the Initiator logic can write and read data with two NVMe SSDs installed within the target system concurrently. Eliminating the necessity for TCP/IP implementation within the Initiator logic simplifies the logic design, allowing it to be exclusively dedicated to supporting NVMe commands. Using a pure-hardwired logic design optimizes data transfer performance, making it possible to upload high-resolution video data to remote storage in real-time.

General Description

Figure 3: TOE100GADV IP Block Diagram

The TOE100GADV IP core serves as a complete offloading engine designed for processing TCP/IP packets. It provides four distinct user interfaces to facilitate the transfer of TCP payload data across four separate TCP sessions. While most network parameters are uniform across these sessions, the TCP port number can be individually configured for each. Moreover, the size of the TCP buffer, for both transmit and receive directions within each TCP session, can be configured independently to accommodate the specific characteristics of each session. The TOE100GADV IP utilizes a 512-bit AXI4-ST interface for Ethernet packet transfer to and from the Ethernet MAC. However, the 100G Ethernet (MAC) Subsystem from Xilinx contains distinct data width and different operative clock. This leads to the necessity to include an adapter logic (MACTx/RxIF) connecting between TOE100GADV IP and the Ethernet Subsystem.

The TOE100GADV IP’s logic can be categorized into three main blocks: Control block, Transmit block, and Receive block. The Control block is connected to the Control I/F of the user interface and is responsible for configuring commands, managing network parameters, and monitoring the operational status of the IP. Within the Control block, the MainCtrl component receives user command requests and stores associated user parameters in Param Regs. After validating the command, MainCtrl generates a command request for the respective TCP engine. The system incorporates four TCP engines, each dedicated to managing the operations of individual TCP sessions.

The Transmit block manages the TCPTx I/F, facilitating the transmission of TCP payload data through a 512-bit AXI4 stream I/F. Data received from the user interface must be re-aligned before being stored in the TxTCP buffer. When a user requests data transmission, the Packet builder retrieves payload data from the TxTCP buffer to construct TCP/IP packets, which are then transferred to the EMAC module.

Conversely, the Receive block manages the TCPRx I/F, receiving TCP payload data from the Ethernet (MAC) Subsystem via a 512-bit AXI4 stream I/F. The operation of the Receive block is reverse of the Transmit block. Received packets undergo validation through Packet Filtering, and the TCP payload data extracted from valid packets is subsequently stored in the RxTCP buffer. When data is available in the RxTCP buffer, and the user is ready to receive it, the RxAligner transfers the received data to the user.

For additional details about each submodule within the TOE100GADV IP, please refer to the following section.

Functional Description

The TOE100GADV IP has two sets of functions: common functions, which are employed for every TCP session, and session-specific functions that operate within each session. The common functions facilitate communication with the Target device, while the session functions are responsible for managing individual TCP session, as illustrated in Figure 4.

Figure 4: TOE100GADV IP Functional diagram

Common States

1. Reset: When the TOE100GADV IP is powered-on, the default state is Reset. This state is designed to wait for the user to configure network parameters for both the host and the target, including MAC address and IP address. Once all parameters, including the initialization mode (DstMacMode), are set, the user can de-assert RstB to 1b to initiate the target’s initialization.

2. Target Initialization: This state is used to obtain the MAC address of the target and store the user-assigned network parameters in internal registers. There are three modes, determined by DstMacMode, for acquiring the target’s MAC address: Client (generating ARP request packet), Server (waiting for ARP reply packet), and Fixed-MAC (user-defined value). Once the target’s MAC address is successfully obtained, the initialization process is completed, and the TOE100GADV IP is ready to establish a connection with the target.

3. Target Ready: After successful target initialization, the main function remains in this state until a reset is asserted. In this state, the user can send command request specifying the active session (Session#0-Session#3) and the operation (Open, Close, and Reset) to be performed by the main function. If the operation request is invalid, such as sending a Close request to an inactive session, the request will be rejected. Valid requests are forwarded to the session-specific function, which returns either a completion or an error status to the main function after executing the operation.

Additionally, it is important to note that the session’s status may also change without any request from the main function if the target device sends a request to open, close, or reset the session.

Session States

1. Session Inactive: Following a reset, each session function enters the Inactive state. In this state, the session function awaits an open session request from the main function (active open) or the target device (passive open). Session establishment is initiated upon receiving the open session request.

2. Create Session: Session creation involves a three-way handshake process using SYN, SYN-ACK, and ACK packets. The first SYN packet can be sent by either the TOE100GADV IP (active open) or the target (passive open). Another device responds with a SYN-ACK packet if it accepts the connection request. Finally, the process is completed when the first device that initiated the SYN packet sends an ACK packet. Once the connection is successfully established, it becomes ready for data transfer, and the session function reports its status upon completing this process.

3. Session Ready: The TCP protocol facilitates bidirectional data transfer, allowing for the simultaneously transmission and reception of data within the same session. This is achieved by assigning unique sequence and acknowledge numbers for each data transfer direction to indicate the position of transmitted and accepted data. However, Figure 4 illustrates the separate representation of ‘Send data’ and ‘Receive data’ to show the fundamental concept of the TOE100GADV IP’s functionality.

4. Send Data: Each session possesses its dedicated data interface, compatible with a 512-bit AXI4-ST interface, designed for transmitting data to the target. However, TOE100GADV IP permits only one session to transmit packets at a time. Each session includes a TxTCP buffer for storing data transmitted by the user. The sequence number of the transmitted packet is updated after data is sent from the TxTCP buffer to the target. The data is removed from the buffer once the target updates the acknowledge number to confirm the amount of received data. If all data is successfully transmitted, the session state returns to the ‘Ready’ state. If the session is terminated, the TxTCP buffer is automatically flushed.

5. Receive Data: Each session includes an RxTCP buffer for storing data received from the target. In accordance with the TCP protocol, the available space in the RxTCP buffer corresponds to the value of the received Window size specified in the transmitted packet. Consequently, the target can continuously send data to TOE100GADV IP without waiting for an updated acknowledge number until the total amount of data equals to the Window size. While receiving data, the sequence number of received packet is monitored to ensure the correctness of the received data sequence. Packet retransmission is triggered if any received data is lost during this process. Once the RxTCP buffer is completely filled with data, the TOE100GADV IP generates a packet to update the acknowledge number, which is then sent back to the target. The RxTCP buffer is automatically cleared during the session establishment process. User can read data from the RxTCP buffer through the 512-bit AXI4-ST interface for each session individually, so simultaneous data reading from multiple sessions is possible.

6. Terminate Session: Session can be terminated either by the TOE100GADV IP (active close) or the target (passive close). The first packet used to initiate connection termination is the FIN packet. If termination request is accepted by another device, it responds with FIN and ACK packets, which can be transmitted in single or multiple packets. Finally, the initiating device sends an ACK packet to complete the process. Upon the successful termination, the session function transitions back to the Inactive state. The session function provides a status update upon the completion of this termination process.

The hardware modules are organized into three distinct blocks: Control, Transmit, and Receive. Each block is responsible for specific functions, as outlined below.

Control Block

· MainCtrl

The main controller, known as MainCtrl, involves the main functions (common block in Figure 4), which were elaborated upon in the preceding section: Reset, Target initialization, and Target Ready. It decodes the user-assigned commands and parameters to determine the subsequent processes.

For instance, it manages distinct sequences for transmitting and receiving packets to accommodate three types of initialization process. It generates various request types for the transmit and receive blocks, facilitating the creation or interpretation of ARP request and ARP reply packets. Once the MAC address of the target is acquired, the result is stored in Param Regs for utilization by other submodules.

In scenarios where a user’s command cannot be supported under specific conditions, the main controller will reject the command and issue an error status.

· Param Regs

The parameters stored at Param Regs can be received from the user assignment or retrieved from the receive block. The user-defined parameters include both the host and the target parameters such as the MAC address, the IP address, and the port number. While the decoded parameters from the receive block are the target parameters such as the MAC address and the port number which are used in specific conditions.

Furthermore, various configuration parameters, including timeout values, initialization mode, and threshold values, are also assigned by user and stored within Param Regs for utilization by the main controller.

· TCP Engine

The TCP Engine manages the operations of each individual TCP session, as described in the session-specific functions shown in Figure 4.

The establishment of a connection can be triggered by either of two devices. In active open mode, the TCP engine prompts the transmit block to send SYN and ACK packets while instructing the receive block to validate a SYN-ACK packet. Conversely, for passive open mode, the packet types of the transmit block and the receive block are reverse.

To send data, once the amount of data in the TxTCP buffer reaches a certain threshold, the TCP engine calculates the sequence number for the transmitted packet. It instructs the transmit block to continuously send data until the total amount of transmitted data reaches the designated value, determined from the latest target window size retrieved by the receive block. The TCP engine will pause the transmit module if the window size and acknowledge number from the target are not updated. In cases of lost transmitted data, which are indicated by the generation of duplicate ACK packets by the target, the TCP Engine will suspend the operation of the transmit block and generate the new parameter sets to initiate data retransmission through the transmit block. TOE100GADV IP provides a status signal indicating the remaining amount of transmitted data yet to be received by the target, allowing the user to monitor the progress of the transmission operation. The send operation is considered completed when there is no remaining transmitted data in the TxTCP buffer.

To receive data, the TCP engine continuously monitors the sequence number of the received packet, retrieved from the receive block. This information is used to calculate the quantity of received data. When the received data reaches a specified threshold, the TCP engine computes the acknowledge number and configures it to the transmit block. This action triggers the sending of a packet, which serves to update the current values of acknowledge number and the window size of the RxTCP buffer. If the sequence number of a received packet is skipped, it indicates a data loss situation. In such cases, the TCP engine instructs the transmit block to generate duplicated ACK packets for the recovery of the missing data. Additionally, the TCP engine offers the user the flexibility to define a threshold value for generating TCP window update packets. This feature is useful when the user logic stops reading data from the RxTCP buffer until it is full. During this period, the target cannot send more data to this session. However, once the user resumes data reading and the free space within the RxTCP buffer increases to the specified threshold value, the TCP engine instructs the transmit block to send an ACK packet, updating the TCP window size based on the RxTCP buffer’s free space. Then, the target can resume data transmission.

The connection termination can be done by either of two devices. In active close mode, the TCP engine prompts the transmit block to send FIN and ACK packets while instructing the receive block to validate FIN and ACK packets. Conversely, for passive close mode, the packet types of the transmit block and the receive block are swapped.

Transmit Block

· Tx Aligner

User logic transmits data to TOE100GADV IP using a 512-bit AXI4-ST interface that is consistently valid for all 512-bit data, except for the last data beat. This last data beat can be configured to specify the number of valid bytes, indicated by the ‘keep’ signal. Consequently, TxAligner is responsible for adjusting the alignment of the subsequent transmitted data when the previous one is not aligned. A Tx Aligner module is shared among all four transmitted data interfaces used by the user, so only one interface can transfer data at a time. Since the same Ethernet MAC is shared by four TCP sessions, one user interface provides sufficient data bandwidth to achieve the maximum performance of 100G Ethernet. The realigned data from TxAligner is then stored in one of four TxTCP buffers, depending on the active user number.

· TxTCP Buffer

The TxTCP buffer serves as the buffer for transmitted data from the user in each TCP session. The size of this buffer is configurable through the parameter ‘TxBufBitWidth<i>’, where ‘i’ corresponds to the session number (ranging from 0 to 3). A mapping table to show the relationship between ‘TxBufBitWidth’ and the actual size of TxTCP buffer is provided in Table 3.

Table 3: TxBufBitWidth and RxBufBitWidth parameter description

Configured value	Buffer size
9	32KB
10	64KB
11	128KB
12	256KB
13	512KB
14	1MB

The maximum data payload size of each transmitted packet is constrained by the Maximum Segment Size (MSS) of the target device, which is obtained by the receive block during the connection establishment process. If the amount of transmitted data exceeds this limitation, the data must be divided and transmitted across multiple packets. Each segmented payload data for each TCP packet is then sent to the Packet Builder to construct complete TCP packets.

The transmitted data remains stored in the TxTCP buffer until the TCP Engine confirms that the data has been received by the target. Consequently, having a large TxTCP buffer size allows for continuous data transmission to the target without waiting for updated ACK packets from the target for extended periods. If the target finishes processing the received data and issues an ACK packet before the TxTCP buffer reaches its capacity, data transmission can be transferred seamlessly without pauses, enabling the achievement of the maximum data rate of 100G Ethernet. However, it is essential to note that this advantage does come at the cost of significant FPGA memory resources.

· Packet Builder

To encapsulate TCP payload data within an Ethernet packet, various layers of header data must be appended, including parameters such as MAC addresses, IP addresses, port numbers, sequence number, acknowledge number, window size, TCP flags, and checksums. This module is responsible for computing the checksums for both the TCP and IP layers and substituting these calculated values into the packet header. Once the header is fully assembled, it is merged with the segmented payload data obtained from the TxTCP buffer before being transmitted to the Ethernet (MAC) Subsystem.

In addition to creating TCP/IP packets, the packet builder must also support for generating ARP request and ARP reply packets, which may be required during the IP initialization process.

Since the Packet Builder is shared among four TCP sessions, it must be capable of switching the active TCP session after transmitting a packet for one session to handle packets for the remaining sessions once their data is ready for transmission.

Receive Block

· RxPac Buffer

Ethernet (MAC) Subsystem can transmit Ethernet packets to TOE100GADV IP at any time, whereas the TOE100GADV IP requires a short pause time while processing each Ethernet packet. Consequently, the RxPac buffer must be included for storing Ethernet packets when the receiving logics of TOE100GADV IP are not ready to accept new data. In specific scenarios, the RxPac buffer may become full, leading to the incomplete storage of incoming packets. As a result, the interrupt is asserted by TOE100GADV IP at TOEMACRx I/F to signal that the most recent packet has been discarded. However, if TCP engine detects the loss of a packet, it will initiate a data retransmission process to recover the missing packets. Increasing the frequency of the user logic’s clock domain (Clk) decreases the likelihood of the RxPac buffer reaching full capacity.

· Packet Filtering

This module is responsible for validating incoming packets. To be considered valid, a packet must meet the following criteria.

1) Network parameters, such as MAC address, IP address, and Port number, must match the user-defined settings established during initialization.

2) The packet must either be an ARP packet or a TCP/IPv4 packet without a data fragment flag.

3) The IP header length must be 20 bytes, and the TCP header length must be within the range of 20 to 60 bytes.

4) Both the IP checksum and TCP checksum must be correct.

5) The data pointer, as determined by the sequence number, must be within a valid range.

6) The acknowledge number must also be within an acceptable range.

7) Differentiated Services Code Point (DSCP) and Explicit Congestion Notification (ECN) values within the IP header must be equal to 0, while Time To Live (TTL) must not be equal to 0.

If a packet meets these validation criteria and contains TCP payload data, the Packet Filtering will extract the TCP payload data and store it in the appropriate RxTCP buffer, determined by the port number value. After the received packet has been fully processed, a signal is generated to notify the TCP Engine to update the current status. In specific scenarios, the TCP Engine may request the Transmit block to generate a packet with updated information, such as acknowledge number and window size, for transmission to the target.

· RxTCP Buffer

The RxTCP buffer serves as the buffer for received TCP payload data from the target in each TCP session. Its size can be configurable through the parameter ‘RxBufBitWidth<i>’, where ‘i’ corresponds to the session number (ranging from 0 to 3). Table 3 provides a mapping table to show the relationship between ‘RxBufBitWidth’ and the actual size of RxTCP buffer.

Data received from the target remains stored in the RxTCP buffer until the user reads it via the 512-bit AXI4-ST I/F. The target continues to transmit TCP payload data until the RxTCP buffer becomes full. Therefore, using a larger RxTCP buffer size increases a chance that the target can continuously send data to the TOE100GADV IP without a pause if the user consistently reads data to free the buffer during operation. As a result, a peak performance can be achieved. Besides, the TCP Engine utilizes the remaining space of RxTCP buffer as the TCP window size in transmitted packets when it needs to update information to the target.

· Rx Aligner

The purpose of the Rx Aligner is to transfer and align received data from the RxTCP buffer to the user via the 512-bit AXI4-ST interface. The 512-bit data from RxTCP buffer can be directly transmitted to the user logic without requiring alignment for every data beat, except for the last beat, which may contain only a portion of valid bytes. Data that follows unaligned transmission to the user logic must realigned by the Rx Aligner. This subsequent data also remains unaligned, with the last beat flag set to facilitate the re-alignment of the data output from the RxTCP buffer. Subsequently, the 512-bit data output from the RxTCP buffer aligns with the 512-bit AXI4-ST interface of the user, restoring it to the same status as before the data unalignment transmission. More details of the Rx alignment function are provided in Figure 23.

User Logic

As illustrated in Figure 3, the user logic is responsible for managing three distinct user interfaces of the TOE100GADV IP. These interfaces include the Control I/F, utilized for network and system parameter configuration and status monitoring; the TCPTx I/F, designed for data transmission; and the TCPRx I/F, intended for the reception of data streams.

Configuring network and system parameters for the TOE100GADV IP, such as timeout values, can be accomplished by specifying constant values or by utilizing user-defined registers. Furthermore, overseeing the signals responsible for initiating command requests and monitoring operational status can be achieved through a state machine. This approach facilitates the customization of command values and enables real-time tracking of operations, including error detection.

Both the Tx and Rx data interfaces employ a 512-bit AXI4-ST bus interface. This design choice offers users flexibility, enabling them to map the Data interface of the TOE100GADV IP to other components that are compatible with the AXI4-ST bus. This facilitates efficient data handling and processing.

100G Ethernet (MAC) Subsystem

The 100G Ethernet (MAC) Subsystem implements the MAC layer with the low-layer protocol, but the interface and features vary depending on the FPGA model. Xilinx offers a 100G Ethernet Subsystem for the UltraScale+ device that implements both the MAC and Physical layers. This interface uses a 512-bit AXI4 stream running at 322.265625 MHz. More information can be found on the following website.

https://www.xilinx.com/products/intellectual-property/cmac_usplus.html

For the Versal device, the 100G Ethernet MAC Subsystem implements the MAC layer and PCS logic but does not include the transceiver. The user interface, when using Non-Segmented mode and independent clock mode, is a 384-bit interface running at 390.625 MHz or higher. More details can be found on the following website.

https://www.xilinx.com/products/intellectual-property/mrmac.html

According to the interfaces of 100G Ethernet (MAC) Subsystem, an adapter logic may be required to seamlessly connect 512-bit AXI4-ST I/F of TOE100GADV IP with the 100G Ethernet Subsystem.

Core I/O Signals

Table 4 provides detailed descriptions of configurable parameters, while Table 5 outlines the I/O signals for TOE100GADV IP. The EMAC interface matches the 512-bit AXI4 stream standard.

Table 4: Core Parameters

Name	Value	Description
NumberSession	1-4	Setting the number of TCP sessions.
TxBufBitWidth0-3	9-14	Setting TxTCP buffer size. Table 3 shows the relation between this value and the actual size.
RxBufBitWidth0-3	9-14	Setting RxTCP buffer size. Table 3 shows the relation between this value and the actual size.

Table 5: User I/O Signals

Signal	Dir	Description
User Control Interface (Common)
RstB	In	Reset IP core. Active Low. Once RstB changes to 1b, the IP initiates the initialization process. Note: Before de-asserting RstB to 1b, the values of DstMacMode, SrcMacAddr, DstMacAddrIn (if used), SrcIPAddr, DstIPAddr, WindowThres, TCPCtlTimeOutSet, and TCPRxTimeOutSet must be stable. Also, these values must be sustained for 4 clock cycles after the de-assertion of RstB.
Clk	In	User clock. Recommended to set a clock frequency of at least 220 MHz to minimize the chance of RxPac buffer being overflow.
IPVersion[31:0]	Out	Represents the IP version number.
TestPin[63:0]	Out	Reserved to be the IP test point.
DstMacMode[1:0]	In	Defines the options for obtaining the MAC address of the target, with additional details provided in Figure 5 - Figure 7. 00b – Client mode. In this mode, the IP generates an ARP request packet during IP initialization, and the target MAC address is extracted from the received ARP reply packet. 01b – Server mode. In this mode, the IP awaits an incoming ARP request packet during IP initialization, and the target MAC address is extracted from the received ARP request packet. 1Xb – Fixed-MAC mode. This mode allows the user to specify the target MAC address using the ‘DstMacAddrIn’ value, assigned by the user. Note: When the host and target are located on different network and connected through a switch, the ‘Fixed-MAC’ mode must be utilized. In such cases, the ’DstMacAddrIn’ value should be set to the MAC address of the host’s switch.
SrcMacAddr[47:0]	In	Defines the host MAC address.
DstMacAddrIn[47:0]	In	Defines the target MAC address, used in Fixed-MAC mode.
DstMacAddrOut[47:0]	Out	Displays the target MAC address. Valid after InitFinish set to 1b (IP initialization is completed).
SrcIPAddr[31:0]	In	Defines the host IP address.
DstIPAddr[31:0]	In	Defines the target IP address.
WindowThres[9:0]	In	Defines the threshold value, measured in 1KB units, to determine when the ‘TCP Window Update’ packet is sent to the target. It is valid within the range of 0 to 1023. The default value is 0, indicating that this feature is disabled. It becomes enabled when other values are set. It is recommended to be configured to one-fourth of the RxTCP buffer size. During receiving data from the target, the IP generates ACK packets containing the TCP window size. If the remaining TCP windows size is insufficient, the target temporarily halts data transmission until the TCP window size increases. Once the user reads data and the expanded size of the RxTCP buffer reaches or surpasses this threshold value, the IP transmits a ‘TCP Window Update’ packet to update the TCP window size. Upon detecting adequate free space, the target resumes transmitting the next data.

Signal	Dir	Description
User Control Interface (Common)
TCPCtlTimeOutSet [31:0]	In	Defines timeout value, ranging from 1 to 0xFFFF_FFFF. The timer operates in sync with the ‘Clk’ signal, with a time unit equivalent to the ‘Clk’ signal’s time period (4 ns for 250 MHz). It counts while awaiting returned packets from the target during each operation. When a timeout occurs, ‘TCPRtrInt’ is asserted, and the status is returned via the ‘TCPRtrIntStatus’ signal, initiating the recovery process. The optimal value depends on system requirements and network characteristics, but it is typically set to 1 second or larger. An example of its usage is presented in Figure 26.
TCPRxTimeOutSet [23:0]	In	Similar to ‘TCPCtlTimeOutSet’, the timeout operates in sync with the ‘Clk’ signal. Its value ranges from 1 to 0xFF_FFFF. The same timeout value is applied for all TCP sessions. This timer counts when the RxTCP buffer holds the last unaligned data (not 512 bits). When a timeout is triggered, the TOE100GADV IP flushes the last data to the user via TCPRx I/F, with ‘TCPRxLast<i>’ asserted to 1b. The optimal value depends on the pause time of the target for transmitting data. The reference design sets this value to 1 msec. An example of its usage is presented in Figure 23.
InitFinish	Out	Represents the completion status of IP initialization process. It is set to 0b when the IP is reset and changes to 1b once the initialization process is completed.
User Control Interface (Session)
TCPConnCmd[3:0]	In	Defines the command value corresponding to the requested session. [1:0] – Command 00b: Active open. The IP initiates connection establishment by sending a SYN packet. 01b: Passive open. The IP waits for the target to initiate connection establishment. Note: Active open and Passive open commands can be issued when ‘InitFinish’=1b, ‘TCPConnOn[i]’=0b, and ‘TCPConnStatus[(64i)+4]’=0b, where ‘i’ represents the session number defined by ‘TCPConnCmd[3:2]’. Once a connection is established successfully, ‘TCPConnOn[i]’ changes to 1b. 10b: Active close. The IP initiates connection termination by sending a FIN packet. This command should be issued when ‘TCPConnOn[i]’=1b, where ‘i’ is session number, and there is no remaining data stored in TxTCP buffer for that session. 11b: Reset. This command allows the user to reset only specified session without interrupting the operation of other sessions to recover from unusual phenomena. [3:2] – Session number to request the command. 00b: Session#0, 01b: Session#1, 10b: Session#2, and 11b*: Session#3. Additional information on how to utilize TCPConnCmd is illustrated in Figure 8 - Figure 17.
TCPSrcPort[15:0]	In	Defines the host port number.
TCPDstPort[15:0]	In	Defines the target port number. This value is not used if operating Passive open.
TCPLastMode[1:0]	In	Configures the last transmission mode to determine how the last packet is sent to accelerate the target’s response. Reducing response time can enhance transfer performance. 00b – Duplicated data. The IP sends the last packet twice, causing the target to recognize the second packet as a retransmission. Consequently, the target responds with an acknowledge packet, thereby completing the data transfer from the host. 1Xb - PSH flag. The IP sets the PSH flag in the TCP header of the last packet to 1b. This flag instructs the target to immediately forward data to the application layer, bypassing any buffering. Consequently, an acknowledge packet is sent to confirm the acceptance of the data. 01b – Normal packet (No PSH or Duplicated packet). In this mode, a standard packet is generated without any additional setting to accelerate the target’s response. Figure 19 and Figure 20 provide the details of the configuration for the last transmission mode.

Signal	Dir	Description
User Control Interface (Session)
TCPConnReq	In	Sets this signal to 1b for a single clock cycle to initiate a command request when ‘TCPConnReady’=1b. During this time, ‘TCPConnCmd’ must contain valid data. Additionally, while setting this signal to 1b and assigning ‘TCPConnCmd’ as Active open or Passive open, ‘TCPSrcPort’, ‘TCPDstPort’ (only used for Active open), and ‘TCPLastMode’ are loaded.
TCPConnReady	Out	Sets to 1b when the IP is ready to accept the new command. Once ‘TCPConnReq’ is set to 1b, this signal is then de-asserted to 0b for 2 clock cycles to prevent any further command requests.
TCPConnCpl	Out	Sets to 1b for a single clock cycle to signal the completion of a command operation. The status of the command completion is conveyed through ‘TCPConnCplStatus’.
TCPConnCplStatus[4:0]	Out	Represents the command completion status. Bit[3:0] aligns with ‘TCPConnCmd’ to indicate which command has been completed or if an error has occurred. [4] – Completion status. 0b: Fail, 1b: Success.
TCPConnOn[3:0]	Out	Represents the active status of individual TCP sessions, with each bit corresponding to a specific session. Bit[0], [1], [2], and [3] show the status of session#0, #1, #2, and #3, respectively. 0b – No connection has been established, 1b – Connection has been established.
TCPConnStatus[255:0]	Out	Represents the status decoded by the TCP engine for individual TCP sessions. For each TCP session, a 64-bit structure is utilized, organized as follows: [63:0], [127:64], [191:128], and [255:192], which correspond to session#0, #1, #2, and #3, respectively. Since the structure of the 64-bit status is similar across all sessions, only the details for bit[63:0] of session#0 are provided as follows. [3:0] – Current State: 0000b: No connection. Awaits a request to open the connection. 0001b: Active open processing. The IP has received an Active open command request. If the three-way handshake (SYN, SYN-ACK, and ACK) is completed, it transitions to 0100b. 0010b: Connection listening. The IP has received a Passive open command request and awaits a SYN packet. Upon receiving the SYN packet, it transitions to 0011b. 0011b: Passive open processing. If the three-way handshake is completed, it transitions to 0100b. 0100b: Connection ready. Data can be transferred in this session. In this state, the user or the target can request connection termination. 0101b: Queuing for Active close processing. This state is temporary and occurs after receiving an Active close command request but before transitioning to 0110b. It may persist for an extended duration if data is still being transmitted to the target. Once all data transmission is successfully completed, it transitions to 0110b. During this time, if a FIN packet is received, it triggers the initiation of a passive close operation. 0110b: Active close processing. The IP initiates packet transfer of FIN, FIN and ACK, and ACK with the target. Upon completion of the operation, it transitions to 0000b. 0111b: Passive close processing. The IP has received a FIN packet from the target while packets for terminating the connection are still being transferred. After the operation is completed, it transitions to 0000b. 1000b: Unusual condition detection. The IP sends RST packet, and its value returns to 0000b. [4] – Packet transmission status. Set to 1b while there are remaining packets for transmission. [15:5] – Reserved

Signal	Dir	Description
User Control Interface (Session)
TCPConnStatus[255:0]	Out	[52:16] – Information about the target, decoded from the received packet during connection establishment. These bits are valid only after the connection is established. · [31:16] – Target port number. · [47:32] – Maximum segment size (MSS) of the target. · [51:48] – Window scaling factor of the target. This value is valid when bit[52] is set to 1b. · [52] – Validation flag for window scaling factor. 0b - Window scaling feature is disabled, 1b – Window scaling feature is enabled. [63:53] - Reserved
TCPRtrInt	Out	Retry interrupt signal triggered when the IP detects lost packets and initiates packet retransmission to recover lost packets. It is set to 1b for a single clock cycle, and status information can be retrieved from ‘TCPRtrIntStatus’.
TCPRtrIntStatus[79:0]	Out	The retry interrupt status signal is divided into two sections: one for common functions shared among all four TCP sessions, assigned to bit[15:0], and the other for session-specific functions, with 16 bits allocated for each session individually. Bit[31:16], [47:32], [63:48], and [79:64] correspond to the status of session#0, #1, #2, and #3, respectively. The details of the 16-bit session status are similar, so only bit[31:16] for session status and bit[15:0] for common status are described as follows. [15:0] - Common functions [0] – Timeout while waiting for an ARP reply packet during initialization in Client mode. The IP retries sending an ARP request packet to the target until it receives an ARP reply packet. [15:1] – Reserved. Figure 26 illustrates an example of the TCPRtrIntStatus assertion for common functions. [31:16] - Session#0 functions [16] – Timeout while waiting for a SYN-ACK packet during an active open command. The IP retransmits a SYN packet. [17] – Timeout while waiting for an ACK packet during passive open operation. The IP retransmits a SYN-ACK packet. [18] - Reserved [19] - Timeout while waiting for an ACK packet during passive close operation. The IP retransmits FIN and ACK packets. [20] – Interrupt triggered by detecting a lost ACK packet or a duplicate ACK packet, which initiates the data retransmission by the IP to prompt the target to return an ACK packet. [21] - Timeout while waiting for a sufficient window size from the target to continue data transmission. In response, the IP transmits a keepalive packet to prompt the target to return an ACK packet, updating the window size value. [22] – Interrupt triggered by detecting lost received data. The IP generates duplicate ACK packets to request the retransmission of the lost data. [23] – Interrupt triggered by detecting a reverse packet. The IP sends ACK packet as a response. [31:24] – Reserved. Figure 27 illustrates an example of the TCPRtrIntStatus assertion for session functions.
TCPRstInt	Out	Reset interrupt signal triggered when the IP detects an unrecoverable issue requiring connection termination. It is set to 1b for a single clock cycle, and status information can be retrieved from ‘TCPRstIntStatus’. The TCPRstInt may be asserted after the 15^th retry process or without any preceding retry attempts. Examples of the reset interrupt assertion are depicted in Figure 29, showcasing the scenario after the retry process, and Figure 30, illustrating the situation without preceding retry attempts.

Signal	Dir	Description
User Control Interface (Session)
TCPRstIntStatus[79:0]	Out	The reset interrupt status signal. Bit[79:16] are assigned to the status of four sessions, with 16 bits allocated per session, mirroring the ‘TCPRtrStatus’ signal structure. Therefore, only bit[31:16] for session#0 status is described as follows. [15:0] – Reserved for common functions. [31:16] - Session#0 functions [16] – The IP transmits RST packet after the 15^th retransmission of TCPRtrIntStatus[16] condition. [17] – The IP transmits RST packet after the 15^th retransmission of TCPRtrIntStatus[17] condition. [18] – The IP transmits RST packet upon detecting a timeout while waiting for FIN and ACK packets during an active close command. [19] - The IP transmits RST packet after the 15^th retransmission of TCPRtrIntStatus[19] condition. [20] - The IP transmits RST packet after the 15^th retransmission of TCPRtrIntStatus[20] condition. [21] – The IP transmits RST packet after the 15^th retransmission of TCPRtrIntStatus[21] condition and the target does not respond any ACK to the IP. [23:22] – Reserved. [24] – The IP receives RST packet from the target, resulting in connection termination. [25] – The IP transmits RST packet after receiving an unsupported packet from the target. For example, a SYN packet is received while the connection has already been established. [31:26] – Reserved
User Tx Data Interface (Use <i> as the session index, ranging from 0 to 3)
TOETxReady<i>	Out	Asserted to 1b to indicate that the ‘TOETxData<i>’ has been accepted. Although four signals are used to control data transfer of each session individually, only one of them can be set to 1b at a time to accept data from a specific session.
TOETxData<i>[511:0]	In	Transmitted data sent to the IP through session#i. It is valid when ‘TOETxValid<i>’ is set to 1b.
TOETxValid<i>	In	Asserts to 1b to transmit the ‘TOETxData<i>’ to the IP.
TOETxLast<i>	In	Asserts to 1b to indicate that this is the last data. This signal is valid when ‘TOETxValid<i>’ is set to 1b. Once this signal is set, the IP will transfer all remaining data stored in the TxTCP buffer to the target without waiting for additional data. The last data is transferred following the ‘TCPLastMode’ configuration.
TOETxKeep<i>[63:0]	In	Byte enable to indicate the number of valid bytes at this transfer cycle. Bit[i] indicates that byte[i] of TOETxData contains valid data. All 512-bit data must be valid during data transfer by asserting all bits of this signal to 1b, except for the last data. At the last data transfer cycle, this signal can be set to specify the number of valid bytes from 1-64. For example, if ‘TOETxKeep<i>’=0000_0000_0000_007Fh (indicating 7 valid bytes), only bit[55:0] of TOETxData<i>’ is considered valid.
TOETxStat<i>[31:0]	Out	Represents the status signal of User Tx Data interface. [19:0] – Represents the amount of data stored in the TxTCP buffer, in byte unit. This value resets to 0 after the target has accepted all the data or when the connection has been terminated. [28:20] – Reserved [30] – Set to 1b during the execution of the connection termination. During termination process, ‘TOETxReady<i>’ needs to be set to 0b to halt additional data transmission. [31] – Set to 1b for a short time to flush data inside the TxTCP buffer after the connection has been terminated. Figure 16 provides an example demonstrating the behavior of TOETxStat during the process of connection termination.

Signal	Dir	Description
User Rx Data Interface (Use <i> as the session index, ranging from 0 to 3)
TOERxReady<i>	In	Asserts to 1b when the ‘TOERxData<i>’ has been accepted.
TOERxData<i>[511:0]	Out	Received data of session#i. It is valid when ‘TOERxValid<i>’ is set to 1b.
TOERxValid<i>	Out	Asserted to 1b to transmit the ‘TOERxData<i>’.
TOERxLast<i>	Out	Asserted to 1b to indicate that this is the last data. At this point, the user should check the amount of valid data by reading ‘TOERxKeep<i>’. This signal is valid when ‘TOERxValid<i>’ is set to 1b.
TOERxKeep<i>[63:0]	Out	Byte enable to represent the number of valid bytes in the current transfer cycle. Bit[i] indicates that byte[i] of TOERxData contains valid data. During data transfer, all 512-bit data must be valid, except for the last data. This value is typically set to all ones for each transfer cycle, except for the last cycle, where it specifies the number of valid bytes from 1-64. For instance, if ‘TOERxKeep<i>’=0000_0000_0000_0003h (indicating 2 valid bytes), only bit[15:0] of ‘TOERxData<i>’ is considered valid.
TOERxStat<i>[31:0]	Out	Represents the status signal of User Rx Data interface. [19:0] – Represents the amount of data stored in the RxTCP buffer, measured in byte unit. This value resets to 0 after the user reads all data or when the new connection has been established. [30:20] – Reserved [31] – Set to 1b for a short time to flush data inside the RxTCP buffer after a new connection has been established. Figure 11 and Figure 12 provide the examples demonstrating the behavior of TOERxStat during the process of connection establishment.
TOEMAC Tx Interface
TOEMACTxReady	In	Asserts to 1b when the ‘TOEMACTxData’ has been accepted.
TOEMACTxData[511:0]	Out	Transmitted data to the Ethernet MAC. It is valid when ‘TOEMACTxValid’ is set to 1b.
TOEMACTxValid	Out	Asserted to 1b to transmit the ‘TOEMACTxData’.
TOEMACTxLast	Out	Asserted to 1b to indicate that this is the last data. During this cycle, some data bytes may be unused, which can be monitored using ‘TOEMACTxKeep’.
TOEMACTxKeep[63:0]	Out	Byte enable to represent the number of valid bytes during this cycle. Bit[i] indicates that byte[i] of TOEMACTxData contains valid data. Generally, all 512-bit data is considered valid, and this signal is typically set to all ones, except for the last data strobe, where it can be set the specific bits to 1b indicating the amount of valid byte from 1-64 bytes.
TOEMAC Rx Interface
TOEMACRxData[511:0]	In	Received data from the Ethernet MAC. It is valid when ‘TOEMACRxValid’ is set to 1b.
TOEMACRxValid	In	Asserts to 1b to transfer the ‘TOEMACRxData’.
TOEMACRxLast	In	Asserts to 1b to indicate that this is the last data.
TOEMACRxKeep[63:0]	In	Byte enable to indicate the number of valid bytes during this cycle. Bit[i] indicates that byte[i] of TOEMACRxData contains valid data. Generally, all 512-bit data is considered valid, and this signal is typically set to all ones, except for the last data strobe, where it can be set the specific bits to 1b indicating the amount of valid byte from 1-64 bytes.
TOEMACRxError	In	Asserts to 1b to indicate that this frame contains an error. This signal is valid when the last data is transferred, as indicated by ‘TOEMACRxLast’=1b.
TOEMACRxInt	Out	Asserted to 1b for a single clock cycle when the RxPac buffer becomes full and the latest received packet is dropped.

Timing Diagram

IP Initialization

The TOE100GADV IP begins the IP initialization process when the reset signal (RstB) transitions from 0b to 1b. Prior to de-asserting the reset signal, users must configure the following common parameters: DstMacMode, SrcMacAddr, DstMacAddrIn (only applicable in Fixed-MAC mode), SrcIPAddr, DstIPAddr, WindowThres, TCPCtlTimeOutSet, and TCPRxTimeOutSet. These common parameters must retain their values for at least 4 clock cycles after the de-assertion of RstB. Once the IP initialization is successfully completed, InitFinish is set to 1b.

The TOE100GADV IP can be initialized using three different modes, determined by DstMacMode. Detailed information about each initialization mode is provided in this section.

Client mode (DstMacMode=00b)

When the target and the host are installed within the same network, the MAC address of the target can be obtained through ARP packet exchange. For simple usage, it is recommended to set the initialization mode of TOE100GADV IP to Client mode. The process of initialization in Client mode is described as follows.

Figure 5: IP Initialization in Client mode

1) Set the common parameters, including DstMacMode, with asserting RstB to 1b. When DstMacMode=00b, the IP initiates the initialization process in Client mode, loading all common parameters into internal registers during this phase. After setting RstB to 1b, user must maintain all inputs for 4 clock cycles to allow for the completion of the parameter loading process. Subsequently, these parameters can be modified as needed.

2) Utilize the MAC address and IP address obtained in step 1) to construct an ARP request packet, which will be transmitted to the target.

3) Upon detecting an ARP request packet, the target generates an ARP reply packet to return its MAC address to the host.

4) Once the ARP reply packet has been received, the target’s MAC address is extracted and loaded into the internal registers. Following this, the IP sets InitFinish to 1b, signifying the completion of the IP initialization process. Additionally, the DstMacAddrOut signal reflects the target MAC address acquired during initialization.

Server mode (DstMacMode=01b)

Server mode is suitable for applications that employ both the TOE100GADV IPs as the host and the target. If the first device is initialized in Client mode, the subsequent device must be configured to Server mode. The initialization process in Server mode is reverse of that in Client mode, described in detail as follows.

Figure 6: IP Initialization in Server mode

1) Set the common parameters, including DstMacMode, with asserting RstB to 1b. When DstMacMode=01b, the IP initiates the initialization process in Server mode, loading all common parameters into internal registers during this period. After setting RstB to 1b, user must maintain all inputs for 4 clock cycles to allow for the completion of the parameter loading process. Subsequently, these parameters can be modified as needed.

2) Utilize the MAC address and IP address obtained in step 1) to validate an incoming ARP request packet received from the target. If it is deemed valid, extract the target’s MAC address and load it into the internal registers.

3) Employ the MAC address and IP address to construct an ARP reply packet, which will be completely transmitted to the target after step 4) where InitFinish is set to 1b, due to the latency involved in the transmission process.

4) The IP sets InitFinish to 1b, signifying the completion of the IP initialization process. Additionally, the DstMacAddrOut signal reflects the target MAC address acquired during initialization.

Fixed-MAC mode (DstMacMode[1]=1b)

The last initialization mode, known as Fixed-MAC mode, is utilized when the host and the target are installed in different networks. In this mode, the MAC address of the target is configured by the gateway’s MAC address, assigned by DstMacAddrIn. The initialization process in Fixed-MAC mode is described in detail as follows.

Figure 7: IP Initialization in Fixed-MAC mode

1) Set the common parameters, including DstMacMode and DstMacAddrIn, with de-asserting RstB to 1b. The DstMacAddrIn is defined by the gateway’s MAC address. When DstMacMode[1]=1b, the IP initializes in Fixed-MAC mode, loading all common parameters into internal registers during this period. After setting RstB to 1b, user must maintain all inputs for 4 clock cycles to allow for the completion of the parameter loading process. Subsequently, these parameters can be modified as needed.

2) The IP sets InitFinish to 1b, signifying the completion of the IP initialization process. Additionally, the DstMacAddrOut signal reflects the target MAC address, which is the same value as DstMacAddrIn.

Connection Establishment

After completing the IP initialization, users can issue a command request to establish a connection for data transfer. The IP offers two options: active open and passive open. If active open is chosen, the IP initiates the connection establishment, commonly referred to Client mode. Conversely, in passive open mode, the target initiates the connection establishment, referred to Server mode.

The TOE100GADV IP has the capacity to manage up to four TCP sessions. Users must specify the session number (ranging from 0 to 3), determined by TCPConnCmd[3:2], when sending the command request to create a connection. Moreover, it needs to configure the session parameters, including TCPSrcPort, TCPDstPort (only used in Active open), and TCPLastMode when requesting the active or passive open command. Upon the successful establishment of the connection, TCPConnOn of the requested session is set to 1b. Comprehensive information regarding the connection establishment is available in this section.

Active open or Client mode (TCPConnCmd[1:0]=00b)

In order to initiate an active open connection, it requires to input both the host and the target port numbers to the IP. Thus, the user must know the port number that the target can accept. For certain target systems, the user needs to execute the application, enabling it to listen on the specified port number before initiating the Client function. The procedure for the active open command is outlined as follows.

1) Configure session parameters, including TCPDstPort, by setting TCPConnReq to 1b for a single clock cycle. When TCPConnCmd[1:0]=00b, the IP initiates the active open process and loads all session parameters into internal registers during this time. The session number for this request is determined by TCPConnCmd[3:2], set to 0 in this example. Subsequently, these parameters can be modified as needed.

Note: Before sending the command request to open the connection, please ensure that InitFinish=1b, TCPConnOn[i]=0b, and TCPConnStatus[(64*i)+4]=0b; where ‘i’ represents the session number (0-3).

2) The IP validates the inputs and initiates the operations. During this phase, new commands cannot be accepted, so TCPConnReady is set to 0b for 2 clock cycles to pause new command requests.

3) The IP allows new command requests by setting TCPConnReady to 1b. Also, the session#0’s state, determined by TCPConnStatus[3:0], changes from 0000b (No connection) to 0001b (Active open processing).

4) The port numbers obtained in step 1) and the network parameters of the requested session (session#0) are utilized to construct a SYN packet for session#0, which is then sent to the target. If the target accepts the connection establishment request, it responds with a SYN-ACK packet for session#0. Subsequently, the IP generates an ACK packet for session#0 to confirm the completion of the connection establishment. Please note that there is latency time in creating each transmitted packet in the TOE100GADV IP. Therefore, the ACK packet is sent to the target after the command completion status is returned to user (step 6). TCPConnStatus[4] is used to indicate the busy status of the packet transmission process. It is set to 1b if there are packets in a queue for transmission and then reset to 0b upon the completion of packet transmission.

5) Upon receiving the SYN-ACK packet and instructing the transmission of the ACK packet, set the connection status of session#0 to 1b, determined by TCPConnOn[0]. The session#0’s state transitions to 0100b (Connection ready). Furthermore, the connection status on the target, including parameters such as maximum segment size and window scaling factor, can be decoded from TCPConnStatus[52:32]. After this, the user can initiate data transfer using session#0.

Figure 8: Active open command on Session#0

6) The IP indicates the command completion by setting TCPConnCpl to 1b for a single clock cycle. Since multiple commands can be requested, and the order of completion may not follow the request order, the details of the command completion are displayed on TCPConnCplStatus[4:0]. In this example, the command has been successfully executed, with bit[4] set to 1b, while bit[3:0] matches the value of TCPConnCmd[3:0].

Passive open or Server mode (TCPConnCmd[1:0]=01b)

Unlike active open, the TOE100GADV IP offers the alternative of establishing a connection in passive mode. In passive mode, the sequence of packet transfer is reversed compared to active mode. In this mode, the target port number is extracted from the SYN packet, not configured by the user. The user must issue the passive open command to enable connection listening before initiating the Client function on another device. The procedure for the passive open command is outlined as follows.

Figure 9: Passive open command on Session#3

1) Configure session parameters by setting TCPConnReq to 1b for a single clock cycle. When TCPConnCmd[1:0]=01b, the IP initiates the passive open process and loads all session parameters into internal registers during this phase. The session number for this request is determined by TCPConnCmd[3:2], set to 3 in this example. Subsequently, these parameters can be modified as needed.

2) The IP validates the inputs and initiates the operations. During this phase, new commands cannot be accepted, so TCPConnReady is set to 0b for 2 clock cycles to pause new command requests.

3) The new command requests for other sessions are enabled by setting TCPConnReady to 1b. Also, the session#3’s state, determined by TCPConnStatus[195:192], transitions from 0000b (No connection) to 0010b (Connection listening). The IP now awaits a SYN packet containing matching parameters.

4) The target transmits a SYN packet for session#3 to initiate the connection establishment.

5) Upon receiving the SYN packet, the session#3’s state transitions to 0011b (passive open processing) to execute the subsequent process: transmitting a SYN-ACK packet for session#3. The network parameters of the SYN-ACK packet comprise the host port number, obtained in step 1), and the target port number, extracted from the SYN packet in step 4). TCPConnStatus[196] is set to 1b during the packet transmission.

6) The IP awaits an ACK packet for session#3 from the target to finalize the connection establishment. Once an ACK packet is received, the session#3’s state changes to 0100b (connection ready). At this time, the connection status of session#3, determined by TCPConnOn[3], is set to 1b. Furthermore, the connection status on the target, including parameters such as the target port number, maximum segment size, and window scaling factor, can be decoded from TCPConnStatus[244:208]. After this, the user can initiate data transfer using session#3.

7) The IP indicates the command completion by setting TCPConnCpl to 1b for a single clock cycle. The details of the command completion are displayed on TCPConnCplStatus[4:0]. In this example, the command has been successfully executed, with bit[4] set to 1b, while bit[3:0] matches the value of TCPConnCmd[3:0].

Multiple command requests

Since TOE100GADV IP can handle up to four TCP sessions, users have the flexibility to issue the multiple command requests to each TCP session, without the need to await the completion of previous commands. Subsequently, users monitor command completions to identify which commands have been executed. Please note that the order in which command completions are returned may differ from the order in which the commands were originally requested.

Figure 10: Multiple command requests with non-sequential command completion

1) Users can sequentially issue multiple commands to open multiple sessions. In this example, two commands are used to open session#1 in passive mode and session#2 in active mode by setting TCPConnReq twice.

2) Upon receiving the first command, the session#1’s state, determined by TCPConnStatus[67:64], transitions to 0010b (connection listening). Upon receiving a SYN packet, it further transitions to 0011b (passive open processing).

3) Similarly, the session#2’s state, determined by TCPConnStatus[131:128], transitions to 0001b (active open processing) when the second command is issued.

4) If the active open command on session#2 completes before the passive open command on session#1, TCPConnOn[2] is set to 1b before TCPConnOn[1]. Additionally, the session’s state changes to 0100b (connection ready).

5) TCPConnCpl is set to 1b twice, signifying the completion of the active command on session#2 and the passive command on session#1, respectively.

Flush operation of the RxTCP buffer during active open command

When users initiate an active open command, the RxTCP buffer undergoes a flush operation. In contrast, the TxTCP buffer is flushed upon the connection termination. The following steps outline how to perform this flush operation when executing an active open.

Figure 11: Flush operation of the RxTCP buffer on Session#0 after active open request

1) When the user issues an active open command for session#0, the IP initiates the operation. The flush flag of the RxTCP buffer for session#0, TOERxStat0[31], is set to 1b. This signal remains asserted to 1b for a minimum of 9 clock cycles before transitioning to 0b.

2) If there is any remaining data in the RxTCP buffer, as indicated by TOERxStat0[19:0] having a non-zero value, all data are freed with two-clock cycle latency after the flush flag has been asserted. Subsequently, TOERxStat0[19:0] is set to zero.

Flush operation of the RxTCP buffer during passive open command

Similarly, during execution of a passive open command for session#0, the RxTCP buffer also undergoes a flush operation. The process for performing this flush operation during a passive open is outlined as follows.

Figure 12: Flush operation of the RxTCP buffer on Session#0 after passive open request

1) Upon the user issuing a passive open command for session#0, the IP initiates the operation, transitioning the session#0’s state to 0010b (connection listening). In this state, the flush flag of the RxTCP buffer for session#0, TOERxStat0[31], is set to 1b. It transitions to 0b when the session#0’s state changes to 0011b (passive open processing).

Connection Termination

Terminating a connection can be initiated by either the TOE100GADV IP (active close) or the target (passive close). For an active close command, users specify the session number and configure the TCPConnCmd value. Once the connection terminates successfully, the TCPConnOn is reset to 0b. It is recommended to issue an active close command after there is no remaining data stored in the TxTCP buffer. However, if the active close command is issued before transmitting data completion, it must wait until all data in the TxTCP buffer is entirely transferred to the target before initiating the close command.

Conversely, if a passive close request is initiated by the target while data transmission is ongoing, the data transmission must be aborted, and the TOE100GADV IP switches to process the connection termination request. In this scenario, users can confirm the completion of transmitting data by checking the number of remaining data in the TxTCP buffer, determined by TOETxStat[19:0]. Multiple commands are supported for controlling each TCP session individually. Additional information about connection termination is provided in this section.

Active close (TCPConnCmd[1:0]=10b)

Before sending an active close command for session#0, ensure that the TCPConnOn[0] is set to 1b, TCPConnStatus[3:0] equals 0100b, and TOETxStat0[19:0] equals 0. The procedure for the active close command is outlined as follows.

Figure 13: Active close command on Session#0

1) Initiate the active close process by setting TCPConnCmd[1:0]=10b and asserting TCPConnReq to 1b for a single clock cycle. In this example, the session number for this request is defined by TCPConnCmd[3:2], which is set to 0. Subsequently, these parameters can be modified as needed.

2) The IP validates the inputs and initiates the operations. During this phase, new commands cannot be accepted, resulting in TCPConnReady being set to 0b for 2 clock cycles to pause new command requests.

3) Following the reception of the active close command, the IP requires a minimum of 16 clock cycles to verify all conditions for initiating the close command. The session#0’s state, determined by TCPConnStatus[3:0], changes from 0100b (Connection ready) to 0110b (Active close processing). During this state transition, the session#0’s state temporarily transitions to 0101b for a few clock cycles. With a specific latency time, the FIN packet for session#0 is transmitted to the target.

4) The IP awaits the transmission of FIN and ACK packets for session#0 from the target to verify the acceptance of the termination request.

5) The session#0’s state transitions to 0000b (No connection), and TCPConnOn[0] is then reset to 0b with a latency time of 4 clock cycles.

6) The IP indicates the command completion by setting TCPConnCpl to 1b for a single clock cycle. The details of the command completion are displayed on TCPConnCplStatus[4:0]. In this example, the command has been successfully executed, with bit[4] set to 1b, while bit[3:0] corresponds to the value of TCPConnCmd[3:0].

7) While ACK transmission was requested in step 5, the transmission of an ACK packet to the target for session#0 is executed after setting TCPConnOn[0] to 0b, due to latency in the transmit block. User can verify the completion of packet transmission by monitoring until TCPConnStatus[4]=0b.

Active close (TCPConnCmd[1:0]=10b) during transmitting data

While it is recommended to wait until there is no remaining data in the TxTCP buffer, as indicated by TCPTxStat[19:0]=0, before issuing the active close command, the IP can accommodate this requirement by following the procedure below.

2) The session#0’s state transitions to 0101b, allowing for the inspection of remaining data in the TxTCP buffer by reading the TOETxStat0[19:0]. TOETxReady0 is set to 0b, suspending additional data transmission for session#0. The IP then waits until all data is successfully transmitted to the target.

3) Once TOETxStat0[19:0] equals zero, ensuring there is no remaining data for transmission, the session#0’s state changes to 0110b, initiating the active close processing. Following this, the procedure aligns with steps 3) – 7), as illustrated in Figure 13.

Figure 14: Active close on Session#0 during transmitting data

Passive close

In passive close, the connection termination is initiated by the target. This results in a reversal of the packet transfer sequence compared to an active close. Several signals are designed to alert the user to the ongoing passive close process, including TCPConnOn[1], TOETxReady1, and TOETxStat1[30] for session#1. The procedure for the passive close command is outlined as follows.

Figure 15: Passive close on Session#1

1) The receive block validates the FIN packet of session#1. If the packet proves valid, the passive close process is initiated, with TOETxStat1[30] being set to 1b to indicate to the user that data transmission must cease due to the connection termination.

2) Subsequently, the session#1’s state, as determined by TCPConnStatus[67:64], transitions to 0111b (passive close processing), and TOETxReady1 is reset to 0b to halt user-initiated data transmission.

3) In step 2, the request to transmit FIN and ACK packets for session#1 is generated. However, the packets are transmitted to the target with a specific latency time. User can monitor the completion of packet transmission by observing when TCPConnStatus[68]=0b.

4) The target acknowledges the receipt of confirmation messages (FIN and ACK packets) and finalizes the connection termination by transmitting an ACK packet of session#1. Upon receipt this packet, the session#1’s state changes to 0000b (no connection).

5) Following this, both TOETxStat1[30] and TCPConnOn[1] are set to 0b, signifying the completion of the passive close operation.

6) TOETxReady1 is then set to 1b, allowing the user to prepare for data transmission on the new connection.

Passive close during transmitting data

While transmitting data to the target, if the target sends a FIN packet to terminate the connection, the IP will immediately halt data transmission and transition into connection termination mode. Any remaining data in the TxTCP buffer will be flushed upon the completion of the connection termination process. The status of the TxTCP buffer can be tracked using the TOETxStat signal. The steps for handling passive close when there is remaining data in the TxTCP buffer are outlined below.

Figure 16: Passive close on Session#1 during transmitting data

1) The procedure for terminating the connection in passive mode during the reception of FIN and ACK packets is similar to steps 1) – 3) in Figure 15. The session#1’s state, as determined by TCPConnStatus[67:64], changes to 0000b (no connection) to signify the completion of the termination process.

2) Once the session#1’s state equals 0000b, the TOETxStat1[31] is set to 1b, indicating that TxTCP buffer is being flushed.

3) TOETxStat1[19:0], indicating the quantity of data stored in the TxTCP buffer for session#1, is reset to 0 during the flush operation.

4) TOETxStat1[31] is subsequently reset to 0b to signify the completion of the flush operation.

Session Reset

In cases where unusual events are detected within a specific session, necessitating a reset process to resolve the issue, a reset session command is made available. This command enables the execution of a reset process for the specific session without interrupting the operation of the remaining sessions.

The first use case for employing the session reset is to cancel a passive open command request that has been remained unanswered by the target for an extended period. Users have the option to initiate a session reset to halt the waiting process and determine the subsequent steps. Another scenario where the session reset proves beneficial is in clearing all data present in the TxTCP buffer that was transmitted by the user before the establishment of the connection.

Further details regarding the reset session command are provided in this section.

Figure 17: Reset request on Session#0

1) Before setting TCPConnReq to 1b to initiate the command request, it is essential to confirm that TCPConnReady is set to 1b. For a reset session request, TCPConnCmd[1:0] should be configured as 11b, and the specific session is denoted by TCPConnCmd[3:2]. In this example, a reset for session#0 is being requested.

2) Subsequently, the reset process for session#0 is executed, necessitating a minimum of 3 clock cycles before completion is asserted. The reset procedure’s completion is indicated by setting TCPConnCpl to 1b, and the command status is returned on TCPConnCplStatus[4:0]. Specifically, bit[4] is set to 1b upon command completion, while bits[3:0] correspond to the value of TCPConnCmd[3:0].

Data transmission

Data transfer can commence once the connection has been established and is not in the process of termination. The TOE100GADV IP provides four TOETx interfaces for transmitting data to the target, utilizing four TCP sessions. These TOETx interfaces follows the AXI4-ST standard, employing TOETxReady to manage data flow. Each time the TOE100GADV IP determines when it should assert a TOETxReady signal for a specific session. It accepts only one set of user data for transmission to the Ethernet MAC. The IP switches to accept data from a new session when the current session pauses data transmission or signals the last data packet.

According to TCP/IP specification, the maximum payload size in each Ethernet packet is constrained by the Maximum Segment Size (MSS). Consequently, if the data transmitted by the user exceeds the MSS value, multiple packets are transmitted to the target. In the event that the user pauses data transmission without asserting the last flag and the remaining data size in the TxTCP buffer falls below the MSS value, the TOE100GADV IP will wait for a specific period. If no additional data arrives until the timeout, the remaining data in the TxTCP buffer is sent to the target.

Typically, the target generates an ACK packet to confirm data reception once the received data reaches the predefined threshold value. Therefore, when the last data is transmitted without a special flag, the target awaits additional data until a timeout occurs before generating the ACK packet. Excessive waiting time in relation to processing time can potentially affect the system’s transfer performance. To mitigate waiting time, the TOE100GADV IP offers two additional features to expedite the ACK packet, configured by TCPLastMode[1:0].

The first TCPLastMode mode involves duplicated data, transmitting the last data packet twice. The target recognizes the second data packet as a data retransmission, which accelerates the target’s response in generating an ACK packet. In our test environment, the response time for an ACK packet in this mode was found to be shorter than the second mode, which is the standard method.

The second mode of TCPLastMode involves the PSH flag. This PSH flag is inserted into the packet header of the last data packet. According to TCP/IP standard, the target must bypass data to the application layer if the PSH flag is found. Therefore, it allows the target to respond with an ACK packet without prolonged waiting.

Further details on transmitting data under various conditions can be found in this section.

Data transmission using a single connection with timeout

Following the AXI4-ST bus standard, data flow control relies on two signals, ‘valid’ and ‘ready’. Successful data transmission occurs when both ‘valid’ and ‘ready’ are set to 1b during each cycle. All 512-bit data must be valid in each transfer cycle, except the last data, which is indicated by setting Last to 1b. Consequently, the ‘Keep’ signal remains at all ones during data transfer, except during the last transfer.

The first example, depicted in Figure 18, illustrates continuous data transfer using a single session, session#0. After transmitting data for a specific duration, the user pauses data transmission without asserting the Last flag. In this scenario, the TOE100GADV IP consistently generates packets with a payload size equal to the MSS value. However, when a timeout is triggered following the user’s pause, with remaining data in the TxTCP buffer, the TOE100GADV IP decides to create a packet containing the remaining data, which is less than MSS value. The steps for transmitting data using a single connection without asserting the last flag are outlined below.

Figure 18: Data transmission using session#0 with timeout

1) The user sets TOETxValid0 to 1b while transferring data to session#0 using the TOETxData0 signal. Throughout the data transfer, the user continuously monitors TOETxReady0 to confirm that the data has been successfully accepted. Additionally, the IP provides TOETxStat0[19:0] signal to indicate the amount of data stored in TxTCP buffer in byte units. There is a latency of approximately 7-8 clock cycles after TOETxValid0 is set to 1b before TOETxStat0[19:0] becomes available.

2) Once the data in TxTCP buffer reaches a sufficient quantity, meeting the MSS value requirement, the transmit process is initiated. This process involves creating an Ethernet packet that includes the TCP payload data, which is then forwarded to the Ethernet MAC. Following this initial transmission, subsequent packets are continuously generated and transmitted to the target.

3) The data bandwidth at user interface must exceed that of the Ethernet MAC interface. As a result, when user data is transmitted continuously without interruption, the TxTCP buffer becomes full. In this example, assuming TxBufBitWidth0 is configured to 14, the full condition is (2^14) – 16 or 16,368. Once this full condition is triggered, TOETxReady0 is reset to 0b, effectively pausing data transmission.

4) During the pause period, all input values from the user, including TOETxValid0, TOETxLast0, TOETxKeep0, and TOETxData0, must be maintained at their current values until TOETxReady0 is set back to 1b, indicating that data transmission can resume.

5) The payload size in each transmitted packet matches the MSS value. However, if the user pauses data transmission until a timeout is triggered, any remaining data in the TxTCP buffer will be assembled and transmitted to the target.

6) After all data has been successfully transferred and accepted by the target, TOETxStat0[19:0] will equals zero, signifying that the buffer is empty.

Data transmission using a single connection with TCPLastMode=00b (Duplicated data)

In situations where there is no further data for transmission, it is advisable to set the Last flag, TOETxLast, to 1b at the last data strobe. When the Last flag is set, all the data stored in the TxTCP buffer is transmitted to the target. TCPLastMode offers three options for transferring the last data. Figure 19 illustrates the duplicated data mode, configured by TCPLastMode=00b. In this mode, the last data packet is transmitted twice to expedite the target’s response. The steps for transmitting data using a single connection with asserting the last flag, configured with TCPLastMode=00b, are outlined below.

Figure 19: Data transmission using session#0 configured by TCPLastMode=00b

1) The last mode is configured during opening the session, as defined by TCPLastMode. In this example, session#0 is opened with setting TCPLastMode[1:0] to 00b, indicating the duplicated data mode. Subsequently, the last packet of session#0 operates under this mode until the session is terminated.

2) The user transfers 512-bit data to the IP, and the IP creates the Ethernet packets containing an MSS-byte payload data for transmitting to the target until the last data is sent by the user. The last data is indicated by setting TOETxLast0 to 1b. During the last cycle, the user can transmit data of any size by specifying the number of valid bytes to TOETxKeep0. In this example, 64 bits or 8 bytes of data are transferred in the last strobe, so TOETxKeep0 is set to 0000_0000_0000_00FFh (bit[7:0] is asserted to 1b to indicate validation of byte#0-byte#7).

3) The last packet, containing the last data (Dn[63:0]), is transmitted twice to trigger data retransmission detection at the target. Upon detecting data retransmission, the receiver responds by sending an ACK packet. After receiving an ACK packet to confirm the successful acceptance of the last data, the TxTCP buffer can remove the accepted data. If there is no additional data from the user, TOETxStat0[19:0] will be equal to 0.

Note: If the user sends additional data after setting the last flag, the IP will automatically cancel the last flag.

Data transmission using a single connection with TCPLastMode[1]=1b (PSH flag)

An effective technique for expediting the return of an ACK packet from the target is to set the PSH flag in the TCP header of the Ethernet packet to 1b. Users can opt for this approach by configuring TCPLastMode[1] to 1b during the open connection command. The steps for transmitting data using this mode are outlined below.

Figure 20: Data transmission using session#0 configured by TCPLastMode[1]=1b

1) During sending open command of session#1, the user sets TCPLastMode[1] to 1b. This mode remains active for the last packet of session#1 until the session is terminated.

2) The user transmits 512-bit data, with TOETxKeep1 set to all ones. The Ethernet packets forwarded to the Ethernet MAC contain an MSS-byte payload data until the last data is transferred. In this example, during the last cycle, 128 bits or 16 bytes of data are transferred, resulting in TOETxKeep1 being set to 0000_0000_0000_FFFFh.

3) The last packet, which includes the last data (Dn[127:0]), is transmitted with the PSH flag in the TCP header set to 1b. After that, the receiver responds by sending an ACK packet, and the TxTCP buffer can remove the accepted data. Without additional data from the user, TOETxStat1[19:0] will be equal to 0. If the user attempts to send additional data after setting the last flag, the IP will automatically cancel the last flag.

Data transmission using multiple connections

While there are four user interfaces available for data transmission, they all share a common TOEMAC Tx interface capable of transmitting only one packet from one session at any given time. Consequently, TOE100GADV IP sets TOETxReady to 1b for only one user interface to receive a data stream for transmission to the Ethernet MAC. In situations where multiple user interfaces simultaneously assert TOETxValid to 1b, the active user interface can switch to the next channels in a sequence order of 0, 1, 2, 3, 0, and so forth. This transition occurs when the active user pauses data transmission by setting TOETxValid to 0b or when it sends the last data by setting TOETxLast to 1b. The following steps outline the process of transmitting data using multiple connections.

Figure 21: Data transmission using multiple sessions

1) Let’s assume that the active session is session#0, continuously transmitting Ethernet packets with MSS#0-byte payload data to the Ethernet MAC.

2) If session#0 pauses data transmission by setting TOETxValid0 to 0b while another session, session#1, requests data transmission by asserting TOETxValid1 to 1b, the TOE100GADV IP decides to switch the active session from session#0 to session#1. TOETxReady0 is then set to 0b, and TOETxReady1 is set to 1b. Once the transmitted data for session#1 reaches the MSS#1 value, a new Ethernet packet containing session#1 data is transmitted.

3) While transmitting session#1 packets, session#0, which still has remaining data in the TxTCP buffer, encounters a timeout condition and initiates packet transmission. In this scenario, the Ethernet packet containing the remaining payload data of session#0 is transmitted, temporarily interrupting the packet transmission for session#1. Afterward, session#1 packet transmission resumes.

4) When the user sends the last data for session#1 by setting TOETxLast1 to 1b, the IP generates packets containing the remaining data of session#1. If there are two sessions requesting data transmission, namely sesson#0 and session#2, the IP gives higher priority to the subsequent session, session#2. Consequently, TOETxReady1 is set to 0b, and TOETxReady2 is set to 1b to initiate data transfer for session#2.

5) The Ethernet packet, containing MSS#2-byte data for session#2, is transmitted to the Ethernet MAC.

Note: The MSS value of each session is individually decoded during connection establishment. Therefore, it may vary depending on the target device or network connection, resulting in different values for each session.

Data reception

The port numbers inside the received packet header are decoded to determine the user interface number, 0-3, responsible for transferring the TCP payload data extracted from incoming Ethernet packets. This payload data is then stored in the RxTCP buffer. When the user sets the TOERxReady signal to 1b, and data is stored in the RxTCP buffer, the transfer of data from TOE100GADV IP to the user is initiated. Each interface is applied to transfer data for each session independently. Therefore, there is a chance to simultaneously transfer data to the user through multiple sessions if these sessions have data stored in the RxTCP buffer and the users activate the TOERxReady for these interfaces by setting it to 1b.

The TCPRx interface, which is the user interface for transferring the received payload data, includes the Last flag. The Last flag can be set to 1b under three conditions: when the last data in the RxTCP buffer aligns with 512 bits, when the last data in the RxTCP buffer is unaligned and reaches a timeout, and when subsequent data follows unaligned data transmission.

For more comprehensive information on receiving data under various conditions, please refer to this section.

Data reception using a single connection with 512-bit alignment

The TOE100GADV IP receives the Ethernet packets through the 512-bit AXI4-ST bus standard without the use of a ‘ready’ signal. As a result, the sender (100G Ethernet Subsystem) can pause data transmission by setting the ‘valid’ signal to 0b. In contrast, the receiver is consistently prepared to accept new data due to the absence of a ‘ready’ signal. Under specific conditions, this configuration may lead to the TOE100GADV IP rejecting an incoming packet and asserting TOEMACRxInt to 1b when the pre-processing Rx buffer within the IP reaches its capacity. Detailed information about the timing diagram of TOEMACRxInt can be found in Figure 25.

During the reception of a packet, the header is extracted and subsequently validated. If the header is deemed valid, the TCP payload data is retrieved, stored in the RxTCP buffer, and then transferred to the user interface, known as TOERx interface. The TOERx interface complies with the 512-bit AXI4-ST bus standard and provides both ‘valid’ and ‘ready’ signals. Users can de-assert TOERxReady to 0b to halt data transmission.

In case the amount of payload data aligns with 512 bits or 64 bytes, all data is transferred to the user until the RxTCP buffer becomes empty, as indicated by TOERxStat[19:0]=0. The steps for receiving data using a single connection with 512-bit alignment are outlined below.

Figure 22: Data reception using session#0 with 512-bit alignment

1) Let’s consider the transmission of an Ethernet packet for session#0 from Ethernet MAC. To initiate data transfer, the TOEMACRxValid is set to 1b, and the first data of the packet (E0) is transmitted to TOEMACRxData.

2) To pause data transmission, TOEMACRxValid is set to 0b. During this phase, the value on TOEMACRxData is discarded.

3) The last data of a packet is transmitted when both TOEMACRxLast and TOEMACRxValid are set to 1b. The keep signal, TOEMACRxKeep, is consistently set to all ones for transmitting 512-bit data via TOEMACRxData, except during the last cycle when it can be set to indicate the amount of valid data from 1-64 bytes in TOEMACRxData (Kn-1 in this example).

4) If the size of transferred data exceeds the maximum limit for an Ethernet packet, the payload data is divided and transferred using multiple packets.

5) Subsequently, the packets undergo validation, and the payload data is stored in the RxTCP buffer. TOERxStat[19:0], indicating the quantity of data in the RxTCP buffer, displays a non-zero value. In this scenario, let’s assume that the total amount of payload data from multiple packets equals M bytes. The transfer of data from the RxTCP buffer to the TOERx interface is initiated by setting TOERxValid0 to 1b. The first payload data (D0) is transmitted to TOERxData0. If the user acknowledges the data by setting TOERxReady0 to 1b, the subsequent data (D1) can be transferred in the next clock cycle.

6) Upon the successful transfer of each data to the user, TOERxStat0[19:0], indicating the data quantity in the RxTCP buffer, is updated in the next two clock cycles. In this example, its value decreases by 64 due to the transfer of 512-bit (64-byte) data.

7) The user can pause data transmission by de-asserting TOERxReady0 to 0b. During this pause, all AXI4-ST outputs of TOERx I/F, including TOERxLast0, TOERxValid0, TOERxKeep0, and TOERxData0, maintain their values.

8) When both TOERxLast0 and TOERxValid0 are set to 1b, the last payload data (Dm-1; where m equals M/64) is transmitted on the TOERxData0 signal. In this case, the payload data size aligns with 512 bits or 64 bytes, resulting in TOERxKeep0 being set to all ones during the last cycle. Following this, TOERxStat0[19:0] is reset to zero, indicating that the RxTCP buffer has become empty.

Data reception using a single connection with 512-bit misalignment

When the total amount of received data is not aligned to 512 bits or 64 bytes, the data is initially transferred to the user with 512-bit alignment. Subsequently, if no additional data is received until the timeout is reached, as configured by TCPRxTimeOutSet during the IP initialization process, the last portion of unaligned data is transmitted to the user with the Last flag set to 1b.

If additional data transmission occurs from the target after this point, the first portion of data is also unaligned, and its length is determined by the number of valid bytes remaining from the previous unaligned data. The combined valid byte count of both pieces of data equals 64 bytes. For instance, if the previous unaligned data is valid for 8 bytes, the next unaligned data will be valid for 56 bytes (64 – 8 = 54 bytes). Following this, the 512-bit data is continuous transferred. The steps for receiving data using a single connection without 512-bit misalignment are outlined below.

Figure 23: Data reception using session#1 with 512-bit misalignment

1) During the IP initialization process, the timeout value for transmitting unaligned data to the user is configured using the TCPRxTimeOutSet signal.

2) Let’s consider a scenario where a received packet for session#1, containing N-byte payload data, is transmitted from the target.

3) Following this, the payload data extracted from the received data is forwarded to the user via TOERx I/F for session#1. The data transmitted during this phase is aligned to 512 bits or 64 bytes, resulting in TOERxKeep1 being equals to all ones.

4) At this point, data transmission is temporarily halted, leaving one piece of data in the RxTCP buffer with a size less than 64 bytes. In this example, there are 8 bytes of remaining data. In the event of no additional data transmission and upon the timer reaching the timeout value, the last data transmission, which is unaligned data, is initiated by setting both TOERxLast1 and TOERxValid1 to 1b. TOERxKeep1 indicates the valid bytes in this cycle, amounting to 0000_0000_0000_00FFh (assertion of bit[7:0] indicates the validation of byte#0-byte#7). The 64-bit data is then transmitted to the TOERxData1[63:0] signal.

5) Now, let’s assume that the target sends an additional packet for session#1, containing (M-N) bytes of payload data.

6) To initiate data transfer, an unaligned data is sent to re-align the remaining data. The valid size of this data is equal to (64 – the valid bytes of the previous unaligned data). In this example, it equals 56 bytes, resulting in TOERxKeep1 for this cycle being equals to 00FF_FFFF_FFFF_FFFFh. The valid data is transmitted to TOERxData1[447:0] signal and TOERxLast1 are set to 1b.

7) The data stream, now aligned to 512 bits, is transferred to user, the same status as step 3).

Data reception using multiple connections

The TOERx I/F, responsible for delivering payload data to the user for each session, can function concurrently. The data transfer for each session commences when the RxTCP buffer contains data, and the user sets TOERxReady to 1b. The following steps outline the procedure for receiving data through multiple connections. Let’s consider a scenario in which the payload data of two sessions, session#2 and session#3, are being transferred, and the data volumes in both sessions align with 512 bits or 64 bytes.

1) An Ethernet packet containing N-byte payload data for session#2 is received and validated. Subsequently, the payload data is extracted and stored in the RxTCP buffer for session#2. However, data transmission is paused, awaiting the assertion of TOERxReady2, which is currently set to 0b.

2) Next, an Ethernet packet with (M-N) bytes of payload data for session#3 is received. Similarly, the payload data is extracted and stored in the RxTCP buffer for session#3. Data transmission is also suspended, awaiting the assertion of TOERxReady3.

3) Upon the assertion of TOERxReady2, the data transmission of session#2 is initiated and continues until completion. If the amount of data aligns with 512 bits, the last data is transmitted with TOERxLast2 set to 1b.

4) Simultaneously, data transmission for session#3 is initiated as soon as TOERxReady3 is asserted. If the data volume aligns with 512 bits, all data is transferred until the RxTCP buffer becomes empty.

Figure 24: Data reception using multiple sessions

Data reception with TOEMacRxInt assertion (RxPac buffer overflow)

When Ethernet packets containing small payload sizes are continuously transferred without interruption for long time, the RxPac buffer within the TOE100GADV IP may become full. Consequently, the TOE100GADV IP must reject incoming packets and assert TOEMACRxInt to 1b, signaling to the user that a packet has been discarded during processing. The user has the option to solve this situation by increasing the ‘Clk’ signal frequency to reduce the chance of packet rejection. The recommended value of 220 MHz is determined based on testing with continuous transfers of 1408-byte packets. The following steps outline and example of this scenario.

1) In this example, each packet is transmitted within two clock cycles, while the receive block requires more than two clock cycles to process each packet. If this pattern persists over an extended period, the RxPac buffer inside TOE100GADV IP will eventually reach its maximum capacity.

2) Once the buffer overflows, an incoming packet is discarded by the receive block. To notify the user of the discarded Ethernet packet, the IP asserts the TOEMACRxInt signal to 1b for one clock cycle after receiving last data of the packet.

3) If the buffer remains full and the subsequent packet is discarded, TOEMACRxInt continues to be asserted to 1b.

4) When the buffer has sufficient free space to store a packet, TOEMACRxInt is not asserted to 1b after receiving the last data of the packet.

Figure 25: Data reception with TOEMACRxInt assertion due to the discarded packet

IP Interrupts

The TOE100GADV IP supports two distinct interrupt types: TCPRtrInt, which is triggered when an error situation is detected and recovery is feasible, and TCPRstInt, which is activated when an error situation is detected, necessitating the transmission of a RST packet to terminate the ongoing process. These interrupt types are determined using 80-bit status signals: TCPRtrIntStatus and TCPRstIntStatus, which are divided into five groups, with each group containing 16 bits. These signals serve a common group and four session groups (session#0 – session#3).

The retry process within the TOE100GADV IP is initiated when the receive block detects a lost packet. To address this loss, the TOE100GADV IP initiates the transmission of certain packets to trigger packet retransmission from the target. In most cases, the retry process sets a maximum retry limit of 15 attempts. If the lost packet remains unresolved after 15 retry attempts, the interrupt type transitions to a reset interrupt, TCPRstInt, initiating the transmission of a RST packet. In other scenarios, such as an ARP reply timeout, the retry process continues indefinitely without a retry limit.

The TOE100GADV IP initiates the transmission of RST packet to terminate a connection when it detects an unrecoverable error or an error that necessitates the immediate termination of the connection, such as receiving a SYN packet while the connection is already established. For a comprehensive understanding of the interrupt signals, please refer the detailed information provided in this section.

Retry Interrupt for lost ARP reply packet (TCPRtrIntStatus[0])

The retry interrupt associated with lost ARP reply packets is designated as bit[0] of TCPRtrIntStatus, the common group. The retry process for this interrupt type operates without a maximum limit on retry attempts. Similarly, bit[22] and bit[23] of TCPRtrIntStatus, session#0 group, are unrestricted, serving as interrupts for lost received data and reverse packet detection, respectively.

TCPRtrIntStatus consists of four session groups, bit[38], bit[54], and bit[70] are allocated for the retry interrupt of lost received data for session#1, session#2, and session#3, respectively. Additionally, bit[39], bit[55], and bit[71] are designated for the retry interrupt associated with reverse packet detection for session#1, session#2, and session#3, respectively. The following steps provide an example illustrating packet transmission to address the recovery of lost packets for these various interrupt types.

Figure 26: TCPRtrIntStatus[0] assertion

1) During the initialization of the IP, a timeout for initiating the retry process is configured using the TCPCtlTimeOutSet signal.

2) In this scenario, let’s assume that the IP is initialized in Client mode, where it transmits an ARP request packet and awaits the return of an ARP reply packet from the target. If the ARP reply packet is lost, and the IP does not receive it within the defined timeout value set in step 1), the TCPRtrInt interrupt is triggered, and TCPRtrIntStatus[0] is asserted to 1b for one clock cycle.

3) The retry process begins by retransmitting the ARP request packet. There is no maximum retry count limit for this interrupt type, so steps 2) and 3) are repeated until the lost packet is successfully received.

4) Upon receiving an ARP reply packet, the retry process is concluded, marking the successful completion of the IP initialization process.

Retry Interrupt for lost SYN-ACK packet (TCPRtrIntStatus[16])

In most cases, the retry process is limited to a maximum of 15 attempts before switching to the TCPRstInt interrupt type. This limitation applies to five distinct interrupt types for each session. Specifically, for session#0, bit[16], bit[17], bit[19], bit[20], and bit[21] of TCPRtrIntStatus are designated as interrupts for the following conditions: lost SYN-ACK packet, lost ACK packet during passive open operation, lost ACK packet during passive close operation, lost ACK packet during data transmission, and lost window update packet during data transmission. For details of bit assignments in other sessions, please refer to Table 6.

Table 6: Bit assignment of TCPRtrIntStatus for session interrupt with maximum limit

Interrupt status	Session#0	Session#1	Session#2	Session#3
Lost SYN-ACK	bit[16]	bit[32]	bit[48]	bit[64]
Lost ACK (passive open)	bit[17]	bit[33]	bit[49]	bit[65]
Lost ACK (passive close)	bit[19]	bit[35]	bit[51]	bit[67]
Lost ACK (data transmission)	bit[20]	bit[36]	bit[52]	bit[68]
Lost Window update	bit[21]	bit[37]	bit[53]	bit[69]

This section provides an example illustrating the scenario of a lost SYN-ACK packet during the establishment of a connection in active mode. The subsequent steps during the execution of the retry process, which occurs multiple times, are outlined below.

Figure 27: TCPRtrIntStatus[16] assertion

1) Let’s consider a scenario where the IP initiates an active open command for session#0 by sending a SYN packet to the target. However, if the SYN-ACK packet is not returned within the specified timeout period, both TCPRtrInt and TCPRtrIntStatus[16] are asserted to 1b for a single clock cycle.

2) Subsequently, the retry process commences by retransmitting the SYN packet for session#0. This packet retransmission, following the assertion of both TCPRtrInt and TCPRtrIntStatus[16], can continue for up to 15 attempts if the SYN-ACK packet is still not received.

3) Upon successfully receiving the SYN-ACK packet for session#0 within 15 retry attempts, the connection establishment process is successfully completed.

Retry Interrupt for lost ACK packet during data transmission(TCPRtrIntStatus[20])

During data transmission, the TOE100GADV IP awaits an ACK packet from the target to confirm the acceptance of data. If the expected ACK packet is not received within the specified time, as defined by TCPCtlTimeOutSet, the IP initiates a retransmission of the data packet, indicated by the assertion of TCPRtrInt and TCPRtrIntStatus[20]. Additionally, upon receiving duplicate ACK packets from the target, signaling a request for the lost data packet, the IP initiates packet retransmission, with counting the retry attempts.

The TOE100GADV IP incorporates a dedicated timer to prevent interrupt assertion from the same duplicate ACK packets in a specific time frame. This duration increases after each subsequent retry attempt. After the 8^th attempt triggered by duplicate ACKs, the blocking period extends to over 1 second, which may exceed the TCPCtlTimeOutSet value. Subsequently, the retry duration is controlled by TCPCtlTimeOutSet.

After 15 attempts, the interrupt type switches to a reset, asserting TCPRstInt and TCPRstIntStatus[20]. The following steps outline an example demonstrating the process of packet retransmission for recovering lost ACK packets.

Figure 28: TCPRtrIntStatus[20] assertion

1) The IP continuously sends data packets until the target’s window size becomes insufficient, leading to a pause in data transmission.

2) The target generates duplicate ACK packets, requesting the retransmission of the lost packets.

3) The duplicate ACK packets triggers the assertion of TCPRtrInt and TCPRtrIntStatus[20] for a single clock cycle.

4) Subsequently, retransmission of the requested packet position, as indicated by the duplicate ACK, commences while keeping track of retry attempts.

5) To prevent rapid packet retransmission due to the same duplicate ACK packets, the IP integrates a dedicated timer that operates within a specific timeframe to discard these duplicated ACK packets. Step 2) – 5) can iterate if duplicate ACK packets persist, and the discard duration, elongated after each retry attempt, remains shorter than the set value of ‘TCPCtlTimeOutSet’.

6) If no expected ACK packet is received within ‘TCPCtlTimeOutSet’ duration, TCPRtrInt and TCPRtrIntStatus[20] are set to 1b.

7) The retry process then initiates by retransmitting the data packet for session#0, while counting each retry attempt. After 15 attempts, if the expected ACK packet is not received, the interrupt type switches to a reset.

Reset Interrupt for lost SYN-ACK packet (TCPRstIntStatus[16])

As previously discussed in the preceding section, the reset interrupt, TCPRstInt, is triggered after the 15^th retransmission without successfully receiving the lost packet. To match the assignment of bits for the retry interrupt status, the same bits are allocated in TCPRstIntStatus, including bit[16], bit[17], bit[19], bit[20], and bit[21], specifically for session#0. For other sessions, the definitions can be found in Table 6. To continue from the previous section, an example of the reset interrupt occurring after the 15^th attempt, without the reception of the SYN-ACK packet, is provided with further details below.

Figure 29: TCPRstIntStatus[16] assertion

1) The first retransmission of the SYN packet for session#0 is initiated following the assertion of the TCPRtrInt and TCPRtrIntStatus[16] signals.

2) In the absence of receiving the SYN-ACK packet, the process outlined in step 1) repeats for a total of 15 iterations. The SYN#0 (Retry#15) packet is transmitted during the final round.

3) Upon reaching the timeout threshold, it triggers the assertion of TCPRstInt and TCPRstIntStatus[16] to initiate the transmission of the RST packet for session#0 to the target.

Reset Interrupt for lost FIN-ACK packet (TCPRstIntStatus[18])

The reset interrupt can be triggered under specific conditions without any attempt at a retry process. Within the TOE100GADV IP, three bits of TCPRstIntStatus are allocated for each session to indicate three distinct types of this reset interrupt. For session#0, bit[18], bit[24], and bit[25] are designated for the reset interrupt, triggered in scenarios where no retry process is initiated. These conditions include the loss of FIN and ACK packets during active close operation, the reception of RST packet, and the reception of an unsupported packet. Further details regarding bit assignments for other sessions can be found in Table 7.

Table 7: Bit assignment of TCPRstIntStatus for session interrupt without retry process

Interrupt status	Session#0	Session#1	Session#2	Session#3
Lost FIN and ACK (active close)	bit[18]	bit[34]	bit[50]	bit[66]
RST packet reception	bit[24]	bit[40]	bit[56]	bit[72]
Unsupported packet reception	bit[25]	bit[41]	bit[57]	bit[73]

This section presents an example that illustrates a situation involving the loss of FIN and ACK packets during the termination of a connection in active mode. The subsequent steps, outlining the transmission of the RST packet following the detection of lost FIN and ACK packets, are detailed below.

Figure 30: TCPRstIntStatus[18] assertion

1) Following the user’s request of an active close command for session#0, a FIN packet is transmitted to the target. However, the target does not transmit FIN and ACK packets until the timeout value, as defined by TCPCtlTimeOutSet signal, is reached.

2) When the timeout duration is reached, both TCPRstInt and TCPRstIntStatus[18] are set to 1b, indicating the activation of the reset interrupt type. Subsequently, the RST packet is dispatched to the target to terminate the connection of session#0.

EMAC Interface

EMAC interface of TOE100GADV IP (TOEMAC I/F) is compatible with the 512-bit AXI4 stream bus, but it does not include a ‘Ready’ signal to pause data transmission in the receiving path (TOEMACRx I/F). This section provides comprehensive information regarding the TOEMAC I/F, covering both transmit and receive directions.

TOEMACTx Interface

When sending a packet, the TOE100GADV IP sets TOEMACTxValid signal to 1b together with the valid value of the associated signals (TOEMACTxData, TOEMACTxKeep, and TOEMACTxLast). Typically, every bit of the ‘Keep’ signal is asserted to 1b to indicate that every byte of transmit data is valid except for the last data cycle. TOEMACTxKeep can indicate the amount of valid bytes of TOEMACTxData from 1-64 bytes at the last data strobe. During data transmission, if the target EMAC is not ready to accept the data, the TOE100GADV IP can temporarily be paused from de-assertion of TOEMACTxReady to 0b. Figure 31 provides additional details about the TOEMACTx interface for the Transmit direction.

Figure 31 Tx Interface of TOEMAC

1) To initiate packet transmission, the first data of packet is sent via TOEMACTxData, and TOEMACTxValid is asserted to 1b. If this data is not the last one, all 512-bit of data will be valid, with TOEMACTxKeep set to all ones. Subsequently, TOEMACTxValid remains asserted at 1b to ensure continuous transfer until the last data of the packet has been successfully transmitted.

2) While transmitting the packet, if the Ethernet (MAC) Subsystem is not ready to receive more data, TOEMACTxReady is set to 0b, temporarily pausing data transmission. During this period, IP output signals including TOEMACTxData, TOEMACTxKeep, TOEMACTxLast, and TOEMACTxValid will maintain their values until TOEMACTxReady is re-asserted to 1b.

3) Upon transmitting the last data of the packet, both TOEMACTxLast and TOEMACTxValid are set to 1b. During the last transfer cycle, TOEMACTxKeep is set to indicate the number of valid bytes in TOEMACTxData. Following this, TOE100GADV IP may proceed with the next packet transmission by setting TOEMACTxValid to 1b or pause packet transmission by setting TOEMACTxValid to 0b.

TOEMACRx Interface

For packet reception, the Ethernet (MAC) Subsystem initiates packet transfer by setting TOEMACRxValid to 1b with valid value of TOEMACRxData, TOEMACRxKeep, TOEMACRxError, and TOEMACRxLast. Packet transmission may be paused temporarily when the Ethernet (MAC) Subsystem is not ready to send the packet by de-asserting TOEMACRxValid to 0b. However, Ethernet MAC has no ability to pause packet transmission, excluding ‘Ready’ signal. If the RxPac buffer within the TOE100GADV IP reaches full while a packet is still streaming in, the TOE100GADV IP will discard the ongoing received packet, as detailed in Figure 25. Furthermore, during the last data transfer cycle, the TOE100GADV IP verifies TOEMACRxError to ensure that the packet does not contain any errors. If an error is asserted, the packet will be discarded by the IP. Additional information about packet reception through the TOEMACRx interface is provided in below.

Figure 32 Rx Interface of TOEMAC

1) TOE100GADV IP detects the new receiving packet when the Ethernet (MAC) Subsystem sets TOEMACRxValid to 1b along with the first data in TOEMACRxData signal. If this is not the last data, TOEMACRxKeep always equals to all ones, indicating that all 512-bit data is valid.

2) If there is no data available, the Ethernet (MAC) Subsystem can temporarily halt the transfer of each packet by de-asserting TOEMACRxValid to 0b.

3) The last data of the packet is received through TOEMACRxData when both TOEMACRxLast and TOEMACRxValid are set to 1b. During this cycle, TOEMACRxKeep may not be equal to all ones if the number of valid bytes of TOEMACRxData is less than 64 bytes. Additionally, TOE100GADV IP checks TOEMACRxError to ensure that it is not asserted to 1b from any errors.

4) After the end of current packet reception, TOE100GADV IP has ability to accept the next packet immediately in the next cycle. In this situation, TOEMACRxValid is continuously asserted to 1b. Otherwise, TOEMACRxValid may set to 0b to end the packet transmission from EMAC.

5) When the incoming packet contains any errors, TOEMACRxError will be set to 1b during the last data transfer cycle, indicated by TOEMACRxLast being set to 1b. In this situation, the TOE100GADV IP will discard this error packet.

Connection termination of unusual case

Figure 33: Terminate connection sequence

The process of terminating a connection in the standard scenario is illustrated in Figure 33, which shows the exchange of four packets between two devices. In this sequence, the first device (Device#0) initiates the connection termination by sending a FIN packet. If the second device (Device#1) agrees to terminate the connection, it responds with both ACK and FIN packets, which may be combined into a single packet or transmitted separately. Finally, Device#0 confirms the termination by sending an ACK packet. However, this section describes the operation of the TOE100GADV IP in some unusual cases.

a) When the TOE100GADV IP receives an active close command from the user, it sends a FIN packet to commence the connection termination process. Subsequently, the TOE100GADV IP awaits FIN and ACK packets that contain specific values for the sequence number (SeqNum) and acknowledge number (AckNum). Assuming that a FIN packet emitted by the TOE100GADV IP assigns SeqNum=N and AckNum=M, the expected values of SeqNum and AckNum in the received FIN and ACK packets are M and N+1, respectively. If these expected packets are not received within the designated timeout period, without any retry attempts, the TOE100GADV IP transmits a RST packet to terminate the connection.

b) Following the execution of an active close command operation, the TOE100GADV IP ceases to accept additional data from the target. Any received packet containing data is discarded, similar to the first case. The IP only accepts FIN and ACK packets that match the expected values of SeqNum and AckNum.

c) If the target initiates the connection termination by sending a FIN packet while the TOE100GADV IP still has remaining data for transmitting, the TOE100GADV IP accepts the termination request and halts data transmission. It responds with FIN and ACK packets to the target, with the SeqNum value configured to match the most recently confirmed data acceptance value. Users can determine the amount of unsent data by monitoring the TOETxStat[19:0] signal. Additional details of this case can be found in Figure 16.

Verification Methods

The TOE100GADV IP Core functionality was verified by simulation and also proved on real board design by using KCU116 and VCK190 board.

Recommended Design Experience

User must be familiar with HDL design methodology to integrate this IP into their design.

Ordering Information

This product is available directly from Design Gateway Co., Ltd. Please contact Design Gateway Co., Ltd. For pricing and additional information about this product using the contact information on the front page of this datasheet.

Revision History

Revision	Date	Description
1.00	13-Feb-24	New release