| High-Speed File-Based Data Logging for 100G Ethernet Sustained — powered
by NVMe-IP for Gen5 and exFAT2-IP. |
|
Design Gateway Hot! News
February 2026
|
|
|
|
|
The Challenge: High-Speed Data Logging
As network and sensor data rates continue to increase, many data logger
systems struggle to maintain sustained recording speed — especially when file system support is required. CPU-based architectures often become the bottleneck, leading
to:
• Dropped data and packet loss
• Complex software stacks
• Limited scalability |
This FPGA-based data logging project demonstrates a standard, reusable
data logger architecture, implemented with 100G Ethernet to clearly showcase high-performance capability.
|
FPGA-Based Reference Architecture
This project is built as an FPGA-based system, where all time-critical
data handling is performed in hardware logic. Incoming data is buffered,
processed, and recorded without relying on a CPU for the data path.
Ethernet Data Logger System
|
To demonstrate a demanding real-world scenario, the system is implemented
as a 100G Ethernet data logger. This closed application allows customers to easily understand system
behavior, performance limits, and design advantages — while still serving
as a flexible base for other applications. Watch the YouTube Demo Video
|
Sustained Recording with DDR Buffering
One of the key challenges in high-speed logging is maintaining a stable,
sustained write rate. In this system:
• Incoming Ethernet data is temporarily buffered in DDR memory
• DDR absorbs burst traffic and rate variations
• Data is written to storage at a steady, sustained speed |
This buffering mechanism ensures continuous recording without packet loss,
even under full line-rate traffic.
|
High-Speed File-Based Recording (PCAPNG)
Unlike raw data dump solutions, this system records data directly into
a standard file system format:
• Packets are converted in FPGA logic into PCAPNG format
• Files are written using exFAT, enabling large-capacity storage
• Recorded files can be opened directly in Wireshark
• No post-processing or conversion is required |
This makes captured data immediately usable for analysis, debugging, and
validation.
|
Built on Proven Storage IP Cores
The system is powered by two key IP cores that work together seamlessly:
| IP Core |
Key Features |
| NVMe-IP (PCIe Gen5) |
High-speed NVMe SSD interface, optimized for sustained continuous write
performance, designed for FPGA-based storage acceleration |
| exFAT2-IP |
File system support directly in hardware, PC-compatible removable storage,
simple data access without special tools |
Together, these IPs combine high-speed recording with easy data retrieval,
creating a smooth workflow from capture to analysis.
|
Packet Filtering via Software Configuration
To improve usability, the system includes packet filtering control through
a user application:
• Users can configure filtering rules easily
• No FPGA redesign is required for filter changes
• Only selected packets are recorded, reducing storage load |
This makes the system practical for real-world logging and monitoring scenarios.
|
Scalable on Agilex 7 FPGA Platform
The demo is implemented on an Altera Agilex 7 FPGA board, and the hardware
resources can be adjusted to match final system requirements.
Examples of scalability include:
| Component |
Demo System |
Board Capability |
| Ethernet |
100G Ethernet |
Up to 400G Ethernet |
| Memory |
Single DDR component |
Multiple DDR channels |
| Storage |
Single NVMe SSD |
Up to 8 NVMe SSDs |
This flexibility allows customers to scale performance without changing
the core architecture.
|
A Reusable Base for High-Speed Data Loggers
While demonstrated as a 100G Ethernet data logger, this FPGA-based system
can serve as a base platform for:
|
Network traffic logging
|
High-speed sensor data recording
|
Test and measurement equipment
|
Telecom and data center monitoring
|
|
Any application requiring high-speed recording with file system support
|
|
Watch the YouTube Demo Video
|
|
|
|
|
| Contact Us |
|
|
| © 2026 Design Gateway Co., Ltd. • Website • YouTube • LinkedIn • Blog • Unsubscribe |