THEMIS is a fairness-maintenance patch for long-haul RDMA networks. It addresses congestion-induced unfairness caused by asymmetric feedback delays between inter-datacenter and intra-datacenter flows. The implementation combines a Proactive Notification Point (PNP) for near-source congestion with a Temporary Reaction Point (TRP) for near-destination congestion.
This repository contains the open-source artifacts for two versions of THEMIS. Use the branch corresponding to the paper you are reproducing:
| Paper | Code branch | Artifact |
|---|---|---|
| ICNP 2025 conference paper | main |
NS-3 simulation and P4/eBPF/RDMA testbed |
| IEEE/ACM Transactions on Networking extended paper | TON |
Extended-version source code |
| Path | Description |
|---|---|
NS3/simulation/ |
NS-3 simulation code and example configurations |
NS3/traffic_gen/ |
Traffic and workload generation utilities |
NS3/analysis/ |
Simulation trace and performance analysis tools |
P4/ |
P4 dataplane and switch control-plane code |
eBPF/ |
eBPF-based daemon used by the testbed prototype |
RDMA_RC_example/ |
RDMA RC example used for host-to-host testing |
Reduces feedback delay for near-source bottlenecks by generating congestion notifications (CNPs) within a single DC.
Temporarily throttles inter-DC flows to address near-destination bottlenecks until the sender receives congestion feedback.
THEMIS is designed to be compatible with existing congestion control algorithms like DCQCN.
THEMIS requires only modifications at the external switches (ESW) connecting the DC to the WAN, ensuring minimal impact on existing infrastructure.
The implementation is optimized to minimize resource overhead on switches, making it deployable on current hardware.
The NS-3 module is developed based on the HPCC project. The project configuration method is consistent with HPCC. Start with the simulation README, then see the traffic generator and analysis guide for workload creation and result processing.
From the repository root, the original example workflow is:
cd NS3/simulation
./waf --run 'scratch/third mix/mix_70%_0.03s/config.txt' > mix/mix_70%_0.03s/out.txt
cd mix/mix_70%_0.03s
python3 annalyze.pyThe figures used in the paper were generated through NS3/analysis.
The testbed prototype uses a P4-capable Barefoot/Tofino switch, the corresponding SDE, Linux hosts with RDMA-capable NICs, and the required RDMA and eBPF software. The switch pipeline and control-plane build targets are documented in P4/Makefile; the RDMA example build and client/server invocation are in RDMA_RC_example/README.md. The daemon entry point is eBPF/daemon.py.
The logical testbed topology is:
Host A --- P4 switch --- Host B
The testbed verifies whether the daemon packet sent via eBPF can be received by the switch CPU in time and whether the flow rule can be installed before the first RDMA data packet arrives.
Hardware setup depends on the installed switch SDE, NIC, and host configuration. Configure those for your environment before running the prototype.
Start the switch:
cd P4
$SDE_INSTALL/bin/bf_kdrv_mod_load $SDE_INSTALL
ls /dev/bf0
make switch
./controlStart the eBPF daemon:
cd eBPF
sudo python daemon.pyBuild and start the RDMA RC transfer. Run the server command on Host B and the client command on Host A:
cd RDMA_RC_example
gcc -Wall -O2 -o RDMA_RC_example RDMA_RC_example.c -libverbs
# Host B (server)
./RDMA_RC_example -g 3 -d mlx5_0
# Host A (client)
./RDMA_RC_example -g 3 -d mlx5_0 <IP of HOST B>The main branch corresponds to the conference paper:
Z. Niu, M. Zhang, J. Zhang, R. Xie, Y. Yang, and X. Hu, "THEMIS: Addressing Congestion-Induced Unfairness in Long-Haul RDMA Networks," in 2025 IEEE 33rd International Conference on Network Protocols (ICNP), Seoul, Republic of Korea, 2025, pp. 1–13, doi: 10.1109/ICNP65844.2025.11192376.
@inproceedings{niu2025themis,
title={THEMIS: Addressing Congestion-Induced Unfairness in Long-Haul RDMA Networks},
author={Niu, Zihan and Zhang, Menghao and Zhang, Jue and Xie, Renjie and Yang, Yuan and Hu, Xiaohe},
booktitle={2025 IEEE 33rd International Conference on Network Protocols (ICNP)},
pages={1--13},
year={2025},
organization={IEEE},
doi={10.1109/ICNP65844.2025.11192376}
}The TON branch corresponds to the extended journal paper:
M. Zhang, Z. Niu, J. Zhang, D. Wang, R. Xie, Y. Yang, M. Xu, and C. Hu, "THEMIS: Addressing Congestion-Induced Unfairness in Long-Haul RDMA Networks," IEEE/ACM Transactions on Networking, 2026, doi: 10.1109/TON.2026.3743177.
@article{zhang2026themis,
title={THEMIS: Addressing Congestion-Induced Unfairness in Long-Haul RDMA Networks},
author={Zhang, Menghao and Niu, Zihan and Zhang, Jue and Wang, Dongyang and Xie, Renjie and Yang, Yuan and Xu, Mingwei and Hu, Chunming},
journal={IEEE/ACM Transactions on Networking},
year={2026},
doi={10.1109/TON.2026.3743177}
}