• SC22-NRE-001 PDFndise sc22

The NDN for Data Intensive Science Experiments (N-DISE) project aims to accelerate the pace of breakthroughs and innovations in data-intensive science fields such as the Large Hadron Collider (LHC) high energy physics program and the BioGenome and human genome projects. Based on Named Data Networking (NDN), a data-centric future Internet architecture, N-DISE will deploy and commission a highly efficient and field-tested petascale data distribution, caching, access and analysis system serving major science programs. The N-DISE project will build on recently developed high-throughput NDN caching and forwarding methods, and containerization techniques, leverage the integration of NDN and SDN systems concepts and algorithms with the mainstream data distribution, processing, and management systems of CMS, as well as the integration with Field Programmable Gate Arrays (FPGA) acceleration subsystems, to produce a system capable of delivering LHC and genomic data over a wide area network at throughputs approaching 100 Gbits per second, while dramatically decreasing download times. N-DISE will leverage existing infrastructure and build an enhanced testbed with high-performance NDN data cache servers at participating institutions.

Goals:

• Integration of NDN with CMS Software Stack
• Joint Caching and Multipath Forwarding
• FPGA Acceleration Subsystems
• SDN Support for N-DISE
• Demonstration Testbed

Highlights from the SC22 N-DISE demo:

1. We demonstrated a record average throughput of 50 Gbps and a peak throughput of 63 Gbps on the N-DISE WAN testbed for the NDN-DPDK forwarder integrated with the NDNc consumer and producer applications (with consumer nodes at SC22 Caltech booth and StarLight Chicago and producer at Caltech).
2. We demonstrated that the VIP joint caching and forwarding algorithm, integrated with the NDN-DPDK forwarder and NDNc consumer and producer applications, achieved uniformly superior performance in terms of throughput, download latency, and cache hit ratios, against multiple baseline algorithms, in two separate topologies (service topology and multipath topology) over the N-DISE WAN testbed.
3. We demonstrated a transparent integration of NDN with the CMS Software components via an NDN based XRootD Open Storage System plugin. Under the same test scenario of demonstration 1, we showed a throughput of 4.8 Gbps for the integrated NDN-CMS system.
4. We demonstrated FPGA acceleration of the hashing computation for the Name Dispatch Table lookup operation of the NDN-DPDK forwarder.  A speedup of at least 2x is demonstrated on a local testbed at UCLA.
5. We demonstrated the integration of content names with Kubernetes clusters using NDN.   Using the same content names at the application, network, and compute layer, we showed the feasibility of a simplified name-based system that harmonizes compute and data for Genomics workflows.