We design AMIS: Advanced Measurement Instrument and Services to achieve flow-granularity network measurement, sustain scalable line rate, meet evolving measurement objectives and derive knowledge for network advancement. The scalable hardware architecture and software-defined measurement framework are key advantages of AMIS. The thrust of the work is to prototype, deploy and apply an advanced measurement instrument and to enable services for accurate network monitoring and in-depth traffic analysis at exchange points of international R&E networks. The instrument will support flow-granularity measurement and monitoring at line rates at up to 40Gbps, and engage software APIs to examine selected flows, with little impact to the performance of user traffic. Besides direct support of existing measurement tools (e.g., perfSONAR), the proposed instrument will provide libraries and applications allowing project-level or Autonomous System (AS)-level flow statistical queries. With scalable hardware and an open source software stack, the measurement services will equip network operators with effective tools to quantify flow-level network performance, and more importantly to enable in-depth flow analysis through software libraries. Such services bring opportunities for big data analysis of measurement results and packet traces, leveraging a high-performance, open computing infrastructure, the Massachusetts Green High Performance Computing Center (MGHPCC). The knowledge derived from the measurement will benefit network operation and provisioning.
The major objectives of AMIS project include the following:
- Design a hardware platform with scalable processing capabilities on network flows at 10, 40, and 100Gbps line rate;
- Design a software defined measurement framework that allows network operators to specify and instantiate measurement tasks and query results;
- Design privacy preserving algorithms to report measurement results while protecting the user privacy;
- Manage and analyze measurement data, make observations and provide insights about network usage patterns for network planning and operation.
Project Team & Partners
- Yan Luo, PI, University of Massachusetts Lowell
- Cody Bumgardner, Co-PI, University of Kentucky
- Gabriel Ghinita, Co-PI, Univ. of Massachusetts Boston
- Michael McGarry, Co-PI, University of Texas El Paso
- Northwestern University
- Florida International University
- Intel Corporation
- Indiana University
Overview of IRNC AMIS Project
We have deployed the prototypes in three locations: UMass Lowell, Univ. of Kentucky and ICAIR of Northwestern Univ. The AMIS instrument at these three locations are fed with 40Gbps and 100Gbps synthetic and real-world network traffic and are operating to serve as testbeds for the designed hardware architecture and software framework.
We are currently planning two new deployment locations. One of them is at Univ. of Texas El Paso as it serves the CUDI, an IRNC link between Mexico and US. The other new location is at Florida International University, which we had tried to setup an AMIS instrument early 2016, however faced challenges in physical connectivity with Tap devices. The recent network upgrade to 100Gbps at FIU/Ampath and availability of 100G Tap devices allow us to revisit the deployment options. We have identified the technical solutions of Tap devices and fiber types, and the purchase orders are being prepared.
Overview of IRNC AMIS Framework
There are five major parts in AMIS project listed as follows.
- AMIS Measurement Functions (UML)
- Packet loss detection
- Packet count
- Flow volume calculation
- Netflow generation
- Privacy Protection (UMB)
- Syntactic Privacy
- Differential Privacy with BigData tools
- Web-based Network Data Visualization and Analytics (UTEP)
- Configure and manage measurement tasks
- Annotate instrument data with auxiliary data for analytics
- Provide data visualization and analytics to support network management
- Plugin Management (UKY)
- Config management
- Dispactch mtask to AMIS nodes
- Query processing on netflow records