Infrastructure: SCinet

SCinet

SCinet Chair
Dennis Duke, Florida State University

SCinet Vice Chair
Jim Rogers, Computer Sciences Corporation

SCinet Vice Chair
Bill Nickless, Argonne National Laboratory

SCinet is the high-performance network built to support the annual International Conference for High Performance Computing and Communications (SC). The SC Conference Series is co-sponsored by ACM SIGARCH and the IEEE Computer Society. SCinet features both a high-performance production-quality network as well as an extremely high performance experimental network, Xnet.

Volunteers from educational institutions, high performance computing centers, network equipment vendors, research networks, and telecommunication carriers work together to design, build, and operate SCinet. Industry vendors and carriers donate much of the equipment and services need to build the LAN and WAN infrastructure. Planning begins more than a year in advance of each SC Conference and culminates with a high-intensity installation just 7 days before the Conference begins.

SC2002 participants will have wide area connectivity to Abilene, ATDnet, DREN, ESnet, HSCC, SuperNet, and vBNS+. Qwest Communications is our most significant partner, delivering substantial WAN services including multiple OC-192c connections and sponsoring the Bandwidth Challenge. The aggregate WAN connectivity available to Industry and Research Exhibitors is expected to exceed 40 billion bits/second (Gbps).

Service Offerings
Wireless [1]

Ubiquitous 802.11b service throughout the meeting rooms and exhibit floor space for people with wireless laptops.
802.11a service in the exhibit floor space and common areas.

Ethernet [2][3]

100BaseFX (multi-mode fiber) (1500-byte Ethernet frames)
1000BaseSX (multi-mode fiber) (1500 or 9000-byte Ethernet frames)
1000BaseLX (multi-mode fiber) (1500 or 9000-byte Ethernet frames)
10GBase-LR (single-mode fiber) (1500 or 9000-byte Ethernet frames)

IP over SONET [4]

OC-48c/STM-16c (single-mode fiber) (4470 byte IP packets)
OC-192c/STM-64c (single-mode fiber) (4470 byte IP packets)

ATM over SONET [5][6]

OC-12c/STM-4c (single-mode fiber) (9180 byte IP packets)
OC-48c/STM-16c (single-mode fiber) (9180 byte IP packets)

SONET [7]

OC-12c/STM-4c (single-mode fiber)
OC-48c/STM-16c (single-mode fiber)
OC-192c/STM-64c (single-mode fiber)

Dark Fiber [8]

single-mode fiber only

Notes:
[1] Native IPv4 and IPv6 Unicast provided by default, with automatic address allocation from the network.
[2] Native IPv4 Unicast, IPv4 Multicast, and IPv6 Unicast provided by default. Router-to-router connections (IPv4 /30) and subnet routing options supported.
[3] Private 802.1q VLAN trunking supported between booths on request.
[4] Native IPv4 Unicast, IPv4 Multicast, and IPv6 Unicast provided by default. Router-to-router connections only.
[5] Wide-area ATM service not guaranteed..
[6] Native IPv4 Unicast and IPv6 Unicast service provided on request, served by a shared OC-12c/STM-4c ATM router port. Router-to-router AAL5-SNAP PVCs only; no LANE support.
[7] Wide-area SONET circuits will be terminated at the Baltimore Convention Center demarc. These circuits carried to the booth by SCinet-managed optical transport equipment or dark fiber, at the discretion of SCinet.
[8] All dark fiber will be run through the SCinet NOC patch panels, where it will be terminated and cross-connected as necessary.

Network Performance Monitoring
SCinet has incorporated a network monitoring infrastructure into the design of this year's network. Monitoring will be used both to watch the internal network for operational purposes and to characterize the high-performance network applications that traverse SCinet, in particular for the Bandwidth Challenge.

Utilization and errors for all external links, and all major SCinet internal links will be monitored for operational purposes. Active techniques will be used to monitor reachability over the external links and latency to key sites.

Internet2® in conjunction with SCinet will provide a "weather map" showing current utilization on all SCinet external links, based on the technology used for the Abilene NOC weather map, developed by the Abilene NOC at Indiana University.

Spirent Communications will provide multiple Adtech AX/4000s to passively monitor each wide area connection and collect statistics. These statistics will include total aggregate traffic counts on each of the connections and total instantaneous traffic counts for use in judging this year's high-Bandwidth Challenge.

Flow data (e.g, NetFlow, cflow) will be collected from routers and visualized using FlowScan, a tool developed by Dave Plonka at the University of Wisconsin at Madison.

Packet Design CNS is providing multiple Route Explorer appliances that provide visibility into IP routes for quickly pinpointing and analyzing router-related problems during network setup and operation. Route Explorer provides a current routing layer view of the IP network and records all routing events which can be used for historical analysis. Its reporting capabilities include web-based reports, a query API, and alerts.

Network Security
The design characteristics that define the SCinet production networks include high bandwidth, low latency, resiliency, and scalability. SCinet peers with the Internet, Agency, and National wide area networks through a series of very high-speed connections. To maximize performance across these interfaces, there are no firewalls. In this regard, the SCinet network is a logical, albeit temporary, extension of the open Internet. Exhibitors and Attendees are reminded that, in this potentially hostile environment, network security is a collective responsibility.

Insecure Applications
The use of insecure applications including TELNET, POP, and FTP is very strongly discouraged. These applications are subject to compromise because they send passwords to remote hosts in human readable cleartext.. Attendees are strongly encouraged to protect their sessions through a mechanism such as Secure Shell (SSH), where all communication is encrypted. SSH implementations are available for little or no cost and are straightforward to install and use. Each Attendee is responsible for ensuring that their communications sessions are protected in accordance with their security requirements

SCinet is providing unencrypted IEEE 802.11a and 802.11b wireless networks for use by Attendees and Exhibitors. Wireless networks are particularly vulnerable by their very nature. The ease of use that makes them attractive is the same feature that is most easily exploited. Wireless networks are open to unauthorized monitoring or snooping by anyone within range of an access point.

Passive Monitoring
SCinet will passively monitor traffic on most external network connections as part of their network performance monitoring activities. In addition, SCinet has a restricted capability to monitor Exhibit floor and external network traffic for evidence of security-related activity including compromise or abuse. However, by no means should this coverage be considered a substitute for safe security practices. Please do your part by being cognizant of network security risks and protecting your systems and sessions.

Expanded Wireless Network Services
In collaboration with Cisco Systems, SCinet will deploy both IEEE 802.11b and the newer IEEE 802.11a wireless networks within the Baltimore Convention Center. Both of the wireless networks are part of the production SCinet network, providing access to the Internet, and many other National and Agency networks. The IEEE 802.11b wireless network, operating at a peak transfer rate of 11Mbps, will be provided on the Exhibit Floor, in the Education Program areas, the Ballroom and meeting rooms, and in many common areas within the Baltimore Convention Center. The IEEE 802.11a wireless network, operating at a peak transfer rate of 54Mbps, will be provided on the Exhibit Floor only.

Free Wireless Access for All Attendees
SCinet provides the wireless networks for use by all Exhibitors and Attendees at no charge. Please refer to the wireless coverage diagram available at the SCinet NOC for specific coverage information for both networks. Known wireless network limitations, such as areas of reduced signal strength, limited client capacity, or other coverage difficulties will be described with additional signage at appropriate locations throughout the Baltimore Convention Center.

DHCP-Enabled Service
IP settings including IP and DNS addresses for wireless clients are automatically provided by SCinet via DHCP. Laptops and other wireless devices configured to request network configuration information via DHCP receive this information automatically upon entering the SCinet wireless coverage area. Wireless devices must conform to the IEEE 802.11a or IEEE 802.11b standards. Please refer to http://www.weca.net/ for more information.

Wireless FAQ Available
SCinet cannot directly support requests for assistance with wireless devices. However, a matrix of network interface cards, operating systems, and access point compatibility is listed on the SCinet web page along with links to wireless equipment vendors, device drivers, and instructions. Wireless network interface cards are available for purchase through the SC2002 conference store.

Wireless Monitoring
SCinet will monitor the health of the wireless networks and maintain this information for Exhibitors and Attendees. The wireless networks are governed by the SCinet Service Level Policy. In summary, while every practical effort shall be made to provide stable reliable network services, there is no explicit service level agreement for any SCinet network, including the wireless networks, nor are there any remedies available in the event that network services are lost.

SCinet Control of the 2.4GHz and 5.2GHz Frequency Radio Spectrum
In order to provide as robust a wireless service as possible, SCinet must control the entire 2.4GHz and 5.2GHz frequency radio spectrum (2.412GHz-2.462GHz) and (5.15GHz to 5.35GHz) within the Baltimore Convention Center. This has important implications for both Exhibitors and Attendees:

Exhibitors and Attendees may not operate their own wireless Ethernet access points anywhere within the Baltimore Convention Center, including within their own booth.
Exhibitors and Attendees may not operate 2.4GHz cordless phones.
Exhibitors and Attendees may not operate 2.4GHz wireless video or security cameras, or any other equipment transmitting in the 2.4GHz or 5.2GHz spectrum.

Successful Wireless Operation is a Community Responsibility.
SCinet wants you to have a successful, pleasant experience at SC2002. This should include the ability to sit down with your wireless-equipped laptop or PDA and check e-mail or surf the Web from anywhere in the wireless coverage area. Please help us achieve this goal by not operating equipment that will interfere with other users. SCinet reserves the right to disconnect any equipment that interferes with the SCinet network.

Xnet
Xnet (eXtreme Net) provides a venue to showcase bleeding-edge, developmental networking technologies and experimental networking applications.

The SCinet Exhibit floor network has evolved into a robust, high-performance, production-quality network that Exhibitors and Attendees depend on for reliable local area, wide area, and commodity network service. Consequently, it has become increasingly difficult for SCinet to showcase bleeding edge, potentially fragile technology. Simultaneously, OEMs have at times been reticent about showcasing bleeding-edge hardware in SCinet, as it became a mission critical, production network.

Xnet provides the solution to this dichotomy by providing a venue which is by definition bleeding-edge, pre-standard, and in which fragility is understood. Xnet thus provides vendors and researcher exhibitors an opportunity to showcase emerging network gear or capabilities, prior to their general commercial availability.

Xnet debuted in Portland, OR at SC’99, where Dense Wavelength Division Multiplexing (DWDM) technology was used in the implementation of OC-48 SONET rings on the conference show floor. At SC2000, Xnet demonstrated pre-production and early delivery 10-Gigabit Ethernet equipment connecting several exhibit floor booths. The SC2001 Xnet expanded the deployment of 10 Gigabit Ethernet using equipment from several vendors and using 10 Gigabit Ethernet in several Bandwidth Challenge Applications, including the highest performance application to date.

Xnet is focusing on novel uses of 10 Gigabit Ethernet and other emerging technologies for 2002. Please refer to materials available at the time of the conference for additional information.

Service Level Policy
The SCinet Committee (SCinet) provides commodity Internet, research, and experimental networks for use by the Exhibitors and Attendees. While every practical effort shall be made to provide stable and reliable network service on each network, there is no explicit service level agreement for any SCinet network, nor are there any remedies available in the event that network services are lost.

SCinet provides a series of networks each year for use by the Exhibitors and Attendees. Each network can be broadly categorized as Commodity Internet, Research, or Xnet infrastructure. In addition, there are significant peering relationships among these networks that allow them to communicate.

Commodity Internet networks include the high bandwidth connection from the convention center to one or more Internet Service Providers, and both wired and wireless networks that connect the exhibit halls, meeting rooms, ballrooms, mail rooms, and other common areas to the Internet.

Research networks include very high bandwidth connections to National and Agency networks including Internet2, ESnet, DREN, NREN, ATDnet, and vBNS. Coupled with the extensive peering relationships that these networks have with other research networks worldwide, SCinet can engineer connectivity to virtually any public IP address in the world. Access to these networks is limited to Exhibitors with network connections to the SCinet core.

Xnet networks are typically experimental and often fragile. These networks connect small numbers of devices at extremely high bandwidth using equipment that is pre-production, pre-standard, or research oriented. In most cases, Xnet networks do not peer with other networks to reduce potential network volatility.

Full Description
SCinet is responsible for the design, engineering, installation, operation, and maintenance of all Commodity, Research, and Xnet networks. These networks must be installed during the week prior to the conference, and removed in their entirety by the day after the conference. The design and engineering phase of SCinet occupies much of the year preceding the conference. In contrast, timelines for the installation, operation, and maintenance are extremely compressed, and introduce significant operational risk that would not necessarily be present in a production environment.

SCinet is primarily organized across functional areas. There are specific points of contact with responsibility for Wide Area Transport, Internet Services, Wireless Infrastructure, Architecture, Routing and Switching Services, and Xnet. Each of the functional area leads has an appropriate staff level to support the installation, operation, and maintenance of that area. Area leads coordinate the interaction among separate groups.

SCinet will make every practical effort to provide uninterrupted service on all networks that it manages. In the event that there is a disruption of service on any network, every practical effort will be made to return that network to service as quickly as possible. Efforts to correct network errors shall be prioritized across the following broad guidelines. Higher priority events are listed first.

1. Failure of core routing services in the SCinet infrastructure.
2. Network service disruptions related to the wide area network transport facilities.
3. Widespread failure of the commodity Internet services.
4. Widespread failure of routing and switching services beyond the core.
5. Widespread failure of wireless Internet services.
6. Disruption of service for individual connections.

While every practical effort shall be made to provide stable and reliable network service on each network, there is no explicit service level agreement for any SCinet network, nor are there any remedies available in the event that network services are lost.