StarT-X: A PCI Arctic Network Interface Unit

The StarT-X Arctic Network Interface Unit (NIU) is a PCI card with hardware support for user-level message passing in a cluster of workstations interconnected by an Arctic Switch Fabric. This is one of the latest developments in a series of StarT parallel processing cluster projects. StarT-X NIU is a direct follow-on to StarT-Jr's FUNi and carries over many of the same message-passing mechanisms. However, StarT-X NIU achieves significantly better performance than FUNi, in both bandwidth and latency, by handling the critical paths in hardware rather than relying on embedded processing. Salient features of StarT-X NIU are: A more detailed description of the StarT-X project is available as slides from a presentation given to Xolas users in April, 1997.

Hardware Status (Comments: This status section is (very) outdated. The activities described were completed succesfully in 1998.)

The first four StarT-X NIU's have been successfully tested with the prototype Arctic Switch Fabric to interconnect a cluster of four Intel Pentium PCs. Bandwidth at 70 MB/sec has been observed for DMA transfers between two Intel Pentium PCs with 430FX chipsets. User-to-user latency is less than 4 usec. StarT-X's performance is only limited by the the performance of the PCI bus on the host system.

StarT-X NIU works with any host platform that supports the 32-bit 33MHz PCI bus standard. StarT-X NIU's are currently installed in LCS's Xolas Cluster of 9 8-processors SUN Enterprise 5000 SMP's. We plan to produce up to 50 StarT-X NIU's to interconnect the other LCS clusters, including the Pleiades Cluster (7x4 DEC Alpha's), and the pending donation of 20+ Intel four-way SMP's.

Software Status (Comments: This status section is (very) outdated. The activities described were completed succesfully in 1998.)

Device drivers are available for Intel and SPARC platforms running Linux and Solaris, respectively. The existing network management software is functional, but extensions to its functionality and user interface are still being added. JAM, an ultra light-weight communication library, provides a mix of C function calls and macros to reflect the hardware-supported message-passing mechanisms with minimal overhead. JAM is ideal for building higher-level libraries. Collaborative efforts to port MPI and distributed Cilk are currently underway.

A StarT-X PCI NIU Card

Two StarT-X Cards in a SUN Enterprise PCI I/O Board

Four Arctic Test Boards (A 4-way switch)

LCS's Xolas Cluster of 9 SUN E5000 SMP's with 8 processors each

Arctic cables (gray ribbons about $100 each) exiting the back panel of a SUN E5000 with 4 StarT-X cards

James C. Hoe (jhoe+www at ece_cmu_edu)

Keywords: StarT-X, StarT-Jr, FUNi, Arctic, network of workstations, parallel processing, network interface, user-level, interprocessor communication