Terascale Computing System installed at Pittsburgh Supercomputing Center

October 02, 2001

PITTSBURGH -- The Terascale Computing System (TCS), the most powerful system in the world committed to unclassified research, is installed on schedule. Developed and implemented by the Pittsburgh Supercomputing Center in collaboration with Compaq Computer Corporation, with funding from the National Science Foundation, the TCS provides computational capability to scientists and engineers nationwide.

They will use it in many areas of research that have wide social impact, including earthquake modeling, storm-scale weather forecasting, global climate change, and protein genomics, modeling that's integral to the development of new drug therapies.

A joint project of Carnegie Mellon University, the University of Pittsburgh and Westinghouse Electric Company, the Pittsburgh Supercomputing Center (PSC) has deployed the TCS to fill a gap in U.S. basic research capability highlighted in a 1999 presidential report. Terascale means computational power beyond a "teraflop" -- a trillion calculations per second. With peak capability of six teraflops, the new system is now by far the most powerful available as an open resource for researchers attacking a wide range of problems.

"The TCS system at Pittsburgh Supercomputing Center continues a history of National Science Foundation support for high-performance computing," said Robert Borchers, director of NSF's Division of Advanced Computational Infrastructure and Research. "Through the NSF's Partnerships for Advanced Computational Infrastructure (PACI) program, this system will increase long-term, fundamental research across all science and engineering disciplines."

"In scale alone, the TCS pushes beyond where open-resource supercomputing technology has been before or would have gone without the NSF PACI program," said PSC scientific directors Michael Levine and Ralph Roskies in a joint statement.

"Compaq committed themselves to the success of the TCS and worked side-by-side with us to make it available on schedule. With storage capacity that's 100,000 times that of most PCs and with 10 million times the communications capability, this system brings significant new research capability to bear on many important problems. While the immediate, direct beneficiaries will be academic scientists, the benefits will flow to the country as a whole, in practical ways we can't forecast."

"Compaq is proud to join with the Pittsburgh Supercomputing Center and the National Science Foundation in delivering TCS," said Bill Blake, Compaq's Vice President of High Performance Technical Computing. "And we're excited that Compaq's AlphaServer SC supercomputing architecture will make such a significant contribution in opening new frontiers in scientific computing."

The TCS represents an unprecedented synthesis of "off-the-shelf" components integrated with an advanced interconnect -- from Quadrics Supercomputers World -- and other technologies to provide a very large-scale system for scientific computing. It comprises 3,000 Compaq Alpha EV68 microprocessors, housed in 750 four-processor AlphaServer systems running Tru64 UNIX. The latest evolution of the widely used Alpha microchip technology, the EV68 has peak floating-point capability of two gigaflops (two billion calculations per second).

Along with six teraflops of processing power, the TCS features 3.0 terabytes of memory, high-bandwidth, low-latency interconnections and remarkable capabilities for large-scale data handling, including the ability to write the entire memory to disk in under 40 seconds. This extremely short system-write time, developed through PSC systems and software engineering, is critical to efficient checkpointing, needed to preserve research data in the event of component failure.

Preparation for the TCS began in October 2000 with installation of a 256-processor prototype system. In August 2001, the first of the new AlphaServer systems arrived at the PSC computer room at Westinghouse Energy Center in Monroeville, Pennsylvania. System components came in multiple deliveries from Compaq facilities in Texas and Scotland.

An on-site team of Compaq, PSC and Westinghouse engineers and technicians -- supported by expert teams at Compaq locations in the United States, Bristol, England and Galway, Ireland -- worked aggressively to meet the Oct. 1 installation date.

"PSC's success in deploying this unprecedented, very large-scale system right on time is a fine achievement," said Jared L. Cohon, president of Carnegie Mellon University. "This is another important chapter in the center's outstanding record of providing the nation's scientists with the most advanced computational tools. This world-class computing system reflects Pittsburgh's international leadership in technology development and is a key component of our region's technology future."

"This computing system is an important advance in assuring the continuation of our nation's leadership in basic research," said University of Pittsburgh Chancellor Mark A. Nordenberg. "There is growing national and international recognition that university research is at the heart of most commercial innovation and much of our recent economic prosperity. And it is significant for our region that Western Pennsylvania is the home of this great national resource -- attesting to the strength of our institutions. When great research universities such as Pitt and CMU partner with industry -- in this case Westinghouse and Compaq -- and the federal government, we are poised to achieve the next great breakthroughs of this new century."

"PSC is to be congratulated on bringing this powerful new technology into being," said Charlie Pryor, president and CEO of Westinghouse Electric Company. "Westinghouse is proud to add its internationally recognized expertise in management excellence and technology leadership to the team. Once again, the Pittsburgh Supercomputing Center has demonstrated its leadership in high-performance computing."

The TCS installation marks the first operation of AlphaServer SC, the system software that ties AlphaServer systems together, on this scale and the first large-scale, multi-level Quadrics switch structure that supports thousands of processors while achieving sustained operation across the system. Standard benchmark software has measured system performance over three teraflops.

The TCS will next go through a period of "friendly user" testing, and by early 2002 it will become available to researchers nationwide through the peer-review process of the NSF PACI program.

PSC and Compaq collaborated on numerous machine enhancements to improve the performance of the TCS, changes that range from the disk controller and file system to wiring optimizations. By careful site planning and redesign of the AlphaServer configurations, PSC engineers reduced the distance between processors, thereby also reducing cabling and minimizing network latency.

Total TCS floor space is roughly that of a basketball court. It uses 14 miles of high-bandwidth interconnect cable to maintain communication among its 3,000 processors. Another seven miles of serial, copper cable and a mile of fiber-optic cable provide for data handling.

The TCS requires 664 kilowatts of power, enough to power 500 homes. It produces heat equivalent to burning 169 pounds of coal an hour, much of which is used in heating the Westinghouse Energy Center. To cool the computer room, more than 600 feet of eight-inch cooling pipe, weighing 12 tons, circulate up to 900 gallons of water per minute, and twelve 30-ton air-handling units provide cooling capacity equivalent to 375 room air conditioners.
For background information, see http://www.psc.edu/publicinfo/terascale/bigiron.html

Compaq and AlphaServer are trademarks of Compaq Information Technologies Group, L.P. The Pittsburgh Supercomputing Center was established in 1986 and is supported by several federal agencies, the Commonwealth of Pennsylvania and private industry.

Michael Schneider, schneider@psc.edu
Sean Fulton, sfulton@psc.edu
Pittsburgh Supercomputing Center

Carnegie Mellon University

Related Technology Articles from Brightsurf:

December issue SLAS Technology features 'advances in technology to address COVID-19'
The December issue of SLAS Technology is a special collection featuring the cover article, ''Advances in Technology to Address COVID-19'' by editors Edward Kai-Hua Chow, Ph.D., (National University of Singapore), Pak Kin Wong, Ph.D., (The Pennsylvania State University, PA, USA) and Xianting Ding, Ph.D., (Shanghai Jiao Tong University, Shanghai, China).

October issue SLAS Technology now available
The October issue of SLAS Technology features the cover article, 'Role of Digital Microfl-uidics in Enabling Access to Laboratory Automation and Making Biology Programmable' by Varun B.

Robot technology for everyone or only for the average person?
Robot technology is being used more and more in health rehabilitation and in working life.

Novel biomarker technology for cancer diagnostics
A new way of identifying cancer biomarkers has been developed by researchers at Lund University in Sweden.

Technology innovation for neurology
TU Graz researcher Francesco Greco has developed ultra-light tattoo electrodes that are hardly noticeable on the skin and make long-term measurements of brain activity cheaper and easier.

April's SLAS Technology is now available
April's Edition of SLAS Technology Features Cover Article, 'CURATE.AI: Optimizing Personalized Medicine with Artificial Intelligence'.

Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.

Post-lithium technology
Next-generation batteries will probably see the replacement of lithium ions by more abundant and environmentally benign alkali metal or multivalent ions.

Rethinking the role of technology in the classroom
Introducing tablets and laptops to the classroom has certain educational virtues, according to Annahita Ball, an assistant professor in the University at Buffalo School of Social Work, but her research suggests that tech has its limitations as well.

The science and technology of FAST
The Five hundred-meter Aperture Spherical radio Telescope (FAST), located in a radio quiet zone, with the targets (e.g., radio pulsars and neutron stars, galactic and extragalactic 21-cm HI emission).

Read More: Technology News and Technology Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.