IEEE Supercomputing 2007
This year was my fourth time attending the IEEE Supercomputing conference. My experiences at Super are almost opposite from my experiences at most other conferences I attend. At SC, I tend to hear information that I can't immediately apply back on campus (deeply scientific presentations based on highly advanced projects conducted on heavily specialized equipment). I'm surrounded by people whose occupations tend to be substantially different from mine (professors and scientists, grid/cluster administrators and scientific programmers). It sure isn't Educause.
So, what am I doing here? Other than sometimes bumping into my colleagues from the CIC Research Computing Group, I'm here as a trend-spotter and as an observer of the field and of faculty. I speak entirely in that capacity, and I'd like to share with you my highly un-scientific observations for your consideration.
Data
One trend I've been tracking actually started back at the first SC I attended, in Phoenix in 2003. Many of the presentations at SC 03 included an observation by the presenter that modern techniques (whether smashing particles together or embedding sensor networks into the US highway system) were producing a vast quantity of data, and further that this quantity of data presented a number of unique challenges which had no real solution at the moment.In subsequent years, discussions of the problem of data and metadata seemed to oscillate between anxious pessimism and shrugging optimism -- either a solution would be found, or it wouldn't. Disks got bigger, memory got faster, and data description standards evolved. This was business as usual. I was disappointed to observe that no one was referring to the historical guardians of data and information architecture: the Library. But that could be a blog post all on its own.
However, in SC06, the game-changing realization seemed to be that a new strategy toward scientific discovery was emerging -- or at least a historical strategy was re-emerging as relevant: the notion that one can make discoveries not only via the strict scientific method (hypothesis leading to an experiment to test that hypothesis leading to refinement of the original hypothesis) but also via data mining and mashing data together without necessarily having a hypothesis or an explicitly designed experiment in the picture. When you have scads of data, former outliers might well turn out to be the signs of some new discovery.
At SC07, I was curious to hear how attitudes toward the data deluge had evolved. The current answer seems to lie in visualization, building on the data-mining trend from last year. This only makes sense as a further step -- if you have so much data that you struggle to store it, surely you also have too much data to look at it number-by-number. In general, the strategy seems to be to visualize data and observe overall trends and behaviors in the visualization, and to convert that observation into further areas for hypothesis and experiment. In addition, there is a growing use of simulations to see behaviors which one cannot (or cannot YET) observe, and then to go looking for that behavior in the world.
The final session of the week (HPC Survivor!) was a humorous look at the storage dilemma, with "experts" describing and defending various future architectures -- expert maintenance (vast disk rooms tended by acolytes); continual expansion via high-capacity portable modules (just keep buying big shipping containers of disk, bury them underground, stick them on planes); the cloud approach, blasting data into cyberspace (the Google approach, zero cost -- hah -- and streamed back to you from some "other" place); and iPods (extreme miniaturization -- from the iPod nano to the iPod femto, 1TB of storage on an earring). One participant asked "Will exaflops of storage be necessary if computers are sufficiently powerful to re-compute the calculations?" I think he was kidding...but...
Hardware/Software
The overall concern for the environmental impact of powering supercomputers was much higher this year compared to last year. This year also marked the rise of the "Green 500" as a contrary measurement to the traditional "Top 500" as a measurement of the most powerful computers.
Another observation I made this year is that there seems to be less concern about hardware capabilities (unlike SC05 & SC06, which was all about the petabyte and the need for petascale computing to address various critical questions) and more about software -- code that will run in an optimal way, which can be written and verified quickly enough to make the computational scale relevant. After all, it's no use having the fastest computer in the world to run your code if you can't code the algorithms you need in a reasonable amount of time. Everything will have moved on before you finish. I take this to be a significant open question -- where will the software for these clusters come from ("the community" is a conveniently nebulous answer), and how will it be written (given the overall dimunition of instruction in Fortran).
The use of non-traditional boards (like the PS3 processor and graphics cards) for particular computations continues. Now that more and more of these boards have double-precision, their usability for scientific calculations has increased. The ability to code directly against the hardware gives you full control over how the processor gets used, but also starts to suggest the need for tools to abstract away the challenges of thinking about millions of cores. The relatively small amount of on-chip memory on cell processors in particular leads to a different way of thinking about problems -- tiny, super-optimized code done in discrete chunks. These new boards are leading to new techniques for developing parallel applications, but the standardization and spread of these techniques is only beginning.
The Bleeding, Bleeding Edge
One of the most fascinating trends I had an opportunity to hear about is a reflection of work by individuals such as MIT's Neil Gershenfeld and David E Shaw of DE Shaw Laboratories -- the transcendence of computers from a thing in a box and into the thing itself.
Shaw's example of this connectedness is a very tangible one -- his laboratory is developing a specialized chip, called Anton. Anton optimizes the solution of a particular group of problems in molecular dynamics (how proteins form and fold). In the case of Anton, the structure of the problem are echoed in the structure of the chip to solve the problem -- the steps of the solution process are in the hardware itself.
The inevitable mixing of concepts that results from this kind of connectedness was readily reflected in the questions that followed Shaw's talk -- when asked about energetics and quantum behavior, it was necessary to carefully distinguish whether the question was about the problem, the algorithms or the hardware. The atoms and molecules being studied are starting to converge with the structure of the thing which studies them. Fascinating.
Neil Gershenfeld's keynote included examples were a step or two further removed from immediate practical use, but gave a further insight into what it might mean for the future. What if programming "bits" -- those 1s and 0s -- and programming "atoms" -- fabricating new materials, components, and structures -- was one and the same? At a basic level, one can think of a computer program as a structured orchestration of logic gates -- pathways of ANDs and ORs, or at a higher level, IFs and NOTs. Progress in programming computers proceeded from directly wiring these gates to expressing the desired wiring paths by means of various input devices -- cards, text, drag and drop -- which converted the input back into the basic gates again. What if we used all we have learned to go back to the beginning?
One example he cited was a "paintable computer" -- if you express the logic gates of a problem in terms of dots on a wall, you might imagine a wallpaper which turns a wall into a supercomputer. If you needed more computational power, you might paint another layer on top. It brings a whole new meaning to redecorating your office! Another example he gave conjectured was a building automation system. Programmable materials would allow the building infrastructure to be composed of materials which constantly maintain a sense of the building's own use of energy, the air flows and temperatures, the status of doors and windows, and which adjusted itself dynamically over the course of a day to avoid waste and pinpoint problems.
Programming in atoms has fabulous and exciting implications for the future. When you smash the idea of the programmable atom with the emerging notion of the "fab lab" -- rapid prototyping systems like 3D printers which are evolving toward the capabilities of the Star Trek replicator -- the picture of a powerful creative force for humanity starts to emerge.
State of the Field
The overall theme of this year's conference was competitiveness and industry, which meant that a number of the invited speakers were representing industries such as automotives, insurance, and entertainment. One of the most interesting presentations I listened to was given by Michael Resch from Stuttgart's HLRS http://www.hlrs.de/, who described the public/private partnership between the university and nearby automotive and machinery companies. The arrangement was somewhat complex, but seemed to balance the needs of all involved. Essentially, the company is a shell for organizing the purchasing, operation, and reselling of cpu cycles. The center is university-owned, the computers are corporate and state purchases, and the company is co-directed by an industry representative and a university representative.
They re-sell cycles to small start-ups as well as the constituent companies. They grant high priority to industry jobs as needed, but at a premium price. The countering benefit to university researchers (who might find their jobs pre-empted by a last-minute automotive simulation) is access to 3x-5x more overall power than could be purchased without the partnership, access to a diverse array of architectures, and shared labor costs.
Although the industry presentations were interesting to listen to, they generally fell into a later part of the technology curve, rather than the bleeding edge of pure research. These accounts of the virtues of semi-commoditized HPC in service of industry was a nice balance to some of the extremes of the pure research presentations. Overall, however, the range of presentations left a rather large hole -- the use of HPC (or just massive but semi-commoditized computation) by researchers outside of physics and industry.
I definitely missed the presence of research presentations from fields like the humanities and social sciences. I hope that this absence does not reflect a trend of the industry as a whole. However, when I shared this observation informally, everyone I spoke with said that even within traditionally HPC-oriented disciplines, sub-fields and talented people are being left behind by the HPC community.
This year's SC was larger than past years (9000 this year), and more inclusive from a socio-demographic perspective than past years -- more women, more minorities -- but the inclusion of projects from disciplines outside of physics/astrophysics was lower than in years past. I think the key challenge for HPC continues to be the spread of HPC to more people in more fields.
