SC11 Journal: News for the Supercomputing Conference
I have a full plate of meetings at Supercomputing 2011 this year. Below are some of the points of interest I am learning about along the way that I wanted to share.
Yesterday I attended the 7:30 breakfast meeting for Platform Computing’s MapReduce. This is a policy-driven workload manager and scheduler that handles mixed types of workloads running on the same cluster. It uses an open architecture to support multiple applications for jobs built with Hadoop MapReduce technology. Applications include Pig, Hive, Java, Oozie, Cumbo, and natively written Java MapReduce programs.
The scheduler can give the high-performance computing (HPC) manager options to schedule job submissions with a Fairshare Scheduler, Preemptive Scheduler, Threshold-based Scheduler or Task Scheduler. It also helps to work with resource draining.
MapReduce sends the application to a Job Controller, which decides what data should be mapped to an input folder and what mapped tasks should go to the local storage. It can split data so only that data that needs to be in the compute schedule goes to the CPU. It also uses Resource Groups to move data between local workstations.
Platform Computing’s MapReduce can allow clusters to be grouped into one large resource. Platform is used as a massive scheduler. The user has control of the job not just when it is in the queue, but while the job is running.
Yesterday afternoon, after the NVIDIA keynote, I met with Intelligent Light’s Roger Rintala. His Fieldview CFD postprocessor has incorporated data management capabilities and has been enhanced to use GPU technology and better memory access. The post-processor has automated workflow features and is accessible by being able to transport XTB files, which are smaller in file size to collaborators within the organization, suppliers or other parties needing access to the results from CFD jobs. This also includes animations and the smaller size or the postprocessor files makes both file storage and sharing easier. Viewers can be purchased in packs of 5, 10, 20, etc.
I next met with Martin Nichols, Altair’s CIO. He said that Altair is growing, with 2011 being a very good year. I asked him what he attributes this to and he said the HyperWorks breadth and depth has become recognized, and the business model is unique. The Unit Model (tokens) enables engineers and analysts to try out new software without more cost. If they find they are using more software, either Altair’s or from their partners, they just purchase more units.
“Altair’s units create access to technology,” he said. “It eliminates the barriers to adopt tools engineers need.”
We talked about the buying cycle for new software and how many layers of management design engineers have to go through to make an investment in new software tools. This can take months to years and is often not completed because the engineer cannot make the case for buying because they haven’t used it. Altair is succeeding because the engineers are already using the program and can easily justify the cost.
Using HyperWorks-on-Demand allows the software user to access the power of HPC with very little cost, Nichols said.
A new portal for HyperWorks Enterprise is scheduled to be released in January. It is already being used by a select group of HyperWorks users. It is a simple interface that allows the user to determine what and how to use the available resources. I be able to get a demo on Thursday.
I met with Silicon Graphics’ new CMO, Frans Aman. He said things are going really well after the acquisition by Rackable Systems in 2009. They have more than $100 million in cash and they are implementing the intellectual property that the original company couldn’t invest in.
“Silicon Graphics is back,” said Aman.
This rountable discussion was based on the use of HPC systems in engineering. Some points that were made:
- For the last 10 years, the HPC industry was on a free ride from Moore’s Law. This is now ending.
- Parallel computing is both good and bad. Programming is not easy. GPUs are not easy. Parallelization is not easy. In the beginning, clusters were a disruptive technology, then GPUs. Something else will appear that we have not thought of.
- Seamless storage has become more and more important in the CAE world. Moving data is difficult.
- There is a great need for the middleware software to take applications like ANSYS and seamlessly parallelize them. This may be done from a diversity of software running on lots of different hardware. Someone will make the layer between software and hardware, where parallelization happens, transparent and easy.
More to come …