With cancer affecting millions of lives each year, Notre Dame scientists are working to develop personalized cancer vaccine therapies with the help of computational modeling. The recent acquisition of a General Purpose Graphics Processing Unit (GPGPU) compute cluster has significantly accelerated output for Notre Dame researchers. Led by Professor Brian Baker (http://chemistry.nd.edu/people/brian-m-baker/), Associate Dean for Research and Graduate Studies in the College of Science and Professor of Chemistry and Biochemistry, an interdisciplinary team of biophysicists, biochemists and immunologists are using the GPGPU cluster to develop new immunotherapeutics. The cluster is maintained and housed by the Center for Research Computing at Union Station Technology Center, downtown South Bend.

Cory Ayres, a biochemistry graduate student who is co-advised by Professor Baker and Professor Steven Corcelli, Department of Chemistry and Biochemistry leads the computational effort. Ayres focuses his research on molecular dynamics studies of T-Cell receptors (TCRs) and major histocompatibility complex proteins bound to peptide antigens (pMHCs). He explains that cells in the human body express proteins, and eventually, after several cellular processes, these proteins are broken up into multiple fragments called peptides. These peptides are then loaded onto an MHC protein and exported to the surface of the cell. Ayres further explained, “Once on the surface of the cell, T-Cells and their T-Cell receptor are able to sample the peptide presented by the MHC protein. If the peptide is recognized as non-self, the T-Cell will initiate an immune response against that cell. This overall process is a large component of what is termed adaptive immunity.”

Ayres uses the GPGPU cluster for molecular dynamics simulation in order to observe the motions of the TCR and pMHC proteins, and to determine the role that these motions play in molecular recognition. The new cluster provides 40 NVIDIA Titan X Cards in 10 Intel based servers. The primary software used for these simulations is the AMBER Molecular Dynamics Suite. “When I first started working on the project, I was working with CPUs only and could simulate about a nanosecond a day.” Ayres said. “Now I’m up to 1.2 microseconds a day, so I am seeing results at a factor of 1,000 times greater from when I started.” The scientists collaborated closely with the ND CRC, AMBER GPU Development Lead Dr. Ross Walker and the EXXACT Corporation.

Not alone in this endeavor, Baker and Ayres also collaborate with fellow researchers at Notre Dame, as well as the lab of Professor Pramod Srivastava from the Department of Immunology, University of Connecticut Health Center.

After sequencing cancer genomes, identifying the mutant proteins that are expressed, and predicting neoepitopes, Dr. Srivastava provides lists of candidate neoepitopes to the Notre Dame team. Ayres is then performs molecular dynamics simulations in order to help ascertain whether a given neoantigen will stimulate an anti-tumor immune response. Ultimately, this will help in identifying neoepitopes that may be useful as cancer vaccine components. In the near future, this work will be a component of a human clinical trial for ovarian cancer. “This cluster is really going to accelerate the throughput of our contribution to the project,” said Ayres. The ultimate goal for researchers is to develop patient specific, personalized vaccines. With the acquisition of this new cluster, their fight against cancer has been substantially accelerated.

For the full research paper, please visit The Journal of Experimental Medicine. The cluster was funded by a grant from the Carole and Ray Neag Comprehensive Cancer Center and Center for Immunotherapy of Cancer and Infectious Disease.

The July 2015 Newsletter has been released with the latest information on activities and events.

Visit the Newsletter page for back issues.

Thanks to additional investment by the University we have been able to replace the aging Panasas (/pscratch) file system with a new system that provides 2x the performance and over 2x the capacity.

On the new filesystem, users will notice significantly faster transfers of large data sets as well as an improved response time on metadata intensive tasks such as listing a directory with many files.

The ND CRC Storage Policy has been updated to reflect the new system. In summary:

  • Individuals can now request (with justification) 1TB of scratch storage (previously 500GB)
  • Scratch storage is no longer counted toward faculty's 4TB allocation of persistent storage

The new scratch storage system is mounted at /scratch365

  • If you previously were granted a /pscratch volume, you now have a /scratch365 volume
  • 365 represents the maximum number of days a file may live in temporary scratch storage
  • Users will receive reminders at 9, 10, and 11 months to remove old data in scratch storage

IMPORTANT: The old /pscratch system will be retired in 3 months (over Labor Day weekend Sept 5-7) and re-purposed as a /scratch30 short term (30 day) unlimited quota file system. Data will NOT be automatically transferred from /pscratch to /scratch365. The file systems are not for persistent data. Users should immediately begin using /scratch365 for new work and begin deleting or moving persistent data from /pscratch into AFS as appropriate.

The CRC team will provide monthly updates and reminders throughout the process. Please contact us with any questions in this regard.

The DHARMA group, headed by Krupali Krusche, had the privilege of being invited to document the Taj Mahal in India during May 2015. The documentation process combines high accuracy 3D pointclouds, from a lidar scanner, with high resolution photographs, from a GigaPan robotic tripod to produce a model of the monument that can be studied by scholars or preservation specialists. A description of previous work at the Roman Forum, Italy, can be found here: DHARMA website

Although there are usually problems documenting monuments (the Indian Army who guard Taj Mahal were suspicious of us and our equipment), we were provided with an unbelivable level of access at Taj, they allowed us to enter the four Minaretts which had been closed to the public for 60 years and also the roof of the mausoleum. The pictures below were my phone snaps taken during the visit.

A brief history from Wikipedia: In 1631, Shah Jahan, emperor during the Mughal empire's period of greatest prosperity, was grief-stricken when his favorite of three wives and beloved companion, Mumtaz Mahal, a Persian princess, died during the birth of their 14th child, Gauhara BegumConstruction of the Taj Mahal began in 1632. The court chronicles of Shah Jahan's grief illustrate the love story traditionally held as an inspiration for Taj Mahal. The principal mausoleum was completed in 1643 and the surrounding buildings and garden were finished about five years later.


The May 2015 Newsletter has been released with the latest information on activities and events.

Visit the Newsletter page for back issues.

Part of the Engineer’s Creed is rooted in the commitment of each engineer to use technical expertise for the common good, to give back to the community in which he or she lives, and to make the world — even a small piece of it — a better place. This is exactly what a group of engineering undergraduates has been doing as part of a service-learning course, Student Engineers Reaching Out (SERO), during the 2014-15 academic year. Under the direction of a faculty adviser, Paul R. Brenner, and coordinated with Matt Willmore in Notre Dame’s Office of Information Technology, the students developed a mini-course sequence for local middle school students who participate in the Robinson Learning Center’s youth development program.

For the full story, and for more College of Engineering Undergraduate Student Spotlights, click here.

91,428; 624,576; 2,322,768. These impressive numbers represent the CPU hour usage over the last 24 hours, the last week, and so far this month… of just one CRC user. Min Zhu, Electrical Engineering Ph.D. student at Xidian University in Xian, China, has been an absolute powerhouse in harnessing compute cores in the Notre Dame Condor Pool to run simulations for her research and looks to be a mainstay at the top of the CRC Computation User List. Zhu, visiting Notre Dame for a year as part of a Chinese government run, all expenses paid program, arrived in September 2014 to work on developing high rate error-correcting codes for optical data communication that can be received at very high speeds using a simple soft decision decoding method. When transmitting data it is imperative to have a very low probability of decoding error, 1 in a trillion or lower, because data is more difficult to decipher than, for example, a wireless voice message that can tolerate a higher percentage of errors, 1 in 10-100,000. Zhu’s massive CPU hour usage comes from her Monte Carlo simulations used to represent a whole system- transmitter, decoder, receiver, and random noise-with various code and decoder parameters to test the decoding error probability.

Zhu’s mentors, Professor Daniel Costello, Bettex Chair Professor Emeritus, Electrical Engineering, and David Mitchell, Visiting Assistant Professor, Electrical Engineering, see significant promise in Zhu’s work. Zhu’s research is a variation of the braided convolutional code approach developed by one of Costello’s former doctoral students. “Min’s approach is different from the original work in it’s focus on high rates and a simplified decoding schedule which allows received information to be decoded with minimal delay. We believe her approach has distinct advantages compared to the other proposed methods,” stated Costello. “If things work out, we expect that her approach would have an impact on the way error-correction is performed for high-speed optical fiber data transmission.”

In addition to optical data transmission, Min’s research is on track to make improvements that will potentially impact digital space and satellite communication and wireless data transmission. Min has submitted a paper on her work to the IEEE Information Theory Workshop (ITW) in South Korea in October.

The University of Notre Dame has been named a GPU Research Center by NVIDIA, the world leader in visual computing, based on the vision, quality, and impact of Notre Dame’s GPU-accelerated computing research.

GPU Research Centers are institutions that embrace and utilize GPU technologies across multiple research fields, and are at the forefront of some of the world’s most innovative scientific research. GPU computing leverages the parallel processing capabilities of GPU accelerators and enabling software to deliver dramatic increases in performance for scientific, analytics, engineering, consumer, and enterprise applications.

With this designation, the University of Notre Dame has access to live online training sessions, designated NVIDIA technical support personnel, and specialized online and in-person training sessions, as well as early access to NVIDIA GPU hardware and software.

Research into the fundamentals of accelerated computing and the number of researchers taking advantage of GPU accelerator techniques and capabilities to accelerate their research are on the rise at Notre Dame. In recent months, multiple grant proposals have been submitted to acquire a GPU cluster at Notre Dame to break barriers in research ranging from genetics to economics, to molecular dynamics and robotics. Collaborations between Computer Science, Chemistry, Chemical Engineering and a multitude of other disciplines on campus would share in the benefit of a GPU cluster acquisition.

This award recognizes outstanding contributions in the areas of computational sciences and visualization. Such contributions may include, but are not limited to: 1) applications of high performance computation and/or visualization technology; 2) development of algorithms, codes, software environments or other tools for better using high performance computing and/or visualization. The nominated work need not have been done using CRC hardware or software.

The winners of the CRC Award for Computational Science and Visualization for 2015 are:

    • Christopher Paolucci, Chemical and Biomolecular Engineering
      advisor: Prof. William Schneider
    • Aparna Bhattacharya, Physics
      advisor: Prof. David Bennett
    • Matthias Wolf and Anna Woodard, Physics
      advisors: Prof. Michael Hildreth and Prof. Kevin Lannon

The awards will be presented at the Graduate School Awards Dinner on Friday, May 15.

The April 2015 Newsletter has been released with the latest information on activities and events.

Visit the Newsletter page for back issues.

The March 2015 Newsletter has been released with the latest information on activities and events.

Visit the Newsletter page for back issues.