Computing Facilities
The Center’s IT infrastructure consists of over 400 CentOS Linux workstations and 150 Windows and Macintosh desktops in offices and labs owned by individual research groups. There is a server farm with over 50 Linux servers that handles central storage, email, web, print, specialized processing and other shared services. The overall storage capacity of the center, including disks in local workstations and central storage, exceeds 4 petabytes.
In the fall of 2020, the Martinos Center installed a new HPC cluster with funding from the Massachusetts Life Science Center. It consists of:
- – four NVIDIA DGX A100 servers with eight NVIDIA A100 GPUs, two AMD EPYC 7742 64 core CPUs, 1TB of memory, and ten 100Gbe network connections
- – five EXXACT servers with ten NVIDIA RTX8000 GPUs, two Intel Xeon Gold 6226R 16 core CPUs, 1.5TB of memory, and one 100Gbe network connection
- – thirty-two DELL R440 servers with two Intel Xeon Silver 4214R 12 core CPUs, 384GB of memory, and one 25Gbe network connection
- – a 1.35PB VAST Storage appliance with 100% solid-state storage and eight 100Gbe network connections
The SLURM batch scheduler is used to manage job submission by users to the cluster. The NVIDIA DGX A100 servers run Ubuntu 18.04 while all EXXACT and Dell servers run CentOS 8.
The Center also has a decade old 92-node computing cluster still in use. Each node consists of two Quad Core Xeon E5472 3.0 GHz CPUs with 32GB of RAM, which together equal a total of 1024 compute cores available for batch jobs. Each node runs CentOS7 and batch submission uses the Torque/Maui batch scheduler. For interactive and large memory jobs, the center also has three Dell PowerEdge R910 with four 8-core Intel Xeon X7560 processors and 256GB of RAM.
The IT facilities are supported by a small IT staff comprising one full-time PhD-level manager, who directs two full-time system administrators and a part-time support technician. The Center also has three full-time programmers who support in-house-developed software for data analysis and management. Available commercial software includes AVS (Advanced Visual Systems, Waltham, MA), MATLAB (The MathWorks, Natick, MA) and MEDx (Sensor Systems, Sterling, VA) for general-purpose computation, simulation and image analysis; and XWIN-NMR (Bruker BioSpin), Origin (OriginLab Corp., Northampton, MA), Nuts (Acorn NMR, Livermore, CA) for analysis of NMR spectra and the Siemens IDEA development environment for pulse sequences and image reconstruction software (Siemens, Erlangen, Germany). A substantial level of internal software development for image and data analysis is ongoing, using LAMP, C, C++, Java, FORTRAN, Ruby, Python, Perl and TCL/TK.
Advanced Computational Image Processing and Analysis Center
The Advanced Computational Image Processing and Analysis Center (ACIPAC) is a satellite of the Martinos Center on the MIT campus, established in collaboration with the MIT Artificial Intelligence (AI) Laboratory. The closely affiliated ACIPAC has extensive resources and expertise for solving practical image processing and analysis issues relevant to biomedical imaging. This Center is an important bridge to the affiliated MIT research community, and allows MIT students a direct avenue to engage in biomedical imaging research at the Martinos Center.