Graphical User Web Based Interface for Batch Processing of Images on a Linux based GPU High Performance Cluster

The latest generation of sCMOS camera based microscopes such as the lattice lightsheet and Andor Dragonfly spinning disc confocal have placed growing demands on researchers to process, analyse and store huge datasets an order of magnitude greater than what was considered normal only a few years ago. Unfortunately the hardware and software systems which have been built to handle such large data (namely Linux based High performance computing, HPC), are typically managed by IT specialists, and are not considered layman friendly. Our goal, was to produce an intuitive Image Processing Portal that our core facility users, with no HPC experience could use with minimal support or training. We present a web portal that is capable of performing large scale, batch processing of microscopy images within a GPU based HPC. This portal provides intuitive web pages allowing users to login remotely from anywhere in the world to submit image processing jobs.

Utilising the portal to submit jobs, image deconvolution is performed using the Microvolution deconvolution engine[1] on our Wiener GPU cluster[2] via multiple Dell r740 nodes each containing two NVIDIA V100 GPUs.

Other utilities being developed for the Image Processing Portal include: A file format conversion page, which allows users to easily batch convert their datasets between image formats. This was originally developed to deal with large Andor Dragonfly “.ims” files which could not be opened in ImageJ/FIJI.

Another feature being developed, which will utilise the MeDiCI (Metropolitan Data Caching Infrastructure) provided by the Research Computing Centre[3], is a GUI interface which will simplify the process of data transfer between the acquisition PC to the HPC via a high-speed Linux-based GPFS parallel file system.

The portal can be modified to add additional tools and run on other hardware configurations.

The portal is currently available to UQ users at: https://imbmicroscopy.rcc.uq.edu.au.

The source code is available via Github at https://github.com/UQ-RCC/IMBPortal