CAMERA: Focussing on instrument based research

Digital instruments such as microscopes, scanners and sequencers now underpin much scientific research, and are a critical element in many modern laboratories. They provide the ability to image and analyse objects across a huge range of modalities, for example optical, electron, magnetic resonance and x-ray imaging, to name a few, including devices such as gene sequencers that use a combination of imaging techniques and chemistry. Data from such instruments is rarely useful as raw data, and often needs to be processed and analysed using powerful high performance computers. Data also needs to be stored and archived for later use by the originating research team and their collaborators. While it is possible to move data manually between instruments, storage systems and computers, it is preferable to make access as transparent as possible to simplify the job of the researchers. Furthermore, while digital instruments have been common for some years, the exponential growth in data volume and velocity is challenging for computational infrastructure, and this demands innovative and powerful solutions be developed.

This paper discussed a new infrastructure called CAMERA, which facilitates the CApture, ManagEment, stoRage and Analysis of data. While CAMERA defines a new framework, it leverages existing open source and commercial software and infrastructure extensively. At the University of Queensland specifically, CAMERA builds on and extends: the UQ Research Data Management platform (RDM) [4] which simplifies the task of requesting storage, the Metropolitan Data Caching Infrastructure (MeDiCI) [5] which simplifies the process of accessing data, and a variety of image repository stacks. CAMERA supports a complete life cycle for instrument gathered data, seamlessly rendering it on a range of instruments, cloud computers, desktops and high performance computers. Importantly, CAMERA encourages best-of- breed of data repositories and meta-data management systems without unnecessary data replication. As data grows exponentially, the latter minimises storage requirements without losing functionality.