A scalable platform for meta-genomic analysis

In support of a new consortium to enhance food safety, IBM Research has developed a software platform, the Meta-genomics Compute and Analytics Workbench (MCAW), to perform rapid and scalable informatics. The system objective is to transform large volumes of raw DNA and RNA sequence data quickly into valid conclusions—with the ultimate goal that potential threats of food-borne illness are detected before they impact public health. MCAW is being used to establish the baseline microbiome of active organisms in samples of food ingredients, to build and share metagenomic  libraries from such samples, and ultimately to detect ingredients that fall outside the normal range.

A MCAW workflow runs informatics programs such as de novo genome assembly on a dataset comprising data files (e.g., raw DNA sequence for a single sample) and laboratory metadata.  MCAW automatically parallelizes across cluster nodes, and generates aggregate statistics and visualization of data from multiple samples, to aid in trend and factor analysis with respect to time, temperature, and biochemical metadata of samples. Processing history is fully recorded so the provenance of results is clear and processing can be reproduced.

We will show MCAW results from assembling and cataloguing genomes  for 4,100 Salmonella entericaisolates sequenced in the 100K Foodborne Pathogen Genome Project[1].  The number of distinct proteins appears to grow as a power (α = 0.465) of the number of isolates, which affirms the importance of integrating a large number of samples in building a pan-genome.

[1] http://100kgenome.vetmed.ucdavis.edu/