Speed up your Data processing with OpenMOLE

Up arrow

Mathieu Leclaire // Romain Reuillon // Mark Hammons

ISCPIF // BioEmergences // FBI

geod
geod
geod

The Data processing

mozaique parameters
Large sets of raw data Large sets of parameters

These processings are time consuming

Computing power


  • Personal computer // 1 to 8 cores
  • Computing server // up to 50 cores
  • Cluster // up to 200 cores
  • Grid // > 2000 cores

European Grid Infrastructure

What does OpenMOLE do ?

It exposes a workflow formalism for describing an experiment
It transparently delegates computational loads to massively parallel environments

Upscaling

Prototype small
Experiment large

Embed your algorithm as a black box

C
R
C++
Java
Scala
Scilab
Octave
Python
Netlogo
...

Workflow topology

A Task runs an executable,
receives and produces Variables.
Variables navigate from one Task to another by means of Transitions
eventually doing Loops.

The design of experiments

mozaique parameters
Design you experiment in OpenMOLE At runtime

OpenMOLE also deals with

Data processing Optimization Sensitivity analysis Design of experiments Calibration

Computing environments

The same Task can be run localy
or by taking leverage of the local multicores
or on a Cluster (PBS for instance)
or on the Grid.
...
A naturally parallel formalism to design experiments

Image specific task



ImageJ2 integration (100% headless required)
Requires Produces
- imageJ2 macro - processed images
- set of images - other results
- set of parameters
- and optionaly imageJ2 plugins

OpenMOLE from the GUI

OpenMOLE from a DSL

val i1 = Prototype[Int]("i1")
val i2 = Prototype[Int]("i2")
val j = Prototype[Int]("j")

val hello = GroovyTask("hello", "j = Model.compute(i1, i2)")

hello addInput i1
hello addInput i2
hello addOutput j
hello addLib "/path/to/model.jar"

val exploration = ExplorationTask(
    "exploration",
    Factor(i1, 0 to 100 by 2 toDomain) x 
    Factor(i2, new UniformIntDistribution take 10)
  )

val ex = exploration -< (hello by 10 on biomed) toExecution        
ex.start

OpenMOLE from a Web server

Download: http://www.openmole.org

@OpenMOLE

The Bioemergences project

C
Image processing

portal access

daily productions
10000 jobs / day
Ralf Mikut et al, Automated Processing of Zebrafish Imaging Data: A Survey, Zebrafish, 2013

The Bioemergences project