Speed up your Data processing with OpenMOLE

Up arrow

Mathieu Leclaire // Romain Reuillon // Mark Hammons

ISCPIF // BioEmergences // FBI

geod
geod
geod

The methods

Data processing Optimization Sensitivity analysis Design of experiments Calibration

These processings are time consuming

Computing power


  • Personal computer // 1 to 8 cores
  • Computing server // up to 50 cores
  • Cluster // up to 200 cores
  • Grid // > 2000 cores

European Grid Infrastructure

What does OpenMOLE do ?

It exposes a workflow formalism for describing an experiment
It transparently delegates computational loads to massively parallel environments

Upscaling

Prototype small
Experiment large

Embed your algorithm as a black box

C
R
C++
Java
Scala
Scilab
Octave
Python
Netlogo
...

Workflow topology

A Task runs an executable,
receives and produces Variables.
Variables navigate from one Task to another by means of Transitions
eventually doing Loops.

The design of experiments

mozaique parameters
Design you experiment in OpenMOLE At runtime

Computing environments

The same Task can be run localy
or by taking leverage of the local multicores
or on a Cluster (PBS for instance)
or on the Grid.
...

OpenMOLE from the GUI

OpenMOLE from a DSL

val i1 = Prototype[Int]("i1")
val i2 = Prototype[Int]("i2")
val j = Prototype[Int]("j")

val hello = GroovyTask("hello", "j = Model.compute(i1, i2)")

hello addInput i1
hello addInput i2
hello addOutput j
hello addLib "/path/to/model.jar"

val exploration = ExplorationTask(
    "exploration",
    Factor(i1, 0 to 100 by 2 toDomain) x 
    Factor(i2, new UniformIntDistribution take 10)
  )

val ex = exploration -< (hello by 10 on biomed) toExecution        
ex.start

OpenMOLE from a Web server

Download: http://www.openmole.org

@OpenMOLE

The Bioemergences project

C
Image processing

portal access

daily productions
10000 jobs / day
Ralf Mikut et al, Automated Processing of Zebrafish Imaging Data: A Survey, Zebrafish, 2013

The Bioemergences project