Skip to content

C
clinicaltrials
Commit 2205e6c0 authored by Romain Loth's avatar Romain Loth
Browse files
Options

use clearer config variables in setup/regcomex.ini and... 

use clearer config variables in setup/regcomex.ini and setup/dockers/docker-compose.yml + make documentation more coherent
parent ffbbabf2
Hide whitespace changes
Inline Side-by-side
Showing with 218 additions and 48 deletions
README.md
View file @ 2205e6c0
......@@ -13,29 +13,67 @@ More info in `doc/` directory
-------
### Setting up the server on an nginx server
### Setting up the servers
#### Running it all via docker
Prerequisites:
- `docker`
- `docker-compose` (>= v. 1.7.0)
Steps to run:
```
cd setup/dockers
# build the components
docker build -t flask_iscpif_regcomex:latest flask_iscpif_regcomex/
docker build -t minidoors:latest minidoors/
docker create mysql
mkdir ../shared_mysql_data
# run them and link them
docker-compose up
```
-------
#### Running in dev
First we need to run the app
Minimal config:
```
# install prerequisites
> sudo apt install python3 docker jq
sudo apt install python3
sudo pip3 install -r setup/requirements.txt
source setup/regcomex.ini
```
Then to run the regcomex app in the simplest way just do:
```
python3 server_comex_registration.py
```
The form server is then accessible locally on `127.0.0.1:5000/regcomex`
-------
Or, to run the app with a real webserver (gunicorn) and a new mysql database:
```
# install more prerequisites
sudo apt install docker jq
cd $INSTALL_DIR
mkdir ../shared_mysql_data
# run the database docker
docker run --detach --name comex_db \
-v $INSTALL_DIR/data/shared_mysql_data:/var/lib/mysql \
--env="MYSQL_ROOT_PASSWORD=very-safe-pass" mysql
# get its IP into the env
export SQLDOCKERIP=$(docker inspect comex_db | jq -r '.[0].NetworkSettings.IPAddress')
export SQL_HOST=$(docker inspect comex_db | jq -r '.[0].NetworkSettings.IPAddress')
# run the app --------------------------------------------
# run the app
gunicorn -b 127.0.0.1:9090 server_comex_registration:app
# -------------------------------------------------------
```
The form server is now accessible locally on `127.0.0.1:9090/regcomex`
The form server is then accessible locally on `127.0.0.1:9090/regcomex`
**Remarks:**
- the default ROUTE_PREFIX is /regcomex, but TODO can be changed in config file
......@@ -43,26 +81,9 @@ The form server is now accessible locally on `127.0.0.1:9090/regcomex`
-------
#### Running it all via docker
Prerequisites:
- `docker`
- `docker-compose` (>= v. 1.7.0)
```
cd setup/dockers
docker build flask_ispcif_light/ -t flask_ispcif_light:latest
docker run -p 9090 --name comex_flask_test flask_ispcif_light
docker-compose up
```
-------
#### Running in prod
TODO
Secondly we ask nginx to reverse-proxy our app
TODO => we ask nginx to reverse-proxy our app
This is a minimal conf (cf [detailed doc](https://github.com/moma/regcomex/blob/master/doc/nginx_conf.md) for real-life conf)
......@@ -80,12 +101,11 @@ server {
### Setting up the doors connection
TODO config file for the doors api routes
The environment variable `DOORS_HOST` must simply be set to the doors server's hostname or IP.
-------
### Connecting the data to *communityexplorer.org*
Currently the data is collected in `data/shared_mysql_data`
- the DB name is `comex_shared`
- the table is `comex_registrations`
......
server_comex_registration.py
View file @ 2205e6c0
......@@ -27,7 +27,9 @@ from traceback import format_tb
# ============= read environ =============
MY_HOST = environ.get('HOST', '0.0.0.0')
MY_DEBUG_FLAG = environ.get('DEBUG_FLAG') == 'true'
MY_SQLDOCKERIP = environ.get('SQLDOCKERIP', '172.17.0.2')
MY_SQL_HOST = environ.get('SQL_HOST', '172.17.0.2')
MY_DOORS_HOST = environ.get('DOORS_HOST', '0.0.0.0')
# TODO add doors port if != 8989
# ============= app creation =============
app = Flask(__name__)
......@@ -71,7 +73,7 @@ ROUTE_PREFIX = "/regcomex"
@app.route(ROUTE_PREFIX+"", methods=['GET','POST'])
def one_big_form():
if request.method == 'GET':
return render_template("base_form.html")
return render_template("base_form.html", doors_host=MY_DOORS_HOST)
elif request.method == 'POST':
# ex: request.form = ImmutableMultiDict([('initials', 'R.L.'), ('email', 'romain.loth@iscpif.fr'), ('last_name', 'Loth'), ('country', 'France'), ('first_name', 'Romain'), ('my-captchaHash', '-773776109'), ('my-captcha', 'TSZVIN')])
# print("GOT ANSWERS <<========<<", request.form)
......@@ -236,7 +238,7 @@ def save_to_db(safe_recs_arr):
db_vals_str = ','.join(db_qstrvals)
# DB is actually in a docker and forwarded to localhost:3306
reg_db = connect( host=MY_SQLDOCKERIP,
reg_db = connect( host=MY_SQL_HOST,
user="root", # TODO change db ownership to a comexreg user
passwd="very-safe-pass",
db="comex_shared"
......
setup/dockers/cheatsheet.md
View file @ 2205e6c0
** Docker commands for advanced testing **
```
# check ports redirection
# docker inspect comex_flask_test | jq '.[0].NetworkSettings'
# docker exec -it comex_flask_test bash
docker inspect comex_flask_test | jq '.[0].NetworkSettings'
docker exec -it comex_flask_test bash
```
setup/dockers/docker-compose.yml
View file @ 2205e6c0
# NB this config assumes we have already built flask_ispcif_light
# NB we must have already built 'flask_iscpif_regcomex' and 'minidoors' images
# ===================== =========
version: '2'
services:
mysql:
mysql_regcomex:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=very-safe-pass
volumes:
- ../../data/shared_mysql_data:/var/lib/mysql
flask_ispcif_light:
# todo simplify path
image: flask_ispcif_light
environment:
- HOST=0.0.0.0 # will bind the server to this address
- DEBUG_FLAG=true
# in the future: external service (=> just remove doors_test & change DOORS_HOST)
doors_test:
image: minidoors
ports:
- "8989:32789"
# POSSIBLE volume for h2 DB itself
flask_ispcif_regcomex:
image: flask_iscpif_regcomex
ports:
- "9090:32790"
# volumes:
# - .:/code
depends_on:
- mysql
# - doors
- mysql_regcomex
- doors_test
links:
- mysql_regcomex:comex_db_backend
- doors_test:doors_backend
environment:
- HOST=0.0.0.0 # script will bind server to this address
- DEBUG_FLAG=false
- SQL_HOST=comex_db_backend # script will read/write sql to this host
- DOORS_HOST=doors_backend # script will log/register users to this host
# doors:
# image: doors
#TODO add php legacy communityexplorer site
setup/dockers/2-create_flask_container/flask_ispcif_light/Dockerfile → setup/dockers/flask_iscpif_regcomex/Dockerfile
View file @ 2205e6c0
FROM ubuntu:xenial
# Regcomex minimal dockerfile
# ----------------------------
# based on https://github.com/egeland/docker-gunicorn-python3-flask by F Egeland
# based on https://github.com/egeland/docker-gunicorn-python3-flask/blob/master/Dockerfile
# MAINTAINER Frode Egeland <egeland@gmail.com>
FROM ubuntu:xenial
# Base
ENV HOST 0.0.0.0
......@@ -13,15 +13,6 @@ ENV DEBIAN_FRONTEND noninteractive
RUN locale-gen fr_FR fr_FR.UTF-8
RUN dpkg-reconfigure locales
# fine-tuning locales (src: gargantext)
# RUN sed -i -e 's/# en_GB.UTF-8 UTF-8/en_GB.UTF-8 UTF-8/' /etc/locale.gen && \
# sed -i -e 's/# fr_FR.UTF-8 UTF-8/fr_FR.UTF-8 UTF-8/' /etc/locale.gen && \
# dpkg-reconfigure --frontend=noninteractive locales && \
# echo 'LANG="fr_FR.UTF-8"' > /etc/default/locale
# ENV LANG fr_FR.UTF-8
# ENV LANGUAGE fr_FR.UTF-8
# ENV LC_ALL fr_FR.UTF-8
RUN apt-get update && apt-get autoclean
RUN apt-get install -y python3-pip git nano
......@@ -29,6 +20,7 @@ RUN apt-get install -y python3-pip git nano
RUN pip3 install --upgrade pip
RUN pip3 install gunicorn flask
# for regcomex itself ----------------------------------------------------------
# pull the comex server from repository
RUN git clone https://github.com/moma/regcomex.git
......@@ -36,15 +28,6 @@ RUN git clone https://github.com/moma/regcomex.git
WORKDIR /regcomex
CMD bash run.sh
# if REALLY needed virtualenv (TODO avoid it)
# RUN apt install python3-virtualenv
# RUN virtualenv --python=/usr/bin/python3 setup/regcomex_venv
# RUN setup/regcomex_venv/bin/activate && pip3 install -r setup/requirements.txt
# Volumes TODO
# VOLUME ["/root/app/logs"]
# Ports
EXPOSE 5000
EXPOSE 9090
setup/dockers/minidoors/Dockerfile 0 → 100644
View file @ 2205e6c0
# Doors minimal dockerfile
# -------------------------
# based on scala and sbt Dockerfile (https://github.com/hseeberger/scala-sbt)
# Pull base image
FROM java:8
ENV SCALA_VERSION 2.11.8
ENV SBT_VERSION 0.13.12
ENV DOORS_COMMIT ef7c5e0
# Install Scala
## Piping curl directly in tar
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /root/ && \
echo >> /root/.bashrc && \
echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' >> /root/.bashrc
# Install sbt
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
WORKDIR /root
# Build Doors ------------------------------------------------------------------
RUN git clone https://github.com/ISCPIF/doors.git
WORKDIR /root/doors
RUN git checkout $DOORS_COMMIT
WORKDIR /root/doors/application
RUN sbt "project lab"
CMD sbt "project lab" run
# ports
EXPOSE 8989
setup/dockers/1-create_sql_container.md → setup/dockers/mysql_prerequisites.md
View file @ 2205e6c0
# to run docker client without sudo
sudo usermod -aG docker
# then full reboot
# to make a docker with the data
# src: hub.docker.com/r/mysql/mysql-server/
#### To make a docker with the mysql instead of using your local mysql server
*source:* hub.docker.com/r/mysql/mysql-server/
```
mkdir ../shared_mysql_data
docker create mysql
docker run --detach --name comex_db \
-v /home/romain/comex/regcomex/data/shared_mysql_data:/var/lib/mysql \
--env="MYSQL_ROOT_PASSWORD=very-safe-pass" mysql
--env="MYSQL_ROOT_PASSWORD=very-safe-pass"\
mysql
# get the ip
export SQLDOCKERIP=$(docker inspect comex_db | jq -r '.[0].NetworkSettings.IPAddress')
# connect --------------------------------------------
mysql -uroot -pvery-safe-pass -h $SQLDOCKERIP -P 3306
# -----------------------------------------------------
# stop it when you're done
docker stop comex_db
export SQL_HOST=$(docker inspect comex_db | jq -r '.[0].NetworkSettings.IPAddress')
```
#### To use any other mysql server
```
export SQL_HOST=localhost # or any other hostname/IP
```
# NB
# also now mysqld --initilize created the following files
~/comex/shared_mysql_data > ll
total 185M
-rw-r----- 1 999 56 2016-11-23 17:49 auto.cnf
-rw-r----- 1 999 1,3K 2016-11-23 17:49 ib_buffer_pool
-rw-r----- 1 999 76M 2016-11-23 17:49 ibdata1
-rw-r----- 1 999 48M 2016-11-23 17:49 ib_logfile0
-rw-r----- 1 999 48M 2016-11-23 17:49 ib_logfile1
-rw-r----- 1 999 12M 2016-11-23 17:49 ibtmp1
drwxr-x--- 2 999 4,0K 2016-11-23 17:49 mysql/
drwxr-x--- 2 999 4,0K 2016-11-23 17:49 performance_schema/
drwxr-x--- 2 999 12K 2016-11-23 17:49 sys/
### Set up the tables
```
# connect to your sql host or container
mysql -uroot -pvery-safe-pass -h $SQL_HOST -P 3306
# --- after connection to mysql
CREATE DATABASE comex_shared ;
USE comex_shared ;
create table comex_registrations (
CREATE TABLE comex_registrations (
doors_uid char(36) not null unique,
-- ISO stamp like 2016-11-16T17:47:07.308Z
last_modified_date char(24) not null,
email varchar(255) not null unique primary key,
initials varchar(7) not null,
......@@ -62,5 +44,6 @@ create table comex_registrations (
interests_text varchar(1200),
community_hashtags varchar(350),
gender char(1),
pic_file blob
pic_file mediumblob
) ;
```
setup/regcomex.ini 0 → 100644
View file @ 2205e6c0
# REGCOMEX INI MINIMAL CONFIG
# ===========================
HOST=localhost
SQL_HOST=localhost
DOORS_HOST=hello_todo
DEBUG_FLAG=false
static/js/comex_reg_form_controllers.js
View file @ 2205e6c0
......@@ -41,10 +41,14 @@ var COLS = [ ["doors_uid", false, 36, 'exact'],
// NB other vars defined in main scope but just before their respective funs
var wholeFormData
var theForm = document.getElementById('comex_reg_form')
var regTimestamp = document.getElementById('last_modified_date')
var uidInput = document.getElementById('doors_uid')
var email = document.getElementById('email')
var doorsHost = document.getElementById('doors_host').value
// captchaHash should be appended by itself if normal submit,
// but we may need to do it ourselves (TODO test)
var captcha = document.getElementById('my-captcha')
......@@ -209,7 +213,7 @@ function callDoors(data, callback, apiAction) {
$.ajax({
contentType: "application/json",
dataType: 'json',
url: "http://localhost:8989/api/" + apiAction,
url: "http://"+doorsHost+":8989/api/" + apiAction,
data: JSON.stringify({
"login": mailStr,
"password": passStr,
......
templates/base_form.html
View file @ 2205e6c0
......@@ -350,6 +350,9 @@
</ul>
</nav>
<!-- hidden input for doors hostname -->
<input id="doors_host" name="doors_host" type="text" hidden>{{doors_host|safe}}</input>
<!-- hidden input for modification date -->
<input id="last_modified_date" name="last_modified_date" type="text" hidden>
</input>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment