Skip to content

gargantext
gargantext
Commit 360d776a authored by c24b's avatar c24b
Browse files
Options

Merge branch 'unstable' of ssh://delanoe.org:1979/gargantext into unstable 

MERGE
parents d673a1ec 94d3f771
Hide whitespace changes
Inline Side-by-side
Showing with 216 additions and 797 deletions
dbmigrate.py 100644 → 100755
View file @ 360d776a
File mode changed from 100644 to 100755
install/01-setup 0 → 100755
View file @ 360d776a
```
#!bin/bash
#name:01-setup
echo "****************SETUP**************************";
for dir in "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
sudo wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo ":::::::::::::::::Done::::::::::::::::::::::::::";
#TODO clone the repo into /srv/gargantext/ and reduce the different steps
#git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
# && cd /srv/gargantext \
# && git fetch origin stable \
# && git checkout stable \
```
install/02-config 0 → 100755
View file @ 360d776a
#!/bin/bash
#configure the base image gargamelle
echo '****************BUILD**********************************'
docker build -t gargamelle:latest ./gargamelle
#2 option with this image:
# configure the container
# run the image with the app in it
echo '::::::::::::::::::::GARGAMELLE IMAGE BUILT:::::::::::::'
echo '*************CONFIG************************************'
sudo docker run \
-v /srv/:/srv/ \
-p 8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "/srv/gargantext/install/gargamelle/psql_configure.sh"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
sudo docker run \
-v /srv/:/srv/ \
-p 8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "/srv/gargantext/install/gargamelle/django_configure.sh"
sudo docker rm -f `docker ps -a | grep -v CONTAINER awk '{print $1 }'`
install/03-run 0 → 100755
View file @ 360d776a
#!/bin/bash
sudo docker run \
-v /srv/:/srv/\
-p 8000 \
-p 5432 \
-it gargamelle:latest \
# /bin/bash -c "service postgresql start; su gargantua -c \'source /env_3-5/bin/activate && /srv/gargantext/manage.py runserver 0.0.0.0:8000\'"
install/MIGRATION.md deleted 100644 → 0
View file @ d673a1ec
# Migration from Gargantext < 3.0.0 versions towards >= 3.*
## Installation
First, install Python 3.5 (see https://www.python.org/downloads/ for
download links).
```bash
cd /tmp
wget https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tar.xz
tar xvfJ Python-3.5.1.tar.xz
cd Python-3.5.1
./configure
make -j4 # option is for multithreading
sudo make install
```
Other components are required:
```bash
sudo pip3.5 install virtualenv
sudo apt-get install rabbitmq-server
```
Then build a virtual environment:
```bash
virtualenv-3.5 VENV
source VENV/bin/activate
pip3.5 install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1
pip3.5 install -U -r requirements.txt
```
## Migrate database
### Django models
```bash
./manage.py makemigrations
./manage.py migrate --fake-initial
```
...or if it fails, try the commandes below:
```bash
./manage.py makemigrations
./manage.py migrate --run-syncdb
```
(see [Django documentation](https://docs.djangoproject.com/en/1.9/topics/migrations/))
### SQLAlchemy models
```bash
./dbmigrate.py
```
## Start the Django server
```bash
./manage.py celeryd --loglevel=INFO # to ensure Celery is properly started
./manage.py runserver
```
install/README.md
View file @ 360d776a
#Install Instructions for Gargantext (CNRS):
## Get the source code
by cloning gargantext into /srv/gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin stable \
&& git checkout stable \
```
The folder will be /srv/gargantext:
* docs containes all informations on gargantext
/srv/gargantext/docs/
* install contains all the installation files
/srv/gargantext/install/
Help needed ?
See [http://gargantext.org/about](http://gargantext.org/about) and [tools](./contribution_guide.md) for the community
Two installation procedure are provided:
1. Semi-automatic installation [EASY]
2. Step by step installation [ADVANCED]
Here only semi-automatic installation is covered checkout [manual_install](manual_install.md)
to follow step by step procedure
##Prerequisites
## Init Setup
## Install
## Run
--------------------
# Semi automatic installation
All the procedure files are located into /srv/garantext/install/
``` bash
user@computer:$ cd /srv/garantext/install/
```
## Prerequisites
* A Debian based OS >= [FIXME]
* At least 35GO in /srv/ [FIXME]
todo: reduce the size of gargantext lib
todo: remove lib once docker is configured
! tip: if you have enought space for the full package you can:
* resize your partition
* make a simlink on gargantext_lib
##Init Setup
Prepare your environnement and make the initial setup.
This initial step creates a user for gargantext plateform along with dowloading additionnal libs and files.
It also install docker and build the docker image and build the gargantext box
``` bash
user@computer:/srv/garantext/install/$ .init.sh
```
### Install
Once the init step is done
* Enter into the docker environnement
Inside folder /srv/garantext/install/
enter the gargantext image
``` bash
user@computer:/srv/garantext/install/$ .docker/enterGargantextImage
```
go to the installation folder
``` bash
root@dockerimage8989809:$ cd /srv/gargantext/install/
```
[ICI] Tester si les config de postgresql et python sont faits en amont à la création du docker file
* Install Python environment
``` bash
root@dockerimage8989809:/srv/garantext/install/$ python/configure
```
* Configure PostgreSql
Inside the docker image, execute as root:
``` bash
root@computer:/srv/garantext/install/$ postgres/configure
```
[Si OK ] enlever ses lignes
Install Gargantext server
* Configure the database
Inside the docker container:
``` bash
service postgresql start
#su gargantua
#activate the virtualenv
source /srv/env_3-5/bin/activate
```
You have entered the virtualenv as shown with (env_3-5)
``` bash
(env_3-5) $ python /srv/gargantext/dbmigrate.py
(env_3-5) $ /srv/gargantext/manage.py makemigrations
(env_3-5) $ /srv/gargantext/manage.py migrate
(env_3-5) $ python /srv/gargantext/dbmigrate.py
#will create tables and not hyperdata_nodes
(env_3-5) $ python /srv/gargantext/dbmigrate.py
#will create table hyperdata_nodes
#launch first time the server to create first user
(env_3-5) $ /srv/gargantext/manage.py runserver 0.0.0.0:8000
(env_3-5) $ /srv/gargantext/init_accounts.py /srv/gargantext/install/init/account.csv
```
FIXME: dbmigrate need to launched several times since tables are
ordered with alphabetical order (and not dependencies order)
* Exit the docker
```
exit (or Ctrl+D)
```
## Run Gargantext
Enter the docker container:
``` bash
/srv/gargantext/install/docker/enterGargantextImage
```
Inside the docker container:
``` bash
#start Database (postgresql)
service postgresql start
#change to user
su gargantua
#activate the virtualenv
source /srv/env_3-5/bin/activate
#go to gargantext srv
(env_3-5) $ cd /srv/gargantext/
#run the server
(env_3-5) $ /manage.py runserver 0.0.0.0:8000
```
Keep it open and outside the docker launch browser
``` bash
chromium http://127.0.0.1:8000/
```
* Click on Test Gargantext
```
Login : gargantua
Password : autnagrag
```
Enjoy :)
See [User Guide](/demo/tuto.md) for quick usage example
install/debian/README_Debian.md deleted 100644 → 0
View file @ d673a1ec
#!/bin/dash
# TODO do apt-get install --force-yes --force-yes
#postgresql3.4-server-dev
#+libxml2-dev
sudo apt-get install --force-yes postgresql
sudo apt-get install --force-yes postgresql-contrib
sudo apt-get install --force-yes rabbitmq-server
sudo apt-get install --force-yes tmux
sudo apt-get install --force-yes uwsgi uwsgi-plugin-python3
#apt-get install --force-yes python-virtualenv
sudo apt-get install --force-yes libpng12-dev
sudo apt-get install --force-yes libpng-dev
sudo apt-get install --force-yes libfreetype6-dev
sudo apt-get install --force-yes python-dev
sudo apt-get install --force-yes libpq-dev
sudo apt-get install --force-yes libpq-dev
#apt-get build-dep python-matplotlib
#apt-get install --force-yes python-matplotlib
#Paquets Debian a installer
# easy_install --force-yes -U distribute (matplotlib)
#lxml
sudo apt-get install --force-yes libffi-dev
sudo apt-get install --force-yes libxml2-dev
sudo apt-get install --force-yes libxslt1-dev
# ipython readline
sudo apt-get install --force-yes libncurses5-dev
sudo apt-get install --force-yes pandoc
# scipy:
sudo apt-get install --force-yes gfortran
sudo apt-get install --force-yes libopenblas-dev
sudo apt-get install --force-yes liblapack-dev
#nlpserver
sudo apt-get install --force-yes libgflags-dev
sudo apt-get install --force-yes libgoogle-glog-dev
# MElt
# soon
## SERVER Configuration
# server configuration
sudo apt-get install --force-yes nginx
# UWSGI with pcre support
sudo apt-get install --force-yes libpcre3 libpcre3-dev
sudo apt-get install --force-yes python3-pip
#pip3 install --force-yes uwsgi
install/debian/asGargantua.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
##MAINTAINER ISCPIF <alexandre.delanoe@iscpif.fr>
#
#git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
# && cd /srv/gargantext \
# && git fetch origin refactoring-alex \
# && git checkout refactoring-alex
#
#cd /srv/gargantext/install \
# && /usr/bin/virtualenv --py=/usr/bin/python3.5 /srv/env_3-5 \
# && /bin/bash -c 'source /srv/env_3-5/bin/activate' \
# && /bin/bash -c '/srv/env_3-5/bin/pip install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1' \
# && /bin/bash -c '/srv/env_3-5/bin/pip install -r /srv/gargantext/install/python/requirements.txt' \
#
## INSTALL MAIN DEPENDENCIES
cd /tmp && wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& chown -R gargantua:gargantua /srv/gargantext_lib
## End of configuration
## be sure that postgres is running
#cd /srv/gargantext && /bin/bash -c 'source /srv/bin/env_3-5/bin/activate' \
# && /srv/gargantext/manage.py shell < /srv/gargantext/init.py
#
echo "Gargantua: END of the installation of Gargantext"
install/debian/asPostgres.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
# ## CONFIGURE POSTGRESQL
psql -c "CREATE user gargantua WITH PASSWORD 'C8kdcUrAQy66U'" && createdb -O gargantua gargandb
install/debian/asRoot.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
#MAINTAINER ISCPIF <alexandre.delanoe@iscpif.fr>
apt-get update && \
apt-get install -y \
apt-utils ca-certificates locales \
sudo aptitude gcc g++ wget git postgresql-9.5 vim
### Configure timezone and locale
echo "Europe/Paris" > /etc/timezone && \
dpkg-reconfigure -f noninteractive tzdata && \
sed -i -e 's/# en_GB.UTF-8 UTF-8/en_GB.UTF-8 UTF-8/' /etc/locale.gen && \
sed -i -e 's/# fr_FR.UTF-8 UTF-8/fr_FR.UTF-8 UTF-8/' /etc/locale.gen && \
echo 'LANG="fr_FR.UTF-8"' > /etc/default/locale && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=fr_FR.UTF-8
## PROD VERSION OF GARGANTEXT
# apt-get install -y uwsgi nginx uwsgi-plugin-python rabbitmq-server
### CREATE USER and adding it to sudo
## USER gargantua cannot not connect with password but SSH key
adduser --disabled-password --gecos "" gargantua \
&& adduser gargantua sudo \
&& echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
# addgroup gargantext here with specific users
## Install Database, main dependencies and Python
## (installing some Debian version before pip to get dependencies)
apt-get update && apt-get install -y \
postgresql-server-dev-9.5 libpq-dev libxml2 \
libxml2-dev xml-core libgfortran-5-dev \
virtualenv python3-virtualenv \
python3.4 python3.4-dev \
python3.5 python3.5-dev \
python3-six python3-numpy python3-setuptools \ # for numpy, pandas
python3-numexpr \ # for numpy performance
libxml2-dev libxslt-dev # for lxml
#if [[ -e "/srv/gargantext" ]]
#rm -rf /srv/gargantext /srv/env_3-5
for dir in "/srv/gargantext"\
"/srv/gargantext_lib"\
"/srv/gargantext_static"\
"/srv/gargantext_media"\
"/srv/env_3-5"\
"/var/www/gargantext"; do \
mkdir $dir
chown gargantua:gargantua $dir
done
echo "Root: END of the installation of Gargantext by Root."
install/docker/config/build deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
#######################################################################
# ____ _
# | _ \ ___ ___| | _____ _ __
# | | | |/ _ \ / __| |/ / _ \ '__|
# | |_| | (_) | (__| < __/ |
# |____/ \___/ \___|_|\_\___|_|
#
######################################################################
sudo docker build -t gargantext .
# OR Get the ID of your container
#ID=$(docker build .) && docker run -i -t $ID
# OR
# cd /tmp
# wget http://dl.gargantext.org/gargantext_docker_image.tar \
# && sudo docker import - gargantext:latest < gargantext_docker_image.tar
install/docker/config/database/database.yml deleted 100644 → 0
View file @ d673a1ec
default: &default
adapter: postgresql
encoding: unicode
pool: 5
host: postgres
port: 5432
username: gargantex
password: gargantex
development:
database: gargantex_dev
test:
database: gargantext_test
install/docker/config/docker-compose.yml deleted 100644 → 0
View file @ d673a1ec
postgres:
image: "postgres:9.5"
volumes_from:
- data
expose:
- 5432
environment:
POSTGRES_PASSWORD: gargantext
POSTGRES_USER: gargantua
web:
command: python manage.py runserver 0.0.0.0:8000
build: .
volumes:
- .:/srv/
ports:
- "8000:8000"
depends_on:
- db
build: .
ports:
- "3000:3000"
links:
- postgres
volumes:
- ../:/srv/
volumes_from:
- data
environment:
HOST: 0.0.0.0
PORT: 3000
data:
image: gargantext
command: echo 'Data Container for PostgreSQL and Initial Data'
volumes:
- /var/lib/postgresql/data
- /bundler
install/docker/config/export.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
sudo docker export $(sudo docker ps -l | awk '{print $1}' | grep -v CONTAINER | head -n 1) > /tmp/gargantext_docker_image.tar
# To import the docker
#sudo docker import - gargantext:latest < data.tar
#sudo cat data.tar | docker import - gargantext
install/docker/docker-compose.yml deleted 100644 → 0
View file @ d673a1ec
postgres:
image: "postgres:9.4"
volumes_from:
- data
expose:
- 5432
environment:
POSTGRES_PASSWORD: gargantext
POSTGRES_USER: gargantua
web:
build: .
ports:
- "3000:3000"
links:
- postgres
volumes:
- ../:/app/
volumes_from:
- data
environment:
HOST: 0.0.0.0
PORT: 3000
data:
image: cogniteev/echo
command: echo 'Data Container for PostgreSQL and Bundler'
volumes:
- /var/lib/postgresql/data
- /bundler
install/docker/enterGargantextImage deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
sudo docker run -i -p 8000:8000 \
-v /srv:/srv \
-t gargantext:latest \
/bin/bash
install/docker/install.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
# Main user of Gargantext is Gargantua (role of Pantagruel soon)!
#sudo adduser --disabled-password --gecos "" gargantua
#######################################################################
# ____ _
# | _ \ ___ ___| | _____ _ __
# | | | |/ _ \ / __| |/ / _ \ '__|
# | |_| | (_) | (__| < __/ |
# |____/ \___/ \___|_|\_\___|_|
#
######################################################################
sudo docker build -t gargantext .
# OR
# cd /tmp
# wget http://dl.gargantext.org/gargantext_docker_image.tar \
# && sudo docker import - gargantext:latest < gargantext_docker_image.tar
function do_cker {
#sudo docker run -d -p 8000:8000 \
sudo docker run -d \
-v /srv2:/srv \
-v /home/alexandre:/home/alexandre \
-t gargantext:latest \
/bin/bash $1
}
#######################################################################
# _____ _ _
# | ___|__ | | __| | ___ _ __ ___
# | |_ / _ \| |/ _` |/ _ \ '__/ __|
# | _| (_) | | (_| | __/ | \__ \
# |_| \___/|_|\__,_|\___|_| |___/
#
#######################################################################
### Create directories in /srv
# Linux only
function create_folders {
for dir in "/srv/gargantext"\
"/srv/gargantext_lib"\
"/srv/gargantext_static"\
"/srv/gargantext_media"\
"/srv/gargantext_data"\
"/srv/env_3-5"; do \
sudo mkdir -p $dir ;\
sudo chown gargantua:gargantua $dir ; \
done;\
sudo chown -R postgres:postgres /srv/gargantext_data/
}
#do_cker "create_folders"
#NOPE
function git_config {
### TODO (soon) : git clone https://gogs.iscpif.fr/gargantext.git
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring
}
#su gargantua -c git_config #NOPE
#######################################################################
## ____ _
## | _ \ ___ ___| |_ __ _ _ __ ___ ___
## | |_) / _ \/ __| __/ _` | '__/ _ \/ __|
## | __/ (_) \__ \ || (_| | | | __/\__ \
## |_| \___/|___/\__\__, |_| \___||___/
## |___/
#######################################################################
#NOPE
function postgres_config {
/usr/lib/postgresql/9.5/bin/initdb -D /srv/gargantext_data/
}
#do_cker "su postgres -c postgres_config"
function postgres_create_db {
sudo /etc/init.d/postgresql start \
&& psql -c "CREATE user gargantua WITH PASSWORD 'C8kdcUrAQy66U'" \
&& createdb -O gargantua gargandb \
&& echo "Root: END of the installation of Gargantexts Database by postgres."
}
#do_cker postgres_create_db
#######################################################################
## _ _ _ _ _
## | | (_) |__ _ __ __ _(_)_ __(_) ___ ___
## | | | | '_ \| '__/ _` | | '__| |/ _ \/ __|
## | |___| | |_) | | | (_| | | | | | __/\__ \
## |_____|_|_.__/|_| \__,_|_|_| |_|\___||___/
##
#######################################################################
#
#######################################################################
### INSTALL MAIN DEPENDENCIES
#######################################################################
###
#### Installing pip version of python libs
#
function install_python_env {
/usr/bin/virtualenv --py=/usr/bin/python3.5 /srv/env_3-52 \
&& /bin/bash -c 'source /srv/env_3-52/bin/activate' \
&& /bin/bash -c '/srv/env_3-52/bin/pip install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1' \
&& /bin/bash -c '/srv/env_3-52/bin/pip install -r /srv/gargantext/install/python/requirements.txt'
}
#do_cker "su gargantua -c install_python_env"
#######################################################################
function init_gargantext {
echo "TODO script pour peupler la base"
}
#do_cker "su gargantua -c init_gargantext"
#######################################################################
### GET CONFIG FILES
function get_libs {
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed"
}
#do_cker get_libs
install/docker/config/Dockerfile → install/gargamelle/Dockerfile 100644 → 100755
View file @ 360d776a
###########################################################
# ____ ____ _____ __ ___ #
# / ___| __ _ _ __ / ___| __ _ _ _|_ _|__\ \/ / |_ #
# | | _ / _` | '__| | _ / _` | '_ \| |/ _ \\ /| __| #
# | |_| | (_| | | | |_| | (_| | | | | | __// \| |_ #
# \____|\__,_|_| \____|\__,_|_| |_|_|\___/_/\_\\__| #
# #
# Gargamelle WEB
###########################################################
######################################################################
#Build an image starting with debian:stretch image
# wich contains all the source code of the app
FROM debian:stretch
MAINTAINER ISCPIF <alexandre.delanoe@iscpif.fr>
######################################################################
#Add the current image into /srv/
ADD . /srv/
#Set the working directory to /srv
WORKDIR /srv/
#Install the debian dependencies
#as root
MAINTAINER ISCPIF <gargantext@iscpif.fr>
# Configure global ENV with deb dependencies
# Configure local ENV requirements
########################################################################
ENV DEBIAN_FRONTEND noninteractive
USER root
#declare 2 environnement
ENV GG_ROOT /srv/gargantext
ENV PYTHON_ENV /srv/env_3-5
### Update and install base dependencies
RUN apt-get update && \
apt-get install -y \
apt-utils ca-certificates locales \
python3-dev \
sudo aptitude gcc g++ wget git postgresql-9.5 vim \
build-essential make
sudo aptitude gcc g++ wget git vim \
build-essential make \
postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 \
postgresql-server-dev-9.5 libpq-dev libxml2 \
postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
RUN echo "############ DEBIAN LIBS ###############"
### Configure timezone and locale
RUN echo "Europe/Paris" > /etc/timezone && \
dpkg-reconfigure -f noninteractive tzdata && \
......@@ -39,43 +34,54 @@ RUN echo "Europe/Paris" > /etc/timezone && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=fr_FR.UTF-8
### Install Database, main dependencies and Python
### (installing some Debian version before pip to get dependencies)
RUN echo "########### LOCALES & TZ #################"
### Install main dependencies and python packages based on Debian distrib
RUN apt-get update && apt-get install -y \
postgresql-server-dev-9.5 libpq-dev libxml2 \
libxml2-dev xml-core libgfortran-5-dev \
virtualenv python3-virtualenv \
libpq-dev \
python3.5 \
python3-dev \
python3-six python3-numpy python3-setuptools \
# ^for numpy, pandas
# ^for numpy, pandas and numpyperf
python3-numexpr \
# ^ for numpy performance
#python dependencies
python3-pip \
# for lxml
libxml2-dev libxslt-dev
# ^ for lxml
#libxslt1-dev zlib1g-dev
RUN echo "############# PYTHON DEPENDENCIES ###############"
#UPDATE AND CLEAN
RUN apt-get update && apt-get autoclean &&\
rm -rf /var/lib/apt/lists/*
#NB: removing /var/lib will avoid to significantly fill up your /var/ folder on your native system
########################################################################
### PYTHON ENVIRONNEMENT (as ROOT)
########################################################################
RUN apt-get install -qy python3.5
RUN apt-get install -qy python3-pip
RUN python3-pip install -r /srv/gargantex/install/python/requirements.txt
RUN adduser --disabled-password --gecos "" gargantua
### PROD VERSION OF GARGANTEXT ONLY
#RUN apt-get install -y uwsgi nginx uwsgi-plugin-python rabbitmq-server
RUN pip3 install virtualenv
RUN virtualenv /env_3-5
RUN echo 'alias venv="source /env_3-5/bin/activate"' >> ~/.bashrc
# CONFIG FILES
ADD requirements.txt /
ADD psql_configure.sh /
ADD django_configure.sh /
## CREATE USER and adding it to sudo >> docker-compose build
## TODO ask user for password
#RUN adduser --disabled-password --gecos "" gargantua
RUN . /env_3-5/bin/activate; pip3 install -r requirements.txt; \
pip3 install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1; \
python3 -m nltk.downloader averaged_perceptron_tagger;
#RUN apt-get install -y sudo && adduser gargantua sudo \
# && echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
RUN chown gargantua:gargantua -R /env_3-5
########################################################################
### POSTGRESQL DATA (as ROOT)
########################################################################
#######################################################################
### CONFIGURE POSTGRESQL
#######################################################################
#in docker database
#RUN sed -iP 's%^data_directory.*%data_directory = '\/srv\/gargantext_data'%' /etc/postgresql/9.5/main/postgresql.conf
######################################################################
RUN sed -iP "s%^data_directory.*%data_directory = \'\/srv\/gargandata\'%" /etc/postgresql/9.5/main/postgresql.conf
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.5/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.5/main/postgresql.conf
#INSTALLATION AND CONFIG python and postgresql module
COPY . /install
CMD [./install/python/configure, ./install/postgres/configure]
EXPOSE 5432 8000
VOLUME ["/srv/",]
install/gargamelle/build.sh 0 → 100755
View file @ 360d776a
#!/bin/bash
#configure the base image gargamelle
docker build -t gargamelle:latest ./gargamelle
#2 option with this image:
# configure the container
# run the image with the app in it
install/gargamelle/configure.sh 0 → 100755
View file @ 360d776a
#!/bin/bash
PATH="/srv/gargantext/install/gargamelle"
#./folders_configure.sh;
#./psql_configure.sh;
/bin/bash "$PATH/django_configure.sh";
/bin/bash -c "Configuration Ok""
install/gargamelle/django_configure.sh 0 → 100755
View file @ 360d776a
#!/bin/bash
##################################################
# __| |(_) __ _ _ __ __ _ ___
# / _` || |/ _` | '_ \ / _` |/ _ \
# | (_| || | (_| | | | | (_| | (_) |
# \__,_|/ |\__,_|_| |_|\__, |\___/
# |__/ |___/
##################################################
#configure django migrations
##################################################
echo "Starting Postgres"
/usr/sbin/service postgresql start
/bin/su gargantua -c 'source /env_3-5/bin/activate \
&& ./srv/gargantext/manage.py makemigrations \
&& ./srv/gargantext/manage.py migrate \
&& ./srv/gargantext/dbmigrate.py \
&& ./srv/gargantext/dbmigrate.py \
&& ./srv/gargantext/dbmigrate.py;'
/usr/sbin/service postgresql stop
install/gargamelle/folders_configure.sh 0 → 100755
View file @ 360d776a
#!/bin/bash
chown -R gargantua:gargantua /srv/gargantext
install/postgres/configure → install/gargamelle/psql_configure.sh
View file @ 360d776a
#!/bin/bash
#######################################################################
## ____ _
## | _ \ ___ ___| |_ __ _ _ __ ___ ___
## ____ _
## | _ \ ___ ___| |_ __ _ _ __ ___ ___
## | |_) / _ \/ __| __/ _` | '__/ _ \/ __|
## | __/ (_) \__ \ || (_| | | | __/\__ \
## |_| \___/|___/\__\__, |_| \___||___/
## |___/
## |___/
#######################################################################
su postgres -c 'pg_dropcluster 9.4 main --stop'
#rm -rf /srv/gargantext_data && mkdir /srv/gargantext_data && chown postgres:postgres /srv/gargantext_data
su postgres -c '/usr/lib/postgresql/9.5/bin/initdb -D /srv/gargantext_data/'
su postgres -c '/usr/lib/postgresql/9.5/bin/pg_ctl -D /srv/gargantext_data/ -l journal_applicatif start'
service postgresql stop
#su postgres -c 'pg_createcluster -D /srv/gargantext_data 9.5 main '
#su postgres -c 'pg_ctlcluster -D /srv/gargantext_data 9.5 main start '
#su postgres -c 'pg_ctlcluster 9.5 main start'
su postgres -c 'pg_dropcluster 9.5 main --stop'
#done in docker but redoing it
if [[ -e "/srv/gargandata" ]]; then
rm -rf /srv/gargandata/*
else
mkdir /srv/gargandata;
chown -R postgres:postgres /srv/gargandata
fi
su postgres -c '/usr/lib/postgresql/9.5/bin/initdb -D /srv/gargandata/'
su postgres -c '/usr/lib/postgresql/9.5/bin/pg_ctl -D /srv/gargandata/ -l journal_applicatif start'
#su postgres -c 'pg_createcluster -D /srv/gargandata 9.5 main '
#su postgres -c 'pg_ctlcluster -D /srv/gargandata 9.5 main start '
service postgresql start
......@@ -24,5 +34,4 @@ su postgres -c "psql -c \"CREATE user gargantua WITH PASSWORD 'C8kdcUrAQy66U'\""
su postgres -c "createdb -O gargantua gargandb"
echo "Postgres configured"
service postgresql stop
install/python/requirements.txt → install/gargamelle/requirements.txt 100755 → 100644
View file @ 360d776a
......@@ -29,5 +29,8 @@ networkx==1.11
pandas==0.18.0
six==1.10.0
lxml==3.5.0
requests-futures==0.9.7
bs4==0.0.1
requests==2.10.0
#testing github
#-e git://github.com/zzzeek/sqlalchemy.git@rel_1_1
install/init.sh deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
echo "Adding user gargantua";
sudo adduser --disabled-password --gecos "" gargantua;
echo "Creating the environnement into /srv/";
for dir in "/srv/gargantext" "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media""/srv/env_3-5"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
echo "Downloading the libs. Please be patient!";
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed";
echo 'Install docker'
sudo apt-get install -y docker-engine
echo 'Build gargantext image'
cd /srv/gargantext/install/
./docker/config/build
#Next steps
#install and configure git
#sudo apt-get install -y git
#clone your SSH key
#cp ~/.ssh/id_rsa.pub id_rsa.pub
#clone the repo
#~ git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
#~ && cd /srv/gargantext \
# get on branch
#~ && git fetch origin unstable \
#~ && git checkout unstable \
#~ echo "Currently on /srv/gargantext unstable branch";
#create your own branch
# git checkout -b my-unstable
install/init/account.csv deleted 100644 → 0
View file @ d673a1ec
gargantua,contact@gargantext,autnagrag,
install/install.sh deleted 100644 → 0
View file @ d673a1ec
#!/usr/bin/bash
echo "Adding user gargantua";
sudo adduser --disabled-password --gecos "" gargantua;
echo "Creating the environnement into /srv/";
for dir in "/srv/gargantext" "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media""/srv/env_3-5"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
echo "Downloading the libs";
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed";
#cp ~/.ssh/id_rsa.pub id_rsa.pub
echo "Cloning the repo";
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring \
echo "Currently on /srv/gargantext refactoring branch";
install/postgres/indexes.sql deleted 100644 → 0
View file @ d673a1ec
-- ____
-- / ___|
-- | | _
-- | |_| |
-- \____|arganTexT
----------------------------------------------------------------------
-- Gargantext optimization of Database --
----------------------------------------------------------------------
--> Manual optimization with indexes according to usages
-- Weakness and Strengths of indexes:
--> it can slow down the insertion(s)
--> it can speed up the selection(s)
--> Conventions for this document:
--> indexes commented already have been created
--> indexes not commented have not been created yet
----------------------------------------------------------------------
-- Retrieve Nodes
----------------------------------------------------------------------
-- create INDEX on nodes (user_id, typename, parent_id) ;
-- create INDEX on nodes_hyperdata (node_id, key);
-- create INDEX on ngrams (id, n) ;
-- create INDEX on ngrams (n) ;
-- create INDEX on nodes_ngrams (node_id, ngram_id) ;
-- create INDEX on nodes_ngrams (node_id) ;
-- create INDEX on nodes_ngrams (ngram_id) ;
-- create INDEX on nodes_ngrams_ngrams (node_id, ngram1_id, ngram2_id) ;
-- create INDEX on nodes_ngrams_ngrams (node_id) ;
-- create INDEX on nodes_ngrams_ngrams (ngram1_id) ;
-- create INDEX on nodes_ngrams_ngrams (ngram2_id) ;
----------------------------------------------------------------------
-- DELETE optimization of Nodes -- todo on dev
-- create INDEX on nodes_nodes_ngrams (node1_id);
-- create INDEX on nodes_nodes_ngrams (node2_id);
-- create INDEX on nodes_nodes (node1_id, node2_id);
-- Maybe needed soon:
-- create INDEX on nodes_nodes_ngrams (node1_id, node2_id);
----------------------------------------------------------------------
-- Analytics
-- create INDEX on nodes_hyperdata (node_id,value_utc); -- remove ?
-- create INDEX on nodes_hyperdata (node_id,key,value_utc);
-- create INDEX on nodes_hyperdata (node_id,key,value_int);
-- create INDEX on nodes_hyperdata (node_id,key,value_flt);
-- create INDEX on nodes_hyperdata (node_id,key,value_str);
----------------------------------------------------------------------
----------------------------------------------------------------------
create index on nodes using GIN (hyperdata);
----------------------------------------------------------------------
install/python/configure deleted 100755 → 0
View file @ d673a1ec
#!/bin/bash
ENV="/srv/env_3-5"
/usr/bin/virtualenv --py=/usr/bin/python3.5 $ENV \
&& /bin/bash -c "source ${ENV}/bin/activate" \
&& /bin/bash -c "${ENV}/bin/pip install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1" \
&& /bin/bash -c "${ENV}/bin/pip install -r /srv/gargantext/install/python/requirements.txt"
install/run_docker.md deleted 100644 → 0
View file @ d673a1ec
#!/usr/bin/bash
#enter the Image
/srv/gargantext/install/docker/enterGargantextImage
#start postgresql
service postgresql start
#change to user
su gargantua
#activate the virtualenv
source /srv/env_3-5/bin/activate
#go to gargantext srv
cd /srv/gargantext/manage.py runserver 0.0.0.0:8000
install/setup.sh deleted 100644 → 0
View file @ d673a1ec
#!/usr/bin/bash
/srv/gargantext/install/docker/enterGargantextImage
/srv/gargantext/install/python/configure
/srv/gargantext/install/postgres/configure
service postgresql start
source /srv/env_3-5/bin/activate
python /srv/gargantext/dbmigrate.py
/srv/gargantext/manage.py makemigrations
/srv/gargantext/manage.py migrate
python /srv/gargantext/dbmigrate.py
python /srv/gargantext/dbmigrate.py
python /srv/gargantext/init_accounts.py /srv/gargantext/install/init/account.csv
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment