Skip to content

gargantext
gargantext
Commit 1897d0c8 authored by delanoe's avatar delanoe
Browse files
Options

Merge remote-tracking branch 'origin/c24b-docker' into merge

parents 75d4a738 47947343
Hide whitespace changes
Inline Side-by-side
Showing with 286 additions and 428 deletions
doc/README.md deleted 100644 → 0
View file @ 75d4a738
#Gargantext
==========
Install Instructions for Gargantext (CNRS):
1. [SETUP](##SETUP)
2. [INSTALL](##INSTALL)
3. [RUN](##RUN)
## Help needed ?
See http://gargantext.org/about and tools for the community
## SETUP
Prepare your environnement
Create user gargantua
Main user of Gargantext is Gargantua (role of Pantagruel soon)!
``` bash
sudo adduser --disabled-password --gecos "" gargantua
```
Create the directories you need
``` bash
for dir in "/srv/gargantext"
"/srv/gargantext_lib"
"/srv/gargantext_static"
"/srv/gargantext_media"
"/srv/env_3-5"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done
```
You should see:
```bash
$tree /srv
/srv
├── gargantext
├── gargantext_lib
├── gargantext_media
│   └── srv
│   └── env_3-5
├── gargantext_static
└── lost+found [error opening dir]
```
## Get the source code of Gargantext
Clone the repository of gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring \
```
**Optionnal**: if you want to contribute clone the repo into your own branch
``` bash
git checkout -b username-refactoring refactoring
```
! TODO (soon) : git clone https://gogs.iscpif.fr/gargantext.git
## SETUP
Build your OS dependencies
2 ways, for each you need to install Debian GNU/Linux dependencies.
1. [EASY] [Docker way](#DOCKER)
2. [EXPERT] [Debian way](#DEBIAN)
### DOCKER
* Install docker
See [installation instruction for your distribution](https://docs.docker.com/engine/installation/)
#### Build your docker image
``` bash
cd /srv/gargantext/install/docker/dev
./build
```
You should see
```
Successfully built <container_id>
```
#### Enter into the docker environnement
``` bash
./srv/gargantext/install/docker/enterGargantextImage
```
#### Install Python environment
Inside the docker image, execute as root:
``` bash
/srv/gargantext/install/python/configure
```
#### Configure PostgreSql
Inside the docker image, execute as root:
``` bash
/srv/gargantext/install/postgres/configure
```
#### Exit the docker
``` exit
```
#### Get main librairies
Can be long, so be patient :)
``` bash
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed"
```
### DEBIAN
[EXPERTS] Debian way (directory install/debian)
## INSTALL Gargantext
### Enter docker container
``` bash
/srv/gargantext/install/docker/enterGargantextImage
```
### Inside docker container configure the database
``` bash
service postgresql start
su gargantua
source /srv/env_3-5/bin/activate
python /srv/gargantext/dbmigrate.py
/srv/gargantext/manage.py migrate
python /srv/gargantext/dbmigrate.py
python /srv/gargantext/dbmigrate.py
echo "TODO: Init first user"
```
FIXME: dbmigrate need to launched several times since tables are
ordered with alphabetical order (and not dependencies order)
### Inside docker container launch Gargantext
``` bash
service postgresql start
su gargantua
source /srv/env_3-5/bin/activate
/srv/gargantext/manage.py runserver 0.0.0.0:8000
python /srv/gargantext/init_accounts.py /srv/gargantext/install/init/account.csv
```
## RUN
### Outside docker container launch browser
``` bash
chromium http://127.0.0.1:8000/
```
Click on Test Gargantext
Login : gargantua
Password : autnagrag
Enjoy :)
doc/install.md deleted 100644 → 0
View file @ 75d4a738
#Gargantext
==========
Install Instructions for Gargantext (CNRS):
1. [SETUP](##SETUP)
2. [INSTALL](##INSTALL)
3. [RUN](##RUN)
## Help needed ?
See http://gargantext.org/about and tools for the community
## SETUP
Prepare your environnement
Create user gargantua
Main user of Gargantext is Gargantua (role of Pantagruel soon)!
``` bash
sudo adduser --disabled-password --gecos "" gargantua
```
Create the directories you need
``` bash
for dir in "/srv/gargantext"
"/srv/gargantext_lib"
"/srv/gargantext_static"
"/srv/gargantext_media"
"/srv/env_3-5"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done
```
You should see:
```bash
$tree /srv
/srv
├── gargantext
├── gargantext_lib
├── gargantext_media
│   └── srv
│   └── env_3-5
├── gargantext_static
└── lost+found [error opening dir]
```
## Get the source code of Gargantext
Clone the repository of gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring \
```
**Optionnal**: if you want to contribute clone the repo into your own branch
``` bash
git checkout -b username-refactoring refactoring
```
! TODO (soon) : git clone https://gogs.iscpif.fr/gargantext.git
## SETUP
Build your OS dependencies
2 ways, for each you need to install Debian GNU/Linux dependencies.
1. [EASY] [Docker way](#DOCKER)
2. [EXPERT] [Debian way](#DEBIAN)
### DOCKER
* Install docker
See [installation instruction for your distribution](https://docs.docker.com/engine/installation/)
#### Build your docker image
``` bash
cd /srv/gargantext/install/docker/dev
./build
```
You should see
```
Successfully built <container_id>
```
#### Enter into the docker environnement
``` bash
./srv/gargantext/install/docker/enterGargantextImage
```
#### Install Python environment
Inside the docker image, execute as root:
``` bash
/srv/gargantext/install/python/configure
```
#### Configure PostgreSql
Inside the docker image, execute as root:
``` bash
/srv/gargantext/install/postgres/configure
```
#### Exit the docker
``` exit
```
#### Get main librairies
Can be long, so be patient :)
``` bash
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed"
```
### DEBIAN
[EXPERTS] Debian way (directory install/debian)
## INSTALL Gargantext
### Enter docker container
``` bash
/srv/gargantext/install/docker/enterGargantextImage
```
### Inside docker container configure the database
``` bash
service postgresql start
#su gargantua
source /srv/env_3-5/bin/activate
python /srv/gargantext/dbmigrate.py
/srv/gargantext/manage.py makemigrations
/srv/gargantext/manage.py migrate
python /srv/gargantext/dbmigrate.py
#create models:
python /srv/gargantext/dbmigrate.py
#created model: `nodes_hyperdata`
echo "TODO: Init first user"
```
FIXME: dbmigrate need to launched several times since tables are
ordered with alphabetical order (and not dependencies order)
### Inside docker container launch Gargantext
``` bash
service postgresql start
su gargantua
source /srv/env_3-5/bin/activate
/srv/gargantext/manage.py runserver 0.0.0.0:8000
python /srv/gargantext/init_accounts.py /srv/gargantext/install/init/account.csv
```
## RUN
### Outside docker container launch browser
``` bash
chromium http://127.0.0.1:8000/
```
Click on Test Gargantext
Login : gargantua
Password : autnagrag
Enjoy :)
docs/about.md
View file @ 1897d0c8
##Welcome to Gargantext documentation
docs/about/index.md 0 → 100644
View file @ 1897d0c8
#Gargantext
Welcome to Garagentext documentation!
docs/contribution-guide.md 0 → 100644
View file @ 1897d0c8
#Contribution guide
## Community
* [http://gargantext.org/about](http://gargantext.org/about)
* IRC Chat: (OFTC/FreeNode) #gargantex
##Tools
* gogs
* server access
* forge
* gargantext box
##Gargantex
* Gargantex box install
(S.I.R.= Setup Install & Run procedures)
* Architecture Overview
* Database Schema Overview
* Interface design Overview
##To do:
* Docs
* Interface deisgn
* Parsers/scrapers
* Computing
## How to contribute:
1. Clone the repo
2. Create a new branch <username>-refactoring
3. Run the gargantext-box
4. Code
5.Test
6. Commit
### Exemple1: Adding a parser
* create your new file cern.py into gargantex/scrapers/
* reference into gargantex/scrapers/urls.py
add this line:
import scrapers.cern as cern
* reference into gargantext/constants
```
# type 9
{ 'name': 'Cern',
'parser': CernParser,
'default_language': 'en',
},
```
* add an APIKEY in gargantex/settings
### Exemple2: User Interface Design
docs/contribution-guide/contribution.md 0 → 100644
View file @ 1897d0c8
#Contribution guide
* A question or a problem? Ask the community
* Sources
* Tools
* Contribution workflow: for contributions, bugs and features
* Some examples of contributions
## Community
Need help? Ask the community
* [http://gargantext.org/about](http://gargantext.org/about)
* IRC Chat: (OFTC/FreeNode) #gargantex
## Source
Source are available throught XXX LICENSE
You can install Gargantext throught the [installation procedure](./install.md)
##Tools
* gogs
* forge.iscpif.fr
* server access
* gargantext box
## Contributing: workflow procedure
Once you have installed and tested Gargantext
You
1. Clone the stable release into your project
Note: The current stable release <release_branch> is: refactoring
Inside the repo, clone the reference branch and get the last changes:
git checkout <ref_branch>
git pull
It is highly recommended to create a generic branch on a stable release such as
git checkout -b <username>-<release_branch>
git pull
2. Create your project on stable release
git checkout -b <username>-<release_branch>-<project_name>
Do your modifications and commits as you want it:
git commit -m "foo/bar/1"
git commit -m "foo/bar/2"
git push
If you want to save your local change you can merge it into your generic branch <username>-<release_branch>
git checkout <username>-<release_branch>
git pull
git merge <username>-<release_branch>-<project_name>
git commit -m "[Merge OK] comment"
##Technical Overview
* Interface Overview
* Database Schema Overview
* Architecture Overview
### Exemple1: Adding a parser
### Exemple2: User Interface Design
docs/discover.md 0 → 100644
View file @ 1897d0c8
#Architecture Overview
#Database Schema
#Website
docs/index.md
View file @ 1897d0c8
#Gargantext
Welcome to Garagentext documentation!
Gargantext is a web plateform to explore your corpora using text-mining[...](about.md)
## Getting started
* [Install](install.md) the Gargantext box
* [Take a tour](demo.md) of the different features offered by Gargantext
##Need some help?
Ask the community at:
* [http://gargantext.org/about](http://gargantext.org/about)
* IRC Chat: (OFTC/FreeNode) #gargantex
##Want to contribute?
* take a look at the [architecture overview](overview.md)
* read the [contribution guide](contribution-guide.md)
## News
## Credits and acknowledgments
docs/install.md
View file @ 1897d0c8
Gargantext
===========
Install Instructions for Gargantext (CNRS): Install Instructions for Gargantext (CNRS):
=> Help needed ? Help needed ?
See [http://gargantext.org/about](http://gargantext.org/about) and [tools]() for the community See [http://gargantext.org/about](http://gargantext.org/about) and [tools]() for the community
1. [SETUP](##SETUP)
2. [INSTALL](##INSTALL)
*.1 with [docker](####DOCKER) [EASY]
*.2 with [debian](####DEBIAN) [EXPERT]
3. [RUN](##RUN)
Prepare your environnement and make the initial installation.
Once you setup and install the Gargantext box. You can use ./install/run.sh utility
to load gargantext web plateform and access it throught your web browser
##SETUP ______________________________
Prepare your environnement
* Create user gargantua 1. [Prerequisites](##Prerequisites)
Main user of Gargantext is Gargantua (role of Pantagruel soon)!
``` bash 2. [SETUP](##Setup)
sudo adduser --disabled-password --gecos "" gargantua
```
* Create the directories you need 3. [INSTALL](##Install)
``` bash 4. [RUN](##RUN)
for dir in "/srv/gargantext" ______________________________
"/srv/gargantext_lib" ##Prerequisites
"/srv/gargantext_static"
"/srv/gargantext_media"
"/srv/env_3-5"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done
```
You should see: * A Debian based OS >= [FIXME]
```bash * At least 35GO in the desired location of Gargantua [FIXME]
$tree /srv todo: reduce the size of gargantext lib
/srv todo: remove lib once docker is configure
├── gargantext
├── gargantext_lib
├── gargantext_media
│   └── srv
│   └── env_3-5
├── gargantext_static
└── lost+found [error opening dir]
``` tip: if you have enought space for the full package you can:
* Get the main libraries * resize your partition
* make a simlink on gargantext_lib
``` bash * A [docker engine installation](https://docs.docker.com/engine/installation/linux/)
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed"
```
* Get the source code of Gargantext ##Setup
Prepare your environnement and make the initial setup.
by cloning the repository of gargantext Setup can be done in 2 ways:
``` bash * [automatic setup](setup.sh) can be done by using the setup script provided [here](setup.sh)
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \ * [manual setup](manual_setup.md) if you want to change some parameters [here](manual_setup.md)
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring \
```
TODO(soon): git clone https://gogs.iscpif.fr/gargantext.git ##Install
TODO(soon): install/setup.sh
* **Optionnal**: if you want to contribute clone the repo into your own branch Two installation procedure are actually proposed:
* the docker way [easy]
``` bash * the debian way [advanced]
git checkout -b username-refactoring refactoring
```
##INSTALL
Build your OS dependencies
2 ways, for each you need to install Debian GNU/Linux dependencies.
####DOCKER WAY [EASY] ####DOCKER WAY [EASY]
...@@ -94,6 +55,7 @@ See [installation instruction for your distribution](https://docs.docker.com/eng ...@@ -94,6 +55,7 @@ See [installation instruction for your distribution](https://docs.docker.com/eng
``` bash ``` bash
cd /srv/gargantext/install/docker/dev cd /srv/gargantext/install/docker/dev
./build ./build
ID=$(docker build .) && docker run -i -t $ID
``` ```
You should see You should see
...@@ -126,12 +88,6 @@ exit (or Ctrl+D) ...@@ -126,12 +88,6 @@ exit (or Ctrl+D)
``` ```
####DEBIAN way [EXPERT]
[EXPERTS] Debian way (directory install/debian)
Install Gargantext server Install Gargantext server
* Enter docker container * Enter docker container
...@@ -159,10 +115,15 @@ python /srv/gargantext/dbmigrate.py ...@@ -159,10 +115,15 @@ python /srv/gargantext/dbmigrate.py
FIXME: dbmigrate need to launched several times since tables are FIXME: dbmigrate need to launched several times since tables are
ordered with alphabetical order (and not dependencies order) ordered with alphabetical order (and not dependencies order)
####Debian way [advanced]
##RUN ##Run Gargantext
* Launch Gargantext * Launch Gargantext
Enter the docker container:
``` bash
/srv/gargantext/install/docker/enterGargantextImage
```
Inside the docker container: Inside the docker container:
``` bash ``` bash
#start postgresql #start postgresql
...@@ -190,6 +151,6 @@ Login : gargantua ...@@ -190,6 +151,6 @@ Login : gargantua
Password : autnagrag Password : autnagrag
``` ```
Enjoy :) Enjoy :)
See [User Guide](./tuto.md) for quick usage example See [User Guide](/demo/tuto.md) for quick usage example
doc/INSTALL.md → docs/manual_install.md
View file @ 1897d0c8
#INSTALL * Create user gargantua
Main user of Gargantext is Gargantua (role of Pantagruel soon)!
## Clone the repositority
(For now git clone ssh://gitolite@delanoe.org:1979/gargantext)
copy the workibg branch
git fetch origin refactoring
create your own branch if you want to contribute
git checkout -b username-refact refactoring
## Installation instruction
are detailled in gargantex/install
create a default user for granatext: gargantua
``` bash ``` bash
sudo adduser --disabled-password --gecos "" gargantua sudo adduser --disabled-password --gecos "" gargantua
``` ```
create the different directory for Gargantex
* Create the directories you need
here for the example gargantext package will be installed in /srv/
``` bash ``` bash
for dir in "/srv/gargantext" for dir in "/srv/gargantext"
"/srv/gargantext_lib" "/srv/gargantext_lib"
...@@ -26,17 +18,42 @@ for dir in "/srv/gargantext" ...@@ -26,17 +18,42 @@ for dir in "/srv/gargantext"
done done
``` ```
You should have: You should see:
```bash ```bash
tree /srv $tree /srv
/srv /srv
├── gargantext ├── gargantext
├── gargantext_lib ├── gargantext_lib
├── gargantext_media ├── gargantext_media
│   └── srv │   └── srv
│   └── env_3-5 │   └── env_3-5
├── gargantext_static └── gargantext_static
└── lost+found [error opening dir] ```
* Get the main libraries
Download uncompress and make main user access to it.
PLease, Be patient due to the size of the packages libraries (27GO)
this step can be long....
``` bash
wget http://dl.gargantext.org/gargantext_lib.tar.bz2 \
&& tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo "Libs installed"
``` ```
* Get the source code of Gargantext
by cloning the repository of gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin refactoring \
&& git checkout refactoring \
```
TODO(soon): git clone https://gogs.iscpif.fr/gargantext.git
See the [next steps of installation procedure](install.md#Install)
docs/overview.md 0 → 100644
View file @ 1897d0c8
#Architecture Overview
#Database Schema
#Website
docs/schemas/ngram_parsing_flow.dot 0 → 100755
View file @ 1897d0c8
// dot ngram_parsing_flow.dot -Tpng -o ngram_parsing_flow.png
digraph ngramflow {
edge [fontsize=10] ;
label=<<B><U>gargantext.util.toolchain</U></B><BR/>(ngram extraction flow)>;
labelloc="t" ;
"extracted_ngrams" -> "grouplist" ;
"extracted_ngrams" -> "occs+tfidfs" ;
"main_user_stoplist" -> "stoplist" ;
"stoplist" -> "mainlist" ;
"occs+tfidfs" -> "mainlist" [label=" TFIDF_LIMIT"];
"mainlist" -> "coocs" [label=" COOCS_THRESHOLD"] ;
"coocs" -> "specificity" ;
"specificity" -> "maplist" [label="MAPLIST_LIMIT\nMONOGRAM_PART"];
"maplist" -> "explore" ;
"grouplist" -> "maplist" ;
}
install/docker/config/Dockerfile
View file @ 1897d0c8
...@@ -26,7 +26,8 @@ ENV PYTHON_ENV /srv/env_3-5 ...@@ -26,7 +26,8 @@ ENV PYTHON_ENV /srv/env_3-5
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y \ apt-get install -y \
apt-utils ca-certificates locales \ apt-utils ca-certificates locales \
sudo aptitude gcc g++ wget git postgresql-9.5 vim sudo aptitude gcc g++ wget git postgresql-9.5 vim \
build-essential
### Configure timezone and locale ### Configure timezone and locale
RUN echo "Europe/Paris" > /etc/timezone && \ RUN echo "Europe/Paris" > /etc/timezone && \
......
install/docker/dev/Dockerfile
View file @ 1897d0c8
...@@ -62,5 +62,11 @@ RUN apt-get install -y sudo && adduser gargantua sudo \ ...@@ -62,5 +62,11 @@ RUN apt-get install -y sudo && adduser gargantua sudo \
##################################################################### #####################################################################
RUN sed -iP "s%^data_directory.*%data_directory = \'\/srv\/gargantext_data\'%" /etc/postgresql/9.5/main/postgresql.conf RUN sed -iP "s%^data_directory.*%data_directory = \'\/srv\/gargantext_data\'%" /etc/postgresql/9.5/main/postgresql.conf
##################################################################### #####################################################################
COPY . /install
CMD [./install/python/configure, ./install/posgres/configure, sudo pip install virtualenvwrapper]
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment