Skip to content

gargantext
gargantext
Commit 3e250ac3 authored by Romain Loth's avatar Romain Loth
Browse files
Options

Merge branch 'unstable' into romain-new (eh oui ça va vite sur unstable cet après midi)

parents 1f77b82b e0cd16a5
Hide whitespace changes
Inline Side-by-side
Showing with 280 additions and 212 deletions
docs/install.md
View file @ 3e250ac3
#Install Instructions for Gargantext (CNRS):
## Get the source code
by cloning gargantext into /srv/gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin stable \
&& git checkout stable \
```
#Install Instructions for Gargamelle:
Gargamelle is the gargantext plateforme toolbox it is a full plateform system
with minimal modules
First you need to get the source code to install it
The folder will be /srv/gargantext:
* docs containes all informations on gargantext
/srv/gargantext/docs/
......@@ -20,151 +13,49 @@ The folder will be /srv/gargantext:
Help needed ?
See [http://gargantext.org/about](http://gargantext.org/about) and [tools](./contribution_guide.md) for the community
Two installation procedure are provided:
1. Semi-automatic installation [EASY]
2. Step by step installation [ADVANCED]
Here only semi-automatic installation is covered checkout [manual_install](manual_install.md)
to follow step by step procedure
##Prerequisites
## Init Setup
## Install
## Run
--------------------
# Semi automatic installation
All the procedure files are located into /srv/garantext/install/
``` bash
user@computer:$ cd /srv/garantext/install/
```
## Prerequisites
* A Debian based OS >= [FIXME]
* At least 35GO in /srv/ [FIXME]
todo: reduce the size of gargantext lib
todo: remove lib once docker is configured
! tip: if you have enought space for the full package you can:
* resize your partition
* make a simlink on gargantext_lib
##Init Setup
Prepare your environnement and make the initial setup.
This initial step creates a user for gargantext plateform along with dowloading additionnal libs and files.
It also install docker and build the docker image and build the gargantext box
``` bash
user@computer:/srv/garantext/install/$ .init.sh
```
### Install
Once the init step is done
* Enter into the docker environnement
Inside folder /srv/garantext/install/
enter the gargantext image
``` bash
user@computer:/srv/garantext/install/$ .docker/enterGargantextImage
```
go to the installation folder
``` bash
root@dockerimage8989809:$ cd /srv/gargantext/install/
```
[ICI] Tester si les config de postgresql et python sont faits en amont à la création du docker file
* Install Python environment
``` bash
root@dockerimage8989809:/srv/garantext/install/$ python/configure
```
* Configure PostgreSql
Inside the docker image, execute as root:
``` bash
root@computer:/srv/garantext/install/$ postgres/configure
```
[Si OK ] enlever ses lignes
## Get the source code
Install Gargantext server
by cloning gargantext into /srv/gargantext
* Configure the database
Inside the docker container:
``` bash
service postgresql start
#su gargantua
#activate the virtualenv
source /srv/env_3-5/bin/activate
```
You have entered the virtualenv as shown with (env_3-5)
``` bash
(env_3-5) $ python /srv/gargantext/dbmigrate.py
(env_3-5) $ /srv/gargantext/manage.py makemigrations
(env_3-5) $ /srv/gargantext/manage.py migrate
(env_3-5) $ python /srv/gargantext/dbmigrate.py
#will create tables and not hyperdata_nodes
(env_3-5) $ python /srv/gargantext/dbmigrate.py
#will create table hyperdata_nodes
#launch first time the server to create first user
(env_3-5) $ /srv/gargantext/manage.py runserver 0.0.0.0:8000
(env_3-5) $ /srv/gargantext/init_accounts.py /srv/gargantext/install/init/account.csv
```
FIXME: dbmigrate need to launched several times since tables are
ordered with alphabetical order (and not dependencies order)
* Exit the docker
```
exit (or Ctrl+D)
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin stable \
&& git checkout stable \
```
## Install
``` bash
# go into the directory
user@computer: cd /srv/gargantext/
#git inside installation folder
user@computer: cd /install
#execute the installation
user@computer: ./install
```
During installation an admin account for gargantext will be created by asking you a username and a password
Remember it to accès to the Gargantext plateform
## Run Gargantext
## Run
Once you proceed to installation Gargantext plateforme will be available at localhost:8000
by running the run executable file
``` bash
# go into the directory
user@computer: cd /srv/gargantext/
#git inside installation folder
user@computer: cd /install
#execute the installation
user@computer: ./run
#type ctrl+d to exit or exit; command
```
Enter the docker container:
``` bash
/srv/gargantext/install/docker/enterGargantextImage
```
Inside the docker container:
``` bash
#start Database (postgresql)
service postgresql start
#change to user
su gargantua
#activate the virtualenv
source /srv/env_3-5/bin/activate
#go to gargantext srv
(env_3-5) $ cd /srv/gargantext/
#run the server
(env_3-5) $ /manage.py runserver 0.0.0.0:8000
```
Keep it open and outside the docker launch browser
Then open up a chromium browser and go to localhost:8000
Click on "Enter Gargantext"
Login in with you created username and pasword
``` bash
chromium http://127.0.0.1:8000/
```
Enjoy! ;)
* Click on Test Gargantext
```
Login : gargantua
Password : autnagrag
```
Enjoy :)
See [User Guide](/demo/tuto.md) for quick usage example
gargantext/constants.py
View file @ 3e250ac3
......@@ -115,17 +115,19 @@ INDEXED_HYPERDATA = {
}
from gargantext.util.taggers import FrenchMeltTagger, TurboTagger
#from gargantext.util.taggers import FrenchMeltTagger, TurboTagger
from gargantext.util.taggers import NltkTagger
LANGUAGES = {
'en': {
#'tagger': EnglishMeltTagger,
'tagger': TurboTagger,
#'tagger': NltkTagger,
#'tagger': TurboTagger,
'tagger': NltkTagger,
},
'fr': {
'tagger': FrenchMeltTagger,
# 'tagger': TreeTagger,
#'tagger': FrenchMeltTagger,
#'tagger': TreeTagger,
'tagger': NltkTagger,
},
}
......
gargantext/util/taggers/NltkTagger.py
View file @ 3e250ac3
#!/usr/bin/python env
from ._Tagger import Tagger
import nltk
#from nltk import pos_tag
from nltk.tag.perceptron import PerceptronTagger
class NltkTagger(Tagger):
'''Require maxtree'''
#import nltk
def __init__(self, *args, **kwargs):
self.tagr = PerceptronTagger()
super(self.__class__, self).__init__(*args, **kwargs)
#~ def __start__(self):
#~ self.tagr = PerceptronTagger()
def tag_tokens(self, tokens, single=True):
return nltk.pos_tag(tokens)
return self.tagr.tag(tokens)
gargantext/util/toolchain/ngrams_extraction.py
View file @ 3e250ac3
......@@ -48,7 +48,7 @@ def extract_ngrams(corpus, keys=('title', 'abstract', ), do_subngrams = DEFAULT_
ngrams_data = set()
# extract ngrams
resource_type_index = corpus.resources()[0]['type']
documents_count = 0
resource_type = RESOURCETYPES[resource_type_index]
default_language_iso2 = resource_type['default_language']
for documents_count, document in enumerate(corpus.children('DOCUMENT')):
......
install/01-setup deleted 100755 → 0
View file @ 1f77b82b
#!/bin/bash
#name:01-setup
#TODO clone the repo into /srv/gargantext/ and reduce the different steps
#git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
# && cd /srv/gargantext \
# && git fetch origin stable \
# && git checkout stable \
echo "****************SETUP**************************";
for dir in "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
#here gargantext_lib.tar.bz2 inside this directory
#sudo wget http://dl.gargantext.org/gargantext_lib.tar.bz2 && \
sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
&& echo ":::::::::::::::::Done::::::::::::::::::::::::::";
install/README.md
View file @ 3e250ac3
# Install
#Install Instructions for Gargamelle:
Gargamelle is a developpement environemment to install configure and run Gargantext plateform.
Gargamelle is the gargantext plateforme toolbox it is a full plateform system
with minimal modules
## Prerequisites
First you need to get the source code to install it
The folder will be /srv/gargantext:
* docs containes all informations on gargantext
/srv/gargantext/docs/
* install contains all the installation files
/srv/gargantext/install/
* MaC or Debian based OS
* Git
* Docker engine
Help needed ?
See [http://gargantext.org/about](http://gargantext.org/about) and [tools](./contribution_guide.md) for the community
## Installation Procedure
## Get the source code
* Clone the repository
If you want to contribute see our [git workflow procedure](../tools/git.md)
clone the repo and switch to your own branch named with username-branchname such as username-unstable for example
Inside the repository (install)
you will find the setup config and run procedure files
cd gargantext/install/
by cloning gargantext into /srv/gargantext
``` bash
git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
&& cd /srv/gargantext \
&& git fetch origin stable \
&& git checkout stable \
```
./01-setup
./02-config #here you will have to choose a username & a password for gargantext
./03-run
```
Open your browser Chrome perferably at localhost:8000
click into enter and login!
## Install
``` bash
# go into the directory
user@computer: cd /srv/gargantext/
#git inside installation folder
user@computer: cd /install
#execute the installation
user@computer: ./install
```
During installation an admin account for gargantext will be created by asking you a username and a password
Remember it to accès to the Gargantext plateform
Enjoy ;)!
## Run
Once you proceed to installation Gargantext plateforme will be available at localhost:8000
by running the run executable file
``` bash
# go into the directory
user@computer: cd /srv/gargantext/
#git inside installation folder
user@computer: cd /install
#execute the installation
user@computer: ./run
#type ctrl+d to exit or exit; command
```
Then open up a chromium browser and go to localhost:8000
Click on "Enter Gargantext"
Login in with you created username and pasword
Enjoy! ;)
install/gargamelle/.01-setup 0 → 100755
View file @ 3e250ac3
#!/bin/bash
#name:01-setup
#TODO clone the repo into /srv/gargantext/ and reduce the different steps
#git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
# && cd /srv/gargantext \
# && git fetch origin stable \
# && git checkout stable
function clone_repo {
echo "* CLONE"
#echo "/home/"$USER"/ssh_keys/"$USER;
read -e -r -p "Please provide you ssh keys full directory:" response;
ssh-agent bash -c 'ssh-add ' $response '';
git clone ssh://gitolite@delanoe.org:1979/gargantext -o /srv/gargantext;
cd /srv/gargantext
git fetch origin stable
git pull
git checkout -b $USER'-stable';
echo "Switching to your own branch";
}
function download_lib {
echo "* DOWNLOAD additionnal lib"
echo "(May take a while, be patient or come back later when finished;)"
sudo wget -P install/ http://dl.gargantext.org/gargantext_lib.tar.bz2
#here gargantext_lib.tar.bz2 inside this directory
}
function uncompress_lib {
cd /srv/gargantext/install/
echo "* UNCOMPRESSING additionnal lib"
sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib
sudo chown -R gargantua:gargantua /srv/gargantext_lib
}
#~ read -r -p "Do you want to clone the project? [y/N] " response
#~ case $response in
#~ [yY][eE][sS]|[yY])
#~ clone_repo
#~ ;;
#~ *)
#~ cd /srv/gargantext/
#~ ;;
#~ esac
echo "::: SETUP ENV :::";
for dir in "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
#here gargantext_lib.tar.bz2 inside this directory
#sudo wget http://dl.gargantext.org/gargantext_lib.tar.bz2 && \
#sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
#&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
#&& echo ":::::::::::::::::Done::::::::::::::::::::::::::";
echo ':::: BUILD ::::'
sudo docker build -t gargamelle:latest ./gargamelle
echo ':::: CONFIGURE ::::'
sudo docker run \
-v /srv/:/srv/ \
-p 8000:8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "./psql_configure.sh; ./django_configure.sh ; exit"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
install/02-config → install/gargamelle/.02-config
View file @ 3e250ac3
#!/bin/bash
#configure the base image gargamelle
#echo '::::: BUILD :::::'
#echo ':::: BUILD ::::'
sudo docker build -t gargamelle:latest ./gargamelle
echo '::::: CONFIG :::::'
echo ':::: CONFIGURE ::::'
sudo docker run \
......@@ -16,4 +13,3 @@ sudo docker run \
/bin/bash -c "./psql_configure.sh; ./django_configure.sh ; exit"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
install/gargamelle/Dockerfile
View file @ 3e250ac3
......@@ -64,9 +64,10 @@ ADD requirements.txt /
ADD psql_configure.sh /
ADD django_configure.sh /
RUN . /env_3-5/bin/activate; pip3 install -r requirements.txt; \
pip3 install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1; \
python3 -m nltk.downloader averaged_perceptron_tagger;
RUN . /env_3-5/bin/activate && pip3 install -r requirements.txt && \
pip3 install git+https://github.com/zzzeek/sqlalchemy.git@rel_1_1 &&\
python3 -m nltk.downloader averaged_perceptron_tagger -d /usr/local/share/nltk_data;
# nltk.data.path.append('path_to_nltk_data')
RUN chown gargantua:gargantua -R /env_3-5
......
install/gargamelle/debug 0 → 100644
View file @ 3e250ac3
#!/bin/bash
sudo docker run \
-v /srv/:/srv/\
-p 8000:8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "service postgresql start; /bin/su gargantua -c 'source /env_3-5/bin/activate && /srv/gargantext/manage.py runserver 0.0.0.0:8000' && bin/bash"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
install/gargamelle/django_configure.sh
View file @ 3e250ac3
......@@ -24,7 +24,5 @@ echo "::::: DJANGO :::::"
./srv/gargantext/dbmigrate.py && \
./srv/gargantext/manage.py createsuperuser'
/usr/sbin/service postgresql stop
install/gargamelle/psql_configure.sh
View file @ 3e250ac3
......@@ -13,6 +13,7 @@ su postgres -c 'pg_dropcluster 9.4 main --stop'
#done in docker but redoing it
rm -rf /srv/gargandata && mkdir /srv/gargandata && chown postgres:postgres /srv/gargandata
su postgres -c '/usr/lib/postgresql/9.5/bin/initdb -D /srv/gargandata/'
#su postgres -c '/usr/lib/postgresql/9.5/bin/pg_ctl -D /srv/gargandata/ -l journal_applicatif start'
su postgres -c '/usr/lib/postgresql/9.5/bin/pg_ctl -D /srv/gargandata/ -l journal_applicatif start'
su postgres -c 'pg_createcluster -D /srv/gargandata 9.5 main '
......@@ -25,5 +26,4 @@ su postgres -c "psql -c \"CREATE user gargantua WITH PASSWORD 'C8kdcUrAQy66U'\""
su postgres -c "createdb -O gargantua gargandb"
echo "Postgres configured"
#service postgresql stop
install/install 0 → 100755
View file @ 3e250ac3
#!/bin/bash
#name:01-setup
#TODO clone the repo into /srv/gargantext/ and reduce the different steps
#git clone ssh://gitolite@delanoe.org:1979/gargantext /srv/gargantext \
# && cd /srv/gargantext \
# && git fetch origin stable \
# && git checkout stable
function clone_repo {
echo "* CLONE"
#echo "/home/"$USER"/ssh_keys/"$USER;
read -e -r -p "Please provide you ssh keys full directory:" response;
ssh-agent bash -c 'ssh-add ' $response '';
git clone ssh://gitolite@delanoe.org:1979/gargantext -o /srv/gargantext;
cd /srv/gargantext
git fetch origin stable
git pull
git checkout -b $USER'-stable';
echo "Switching to your own branch";
}
function download_lib {
echo "* DOWNLOAD additionnal lib"
echo "(May take a while, be patient or come back later when finished;)"
sudo wget -P install/ http://dl.gargantext.org/gargantext_lib.tar.bz2
#here gargantext_lib.tar.bz2 inside this directory
}
function uncompress_lib {
cd /srv/gargantext/install/
echo "* UNCOMPRESSING additionnal lib"
sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib
sudo chown -R gargantua:gargantua /srv/gargantext_lib
}
#~ read -r -p "Do you want to clone the project? [y/N] " response
#~ case $response in
#~ [yY][eE][sS]|[yY])
#~ clone_repo
#~ ;;
#~ *)
#~ cd /srv/gargantext/
#~ ;;
#~ esac
echo "::: SETUP ENV :::";
for dir in "/srv/gargantext_lib" "/srv/gargantext_static" "/srv/gargantext_media"; do
sudo mkdir -p $dir ;
sudo chown gargantua:gargantua $dir ;
done;
#here gargantext_lib.tar.bz2 inside this directory
#sudo wget http://dl.gargantext.org/gargantext_lib.tar.bz2 && \
#sudo tar xvjf gargantext_lib.tar.bz2 -o /srv/gargantext_lib \
#&& sudo chown -R gargantua:gargantua /srv/gargantext_lib \
#&& echo ":::::::::::::::::Done::::::::::::::::::::::::::";
echo ':::: BUILD ::::'
sudo docker build -t gargamelle:latest ./gargamelle
echo ':::: CONFIGURE ::::'
sudo docker run \
-v /srv/:/srv/ \
-p 8000:8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "./psql_configure.sh; ./django_configure.sh ; exit"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
install/run 0 → 100755
View file @ 3e250ac3
#!/bin/bash
sudo docker run \
-v /srv/:/srv/\
-p 8000:8000 \
-p 5432 \
-it gargamelle:latest \
/bin/bash -c "service postgresql start; /bin/su gargantua -c 'source /env_3-5/bin/activate && /srv/gargantext/manage.py runserver 0.0.0.0:8000'"
sudo docker rm -f `docker ps -a | grep -v CONTAINER | awk '{print $1 }'`
templates/pages/menu.html
View file @ 3e250ac3
......@@ -229,7 +229,7 @@
<p>
Gargantext
<span class="glyphicon glyphicon-registration-mark" aria-hidden="true"></span>
, version 3.0.4.3,
, version 3.0.4.4,
<a href="http://www.cnrs.fr" target="blank" title="Institution that enables this project.">
Copyrights
<span class="glyphicon glyphicon-copyright-mark" aria-hidden="true"></span>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment