Commits · simon-data-import · humanities / gargantext

29 Mar, 2018 15 commits
- wip · 6a172480
  sim authored Mar 29, 2018
  
  6a172480
- Keep raw files processed by scrapers in UPLOAD_DIRECTORY · 0be8f66f
  sim authored Mar 29, 2018
  
  0be8f66f
- Migration: remove legacy table nodes_hyperdata · ef08aad6
  sim authored Mar 29, 2018
  
  ef08aad6
- Model: rename typename, date, hyperdata of Node to type, created and data · e6a6f379
  sim authored Mar 29, 2018
  
  e6a6f379
- Migration: rename typename, date, hyperdata of Node to type, created and data · 7110ae5b
  sim authored Mar 29, 2018
  
  7110ae5b
- Alembic: move utilities to gargantext/utils · babbc0c9
  sim authored Mar 29, 2018
  
  babbc0c9
- Alembic: fix stored procedures dropping · 323b1d6e
  sim authored Mar 29, 2018
  
  323b1d6e
- Alembic: a ReplaceableObject can be designated by its name · 2b7bc9d5
  sim authored Mar 29, 2018
  
  2b7bc9d5
- All scrapyd logs are kept by default · 6cb54246
  sim authored Mar 29, 2018
  
  6cb54246
- Change default scrapyds logs dir to /var/log/gargantext/scrapyd · 1f418a48
  sim authored Mar 29, 2018
  
  1f418a48
- Fix setup of scrapyd logs directory · 1019b95a
  sim authored Mar 29, 2018
  
  1019b95a
- Automatically handle scrapyd configuration file · 06dfaf2f
  sim authored Mar 29, 2018
  
  06dfaf2f
- Add HAL scraper · 1b34fc89
  sim authored Mar 29, 2018
  
  1b34fc89
- Store scraping 'resources' in CorpusNode · 40620d80
  sim authored Mar 28, 2018
  
  40620d80
- Scrapers: major refactor and many new features · ce2c9c78
  sim authored Mar 28, 2018
  
  ce2c9c78
28 Mar, 2018 15 commits
- Add urllib3 module for scrapers error reporting · 8efa9762
  sim authored Mar 28, 2018
  
  8efa9762
- Delegate RIS parsing to RISParser · 76bf421a
  sim authored Mar 28, 2018
  
  76bf421a
- Fix response type classes hierarchy · 238bf26d
  sim authored Mar 28, 2018
  
  238bf26d
- Add language detection and naming utilities · 2754c5ea
  sim authored Mar 28, 2018
  
  2754c5ea
- Add type conversion utilities · 4695d417
  sim authored Mar 28, 2018
  
  4695d417
- Change default datetime to last day of year 1900 · 047dc8de
  sim authored Mar 28, 2018
  
  047dc8de
- Better date parsing · 2fcbf674
  sim authored Mar 28, 2018
  
  2fcbf674
- Add Node.related many-to-many relationship via nodes_nodes · 249469c3
  sim authored Mar 28, 2018
  
  249469c3
- Migration: add privileges and policies on nodes_nodes table · 7b4dd6b4
  sim authored Mar 28, 2018
  
  7b4dd6b4
- Better handling of hyperdata modification · d73f94d5
  sim authored Mar 28, 2018
  
  d73f94d5
- Add Node.NAME_MAXLEN constant · 6ebdc37e
  sim authored Mar 28, 2018
  
  6ebdc37e
- Add explicit DocumentNode definition · 83934674
  sim authored Mar 28, 2018
  
  83934674
- Clean legacy stuff in Node model · 3ffad0ed
  sim authored Mar 28, 2018
  
  3ffad0ed
- Add RIS parser · f7a2666f
  sim authored Mar 28, 2018
  
  f7a2666f
- Upgrade dependencies · ba4e11d8
  sim authored Mar 28, 2018
  
  ba4e11d8
21 Mar, 2018 3 commits
- Pubmed scraper: use JournalIssue to get publication date · b053ed96
  sim authored Mar 21, 2018
  
  b053ed96
- Pubmed scraper: don't yield result count · ed572281
  sim authored Mar 21, 2018
  
  ed572281
- Better utilities to parse and convert to datetime · e2243575
  sim authored Mar 21, 2018
  
  e2243575
19 Mar, 2018 2 commits
- Upgrade dependencies · d76f0f82
  sim authored Mar 14, 2018
  
  d76f0f82
- Install scrapyd server to run scrapers · 1a7b5bf3
  sim authored Mar 13, 2018
  
  1a7b5bf3
14 Mar, 2018 5 commits
- Better log verbose format · 4fc0ec1a
  sim authored Mar 14, 2018
  
  4fc0ec1a
- Add ris scraper · 5c520cc8
  sim authored Mar 14, 2018
  
  5c520cc8
- Add pubmed scraper · c816759e
  sim authored Mar 14, 2018
  
  c816759e
- Brand new scrapers infrastructure using scrapy · 68aef175
  sim authored Mar 14, 2018
  
  68aef175
- Add json_loads function to gargantext.utils.json · b68c1ae0
  sim authored Mar 14, 2018
  
  b68c1ae0