Skip to content

gargantext
gargantext
Commit 1e39ea34 authored by Romain Loth's avatar Romain Loth
Browse files
Options

copy of WIP annotations

parent dc10840a
Hide whitespace changes
Inline Side-by-side
Showing with 395 additions and 38 deletions
annotations/romain_infos.md 0 → 100644
View file @ 1e39ea34
ANNOTATIONS
===========
2016-01
## Routines de manipulation de ngrams dans les listes
#### Trajectoire globale des actions choisies
1. angular: ngramlist.js (user input) or highlight.js (user menu controller)
2. angular: http.js configuration object
{ 'action': 'post', 'listId': miamlist_id, ..}
3. AJAX POST/DELETE
4. "API locale" (=> annotations.views)
5. DB insert/delete
Remarque:
Dans le code annotations d'Elias, il y a une "API locale" qui transmet les actions client vers le serveur.
=> l'interconnexion est configurée pour angular dans annotations/static/annotations/app.js qui lance son propre main sur la fenêtre en prenant les paramètres depuis l'url et en s'isolant de django
=> les routes amont sont définies pour django dans annotations.urls et reprises pour angular dans http.js
#### Par ex: l'étape AJAX pour suppression
`curl -XDELETE http://localhost:8000/annotations/lists/7129/ngrams/4497`
via annotations.views.NgramEdit.as_view())
#### ajout d'un ngram
```
curl -XPOST http://localhost:8000/annotations/lists/1866/ngrams/create \
-H "Content-Type: application/json" \
-d '{"text":"yooooooooo"}' > response_to_ngrams_create.html
```
## Points d'interaction côté client (GUI)
Add Ngram via dialog box :
- controller:
ngramlist.annotationsAppNgramList.controller('NgramInputController')
- effect:
1. NgramHttpService.post()
Add Ngram via select new + menu
- controller:
highlight.annotationsAppHighlight.controller('TextSelectionMenuController')
1. toggleMenu (sets action = X)
2. onMenuClick
- effect:
1. NgramHttpService[action]
annotations/static/annotations/activelists.js
View file @ 1e39ea34
......@@ -24,11 +24,14 @@
});
});
// FIXME: est-ce qu'on ne pourrait pas directement utiliser lists
// au lieu de recopier dans allListsSelect ?
$rootScope.$watchCollection('lists', function (newValue, oldValue) {
if (newValue === undefined) return;
// reformat lists to allListsSelect
var allListsSelect = [];
console.log($rootScope.lists)
// console.log($rootScope.lists)
angular.forEach($rootScope.lists, function(value, key) {
this.push({
'id': key,
......@@ -45,7 +48,7 @@
$timeout(function() {
$('.selectpicker').selectpicker();
$('.selectpicker').selectpicker('val', ['MAINLIST']);
$('.selectpicker').selectpicker('val', ['MAPLIST']);
});
});
......
annotations/static/annotations/app.css
View file @ 1e39ea34
......@@ -6,14 +6,23 @@
*/
.MAPLIST {
color: black;
/* green */
background-color: rgba(60, 118, 61, .7);
cursor: pointer;
}
.MAINLIST {
color: black;
background-color: rgba(60, 118, 61, 0.5);
/* background-color: rgba(60, 118, 61, 0.5); */
background-color: orange;
cursor: pointer;
}
.STOPLIST {
color: black;
/* grey */
background-color: rgba(169, 68, 66, 0.2);
cursor: pointer;
}
......
annotations/static/annotations/document.js
View file @ 1e39ea34
......@@ -8,6 +8,8 @@
// dataLoading = signal pour afficher wait
$scope.dataLoading = true ;
console.log("annotations.document.DocController.DocumentHttpService.get():before")
$rootScope.documentResource = DocumentHttpService.get(
{'docId': $rootScope.docId},
......@@ -21,6 +23,7 @@
$rootScope.docId = data.id;
$rootScope.full_text = data.full_text;
$rootScope.abstract_text = data.abstract_text;
console.log("annotations.document.DocController.getannotations")
// GET the annotationss
NgramListHttpService.get(
{
......
annotations/static/annotations/highlight.js
View file @ 1e39ea34
......@@ -9,19 +9,34 @@
annotationsAppHighlight.controller('TextSelectionController',
['$scope', '$rootScope', '$element',
function ($scope, $rootScope, $element) {
// dbg: apparently no data sent throught local scope, just using $element[0] attribute uuid to attach
// console.log('TextSelectionController $scope.$id: ' + $scope.$id)
// grand parent should be the rootscope
//console.log($scope.$parent.$parent.$id)
// (prepared once, after highlight)
// (then used when onClick event)
// retrieve corresponding ngram using element attr uuid in <span uuid="42">
// FIXME maybe use angular.copy of the annotation
var keyword = _.find(
$rootScope.annotations,
function(annotation) { return annotation.uuid.toString() === $element[0].getAttribute('uuid').toString(); }
);
// attach the annotation scope dynamically
if (keyword) {
// console.log('TextSelectionController found highlighted keyword annotation: ' + keyword.text)
keyword.romdebuginfo = "source = TextSelectionController" ;
$scope.keyword = keyword;
}
$scope.onClick = function(e) {
$rootScope.$emit("positionAnnotationMenu", e.pageX, e.pageY);
$rootScope.$emit("toggleAnnotationMenu", $scope.keyword);
// $rootScope.$emit("toggleAnnotationMenu", {'uuid':42,'list_id':1,'text':'gotcha'});
console.log("EMIT toggleAnnotationMenu with \$scope.keyword: '" + $scope.keyword.text +"'")
e.stopPropagation();
};
}]);
......@@ -71,13 +86,30 @@
function toggleMenu(context, annotation) {
$timeout(function() {
$scope.$apply(function() {
// £TODO check
var miamlist_id = _.invert($rootScope.lists).MAINLIST;
var stoplist_id = _.invert($rootScope.lists).STOPLIST;
var maplist_id = _.invert($rootScope.lists).MAPLIST;
// if called from highlighted span
// - annotation has full {ngram}
// - context has properties:
// name,targetScope,currentScope, ...
// if called from new selected text
// - annotation has just a string
// - context is null
// variable used in onClick
$scope.selection_text = angular.copy(annotation);
// debug
// console.log("toggleMenu with \$scope.selection_text: '" + JSON.stringify($scope.selection_text) +"'") ;
if (angular.isObject(annotation) && !$element.hasClass('menu-is-opened')) {
// existing ngram
console.log("toggleMenu.annotation: '" + JSON.stringify(annotation) +"'")
// Delete from the current list
$scope.menuItems = [
{
......@@ -87,8 +119,24 @@
'listName': $rootScope.lists[annotation.list_id]
}
];
// Context menu proposes 3 things for each item of list A
// - deletion from A
// - adding/moving to other lists B or C
// Because of logical dependencies b/w lists, these
// menu actions will also trigger todo_other_actions
// cf. forge.iscpif.fr/projects/garg/wiki/Ngram_Lists
// TODO disambiguate annotation.list_id for highlighted MapList items
// -------------------------------------------------------------
// Because MapList annotations are also MiamList,
// we should ensure that list_id is indeed "MapList"
// (ie that it was added last in CompileNgramsHtml)
// otherwise the "if" here will propose MiamList's options
if ($rootScope.lists[annotation.list_id] == "MAPLIST") {
// Add to the alternative list
// Add to the other lists
$scope.menuItems.push({
'action': 'post',
'listId': stoplist_id,
......@@ -130,6 +178,80 @@
});
}
// £TODO CHECK
// if ($rootScope.lists[annotation.list_id] == "MiamList") {
// // Add to the other lists
// $scope.menuItems.push({
// 'action': 'post',
// 'listId': maplist_id,
// // "Add" because copy into MapList
// 'verb': 'Add to',
// 'listName': $rootScope.lists[maplist_id]
// },
// {
// 'action': 'post',
// 'listId': stoplist_id,
// // "Move"
// // £dbg: TODO for instance pass conditional dependancy as info
// // 'todo_other_actions': 'remove from miam',
// 'verb': 'Move to',
// 'listName': $rootScope.lists[stoplist_id]
// });
// } else if ($rootScope.lists[annotation.list_id] == "StopList") {
// // Move from stop to the "positive" lists
// $scope.menuItems.push({
// 'action': 'post',
// 'listId': miamlist_id,
// // 'todo_other_actions': 'remove from stop',
// 'verb': 'Move to',
// 'listName': $rootScope.lists[miamlist_id]
// },
// {
// 'action': 'post',
// 'listId': maplist_id,
// // 'todo_other_actions': 'remove from stop, add to miam',
// 'verb': 'Move to',
// 'listName': $rootScope.lists[maplist_id]
// });
// } else if ($rootScope.lists[annotation.list_id] == "MapList") {
// // No need to add to miam, just possible to move to stop
// $scope.menuItems.push({
// 'action': 'post',
// 'listId': stoplist_id,
// // 'todo_other_actions': 'remove from miam and from map'
// 'verb': 'Move to',
// 'listName': $rootScope.lists[stoplist_id]
// });
// }
//
//
//
var pos = $(".text-panel").position();
function positionElement(context, x, y) {
......@@ -160,6 +282,9 @@
/*
* Toggle the menu when clicking on an existing ngram keyword
*
* £TODO test: apparently this is never used ?
* (superseded by TextSelectionController.onClick)
*/
$(".text-container").delegate(':not("#selection")', "click", function(e) {
// if ($(e.target).hasClass("keyword-inline")) return;
......@@ -175,7 +300,14 @@
* Menu click action
*/
$scope.onMenuClick = function($event, action, listId) {
// TODO interpret context menu chosen actions
// as implicit API+DB actions
// ex: add to map (+ add to miam + delete from stop)
if (angular.isObject($scope.selection_text)) {
console.log("requested action: " + action + "on scope.sel: '" + $scope.selection_text + "'")
// action on an existing Ngram
NgramHttpService[action]({
'listId': listId,
......@@ -246,10 +378,19 @@
* Replace the text by an html template for ngram keywords
*/
function replaceTextByTemplate(text, ngram, template, pattern) {
// TODO prevent matching inside XML tags
// (now if term=='class' it highlights XML class attrs ><)
// other interfering terms: ng, span, click, keyword, inline...
// suggestion by mrodic: take the original text and replace for all act
// but problem: it's not multiple lists it's previous replacements in t
// solution: use a tag ersatz here like <xmlspan> </xmlspan>
// then replace all afterwards with real xml <span ...> tags
return text.replace(pattern, function(matched) {
var tpl = angular.element(template);
tpl.append(matched);
tpl.attr('title', ngram.tooltip_content);
tpl.attr('title', "Click to add/remove");
tpl.attr('uuid', ngram.uuid);
/*
* Add CSS class depending on the list the ngram is into
......@@ -260,6 +401,14 @@
});
}
/* Escape text before it's inserted in regexp (aka quotemeta)
* ex: "c++" => "c\+\+"
*   so the '+' won't act as regexp operator
*/
function escapeRegExp(string){
return string.replace(/([.*+?^${}()|\[\]\/\\])/g, "\\$1");
}
/*
* Sorts annotations on the number of words
* Required for overlapping ngrams
......@@ -288,6 +437,8 @@
var middlePattern = "(?:<\/span>)*\\s(?:"+templateBeginRegexp+")*";
var endPattern = "(?:<\/span>)*)\\b";
console.log("highlight annotations length: " + annotations.length)
var sortedSizeAnnotations = lengthSort(annotations, "text"),
extraNgramList = angular.copy($rootScope.extraNgramList);
......@@ -296,34 +447,56 @@
extraNgramList[id] = [];
});
// rl: £dbg counters
var i = 0 ;
var j = 0 ;
var k = 0 ;
angular.forEach(sortedSizeAnnotations, function (annotation) {
i ++ ;
// console.log('----------------\n')
// console.log('sortedSizeAnnotations n° ' + i + ': \n ' + JSON.stringify(annotation) +'\n')
// exclude ngrams that are into inactive lists
if ($rootScope.activeLists[annotation.list_id] === undefined) return;
j ++ ;
// used to setup css class
annotation.listName = $rootScope.lists[annotation.list_id];
// regexps
var words = annotation.text.split(" ");
// regexps (with escaped content)
var words = annotation.text.split(" ").map(escapeRegExp);
var pattern = new RegExp(startPattern + words.join(middlePattern) + endPattern, 'gmi');
var textRegexp = new RegExp("\\b"+annotation.text+"\\b", 'igm');
var isDisplayedIntraText = false;
var textRegexp = new RegExp("\\b"+escapeRegExp(annotation.text)+"\\b", 'igm');
// highlight text as html
// -----------------------
// text content taken in argument textMapping:
// {'#title': 'some text',
// '#abstract-text': 'some text',
// '#full-text': 'some text' }
angular.forEach(textMapping, function(text, eltId) {
// console.log('textMapping for "' + eltId + '" : ' + JSON.stringify(text))
// £TODO pas besoin de pattern.test avant pattern.sub !
if (pattern.test(text) === true) {
textMapping[eltId] = replaceTextByTemplate(text, annotation, template, pattern);
isDisplayedIntraText = true;
k ++ ;
textMapping[eltId] = replaceTextByTemplate(
text,
annotation,
template,
pattern);
}
});
if (!isDisplayedIntraText) {
// add extra-text ngrams that are not already displayed
if ($.inArray(annotation.uuid, extraNgramList[annotation.list_id].map(function (item) {
return item.uuid;
})) == -1) {
// push the ngram and sort
extraNgramList[annotation.list_id] = extraNgramList[annotation.list_id].concat(annotation);
}
}
// rloth: for now let's show *all* ngrams of the active list
// in the left side
extraNgramList[annotation.list_id] = extraNgramList[annotation.list_id].concat(annotation);
});
// rl: £dbg
console.log('---- compileNgramsHtml created '
+ k + ' series of highlighted spans from '
+ j + ' annotations in activeLists (of ' + i + ' annotations total) ----\n')
// update extraNgramList
$rootScope.extraNgramList = angular.forEach(extraNgramList, function(name, id) {
extraNgramList[id] = lengthSort(extraNgramList[id], 'text');
......@@ -333,6 +506,7 @@
}
$rootScope.refreshDisplay = function() {
console.log("annotations.highlight.refreshDisplay()")
if ($rootScope.annotations === undefined) return;
if ($rootScope.activeLists === undefined) return;
if (_.keys($rootScope.activeLists).length === 0) return;
......
annotations/static/annotations/http.js
View file @ 1e39ea34
......@@ -8,7 +8,28 @@
$httpProvider.defaults.xsrfCookieName = 'csrftoken';
}]);
/*
* Read Document
* DocumentHttpService: Read Document
* ===================
*
* route: annotations/documents/@d_id
* ------
*
* exemple:
* --------
* {
* "id": 556,
* "publication_date": "01/01/66",
* "title": "Megalithic astronomy: Indications in standing stones",
* "abstract_text": "An account is given of a number of surveys of
* stone circles, alignments, etc., found in Britain.
* The geometry of the rings is discussed in so far
* as it affects the determination of the azimuths
* to outliers and other circles.",
* "full_text": null,
* "journal": "Vistas in Astronomy",
* "authors": "A. Thom"
* }
*
*/
http.factory('DocumentHttpService', function($resource) {
return $resource(
......@@ -26,7 +47,28 @@
});
/*
* Read all Ngrams
* NgramListHttpService: Read all Ngrams
* =====================
*
* route: annotations/corpora/@c_id/documents/@d_id
* ------
*
* json return format:
* -------------------
* corpus_id : {
* lists: {(list_id:name)+}
* doc_id : [ngrams_objects]+,
* }
*
* exemple:
* --------
* "554": {
* "lists": { "558": "StopList", "564": "MiamList", "565": "MapList" }
* "556": [{ "uuid": 2368, "occurrences": 1.0, "text": "idea", "list_id": 564 },
* { "uuid": 5031, "occurrences": 1.0, "text": "indications", "list_id": 564},
* { "uuid": 5015, "occurrences": 3.0, "text": "star", "list_id": 565 },
* ... ],
* }
*/
http.factory('NgramListHttpService', function ($resource) {
return $resource(
......@@ -45,7 +87,21 @@
});
/*
* Create, modify or delete 1 Ngram
* NgramHttpService: Create, modify or delete 1 Ngram
* =================
*
* £TODO REACTIVATE IN urls.py
*
* if new ngram:
* -> ngram_id will be "create"
* -> route: annotations/lists/@node_id/ngrams/create
* -> will land on views.NgramCreate
*
* else:
* -> ngram_id is a real ngram id
* -> route: annotations/lists/@node_id/ngrams/@ngram_id
* -> will land on views.NgramCreate
*
*/
http.factory('NgramHttpService', function ($resource) {
return $resource(
......
annotations/static/annotations/ngramlist.js
View file @ 1e39ea34
......@@ -24,7 +24,13 @@
'docId': $rootScope.docId
},
function(data) {
// $rootScope.annotations
// ----------------------
// is the union of all lists, one being later "active"
// (then used for left-side flatlist AND inline annots)
$rootScope.annotations = data[$rootScope.corpusId.toString()][$rootScope.docId.toString()];
// TODO £NEW : lookup obj[list_id][term_text] = {terminfo}
// $rootScope.lookup =
$rootScope.refreshDisplay();
},
function(data) {
......@@ -87,7 +93,28 @@
var value = angular.element(inputEltId).val().trim();
if (value === "") return;
// £TEST locally check if already in annotations NodeNgrams ------
// $rootScope.annotations = array of ngram objects like:
// {"list_id":805,"occurrences":2,"uuid":9386,"text":"petit échantillon"}
console.log('looking for "' + value + '" in list:' + listId)
var already_in_list = false ;
angular.forEach($rootScope.annotations, function(annot,i) {
// console.log(i + ' => ' + annot.text + ',' + annot.list_id) ;
if (value == annot.text && listId == annot.list_id) {
console.log('the term "' + value + '" was already present in list')
// no creation
already_in_list = true ;
}
}
);
if (already_in_list) { return ; }
// ---------------------------------------------------------------
// will check if there's a preexisting ngramId for this value
// TODO: if maplist => also add to miam
NgramHttpService.post(
{
'listId': listId,
......@@ -97,6 +124,7 @@
'text': value
},
function(data) {
console.warn("refresh attempt");
// on success
if (data) {
angular.element(inputEltId).val("");
......@@ -108,6 +136,11 @@
},
function(data) {
$rootScope.annotations = data[$rootScope.corpusId.toString()][$rootScope.docId.toString()];
// TODO £NEW : lookup obj[list_id][term_text] = {terminfo}
// $rootScope.lookup =
$rootScope.refreshDisplay();
},
function(data) {
......
annotations/templates/annotations/main.html
View file @ 1e39ea34
......@@ -55,10 +55,14 @@
</div>
</div>
</div>
<div>
<div class="list-selector">
<h5>Select lists</h5>
<select class="selectpicker" multiple ng-change="activeListsChange()" ng-model="lists" ng-controller="ActiveListsController">
<option ng-repeat="item in allListsSelect" id="list---{[{item.id}]}" ng-disabled="{[{ item.label == 'MAINLIST' }]}">{[{item.label}]}</option>
<option ng-repeat="item in allListsSelect" id="list---{[{item.id}]}">{[{item.label}]}</option>
<!-- to disallow unchecking MapList add this into <option> element: ng-disabled="{[{ item.label == 'MapList' }]}" -->
</select>
</div>
</div>
......
annotations/urls.py
View file @ 1e39ea34
......@@ -10,6 +10,10 @@ urlpatterns = [
# publication_date
# abstract_text,full_text
url(r'^documents/(?P<doc_id>[0-9]+)$', views.Document.as_view()), # document view
# GET:
# was : lists ∩ document (ngram_ids intersection if connected to list node_id and doc node_id)
# fixed 2016-01: just lists (because document doesn't get updated by POST create cf. ngram.lists.DocNgram filter commented)
url(r'^corpora/(?P<corpus_id>[0-9]+)/documents/(?P<doc_id>[0-9]+)$', views.NgramList.as_view()), # the list associated with an ngram
# 2016-03-24: refactoring, deactivated NgramEdit and NgramCreate
......
annotations/views.py
View file @ 1e39ea34
......@@ -34,33 +34,51 @@ def main(request, project_id, corpus_id, document_id):
}, context_instance=RequestContext(request))
class NgramList(APIView):
"""Read and Write Annotations"""
"""Read the lists of ngrams (terms) that will become annotations"""
renderer_classes = (JSONRenderer,)
def get(self, request, corpus_id, doc_id):
"""Get All for a doc id"""
corpus_id = int(corpus_id)
doc_id = int(doc_id)
# our results: ngrams for the corpus_id (ignoring doc_id for the moment)
doc_ngram_list = []
lists = {}
for list_type in ['MAINLIST']:
for list_type in ['MAINLIST', 'MAPLIST', 'STOPLIST']:
corpus_nod = cache.Node[corpus_id]
list_nod = corpus_nod.children(typename=list_type).first()
list_id = list_nod.id
lists["%s" % list_id] = list_type
# ngrams for the corpus_id (ignoring doc_id for the moment):
doc_ngram_list = [(obj.id, obj.terms, w) for (w,obj) in list_nod.ngrams.all()]
# add to results
doc_ngram_list += [(obj.id, obj.terms, w, list_id) for (w,obj) in list_nod.ngrams.all()]
print("annotations.views.NgramList.doc_ngram_list: ", doc_ngram_list)
data = { '%s' % corpus_id : {
'%s' % doc_id : [
{
'uuid': ngram_id,
'text': ngram_text,
'occurrences': ngram_occurrences,
'list_id': list_id,
}
for ngram_id, ngram_text, ngram_occurrences in doc_ngram_list],
'%s' % doc_id :
[
{'uuid': ngram_id,
'text': ngram_text,
'occurrences': ngram_occurrences,
'list_id': list_id,}
for (ngram_id,ngram_text,ngram_occurrences,list_id) in doc_ngram_list
],
'lists': lists
}}
# format alternatif de transmission des "annotations", classé par listes puis ngram_id
# { 'corpus_id' : {
# list_id_stop: {term_stop1: {term_data}, term_stop2: {term_data}..},
# list_id_miam: {term_miam1: {term_data}, term_miam2: {term_data}..},
# list_id_map: {term_map1: {term_data}, term_map2: {term_data}..},
# }
# 'lists' : {"list_id" : "list_type" ... }
# }
# NB 3rd possibility: unicity of ngram_text could also allow us to use it
# as key and could enhance lookup later (frequent checks if term exists)
return Response(data)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment