Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Mohamed Khemakhem

Luca Foppiano

Laurent Romary

Date de publication:

01 Avril 2017

Mots-clefs:

Machine learning
CRF
TEI
automatic structuring
digitized dictionaries

Publié ici :

19 Avril 2017

Mis à jour le :

21 Juin 2017

Langue de publication:

Anglais

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Mohamed Khemakhem, Luca Foppiano, Laurent Romar

An important number of digitized lexical resources remain unexploited due to their unstructured content. Manually structuring such resources is a costly task given their multifold complexity. Our goal is to find an approach to automatically structure digitized dictionaries, independently from the language or the lexicographic school or style. In this paper we present a first version of GROBID-Dictionaries1, an open source machine learning system for lexical information extraction. Our approach is twofold: we perform a cascading structure extraction, while we select at each level specific features for training. We followed a ”divide to conquer” strategy to dismantle text constructs in a digitized dictionary, based on the observation of their layout. Main pages (see Figure 1) in almost any dictionary share three blocks: a header (green), a footer (blue) and a body (orange). The body is, in its turn, constituted by several entries (red). Each lexical entry can be further decomposed (see Figure 2) as: form (green), etymology (blue), sense (red) or/and related entry. The same logic could be applied further for each extracted block but in the scope of this paper we focus just on the first three levels. The cascading approach ensures a better understanding of the learning process’s output and consequently simplifies the feature selection process. Limited exclusive text blocks per level helps significantly in diagnosing the cause of prediction errors. It allows an early detection and replacement of irrelevant selected features that can bias a trained model. In such a segmentation, it becomes more straightforward to notice that, for instance, the token position in the page is very relevant to detect headers and footers and has almost no pertinence for capturing a sense in a lexical entry which is very often split on two pages. To implement our approach, we took up the available infrastructure from GROBID [7], a machine learning system for the extraction of bibliographic metadata. GROBID adopts the same cascading approach and uses Conditional Random Fields (CRF) [6] to label text sequences. The output of Grobid dictionary is planned to generate a TEI compliant encoding [2, 9] where the various segmentation levels are associated with an appropriate XML tessellation. Collaboration with COST ENeL are ongoing to ensure maximal compatibility with existing dictionary projects. Our experiments justify so far our choices, where models for the first two levels trained on two different dictionary samples have given a high precision and recall with a small amount of annotated data. Relying mainly on the text layout, we tried to diversify the selected features for each model, on the token and line levels. We are working on tuning features and annotating more data to maintain the good results with new samples and to improve the third segmentation level. While just few task specific attempts [1] have been using machine learning in this research direction, the landscape remains dominated by rule based techniquess [4, 3, 8] which are ad-hoc and costly, even impossible, to adapt for new lexical resources.

Dernières actualités de l'IFRE

CMB Political violence D’un savoir en crise à un savoir indiscipliné... À travers la variété de formes qu’il est susceptible d’ad Lire la suite	CMB offre d'emploi Directeur adjoint d'établissement cultur... Description synthétique du poste Lire la suite	CMB La scolarité des enfants placés : quels levie... Les recherches nationales et internationales convergent p Lire la suite
CMB Camus et la montée aux extrêmes. Commentaires... Les controverses sur le rapport d’Albert Camus à la viole Lire la suite	CMB Que faire des interventions militaires dans l... Des segments du champ académique sont en partie financés Lire la suite	CMB Work in London, love in Paris: middle class m... Our paper contributes to the analysis of the transformati Lire la suite
CMB La scolarité des enfants placés : quels levie... Les recherches nationales et internationales convergent p Lire la suite	CMB L'éclatement de la norme d'emploi e... Cet article propose une analyse comparative des transform Lire la suite	CMB appel à contributions Faire face à la défaite (1870-1945) Soixante-... 05 octobre 11h00 Lire la suite

Derniers contenus par mots-clefs

CMB Onomastics Large-scale diversity estimation through surname... By Antoine Mazieres (CMB; LISIS), Camille Roth (CMB;... Lire la suite	CMB TEI Enhancing Usability for Automatically Structuring... Enhancing Usability for Automatically Structuring Digitised... Lire la suite	CMB digitized dictionaries Automatic Extraction of TEI Structures in Digitized... Automatic Extraction of TEI Structures in Digitized Lexical... Lire la suite
CMB LMF ALIF editor for generating Arabic normalized lexicons ALIF editor for generating Arabic normalized lexicons... Lire la suite	CMB regresión Symbolic regression of generative network models International audience Lire la suite	CMB regresión Symbolic regression of generative network models International audience Lire la suite
CMB regresión [halshs-01101038] Symbolic regression of generative... Networks are a powerful abstraction with applicability to a... Lire la suite	CMB regresión [hal-01101023] Symbolic regression of generative... Networks are a powerful abstraction with applicability to a... Lire la suite

UMIFRE

Unités mixtes des instituts français de recherche à l'étranger

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Date de publication:

Mots-clefs:

Publié ici :

Mis à jour le :

Langue de publication:

Visiter le site web

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Dernières actualités de l'IFRE

Derniers contenus par mots-clefs

Formulaire de recherche

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Date de publication:

Mots-clefs:

Publié ici :

Mis à jour le :

Langue de publication:

Visiter le site web

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Dernières actualités de l'IFRE

Derniers contenus par mots-clefs

Twitter

Facebook