Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Mohamed Khemakhem

Date de publication:

29 Août 2017

Mots-clefs:

digitized dictionaries
automatic structuring
CRF
TEI
machine
Learning

Publié ici :

30 Août 2017

Mis à jour le :

05 Septembre 2017

Langue de publication:

Anglais

Zone géographique:

Europe

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Par Mohamed Khemakhem
Communication dans un congrès, electronic lexicography, eLex 2017, Sep 2017, Leiden, Netherlands

Abstract : An important number of digitized lexical resources remain unexploited due to their unstructured content. Manually structuring such resources is a costly task given their multifold complexity. Our goal is to find an approach to automatically structure digitized dictionaries, independently from the language or the lexicographic school or style. In this paper we present a first version of GROBID-Dictionaries1, an open source machine learning system for lexical information extraction. Our approach is twofold: we perform a cascading structure extraction, while we select at each level specific features for training. We followed a ”divide to conquer” strategy to dismantle text constructs in a digitized dictionary, based on the observation of their layout. Main pages (see Figure 1) in almost any dictionary share three blocks: a header (green), a footer (blue) and a body (orange). The body is, in its turn, constituted by several entries (red). Each lexical entry can be further decomposed (see Figure 2) as: form (green), etymology (blue), sense (red) or/and related entry. The same logic could be applied further for each extracted block but in the scope of this paper we focus just on the first three levels. The cascading approach ensures a better understanding of the learning process’s output and consequently simplifies the feature selection process. Limited exclusive text blocks per level helps significantly in diagnosing the cause of prediction errors. It allows an early detection and replacement of irrelevant selected features that can bias a trained model. In such a segmentation, it becomes more straightforward to notice that, for instance, the token position in the page is very relevant to detect headers and footers and has almost no pertinence for capturing a sense in a lexical entry which is very often split on two pages. To implement our approach, we took up the available infrastructure from GROBID [7], a machine learning system for the extraction of bibliographic metadata. GROBID adopts the same cascading approach and uses Conditional Random Fields (CRF) [6] to label text sequences. The output of Grobid dictionary is planned to generate a TEI compliant encoding [2, 9] where the various segmentation levels are associated with an appropriate XML tessellation. Collaboration with COST ENeL are ongoing to ensure maximal compatibility with existing dictionary projects. Our experiments justify so far our choices, where models for the first two levels trained on two different dictionary samples have given a high precision and recall with a small amount of annotated data. Relying mainly on the text layout, we tried to diversify the selected features for each model, on the token and line levels. We are working on tuning features and annotating more data to maintain the good results with new samples and to improve the third segmentation level. While just few task specific attempts [1] have been using machine learning in this research direction, the landscape remains dominated by rule based techniquess [4, 3, 8] which are ad-hoc and costly, even impossible, to adapt for new lexical resources.

digitized dictionaries automatic structuring CRF TEI machine learning

Dernières actualités de l'IFRE

CMB Political violence D’un savoir en crise à un savoir indiscipliné... À travers la variété de formes qu’il est susceptible d’ad Lire la suite	CMB offre d'emploi Directeur adjoint d'établissement cultur... Description synthétique du poste Lire la suite	CMB La scolarité des enfants placés : quels levie... Les recherches nationales et internationales convergent p Lire la suite
CMB Camus et la montée aux extrêmes. Commentaires... Les controverses sur le rapport d’Albert Camus à la viole Lire la suite	CMB Que faire des interventions militaires dans l... Des segments du champ académique sont en partie financés Lire la suite	CMB Work in London, love in Paris: middle class m... Our paper contributes to the analysis of the transformati Lire la suite
CMB La scolarité des enfants placés : quels levie... Les recherches nationales et internationales convergent p Lire la suite	CMB L'éclatement de la norme d'emploi e... Cet article propose une analyse comparative des transform Lire la suite	CMB appel à contributions Faire face à la défaite (1870-1945) Soixante-... 05 octobre 11h00 Lire la suite

Derniers contenus par mots-clefs

CMB TEI Enhancing Usability for Automatically Structuring... Enhancing Usability for Automatically Structuring Digitised... Lire la suite	CMB Machine learning Automatic Extraction of TEI Structures in Digitized... Automatic Extraction of TEI Structures in Digitized Lexical... Lire la suite	CMB LMF ALIF editor for generating Arabic normalized lexicons ALIF editor for generating Arabic normalized lexicons... Lire la suite
CEDEJ Health facilities SEMINAR - “Interaction between health Institutions in... On Sunday 31st January, Mariam Sharif presented in CEDEJ... Lire la suite	CEDEJ Health facilities Interaction between health Institutions in knowledge... On Sunday 31st January, Mariam Sharif has presented in... Lire la suite	CEDEJ Gestion Perceptions and management of gender roles and... A number of anti-sexual harassment and anti-sexual violence... Lire la suite
IFRA/SHS Fruit Gerrit Deutschländer, Dienen lernen, um zu herrschen.... Fruit d’une Dissertation soutenue en 2009 à l’université de... Lire la suite	CEFC Chine China and the WTO Dispute Settlement System Since its accession to the WTO on 11 December 2001, China... Lire la suite	CMB copia How do we copy and paste? The semantic drift of... We describe reformulation processes within a large... Lire la suite

UMIFRE

Unités mixtes des instituts français de recherche à l'étranger

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Date de publication:

Mots-clefs:

Publié ici :

Mis à jour le :

Langue de publication:

Zone géographique:

Visiter le site web

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Dernières actualités de l'IFRE

Derniers contenus par mots-clefs

Formulaire de recherche

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Auteur:

Date de publication:

Mots-clefs:

Publié ici :

Mis à jour le :

Langue de publication:

Zone géographique:

Visiter le site web

Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields

Dernières actualités de l'IFRE

Derniers contenus par mots-clefs

Twitter

Facebook