Chapter 3 Doccano-Semantics
Having outlined the key ingredients for the data transformation pipeline1, this chapter focuses on the various system components, up to the final transformation of the legal provisions in the Lynx format specification (Rodríguez-Doncel et al. 2023). Finally, it discusses the results and compares the data transformation pipeline with other existing tools for the semantic annotation of legal texts.
3.1 System description
The data transformation pipeline, as illustrated in Figure 3.1, is designed to enable seamless end-to-end semantic annotation. It encompasses the following steps:
Parse files from Akoma Ntoso format: In this initial stage, we extract the legal provisions at the paragraph level from files structured in Akoma Ntoso, while preserving relevant metadata, such as paragraph, article, and chapter number, the presence of insertions or reference, and others. This components outputs the provisions in a JSONLines file2, required for the upload via the Doccano web interface.
Parse relevant ontologies and controlled vocabularies into a Doccano-compatible format (JSON): Doccano does not support ontological classes and controlled vocabularies out of the box, but it has an "import labels" functionality that requires JSON files with the following attributes: label, suffix_key, text_color, and background_color.
Annotate using the Doccano tool: Doccano provides a user-friendly interface where labels and relationships are easy to find in the respective ontologies. Appropriate user-interface and database adaptations of Doccano’s open-source code were made necessary, as the tool did not initially support more than 26 different labels. This increased the complexity of the work, as it required an analysis of the architecture of the tool. The tool’s user-friendly interface enables the annotator to mark and categorize important information according to their needs.
Map the Doccano annotated output file to a Lynx format: we transform the annotated JSONL file from Doccano into the Lynx format, an offset-based document format for legal compliance.
3.2 System’s components
In this section, we will present in detail the various components of the data transformation pipeline.
3.2.1 Akoma Ntoso preprocessing
Akoma Ntoso is one of the major standards used for structuring machine-readable legal documents across a variety of legislation. While some applications support it out-of-the box, Doccano is not among them. Therefore, it is key to transform these files in a format suitable for it (JSONLines). This takes place by calling the AkomaNtosoParser class, which is instantiated with the file path of the Akoma Ntoso file and then leverages the ElementTree library to parse it. The component undergoes the following sequential steps:
Parsing Chapters: the extract_chapters method parses chapters’ titles and ids and stores them as a DataFrame in an attribute of the class instance
Parsing Articles: the extract_articles method parses the Akoma Ntoso file at the article level and stores them as a list in an attribute of the class instance
Parsing Paragraphs: the extract_paragraphs method parses the paragraph attribute of the class instance and stores them as a DataFrame in an attribute of the class instance
Creating the DataFrame: the create_dataframe method merges all the information regarding chapters, articles and paragraph in a single DataFrame and stores it in an attribute of the class instance
Data Export: the export_jsonl method converts the stored DataFrame in JSON writes it to a JSONLines file.
3.2.2 Label preprocessing: OWL and SKOS
3.2.2.1 SKOS
SKOS is a widely-used RDF vocabulary for representing and managing knowledge organization systems. SKOS files can contain valuable information, and extracting top concepts is crucial for effective knowledge representation and visualization. This work introduces SKOSParser, a Python component designed to parse SKOS files and identify top concepts.
The SKOSParser class is instantiated with the file path to the SKOS file in XML format. The component employs the rdflib library to create a graph object, which parses and represents the SKOS file. Subsequently, the SKOSParser performs the following key functionalities:
Parsing and Extracting Top Classes: The parse_skos method parses the SKOS file, extracting essential concepts, and filters out top classes by considering SKOS:Concept instances without SKOS:topConceptOf relationships. The parser retrieves the concept’s URI, preferred label (in English), and an associated prefix. The top classes are then stored in a structured format as a list of dictionaries.
Enrichment with Random Colors: The enrich_json method processes the extracted top concepts, appending a human-readable label by concatenating the concept’s prefix and preferred label. The method also assigns random background colors to each SKOS file to enhance visual representation during visualization. The enriched data is stored in a standardized JSON format.
Data Export: The SKOSParser class includes a write_to_file method that exports the enriched data to a JSON file compatible for the upload in the Doccano annotation tool.
The SKOSParser component demonstrates a proficient solution for extracting top concepts from SKOS controlled vocabularies. By leveraging Python’s rdflib library, the component efficiently navigates and processes RDF graph data, yielding structured information about top classes.
3.2.2.2 OWL
The OntologyParser is a Python class which is instantiated with the file path to the RDF ontology in Turtle format. The parser utilizes rdflib to create a graph object, subsequently parsing the ontology file to populate the graph. The parser then employs the random_color method to assign a random color to each ontology.
The key functionalities of the OntologyParser are as follows:
Class Extraction: The component traverses the ontology graph to identify classes with RDF type OWL.Class. Utilizing a predefined set of namespaces, the parser associates each class with a human-readable and concise class name, while preserving its full URI reference. The parser stores class information in a structured format, including class names, suffix keys, background colors, text colors, and URIs.
Property Extraction: The parser further identifies RDF Object Properties in the ontology, extracting and organizing them similarly to classes. Utilizing the graph’s triples, the component extracts property names and stores them alongside relevant attributes, such as suffix keys and background colors.
Data Export: The Ontology Parser provides methods to obtain and export extracted class and property data as JSON files, supported for their loading in the Doccano annotation tool.
3.2.3 Adaptation of the Doccano annotation tool
To overcome the challenge of accommodating numerous classes and properties stored in ontologies, we had to adapt a small portion of the Doccano source code. The original web app only supports a limited number of labels (26).
To address this, the Django model LabelType had to be modified by extending the the maximum length of the suffix_key field to 254 to allow for up to 256! different combinations.
This enabled the adoption of the URIs of ontological classes as unique identifiers in the database, thus promoting compatibility with ontological data sources.
3.2.4 Manual annotation
Since the development of a semi-automated system for semantic annotation is beyond the scope of this project, annotations were manually done by the researcher. The main goal was to pinpoint elements in the legislation offering a legal foundation for the copyright ontology. This entailed a methodical approach, aided by the thematic organization of the Italian legislation. For instance, the section "Opere Protette" (Protected Works) outlines what qualifies as a subject of copyright protection, essentially, what is viewed as a creative work. In this section, only two ontological classes were assigned to specific provisions: schema1:CreativeWork and cro:Work. However, the subsequent section, "Soggetti del Diritto" (Subjects of Law), delves into the act of creation, remaining within the Creative Model of the Copyright ontology.
Key observations from the annotation process include the realization that while most declarative provisions aligned directly with Copyright Ontology classes, those indicating specific rules and procedures necessitated a distinct representation, especially concerning elements tied to deontic logic. This indicates a potential need to integrate with other ontologies like LegalRuleML or LKIF (Legal Knowledge Interchange Format). Such integration implies a layered, multi-step annotation process, enabling diverse machine-readable representations to accurately capture nuances in natural language. Legal texts’ inherent ambiguity allows them to abstractly convey intricate information, presenting a challenge when trying to express such content explicitly.
The resulting annotated dataset primarily aids in targeted retrieval. Enriching the original Copyright Ontology with foundational details from Italian legislation streamlines the search process, though the benefits currently appear modest. However, expanding the annotation to multiple legislations would likely prove useful, granting legal scholars direct access to shared concepts across different jurisdictions.
3.2.5 Representing Legal concepts in Lynx
After having completed the semantic annotation of the examined legal documents, the last step of the pipeline concerns their export in a machine-readable a reusable format. In this context, we identified the Legal Knowledge Graph Ontology (Rodríguez-Doncel et al. 2023), developed in the context of the Lynx project (Victor Rodriguez-Doncel 2023) as a promising candidate format for encoding the annotations in a machine-readable format.
The Lynx specification was developed to provide a suitable standard for representing documents related to compliance, through the class LynxDocument. Lynx Documents adhere to the NIF (NLP Interchange Format) specification and extensively incorporate ELI metadata elements. Additionally, they can be assembled into collections and augmented with annotations.
This system component is centered around the LynxDocument class. It extracts entities and their metadata from annotated text documents and convert them into a RDF format according to the Lynx specification. The key functionalities are as follows:
Annotation Conversion: LynxDocument reads and processes Akoma Ntoso and SKOS elements from a JSON file, converting them into a format suitable for RDF graph representation. The Akoma Ntoso elements define metadata, while the SKOS elements represent predefined concepts and their URIs.
Graph Initialization: LynxDocument initializes an RDF graph, binding relevant prefixes to namespaces used in the graph representation. The SKOS and Akoma Ntoso elements are then loaded into the graph.
Document Representation: LynxDocument generates a URI for each document based on its identifier and type. The tool adds document metadata, such as language and identifiers at the paragraph level, if available in the Akoma Ntoso elements.
Text and Annotation Addition: LynxDocument extracts the text from the annotated document and adds it to the RDF graph. The tool annotates entities within the text by creating RDF nodes for each entity span.
Serialization: LynxDocument serializes the generated RDF graph into Turtle format.
3.3 Discussion
3.3.1 Limitations
The data pipeline named doccano-semantics, has several limitations:
Annotation capabilities: as of now, the system supports only manual annotation. While this ensures precision and attention to detail, we recognize the potential for automated or semi-automated annotation tools in the future.
User interface: the current user interface in Doccano is functional, but there’s room for further refinement, in particular for the management of the ontology labels and the integration of the command line utilities directly in the user interface.
Limited support for Akoma Ntoso documents: the system is tailored to parse Italian Akoma Ntoso documents, as there are still slighly different implementations of the standard across jurisdiction.
Limited support for output format: currently, the only output format available is the Lynx format. This can be restrictive for users who are looking for different markup languages. Expanding our output format options would increase the overall utility of the system.
3.3.2 Overview of existing platforms
Several annotation tools exist for the legal domain, therefore extensive comparison with equivalent platforms would be a complex and time-intensive task, therefore for the purpose of this project work, it was conducted a literature review and comparison of functionalities and markup languages supported.
The research group working in the University of Bologna has been developing a series of utility tools3 supporting their sponsored specifications4 as well as annotation platforms.
RAWE is a web-based WYSIWYG editor designed for legal experts. It enables the modeling of Akoma Ntoso legal texts in deontic rules structured as valid LegalRuleML, as well as the inclusion of other external rules (Palmirani et al. 2013). While promising, it is does not allow the upload of user-defined ontologies, as doccano-semantics.
Notable is also the LIME platform, which is able to parse Akoma Ntoso documents and provide some form of NER-based automatic annotation. However, there is little support for manual annotation and the inclusion of user-defined ontologies or vocabularies, a core feature of doccano-semantics.
An interesting development is the University of Auckland’s "LegalRuleML editor with transformer-based autocompletion" (Fuchs et al. 2023), showcased in 2023 at the European Conference on Computing in Construction. However, although this platform provides semantic-aware translation of legal rules in a machine-consumable language, it does not really annotate the legal texts (as doccano-semantics does), which is a key element of explainability. In any case, the use of LLMs for semantic annotation is a promising field of research.
Finally, the Maastrict University’s Law & Tech lab is carrying on the Lawnotation project (2022-2024), which aims to develop an infrastructure that allows researchers to making legal data and annotation schemes (current and future) accessible for annotation and analysis purposes, and to develop an annotation platform for analyzing the linguistic and legal characteristics of legal documents. There is a proof-of-concept but access to external users is still limited. However, the tool shares the same goals are in line with the ones of doccano-semantics and it leverages the same frontend framework (Nuxt.js),
While these platforms are considered promising, the complexities of legal texts are numerous and at this stage, we lack a unified platform that adequately addresses them (Huang, Lin, and Liu 2022; Reeve and Han 2005).
3.3.3 Ethical aspects
The doccano-semantics system, at its current stage, does not necessitate an ethical impact assessment as it is not fundamentally based on AI technologies, rather on manual annotation. The system primarily aims to facilitate best practices of data quality for subsequent AI-based applications. Its focus is on data preparation and organization rather than autonomous decision-making or predictive analytics.
However, should the system evolve to include semi-automatic or automatic semantic annotation functionalities in the future, an ethical impact assessment may become required. In the case of the usage of data produced by doccano-semantics in downstream AI systems, an AI ethical impact assessment may be considered.
3.3.4 Future Work
In this study, we present a comprehensive end-to-end pipeline, yet several enhancements warrant future exploration. Specifically:
Akoma Ntoso Version Support: The current AkomaNtosoParser class is tailored to the Italian legislative implementation. However, preliminary trials with the Luxembourg and France versions have indicated discrepancies. Refinement in extraction methodologies is essential.
User Interface Integration: Currently, the conversion to Doccano-compatible formats and file exports take place through a command-line interface. Integrating these processes within the Doccano web interface would significantly enhance user experience.
Semi-automated Annotation: The Doccano platform permits automated annotations via external API services. Implementing a diverse set of semantic annotation algorithms could expedite the annotation rate and increase the time experts can spend on high-value decisions.
Scaling up: A plug and play extension model that connects the academics and open source communities to share legal semantic annotation algorithms at scale could largely improve the interchange of semantic annotation algorithms. APIs guarantee the technical interoperability of such a model. Some identified algorithms are the followings: Fuchs et al. (2023; Ceci et al. 2012; Lesmo, Mazzei, and Radicioni 2009; Savelka 2023; Camilleri, Gruzitis, and Schneider 2016)
Expand the set of export representations: While the selected annotation specification is Lynx, optimized for embedding the semantic annotations of legal documents in Turtle format, it may not be the optimal tool for delineating procedural rules and deontic logic. Exploring alternative specifications, such as Akoma Ntoso (Palmirani and Vitali 2015), LegalRuleML (Palmirani and Governatori 2021), or Catala (Merigoux, Chataing, and Protzenko 2021), could offer more comprehensive representations.
3.3.5 Results
This research adheres to the FAIR principles (Wilkinson et al. 2016) and it produced the following results:
This thesis: this document describes the research objectives, methodology, and results of the project work.
A data pipeline for semantic annotation of legal texts in Akoma Ntoso format, as well as the description of the internal components. The software code is freely accessible in the Github Repository at the following address https://github.com/AlessioNar/semanticannotation.
A modified version of doccano, named doccano-semantics. The fork of the doccano repository is available at the following address https://github.com/AlessioNar/doccano-semantics
A partial annotation of the Italian copyright legislation, the dataset is available for download in Zenodo at the following address https://doi.org/10.5281/zenodo.8312755
A partial representation of the Italian copyright legislation in the Lynx format, the dataset is available for download in Zenodo at the following address https://doi.org/10.5281/zenodo.8312759
3.4 Conclusion
Throughout this chapter, we have presented the data transformation pipeline built for streamlining the semantic annotation of legal documents by leveraging the Doccano tool and the Lynx specification. We presented the different components of the system and provided an initial evaluation of the system. Finally, we presented and discussed the results of the work and compared it with other existing platforms.
References
Namely, a legislative text (Law 633 of 1942, (Gazzetta Ufficiale 1941)), a set of potential labels (the Copyright Ontology (García and Gil 2006)) and the annotation tool (Doccano (Nakayama et al. 2018))↩︎
A file containing lines of JSON files↩︎
Among the conversion tools we can list Formex2AKN, Norme in Rete to AKN, Text2AKN↩︎
Akoma Ntoso and LegalRuleML↩︎