Chapter 1 Semantic annotation
The maturation of digital technologies has opened up countless new opportunities for legal activities. Across the years, they have enabled for improved methodologies for handling, exploring, analyzing and implementing legal resources, greatly impacting the legal sector at different levels (Katz, Dolin, and Bommarito 2021; Sharma et al. 2021). In this context, the field of legal informatics has thrived, with more and more scholars focusing on the different aspects of innovative legal technologies, from the development of legal ontologies and semantic annotation of legal texts, to automated regulatory compliance checks and enhanced legal reasoning (Katz, Dolin, and Bommarito 2021; Loutsaris and Charalabidis 2020; Victor Rodriguez-Doncel 2023).
This was largely enabled by the development of new ways of storing legal information, which allowed for an easier retrieval and reuse of the legal concepts present in the natural language texts such as the competent jurisdiction, temporal elements, and deontic rules (Sharma et al. 2021; Loutsaris and Charalabidis 2020).
1.1 What is semantic annotation?
Among the array of techniques that have emerged in recent years, a prominent approach involves semantically annotating legal texts (Soavi et al. 2022b; Nazarenko, Lévy, and Wyner 2021; Tang et al. 2020; Ghijsen et al. 2013). This technique entails "connecting these texts with established ontologies by to identifying concepts from that ontology that are relevant to the document or that are referred to by it, as well as identifying specific passages in the document where the concepts in question are mentioned" (Brank, Leban, and Grobelnik 2018).. In other words, semantic annotation can be defined as the process of mapping texts to ontologies, enabling further processing by machines based on meaning instead of syntax (Kiryakov et al. 2003).
According to Adebayo, Di Caro, and Boella (2018) semantic annotation can be formalized "as a 4-tuple (a, b, c, d), where a is the subject of the annotation, b is the object of the annotation, c is the predicate which defines the type of relationship between a and b, while d signifies the context in which the annotation is made."
1.1.1 Benefits of semantic annotation
Semantically annotating legal texts has several advantages. In particular, it can power the retrieval of legal documents using detailed natural language queries. Traditional annotation-based retrieval systems rely on word matching and they often fall short in supporting complex legal query types. With a semantic understanding of query sentences and legal annotations, retrieval can become more precise and efficient (Soavi et al. 2022a).
Additionally, semantic annotations minimizes the ambiguity of the meaning of legal terms by making them explicitly defined (Soavi et al. 2022a). However, this is often complex to achieve, as in many cases legislators draft legislation which is "ambiguous-by-design" in order to strike a deal.
Moreover, semantic annotations could be reused in data-intensive applications downstream. An example of such applications is Kantoorbelasting, a proof-of-concept of the Flemish Government that leverages semantically annotated municipal regulations to enable citizens to estimate the taxes on commercial buildings depending on the city they plan to set their business in (Digitaal Vlaanderen 2023). Automated compliance checks are also another well-explored use-case (Kiyavitskaya et al. 2006). Among many, a notable example is the Lynx project, focused on enhancing access to digital regulatory compliance documents that target SMEs. The pilot developed regulatory compliance assurance services in three areas: contract compliance, labour law and fossil fuels (Victor Rodriguez-Doncel 2023). These applications are built on top of the semantic information contained in the legal corpora, thus increasing the integration among legal resources and the business logic and data access layers of the IT systems involved (Digitaal Vlaanderen 2023; Victor Rodriguez-Doncel 2023).
1.1.2 Challenges of semantic annotation
However, annotating legal documents is a complex task due to the discursive nature of legal information. This is often referred to as the "natural language barrier," which involves translating natural language sentences into semantic interpretations (Ceci et al. 2012; Soavi et al. 2022a). Transforming these texts into solid formal specifications often requires a multi-step process with multiple iterations that ideally would need to be validated by the legislators (Soavi et al. 2022a).
Additionally, as mentioned above, legal resources often purposefully leave room for diverse and/or conflicting interpretations, so that the issue is dealt in secondary legislation such as implementing acts or by the competent judicial court.
Finally, achieving high-level semantic meanings in the legal domain often requires integrating extracted relations with external legal knowledge sources, which can be vast and varied.
1.2 Considerations for the semantic annotation of legal texts
As described above, annotating a legal text is an open field of research. However, with the evolution of legal informatics, some well-known concepts and considerations have been pinpointed by the scientific community as crucial for achieving solid results.
1.2.1 Legal isomorphism
The unique characteristics of the legal domain demand accountability, traceability, and explainability of legally binding decisions and provisions. Consequently, it is essential that a machine-readable representation of a legal text can be traced back to its original form in natural language. This element is crucial because, according to prevailing legal provisions in most legal systems, the natural language version stands as the sole authoritative source of truth recognized by the courts. As a consequence, much effort has been devoted to finding ways of encoding legal information in a way that is both human readable and machine readable (Santos et al. 2018; Víctor Rodriguez-Doncel et al. 2018).
This correspondence is referred to with the term isomorphism, which can be defined as the precise correspondence between the natural language of a text and its representation (Bench-Capon and Coenen 1992). Karpf and Københav (1989) identifies five conditions for achieving this correspondence:
The individual representation of each legal source.
The preservation of each source’s structure.
The maintenance of mutual relations, references, and connections between sources.
A distinct representation of legal sources from other system components.
The inclusion of both material and procedural rules (if the model covers procedural law).
In other words, isomorphism demands a clear link between the source material items and items in the knowledge base. Originally, the key benefits of isomorphism were believed to be in knowledge base verification, validation, and maintenance. However, further application revealed broader advantages, impacting the system’s entire lifecycle (Bench-Capon and Coenen 1992).
Legal knowledge bases, due to the dynamic nature of laws and regulations, necessitate frequent updates. An isomorphic approach allows for pinpointed updates, meaning that minimal changes in the source material lead to localized adjustments in the knowledge base (Bench-Capon and Coenen 1992).
Lastly, from a user’s perspective, isomorphism brings forth several advantages, such as simplified system learning and clarity in understanding the system’s rationale. If users base their understanding on specific source documents, an isomorphic system aligns with this perspective, making interactions more intuitive (Bench-Capon and Coenen 1992).
1.2.2 Provision-centric approach
A notable perspective emerging from multiple studies is viewing legislation through a "provision-centric" lens. Legislation, in essence, transports rules or provisions, carried by linguistic acts. By focusing on provisions, researchers aim to offer a more systematic view of legal systems. Provisions, in their various forms depending on legislative intent, can be described using metadata schemes (C. Biagioli et al. 2005; Francesconi and Passerini 2007).
In contractual and legal contexts, a provision is often understood as a stipulation, term, or condition that delineates specific aspects of an agreement. These provisions, as highlighted by Francesconi and Passerini (2007), serve as the cornerstone of many agreements, outlining the obligations, rights, and responsibilities of the agents involved. By focusing on provisions as the central component rather than mere isolated terms, this approach offers a holistic perspective on the entire contractual structure (Francesconi and Passerini 2007). Provisions, when semantically annotated, transcend from being static textual entities to dynamic, interconnected, and easily navigable data points. This amalgamation not only streamlines the processing and management of legal documents but also ensures greater clarity and reduced ambiguities (C. Biagioli et al. 2005).
1.2.3 Deontic logic
Deontic logic can be defined as "the logic of normative expressions: expressions pertaining to the obligations, permissions and rights of agents" (Jones and Sergot 1992; Wright 1951). As such, it is highly relevant for the formal representation of legal knowledge, as it is one of the key elements that could be leveraged for modeling legal rules. Its significance becomes even more pronounced when we delve into the semantic annotation of legal texts, a process that attaches deeper, structured meanings to textual content.
This is not to say that legal provisions cannot be modeled without taking into account deontic logic. Jones and Sergot (1992) reflects on the fact that while many legal fragments can be modeled without such operators, certain situations, especially those involving potential violations and consequential states, demand its inclusion for achieving an accurate and reusable representation.
The semantic richness of legal texts often goes beyond mere linguistic expressions. The real challenge lies in discerning whether a provision can be violated and if such a violation leads to a state regulated by another legal provision (Jones and Sergot 1992). This discernment is crucial for semantic annotation, ensuring that the underlying obligations, permissions, or prohibitions are captured accurately.
1.2.4 Interpretation
Semantically annotating legal texts means diving deep into the meanings and contexts of legal language. Instead of just labeling words, sentences or paragraphs, this process aims to understand and map out the deeper ideas, relationships, and intentions in legal documents. It’s about making sense of complex legal terms and determining their relationships within the broader legal framework (Ceci et al. 2012; Soavi et al. 2022b).
Legal texts, more often than not, are characterized by having an open texture (Waismann 1947), that is the natural uncertainty or lack of clarity in various legal rules and guidelines. This ambiguity emerges because the nuances of language can’t always accurately reflect all facets of human experiences. When framing laws, it’s impossible for lawmakers to predict every circumstance or situation where a law might come into play. Consequently, these legal documents frequently include terms that invite interpretation. Such undefined spaces or "grey zones" in the text offer adaptability, ensuring that the law remains relevant in evolving societal situations and unexpected events. As a consequence, while the goal is to remain as objective as possible, this inherent nature of legal language means that some degree of interpretation is inevitable. Annotators must discern the latent intentions behind provisions, predict potential areas of contention, and preemptively address ambiguities (Stegmeier et al. 2021; Ma and Wilson 2021).
However, the ultimate authority on interpretation traditionally rests with the courts. Annotators, regardless of their expertise, are navigating a terrain where they must tread carefully. Every legal provision or statute is a product of extensive deliberations, negotiations, and compromises. Thus, the language used often carries with it historical, political, and socio-economic nuances that might not be immediately evident (Stegmeier et al. 2021; Athan et al. 2014).
Moreover, court decisions, which serve as primary interpretive guides, evolve over time. Precedents may be overturned, and interpretations might shift based on societal changes, evolving legal philosophies, or the particular composition of a judicial bench. Annotators, therefore, face the challenge of ensuring their annotations remain relevant and accurate in light of an ever-evolving body of case law. They might be confronted with situations where different courts provide varied interpretations of the same provision. Deciding which interpretation to align with, or how to present multiple interpretations without bias, can be daunting (Stegmeier et al. 2021; Athan et al. 2014).
Additionally, the risk of personal bias is ever-present. Annotators, being human, come with their own set of beliefs, values, and experiences that can inadvertently influence their interpretation. While the goal is always to remain neutral, the subjective nature of some legal provisions can make this challenging. For instance, terms like "reasonable" or "undue hardship" in legal statutes are inherently open to interpretation and don’t have fixed, universally accepted definitions (Ma and Wilson 2021; Athan et al. 2014). This requires a traceability of the agent that performed the annotation in the annotation itself, in order to be able to ask for clarifications or interpretative keys, should the need arise.
Lastly, the stakes are high. Misinterpretation or oversimplification during annotation can lead to the misrepresentation of legal provisions, that can have tangible consequences. This is especially true for systems that rely on these annotations for the automation of decision-making processes.
1.3 An overview of semantic annotations techniques
So far, we have explored some considerations that needs to be taken into account when establishing an annotation strategy. However, we have not dwelt yet into how the annotation itself can take place and the role of automated systems in supporting or carrying out the annotation effort. Broadly speaking, the techniques that have been employed for this task can be subdivided in three main categories: manual annotation, assisted or semi-automated annotation, and automated annotation.
1.3.1 Manual Annotation
Legal texts, as repositories of historical, cultural, and social contexts, require an interpretative finesse that automated approaches often lack. Because of this, semantic annotation requires the support of legal scholars and domain experts who engage in a meticulous examination of the texts. Through this process, they assign relevant semantic tags or metadata based on their expertise, interpretation and understanding of the legal text (Nazarenko, Lévy, and Wyner 2018; Governatori 2005).
The main advantage of manual annotation lies in its precision. The complexity of the legal language often elude computational algorithms. As a result, manual annotation ensures that annotations encompass both the overt and covert meanings enshrined within legal provisions. Furthermore, as mentioned above, certain legal provisions or judgments can be open to multiple, potentially conflicting interpretations (Stegmeier et al. 2021).
While the meticulous nature of manual annotation underscores its strength, it is simultaneously a source of limitation. The time and effort it demands make it less viable for managing extensive databases of legal documents, a concern amplified in an era characterized by exponential digital expansion (Nazarenko, Lévy, and Wyner 2018).
Nazarenko, Lévy, and Wyner (2021) worked on a methodology to semantically annotate legal documents. They introduce a coarse-grained, interpretation-neutral annotation layer to enrich legal texts, striking a balance between traditional statistical retrieval and full textual rule formalization. Their methodology is exemplified through the semantic annotation of the French version of the GDPR, utilizing the Core Legal Annotation Language (CLAL) formalized in XML, establishing a gold standard for future semantic annotations in the legal domain.
Another issue with manual annotation is that, although it enriches the depth of analysis, it introduces an element of subjectivity. Given the interpretational diversity among annotators, disparities in annotations can arise, particularly when dealing with multiple experts (Stegmeier et al. 2021).
In light of its reliance on human expertise, manual annotation necessitates substantial resources, both in terms of time and specialized knowledge. This, in turn, can render it less tenable for projects constrained by limited resources or stringent timelines.
1.3.2 Semi-automated or assisted annotation
Semi-automated annotation seeks to strike a balance between human expertise and the speed of automated tools. Generally, by using this approach, computational tools propose initial annotations, which are subsequently reviewed and refined by human experts. By amalgamating human judgment with computational speed, semiautomatic annotation offers an efficient means of annotating texts, ensuring a harmonious blend of accuracy and efficiency.
Soavi et al. (2022b) developed a tool named "ContrattoA" that semi-automatically conducts semantic annotation of legal contract text leveraging a domain ontology. To achieve this, they first leveraged lexical patterns to recognize some ontological concepts in the legal text. Then, the effectiveness of these patterns was evaluated in an empirical study where one group of subjects was asked to annotate legal text manually, while a second group edited the annotations generated by ContrattoA. Subsequently, they focused on the core contract concepts of obligation and power where the results from the first iteration were mixed. Using an extended set of sample contracts, new lexical patterns were derived. These patterns were shown to significantly improve the performance of ContrattoA.
Ceci et al. (2012), worked on designing a system to assist human experts in the annotation of normative modifications. As in the previous example, the automated system proposes a label, which is then validated or modified by the human annotator.
1.3.3 Automated annotation
The domain of legal texts presents unique challenges for automated semantic annotation due to its intricate language, layered references, and the high stakes involved in accurate interpretation. The most salient advantage of automatic annotation is its speed. Algorithms can process vast volumes of data at rates unattainable by humans. Yet, the very strength of automatic annotation is also its weakness. The absence of human oversight can lead to misinterpretations, especially given the nuanced and context-sensitive nature of legal language. Algorithms, while efficient, often lack the depth to fully grasp the multifaceted interpretations inherent in legal provisions at the current state of technology. Over the years, several researchers have delved into the task of automating various aspects of legal text processing.
Palmirani, Brighi, and Massini (2003) worked on the automatic extraction of normative references in legal texts from the Italian acquis. They claimed that standardized legal documents, marked up under uniform formats and structures, can facilitate a seamless integration between distinct legal texts, making them easier to reference, find, and process. A significant contribution of their work lies in the identification and marking of various components of legal texts, such as partitions (paragraphs, articles and sections), and the normative references they contain. Their project sought to craft a model adept at recognizing, understanding, and normalizing these normative references, a step forward in ensuring semantic interoperability among various legal information systems.
Recognizing the magnitude and intricacy of legal documents, C. Biagioli et al. (2005) emphasized the need for automated tools that can support the annotation process. They worked on the identification of regulatory provisions by detecting actors, rights, obligations and other entities integral to the legal discourse. They structured their approach in two steps: a provision identification module, that detects and classifies fragments of normative texts automatically based on their provision type, and an argument extraction module, that extracts the associated arguments from these provisions. They encountered several challenges tied to the unique lexicon and structure of the legal language, as they often carry weighty consequences.
Zeni et al. (2008, 2013) worked on automating the annotation of the Stanca law on web accessibility by adapting the Cerno framework in the Italian language. This framework provides a structured method, encompassing the identification of specific text fragments in regulatory documents, constructing a cohesive semantic model from these annotations, and subsequently transforming this model into distinct functional and non-functional requirements by focusing on concepts of obligations, rights, anti-rights, actor, and others.
Asooja et al. (2015) focused on creating an automated approach for annotating financial regulations. Their aim was to enrich a formal ontology using segments from legislative texts. The method employed not only domain-centric semantics but also general ones, and integrated deontic logic for prescriptive clauses. Their technique utilized a multi-label classification strategy, powered by several binary classifiers. This system was trained using provision types manually annotated by subject matter experts, adopting a supervised learning approach.
Adebayo, Di Caro, and Boella (2018) leveraged the TextTiling algorithm, enhancing it with Latent Dirichlet Allocation (LDA) for refined topic modeling. By integrating Semantic Textual Similarity principles, legal texts are segmented to mirror inherent subtopics. Their aim was to semantically connect concept descriptors with topical segments across a document. Their methodology unfolds in three stages: topical document segmentation, concept profiling, and mapping of concepts to segments.
Humphreys et al. (2020) harness advanced techniques to automate the semantic annotation of legal documents, specifically focusing on extracting norms and their elements. Central to their approach is combination of paraphrasing tools within unsupervised information extraction systems to produce semantically cohesive components from natural language. They applied Semantic Role Labeling (SRL) to identify and annotate specific elements such as "Situation", "Result", and "Condition" in legal texts. These annotated roles are then systematically mapped to slots in legal ontologies.
In Sleimi et al. (2020), an automated system that utilizes NLP and ML to extract designated metadata from texts was developed. They tested the system’s effectiveness through two case studies centered on Luxembourgish legislation. The outcomes were encouraging: precision rates reached 97.2% and 82.4%, with recall at 94.9% and 92.4%. This evaluation was anchored on 200 randomly selected statements from traffic laws. Their approach could be divided into three components: semantic metadata extraction (at both phrase and statement levels) and a named entity recognition algorithm used to identify the agents involved (Commission, Directorate-General for Financial Stability, and Union 2021).
In the study by Savelka (2023), the use of generative pre-trained transformers (GPT) was explored for the semantic annotation of short text snippets from various legal documents. While there have been discussions regarding the potential applications of this technology in the legal domain, a thorough analysis was missing, especially concerning sentence-level semantic annotation in zero-shot learning settings. Savelka (2023)’s research addressed this gap, evaluating the LMM’s ability to annotate small batches of short text snippets based solely on concise definitions of the semantic types. The results showed that the GPT model performed in zero-shot settings across different document types, achieving \(F_1\) scores of 73% on court opinions, 86% for contracts, and 54% for statutes and regulations.
1.4 Conclusion
The complex nature of legal resources, rich in subtle linguistic patterns and vague language, compels a well-though approach to semantic annotation. While manual annotation offers a depth unattainable by machines, it often falls short in scalability. On the other hand, while automatic annotation offers unparalleled speed, it sometimes lacks the finesse required for precise interpretation. Semi-automatic annotation, on the other hand, offers a promising middle ground.
As the domain of legal informatics evolves, a holistic understanding of these techniques is imperative. Future endeavors in this realm will undoubtedly seek to further harmonize the strengths of each method, paving the way for more refined, efficient, and accurate semantic annotations.
In conclusion, the semantic annotation of legal texts remains an evolving field, characterized by diverse methodologies and techniques. These methodologies, while differing in their approaches, collectively underscore the significance of structured, systematic, and semantically rich representation of legal texts for enhanced accessibility, interpretation, and applicability.