Keywords

1 Introduction

Although the reasoning of judgment documents cannot solve the actual problems of judicature, it is the touchstone of justice [1]. The reasoning of judgment documents is not purely a legal or technical issue, but it is a political and institutional issue. Attaching importance to the reasoning of judgment documents is essentially the embodiment of judiciary civilization, especially the embodiment of judicial democracy and judicial rationality [2]. In the latest reform outline and project of information development released by the Chinese People’s Court, promoting the reform of judgment documents reasoning and improving the level of judgment documents reasoning are both mentioned. In order to achieve the goals, we have to study on the technology of automated judgment documents reasoning evaluation.

The reasoning of judgment documents contains four aspects, which are fact of case, law of case, emotion of document and style of writing. The aspect of fact focuses on the description of evidence, fact and the relationship between them [3]. For example, how can judicial evidence become the basis of judgement? Judicial evidence must be objective, relevant and legal. These three characteristics of judicial evidence are not only related to the theoretical problem, but also should be reflected in judgment documents [4]. The collection of evidence, the relationship between evidence and fact and the relationship between different evidence is called evidence chain. How to build evidence chain relational model is the basis and key to automated judgment documents reasoning evaluation.

The Study of judgment documents reasoning evaluation is mainly based on the extraction and analysis of evidence chain. Our research uses the legal language (language with the characteristics of legal terminology) and Chinese natural language processing technology, extracts key paragraphs of reasoning, builds evidence chain relational model and uses computer visualization technology to display the model. To highly efficiently handle historical big data, we need to use some big data processing technologies such as Hadoop or Spark.

In this paper, we propose an approach to build evidence chain relational model based on Chinese judgment documents. Our work includes: (1) propose an automated method to do text preprocessing based on Chinese judgment documents, convert unstructured judgment documents into semi-structured XML format files, extract evidence set and fact set, (2) propose a strategy of extracting key elements, obtain the keywords of evidence and facts, (3) propose an approach to create the connection points of evidence chain relational model, including finding the relationship between evidence and fact and finding the relationship between different evidence, (4) realize tabular display and graphical display of the evidence chain relational model.

To evaluate the performance of our approach, we use civil documents, criminal documents and administrative documents to test the system. We will prove the contribution of this paper by answering the research questions as follows:

  1. (1)

    How to do the automated text preprocessing for Chinese civil documents, criminal documents and administrative documents? Which information should be included in the preprocessed semi-structured files?

  2. (2)

    Which kinds of key elements should be selected? How to extract those key elements?

  3. (3)

    How to find the relationship between evidence and fact? How to find the relationship between different evidence?

  4. (4)

    How to display evidence chain relational models clearly?

The remainder of this paper is laid out as follows. Section 2 introduces related work of this paper. Section 3 introduces our approach in detail. Section 4 concludes with a discussion of future work.

2 Related Work

For a long time, the study of judgment documents had been limited to the relevant legal experts, until the development of computer automation technology. Some Speculation about Artificial Intelligence and Legal Reasoning published by Buchanan in 1970 marks the birth of a branch research on artificial intelligence and legal reasoning [5]. In recent years, as the artificial intelligence algorithm and natural language processing technology become a boom once again, a large number of domestic and international researches on judgment documents and semantic information spring up. The modeling of legal reasoning about evidence within general theories of feasible reasoning and argumentation was studied in 2003 [6]. An approach to formalize argumentative story-based analysis of evidence was proposed in 2007 [7]. In 2009, researchers explored the potential of using extended belief change operators for modeling the evolution of legal evidence [8]. Recent research has shown that argumentation can inform the construction of Bayesian networks. A position paper presents an investigation into the similarities, differences and synergies between Bayesian networks and argumentation diagrams and shows a first version of an algorithm to extract argumentation diagrams from Bayesian networks [9]. Because of the particularity of Chinese and Chinese legal system, many foreign research methods and achievements cannot be directly applied to the analysis of the Chinese judgment documents. In recent years, the domestic research on automated Chinese judgment documents processing gradually increases, research contents include the informationalized storage and management of archives, recognition and classification of judgment documents, detection and protection of privacy content in judgment documents and so on.

Evidence chain is a long-standing question in the field of law. In Rationale of Judicial Evidence, the author makes a thorough analysis of the rationality of judicial proof based on various kinds of judicial evidence, and puts forward the guidance of constructing reasonable evidence chain [10]. The Study on Criminal Evidence Chain is the representative work of the research on evidence chain in China. The author defines the basic conception of chain unit, main part of unit (chain unit body), key of unit (chain unit head) and connection point in evidence chain, and proposes different kinds of link, for example simple link, multiple link, net link, etc. [11]. There are also some researches on the combination of computer technology and evidence chain. A few researchers focus on the analysis of computer forensics, evidence from micro computer technology to refine the process chain of evidence, summarizing the main phase of formation of the acquisition and curing, identification, adjudication and archiving in the evidence chain [12]. An electronic judicial identification model based on three-dimensional trusted electronic evidence acquisition model is proposed. The model implements linear process control and evidence supervision. The model ensures evidence’s own safety and legal effect on evidence supervision chain [13].

A very important step of our approach to build evidence chain relational model is extracting key elements. We refer to some existing research achievements during our work. The authors of Extracting 5W1H Event Semantic Elements from Chinese Online News propose a verb-driven approach to extract 5W1H (Who, What, Whom, When, Where and How) event semantic information from Chinese online news. This approach extracts event facts (i.e. 5W1H) by applying a rule-based method (verb-driven) and a supervised machine-learning method (SVM) [14]. The researchers who published Chinese News Event 5W1H Semantic Elements Extraction for Event Ontology Population propose a novel approach of 5W1H (Who, What, Whom, When, Where, How) event semantic elements extraction for Chinese news event knowledge base construction. The approach comprises a key event identification step, an event semantic elements extraction step and an event ontology population step [15]. Although the application scenarios of these papers are not as the same as this one, they are all based on Chinese text.

3 Approach

In this section, we present our approach to build evidence chain relational model based on Chinese judgment documents in detail as follows. Section 3.1 introduces evidence chain relational model and presents an overview of the flow diagram for Chinese judgment documents analysis. Section 3.2 introduces text preprocessing for Chinese judgment documents. Section 3.3 introduces the strategy of extracting key elements from evidence and facts. Section 3.4 introduces an approach to create the connection points. Section 3.5 introduces methods of model display.

3.1 Overview

Figure 1 presents the sample graph of evidence chain relational model. An evidence chain relational model has at least two different chain units. Each chain unit contains chain unit body and head. One chain unit body corresponds to certain evidence, and head is the key element of this evidence. Connecting several chain unit heads forms connection point which presents the relationship between different evidence. Evidence set is also connected to fact by connection point.

Fig. 1.
figure 1

Sample graph of evidence chain relational model

Full size image

Figure 2 presents the flow diagram of Chinese judgment document analysis. Because the original judgment document is pure text format, in order to complete building evidence chain relational model, we have to do text preprocessing firstly. In Sect. 3.2.1 we will introduce the method we use to convert unstructured judgment documents into semi-structured XML format files and the evaluation results of this method. In the process of making court verdicts, the parties, agent ad litem and third parties can debate authenticity, legality, relevance of evidence they submit. The judge has to show the result of debate in his judgment document. Inadmissible evidence apparently can’t be included in evidence chain relational model. We will introduce the approach to label admissibility of evidence in Sect. 3.2.2, and then we will introduce how to extract evidence set and fact set in Sect. 3.2.3.

Fig. 2.
figure 2

Flow diagram for chinese judgment documents analysis

Full size image

Connection points in evidence chain relational model are formed by connecting chain unit heads that mean key elements of evidence. So the most direct way to obtain correlation of different text is calculating contact ratio of key elements, therefore the strategy of extracting key elements is very important. Key elements have to embody characteristics of evidence and fact. We select 4W1H (What, Where, When, Who and How Much) as five key elements. In Sect. 3.3 we will introduce how to extract WHAT, WHERE, WHEN, WHO and HOW MUCH in turn and show the evaluation results of these methods.

As mentioned, how to organize statement while writing reasoning has no standard, so the judge may not express the relationship between evidence and fact it supports in his judgment document. We can obtain some clear relationships while we do text preprocessing, but unclear relationships need to be calculated. In Sect. 3.4.1 we will introduce the calculating method. We define the chain unit head as the key element of evidence, in other words, a chain unit head means a proof point. Each chain unit is allowed to have many heads and only one body. For example, a knife with fingerprints on it at the scene of the crime has two proof points. One is that the weapon of this crime is a knife, and another is criminal suspect’s fingerprints. Different chain units may have same heads that embody the relationship between them. For example, an expert conclusion provided by an accreditation agency also proves fingerprints of criminal suspect. Section 3.4.2 will introduce the method to build the relationship between different evidence.

The final goal of building evidence chain relational model is to evaluate judgment document reasoning. For this purpose, the model should be easy to read, examine and evaluate. We propose two ways to display evidence chain relational model. The way of using Excel sheets is introduced in Sect. 3.5.1, and the way of using graphical interfaces is introduced in Sect. 3.5.2.

3.2 Text Preprocessing

In this section, we mainly focus on introducing the methods used to preprocess Chinese judgment documents.

3.2.1 Convert Judgment Document into XML File

According to case nature, judgment documents can be classified into three types: Civil, criminal and administrative. Moreover, judgment documents can be classified into certain types by trail procedure. For example, first instance, second instance and so on. For this reason, we set up six kinds of XML templates for all types of judgment documents.

First step of converting is paragraph splitting. According to the position of a paragraph and regex match results of keywords from first sentence; all paragraphs can be classified into seven parent sections: headline, party, litigation record, basics of case, trail process, verdict and end. Deeper analysis of each parent section is used for splitting child sections. For building evidence chain relational model, the most important parent section is basics of case. In first instance over civil judgment documents, this section includes seven child sections at most, such as plaintiff’s appeal, defendant’s defense, evidence section, fact section, etc. Take evidence section as an example, we are going to introduce how to identify whether a paragraph is evidence section in detail. If a paragraph contains “提出如下证据:”, “证据如下:” or other similar phrases, it must be an evidence section. If a paragraph contains “证据”, “证明”, “证言” or other similar keywords, it may be an evidence section. In this case, other keywords need to be verified. If keyword “查明” is also contained in this paragraph, then this paragraph belongs to fact section rather than evidence section. In other cases, characteristics of context need to be considered. Furthermore, when we extract evidence sections and fact sections from basics of case, the relationship between evidence and fact mentioned in the judgment document will be recorded. For instance, if an evidence section starts with “认定上述事实的证据有”, the fact in the section before it will be assigned to the same group as the evidence in it. We call the group “evidence-fact group”, which will be mentioned again in Sect. 3.2.3.

Second step of converting is information item extraction. According to the requirement of judgment document analysis and the writing standard of judgment document, we set up about 630 information items that need to be extracted. Because judges have many undefined “hidden” habits while writing judgment documents, we have summarized these habits by reading a large number of judgment documents. In our work, information items are extracted by regular expression in light of writing standards and habits. Table 1 presents some samples of regular expressions we have studied and summarized for extracting information items.

Table 1. Some samples of regular expressions for extracting information items
Full size table

To evaluate the accuracy of our approach, we use two ways: both existing tools and human labeling, to build test dataset that contains more than 5000 judgment documents. By comparing each section and each information item, the evaluation result shows that overall accuracy of our approach is 93.5%. As Table 2 presents, accuracies of splitting basics of case, which is the most important parent section for building evidence chain relational model are all higher than 90%, even up to 97%.

Table 2. Accuracies of splitting seven parent sections
Full size table

As Table 3 presents, accuracies of evidence section extraction and fact section extraction are all over 70%, even up to 98%.

Table 3. Accuracies of evidence section extraction and fact section extraction
Full size table

Our approach can reach the level of mature tools in the industry. Furthermore, we can meet the requirements formulated for the evaluation of judgment document reasoning. Our approach can do more in-depth analysis of reasoning sections and realize some functions that other tools don’t have.

3.2.2 Label Admissibility of Evidence

In face of much evidence, three characteristics are supposed to be considered as filter criteria. They are authenticity, legality and relevance. Evidence without these characteristics cannot be brought into evidence chain relational model [16].

The admissibility of evidence is always written at the end of evidence section or after each evidence. First of all, we have to extract the description of admissibility. The method to do this work is similar to the approach to extract child section from parent section.

Due to the improvement and development of Chinese criminal justice, the judicial interpretation is mainly concentrated on evidence rules of criminal proceeding [17]. It’s harder to label the admissibility of evidence extracted from civil or administrative cases, because the situation that part of evidence can be adopted meanwhile the other part can’t always occurs. In this case, processing requires following three steps:

  1. (1)

    Divide the description of admissibility into single sentences and each sentence only embodies the admissibility of one or a set of evidence. Judges always use Arabic numerals or Chinese character numbers as identifiers. A set of evidence means identifiers of those evidence are connected by punctuation connectors such as “,”, “~”, “−” or Chinese connectors such as “和”, “至”, “到”. If there are no obvious identifiers, word segmentation is needed. Segmented words are used to build a dictionary which is the basement of matching in the next step.

  2. (2)

    If there are identifiers, use them in each sentence to relate the sentence to corresponding evidence. Otherwise, match words in each sentence based on the dictionary and find out corresponding evidence.

  3. (3)

    Use keywords as characteristics to classify the result of admissibility. For example, “不采信”, “不采纳”, “不予采信” and “不予采纳” mean not adopted, on the contrary, those keywords without negative expression mean adopted.

3.2.3 Extract Evidence Set and Fact Set

Building evidence chain relational model needs every single fact and evidence, so extracting them from the section they belong to is required. Concrete steps are as follows:

  1. (1)

    Read every evidence-fact group. If there is a fact section in it, we put the fact into fact set, and turn to step (2). Otherwise, turn to step (3).

  2. (2)

    Put all evidence in this evidence-fact group into an evidence set related to the fact set we build in step (1).

  3. (3)

    Put all evidence in this evidence-fact group into a specific evidence set called “unrelated evidence set”.

3.3 Extract Key Elements

Because 4W1H have different characteristics, each kind of element needs respective method. In this section, we mainly focus on introducing the methods used to extract key elements. Same word may repeat in one text, so de-duplication must be considered. Make sure that the set of each kind of element doesn’t contain repeated words.

3.3.1 Extract What

WHAT refers to entitative things mentioned in evidence and facts. For example, WHAT usually refers to drug name in drug cases, and refers to type and brand of vehicle in traffic cases. Words of WHAT have two characteristics, one is all of them are nouns; the other is most of them belong to subject or object.

We use two methods to extract WHAT:

  1. (1)

    RegExp (Regular Expression). For things have regular structure, such as file name between “《’’ and “》”, use corresponding regular expression to extract.

  2. (2)

    Semantic Relation. We use constraint-based Chinese dependency parsing method to analyze sentence structure automatically based on maximum entropy model and maximum spanning tree. Select word itself, part of speech, distance between two words and dependency of two words as features to train the model.

As mentioned above, subject and object are likely to be elements we look for. After statement structure analysis, obtain all of subject and object, filter out sites, names and words that are not noun. Chinese attribute V-N compounds which can be interpreted as nominals with V as the modifier and nominal phrases with a verbal modifier both need to be considered to select an appropriate completion for rest words afterwards [18].

3.3.2 Extract Where

WHERE refers to sites mentioned in evidence and facts. Words of WHERE have two characteristics, one is all of them are nouns or location words; the other is most of them appear after prepositions.

We use two methods to extract WHERE:

  1. (1)

    POS (Part of Speech). Unlike English, Chinese text requires word segmentation. We use Chinese tokenizer based on n-Gram, CRF (Conditional Random Field) and HMM (Hidden Markov Model) to segment words. After that, nature of every word is marked. If a word is labeled as “S” which means site, it is what we look for.

  2. (2)

    Semantic Relation. In consideration of error rate of word segmentation, semantic relation is also required. We build a list of common prepositions to extract phrases after them.

3.3.3 Extract When

WHEN refers to time mentioned in evidence and facts. Words of WHEN have two characteristics, one is containing Arabic numerals or Chinese character numbers; the other is they have standard structure.

We use RegExp (Regular Expression) to extract WHEN. Regular expressions covered from the most precise time “X年X月X日X时X分” to the simplest time “X年” can match time phrases with different precision.

3.3.4 Extract Who

WHO refers to parties mentioned in evidence and facts. Parties are not only people but also companies and administrative organs.

We use POS (Part of Speech) to extract WHO. Names of people and organizations are both extracted. Furthermore, in consideration of the problem of excessive word segmentation, we use a similar method that is similar to the method mentioned above when we introduce how to extract WHAT to select an appropriate completion.

3.3.5 Extract How Much

HOW MUCH refers to quantitative phrases mentioned in evidence and facts. Words of HOW MUCH have two characteristics, one is containing Arabic numerals or Chinese character numbers; the other is they always appear before quantifiers.

We use RegExp (Regular Expression) to extract HOW MUCH. We build a list of common quantifiers to extract phrases before them.

3.3.6 Evaluation

To evaluate our methods, we conduct an experiment. The experiment is conducted on a self-constructed dataset which comprises of 1692 evidence and 337 facts from Chinese judgment documents. The evaluation results are shown in Table 4.

Table 4. Evaluation results of key elements extraction
Full size table

As the table presents, recall and precision are almost more than 70%. The evaluation results prove that our methods to extract key elements can lay a solid foundation for finding out connection points and calculating chain unit heads in later steps.

However, what we should not ignore is that results of extracting key elements based on semantic relation and part of speech are not very satisfactory. Especially, how to extract WHAT can be further optimized.

One way to solve this problem is to change features for maximum entropy model training. Those features we select now are too focused on words themselves. Another way is changing a better corpus. Furthermore, we would like to try to use CRF (Conditional Random Field) or SVM (Support Vector Machine) as our main algorithm.

3.4 Create Connection Points

In this section, we mainly focus on introducing the methods used to create connection points in evidence chain relational model. Even for legal experts, it’s not easy to create logical links when building evidence chain relational model. Our approach to create connection points cannot possibly obtain completely correct results. What we do is providing the most probable result.

3.4.1 Build Relationship Between Evidence and Fact

Evidence chain relational model we define contains all facts of a case, and each fact connects to related evidence chain by connection points, therefore, we have to build relationship between all of evidence and facts before building evidence chains. Because one of characteristics evidence chain has is uniqueness which means every chain proves only one fact [16]. For each of evidence in “unrelated evidence set”, we find out the most probable relevant fact.

A method to obtain correlation of different Chinese texts is calculating similarity coefficient based on characteristics of literal words, such as statistical character and semantic feature [19]. We build relationship between evidence and fact according to the number of same words they have. More same words mean more possibilities, so we choose the fact that has most same words with the evidence and build relationship between them. In this method, two points need to be paid attention to:

  1. (1)

    Semantic similarity of words. Two words to be compared must be the same kind of key elements.

  2. (2)

    Weight factor of words. Different weights can be set for different kinds of key elements according to the cause of action. For example, in drug cases, criminal facts are always that a group of drug dealers sells a drug at different times. The separation of facts in this circumstances is WHEN instead of WHAT, so WHEN requires high weights.

3.4.2 Build Relationship Between Different Evidence

Relationship between different evidence means the mutual corroboration based on chain unit heads. In other words, the same key elements different evidence has been able to be chain unit heads. The algorithm we use to find out same key elements is Hash algorithm. We use key elements as keys of a HashMap, if evidence contains a certain key element, the identifier of this evidence will be saved into the value-list of relevant key. After completing mapping, the value-list of each key is read in turn to find out candidate set of chain unit heads.

There are many jargon terms in Chinese judgment documents. These words appear in lots of evidence, but as we all know they are not chain unit heads. Although many published sets of stop words can be used, such stop words cannot meet our requirements. For example, “证据” (evidence), “事实” (fact), “原告” (plaintiff), “被告” (defendant), etc. occur in almost every judgment documents, but they are not common stop words. Besides that, some words that are not jargon terms also cannot be chain unit heads. For example, “行为” (behavior), “内容” (content), “过程” (process), etc. These words don’t have practical significance to be proof points.

To construct a stop words list that meets requirements talked above, we use 13571 judgment documents as our corpus and calculate IDF of each unique word. IDF (Inverse Document Frequency) refers to the inverse fraction of documents that contain a specific word. Words with low IDF can be regarded as potential stop words. After scanning all judgment documents and filtering common stop words, we calculate IDF of rest words and sort words in ascending order. Top N words have been chosen as stop words. The number N is chosen manually after we scan the result of sorting. Stop words we choose are added into the set of common stop words to construct the stop words list we use to filter candidate set of chain unit heads.

3.5 Display Evidence Chain Relational Model

In this section, we mainly focus on introducing the methods used to display evidence chain relational model. The Excel sheets can clearly present the facts and the contents of each evidence in detail and the graph can show the relationship in the model directly and comprehensively.

3.5.1 Generate Excel File

Evidence chain relational model should be saved as a format which is easy to store, read, understand and exchange, so we choose Excel. An evidence chain relational model built based on a judgment document is saved as an Excel file with two sheets:

  1. (1)

    Evidence sheet only focuses on evidence in the judgment document. Identifier, name, content, type, submitter, admissibility result, reason of admissibility result, chain unit heads and relevant key text of each evidence are set out in the sheet in turn. The relationship between chain unit heads and key text is one to one. Each chain unit head is a word which contains few information. If readers want to know the real meaning of a chain unit head, they have to find its location in the text of evidence and read the context. To provide convenience for readers, we extract the clause with a chain unit head in it as the relevant key text of this chain unit head.

  2. (2)

    Fact sheet focuses on not only facts but also relationship between evidence and fact. Identifier, name, content of each fact are set out in the sheet in turn. At the same time, relevant evidence and their chain unit heads are displayed after every fact.

3.5.2 Graphical Display

Actually, the research on evidence chain relational model has two directions. This paper introduces the part of reverse direction research. Forward direction research focuses on how to implement a system for judges to build evidence chain relational models by themselves conveniently and how to translate graphical models into reasoning sentences automatically.

As Fig. 3 presents, we can use the system with evidence chain relational model in Excel format as input. The system analyzes the Excel file and extracts the model. After automatic typesetting, the evidence chain relational model we built can be displayed on the interface clearly.

Fig. 3.
figure 3

System interface of forward direction research

Full size image

4 Conclusion

In this paper, we propose an approach to build evidence chain relational model based on Chinese judgment documents. We can extend our study on reasoning modeling in two ways. First, combine our evidence chain relational model with evidential Bayesian network. Bayesian network provides a means to identify and evaluate the hypotheses that may have produced the available evidence in a case and to assess their plausibility [20]. Evidential Bayesian provides a means to estimate how much more strongly evidence supports one hypothesis over another, and how strongly the addition or removal of evidence would affect the relative level of support for alternative hypothesis [21]. So we can do the numerical evaluation of argumentation and the credibility of reasoning. Second, combine the analysis of main section of verdict to construct the reasoning logic diagram. The diagram contains every step of reasoning. Each step has relevant judicial evidence, fait juridique, law and so on. The diagram shows the relationship between evidence and inference, shows each layer of reasoning, so as to achieve the visualization of reasoning in judgment documents.

When we complete the study that we talk about above, we will make a solid foundation for the evaluation of judgment documents reasoning, we can also achieve the goal of evaluating ‘fact of case’ which is one of the aspects of reasoning. If there are other methods to evaluate ‘laws of case’ and ‘style of writing’, the whole system can do a comprehensive, accurate and efficient evaluation of judgment documents reasoning, to provide technical support for improving the level of judgment documents reasoning to the Chinese People’s Court.