RuleRS: a rule-based architecture for decision support systems

Abstract

Decision-makers in governments, enterprises, businesses and agencies or individuals, typically, make decisions according to various regulations, guidelines and policies based on existing records stored in various databases, in particular, relational databases. To assist decision-makers, an expert system, encompasses interactive computer-based systems or subsystems to support the decision-making process. Typically, most expert systems are built on top of transaction systems, databases, and data models and restricted in decision-making to the analysis, processing and presenting data and information, and they do not provide support for the normative layer. This paper will provide a solution to one specific problem that arises from this situation, namely the lack of tool/mechanism to demonstrate how an expert system is well-suited for supporting decision-making activities drawn from existing records and relevant legal requirements aligned existing records stored in various databases.We present a Rule-based (pre and post) reporting systems (RuleRS) architecture, which is intended to integrate databases, in particular, relational databases, with a logic-based reasoner and rule engine to assist in decision-making or create reports according to legal norms. We argue that the resulting RuleRS provides an efficient and flexible solution to the problem at hand using defeasible inference. To this end, we have also conducted empirical evaluations of RuleRS performance.

1 Introduction

Decision makers in governments and enterprises often have to take decisions and create reports based on regulations (and other normative and legislative documents) and organisation specific data, where the organisation data is generally stored in enterprise (relational) databases. For example, Australian financial institutions are subject to Financial Sector (Collection of Data) Act 2001 for what (financial) information to report to the relevant regulators (e.g., Australian Prudential Regulator Authority), and government departments and agencies are required to comply with the Public Governance Performance and Accountability Act 2013 and Public Governance Performance and Accountability Rule 2014 for their annual financial reporting. The requirements about what, when and in what forms to report (and related exceptions) are given in the (relevant) regulations while the (financial and related) data is stored in the databases of the institutions that have to generate reports about the data.

Accordingly, in these scenarios, one has to perform some legal reasoning (for example to understand what are the actual requirements that apply in a given case) based on the information from their enterprise databases. Legal reasoning has two distinctive features: the first is that norms, generally, set base guidelines, but then they are open to exceptions, where the exceptions are expressed themselves as norms. Also, norms prescribe behaviours using concepts such as obligations, prohibitions, and permissions. Obligations and prohibitions can be violated, and legal frameworks provide conditions to compensate for violations. Relational databases are essentially first-order logic theories, where the tables are grounded instances of predicates and queries are first-order logic expressions about the predicates stored by the tables; as such they are not suitable to reason with deontic concepts and violations (Herrestad 1991), also, Governatori (2015b) argues that most existing Deontic Logics are not able correctly handle compensatory norms, and norms where deontic concepts imply other deontic concepts. Consequently, we need a logic that can handle exceptions and complex deontic notions. Defeasible Deontic Logic (Governatori et al. 2013) is a rule-based computationally oriented logic that proved suitable to handle legal reasoning and that does not suffer from the problem affecting other Deontic Logics (Governatori 2015a).

In this paper we investigate (and discuss the implementation and empirical evaluation of) a methodology to combine Defeasible Deontic Logic to handle the reasoning about the legal requirements, and relational databases to retrieve information from enterprise databases. The key intuition is that the norms governing the scenario are encoded as rules in the logic, while the facts are encoded as database queries. Accordingly, the focus of the paper is not to study whether the chosen logic is able to properly represent norms and legal reasoning, but whether the proposed framework is scalable enough to handle massive (enterprise) databases, to provide responses in a “reasonable” time.¹ In our opinion, the work addresses a shortcoming in the field of Artificial Intelligence and Law. Most approaches to legal reasoning have been tested on a very small scale scenarios and data. To be best of our knowledge, this is the first work testing legal reasoning on large scale databases, and thus can provide benchmarks for future developments.

For the evaluation of the framework we are going to examine two case studies: the first is based on New South Wales Mandatory Reporting Guidelines (NSW Government 2016), mandating child care centres (and other professionals) to report to appropriate authorities potential child abuse or neglect cases, where the determination where a case is to a decision support system for the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS, formerly AERS) (US Food and Drug Administration 2016). For the first case study, do the sensitive nature of the application combined to privacy and legal issues, it was not possible to use real data. Thus, instead of artificially creating a synthetic database, we decided to use an existing database (from a different domain) and test the set of rules and the database against different types of queries just to measure the performance. Therefore, the database queries were created for their structure not for their content. The second case study was evaluated for content and performance. Specifically, we use a sample database that contains information on adverse event and medication error reports submitted to the FDA generated from the publicly available FDA datasets. The datasets consist of (US Food and Drug Administration 2015) postmarket drug and biologic safety evaluations based on records and reports concerning adverse drug experiences on marketed prescription drugs for human use without approved new drug applications and the relevant regulation (US Government 2014). We illustrate the system for decision-making towards compliant or non-compliant information, in particular, on adverse event and medication error reports submitted to FDA using RuleRS system architecture and its internal components.

The paper is organised in seven sections. Section 2 briefly outlines defeasible logic. Section 3 introduces the RuleRS architecture. We include the case studies in Sect. 4 and the evaluation using case study data in Sect. 5. The following one describes some review of related works. Finally, we conclude with a discussion of implications for future research.

2 Defeasible logic

RuleRS is developed in the setting of Defeasible Logic (DL). Why DL? Indeed, DL is a formalism that has been successfully used for legal reasoning [and it has been proved that other formalisms successful in legal reasoning correspond to variants of DL (Governatori 2011)]. Defeasible Deontic Logic has been successfully used for applications in legal reasoning (Antoniou et al. 2001; Governatori 2005; Governatori and Shek 2013; Governatori 2015a) and it is has been shown that it does not suffer from problems affecting other logics used for reasoning about norms and compliance (Governatori and Hashmi 2015; Governatori 2015a). Thus Defeasible Deontic Logic is a conceptually sound approach for the representation of regulations and at the same time, it offers a computationally feasible environment to reason about them [Governatori and Rotolo (2008) proved that the logic is computationally feasible since since we can compute the extension of a theory in linear time]. In this paper, we are going to evaluate whether the proposed combination offers a practical solution for decision systems in the legal domain combining databases and norms (represented as rules). To this end, we examine two practical systems’ response time to identify a child at risk (NSW Government 2016) involving in a sample Child Care Management Systems database, and an adverse report is compliant with the relevant regulation system using two large sample databases generated from FDA datasets (US Food and Drug Administration 2015).

Defeasible Logic is “skeptical” nonmonotonic logic (meaning that it does not support contradictory conclusion), was originally proposed by Nute (1994). Since then it has been significantly used in the legal domain or closely related areas, such as modelling regulations (Antoniou et al. 1999), e-contracting (Governatori 2005; Grosof 2004), business processes compliance (Sadiq and Governatori 2015; Governatori and Rotolo 2010) and automatic negotiation system (Skylogiannis et al. 2007). The modelling of regulations in DL also offers support for “Decision support”, “Explanation”, “Anomaly detection”, “Hypothetical reasoning” and “Debugging” tasks. Decision support is used to infer a correct answer from given rules and regulations. DL is one of the possible solutions since regulations may contradict one another using the defeasible rules do not necessarily in force; instead they may be blocked by other rules with contrary conclusions (Antoniou et al. 1999).

A defeasible theory D [a knowledge base in defeasible logic, or a defeasible logic program (Maher 2001)], consists of five different kinds of knowledge: facts, strict rules, defeasible rules, defeaters, and a superiority relation. D is a triple $(F,R,\succ )$ where F and R are finite sets of facts and rules respectively, and $\succ$ is a superiority relation on R.

The language of DL consists of a finite set of literals, where a literal is either an atomic proposition or its negation. Given a literal $l, \sim\kern-2pt l$ denotes its complement. That is, if $l=p$ then $\sim\kern-2pt l=\lnot p$, and if $l=\lnot p$ then $\sim\kern-2pt l=p$.

Facts are logical statements describing indisputable facts, represented either in the form of states of affairs [literal or modal literal (refer to Sect. 2.1)] or actions that have been performed, and are considered to be always true. For example, “John is a human” is represented by: human(John).

A rule r, on the other hand, describes the relations between a set of literals [the antecedent A(r), which can be empty] and a literal [the consequence C(r)]. We can specify the strength of the rule relation using the three kinds of rules supported by DL, namely: strict, defeasible, and defeater.

Strict rules are rules in the classical sense: whenever the premises are indisputable (e.g. a fact) then so is the conclusion. For example,

$$\begin{aligned} human(X) \rightarrow mammal(X) \end{aligned}$$

which means “Every human is a mammal”.

It is worth mentioning that strict rules with empty antecedents can be interpreted the same way as facts. However, in practice, facts are more likely to be used to describe contextual information; while rules, on the other hand, are more likely to be used to represent the reasoning underlying the context.

Defeasible rules are rules that can be defeated by contrary evidence. For example, typically mammal cannot fly, written formally:

$$\begin{aligned} mammal(X) \Rightarrow \lnot flies(X) \end{aligned}$$

The idea is that if we know that X is a mammal, then we may conclude that it cannot fly unless there is other, not defeated, evidence suggesting that it may fly (for example that the mammal is a bat). A defeasible rule with an empty antecedent can be considered as a presumption.

Defeaters are rules that cannot be used, on their own, to draw any conclusions. Their only use is to prevent some conclusions, i.e., to defeat some defeasible rules by producing evidence to the contrary. For example the rule:

$$\begin{aligned} heavy(X)\leadsto \lnot flies(X) \end{aligned}$$

states that an animal is heavy is not sufficient enough to conclude that it does not fly. It is only evidence against the conclusion that a heavy animal flies. In other words, we do not wish to conclude that $\lnot flies$ if heavy, we simply want to prevent a conclusion flies.

A full definition of the proof theory can be found in Antoniou et al. (2001) and Billington et al. (2010). Roughly, the rules with conclusion p form a team that competes with the team consisting of the rules with conclusion $\lnot p$. If the former team wins p is defeasibly provable, whereas if the opposing team wins, p is non-provable.To conclude, let us consider D as a theory in DL (as described above). A conclusion of D is a tagged literal and can have one of the following four forms: $+\Delta q$ meaning that q is definitely provable in D (i.e., using only facts and strict rules); $-\Delta q$ meaning that we have proved that q is not definitely provable in D; $+\partial q$ meaning that q is defeasible provable in D; and $-\partial q$ meaning that we have proved that q is not defeasible provable in D.

Strict derivations are obtained by forward chaining of strict rules while a defeasible conclusion p can be derived if there is a rule whose conclusion is p, whose prerequisites (antecedent) have either already been proved or given in the case at hand (i.e., facts), and any stronger rule whose conclusion is $\lnot p$ has prerequisites that fail to be derived. In other words, a conclusion p is (defeasibly) derivable when: p is a fact, or there is an applicable strict or defeasible rules for p, and either all the rules for $\lnot p$ are discarded (i.e., not suitable) or every rule for $\lnot p$ is weaker than an applicable rule for p.

2.1 Defeasible Deontic Logic

It has been argued that legal reasoning requires two types of rules: constitutive and prescriptive rules. Constitutive rules are used to model definition of terms and parameters specific to legal documents, for example, the definitions of terms in an act, whereas prescriptive rules are applied for encoding the obligations, prohibitions, permissions, ..., and the conditions under which they enter into force according to a specific legal document. To correctly model the provision corresponding to prescriptive norms, we have to supplement the language with deontic operators. In this respect we follow the classification proposed by Governatori (2013) and Hashmi et al. (2015). In addition, the logic has mechanisms to terminate and remove obligations [see Governatori and Rotolo (2010) for full details]. For obligations and permission we use the following notation:

• $[\hbox {P}]p$: p is permitted;

• $[\hbox {OM}]p$: there is a maintenance obligation for p²;

• $[\hbox {OAPP}]p$: there is an achievement preemptive and perdurant obligation for p;

• $[\hbox {OAPNP}]p$: there is an achievement preemptive and non-perdurant obligation for p;

• $[\hbox {OANPP}]p$: there is an achievement non preemptive and perdurant obligation for p;

• $[\hbox {OANPNP}]p$: there is an achievement non preemptive and non-perdurant obligation for p.

Compensations are implemented based on the notion of ‘reparation chain’ (Governatori and Rotolo 2006). A reparation chair is an expression

$$\begin{aligned}{}[\hbox {O}_{1}]c_{1}\otimes [\hbox {O}_{2}]c_{2}\otimes \cdots \otimes [\hbox {O}_{n}]c_{n}, \end{aligned}$$

where each $[\hbox {O}_{i}]$ is an obligation, and each $c_{i}$ is the content of the obligation (modelled by a literal). The meaning of a reparation chain is that we have that $c_{1}$ is obligatory, but if the obligation of $c_{1}$ is violated, i.e., we have $\lnot c_{1}$, then the violation is compensated by $c_{2}$ (which is then obligatory). But if even $[\hbox {O}_{2}]c_{2}$ is violated, then this violation is compensated by $c_{3}$ which, after the violation of $c_{2}$, becomes obligatory, and so on.

Defeasible Deontic Logic allows deontic expressions (but not reparation chains) to appear in the body of rules. Thus we can have rules like:

$$\begin{aligned} restaurant, [\hbox {P}]sellAlcohol \Rightarrow [\hbox {OM}] showLicense \otimes [\hbox {OAPNP}] payFine \end{aligned}$$

The rule above means that if a restaurant has a license to sell alcohol (i.e, it is permitted to sell it, $[\hbox {P}]{sellAlcohol}$), then it has a maintenance obligation to expose the license $([\hbox {OM}] {showLicense})$, if it does not then it has to pay a fine $([\hbox {OAPNP}] {payFine})$. The obligation to pay the fine is non-pre-emptive (meaning that it cannot be paid before the violation). The logic is equipped with a binary relation over rules, called superiority relation, that allows us to handle rules with conflicting conclusions: for example a rule r setting a general prohibition and a second rule s that derogates the prohibition permitting the conclusions. This type of situation is common in legal reasoning and can be modelled by saying that s is “stronger” than r, in symbols $s>r$. If both rules apply, we will say that s defeats r. For full a description of the logic and its features, see Governatori (2005), Governatori and Rotolo (2010) and Governatori et al. (2013).

The reasoning to determine what obligations, prohibitions, and permissions are derivable from a set of facts and a setof rules is as follows.

An obligation $[\hbox {O}]p$ (where $[\hbox {O}]$, $[\hbox {O}_x]$ and $[{{\mathsf {D}}}_y]$, in the description below, are placeholders for the obligations described above) is derivable if:

• 1. $[\hbox {O}]p$ is given as one of the facts, or

• 2. there is a rule

  $$\begin{aligned} r: a_1,\dots a_n\Rightarrow [\hbox {O}_1]p_1\otimes [\hbox {O}_m]p_m\otimes [\hbox {O}]p\dots \end{aligned}$$

  such that

  • (a) for all $1\le i\le n$, $a_i$ is provable, and

  • (b) for all $1\le j\le m$, $[\hbox {O}_j]p_j$ and $\lnot p_j$ are provable, and

  • (c) for all rules

    $$\begin{aligned} s: b_1,\dots ,b_k\Rightarrow [{{\mathsf {D}}}_1]q_1\otimes [{{\mathsf {D}}}_l]q_l\otimes [{{\mathsf {D}}}]p' \end{aligned}$$

    such that $p'$ is the negation of p, either

    • (i) exists $1 \le i\le k$ such that $b_i$ is not provable, or

    • (ii) exists $1\le j\le l$ such that either $[{{\mathsf {D}}}_j]q_j$ or $\lnot q_j$ is not provable, or

    • (iii) r defeats s.

The idea is that there must be a rule that fires: so all the elements in the antecedents are provable (a), and in case the conclusion is an obligation for a reparation, all the obligations before it have to be violated. Thus, the violated obligations were in force (thus the obligations were provable) and we have evidence that it was violated (thus the negation of the content of each violated obligation is provable) (b).Also, we have to ensure that there are no rules for the opposite that fire (c), and if they do, these rules are weaker than the rule for the obligation we want to conclude.

For permission, we have the same conditions, but where we use $[\hbox {P}]p$ instead of $[\hbox {O}]p$; also, we conclude $[\hbox {P}]p$ if we can conclude $[\hbox {O}]p$. Due to space reasons, readers interested in understanding the semantics, deontic operator conversions, conflict detections, conflict resolutions, and algorithm implementing this rule-based system are referred to Governatori and Rotolo (2004), Governatori and Rotolo (2008), Governatori and Rotolo (2010) and Governatori et al. (2013) for details.

3 RuleRS architecture

In this section, we will introduce the RuleRS design architecture and technical details (refer to Fig. 1). Overall, RuleRS consists of four main system components. In particular, the key system components of RuleRS are: (1) I/O Interface, (2) Predicates, (3) Formal Rules , and (4) SPINdle Reasoner. In the rest of this section, we give a short outline of the RuleRS internal components and their functions (refer to Fig. 1).

Fig. 1

Rule-based reporting system (RuleRS)

3.1 I/O interface

The I/O interface is implemented in Java to bridge RuleRS components and interacting with each other. The I/O interface is used to compile predicates (SQL or JSON files) and to generate facts and contexts in formal notation in DFL syntax, and SPINdle reasoner receives this as a parameter. The I/O interface also displays the final remarks or comments for each of the incidents and predicates. Figure 2 includes a sample RuleRS interface.

Fig. 2

Generate SQL/JSON for predicate

3.2 Predicates

The values encoding a regulation and the databases (schemas) used in conjunction with the rules are in general developed independently are likely have a different vocabulary in general. There is no direct correspondence between the literals used by the rules and the table/attributes of the database schema. Accordingly, we have to establish a mapping among them to enable the integration of rules and instances in the database. The fundamental idea behind predicates is that data stored in the database corresponds to facts in a defeasible theory and these facts can be retrieved from the database using queries. Thus each fact corresponds to a (SQL) query and a predicate is a statement that can be true or false depending on the value of its arguments/variables. A predicate with n arguments is just an n-ary relation.

A predicate in RuleRS corresponds to a database view, i.e., a named query, where the name is literal to be used by the defeasible rules. The details are the query to be run to determine if the predicate is true or false for a given set of parameters. In case the output of the query is not empty, the predicate is true and is passed to the defeasible theory as fact.

In RuleRS, predicate consists of two components: (1) predicate name and (2) predicate details. Predicate name represents the action(s), condition(s) or indisputable statement(s), and passed on to the rule engine, SPINdle as a defeasible fact (literal and modal literal) (Governatori and Rotolo 2004, 2008, 2010) or actions that have been performed. For example, the fact “There is a risk for an incident” is represented by “riskForIncident” and passed as “$>>riskForIncident$” to SPINdle if it is returned as true from the relational database. “Predicate details” includes the “incident details” and is stored either as an SQL Statement or (after conversion) in lightweight data-interchange JSON (JavaScript Object Notation) syntax³ (easy for humans to read and write, easy to generate, and easy to parse for machines) to create a bridge between the data stored in the database and the terms passed as predicates (input case) to the rule engine. The SQL or JSON statements can be created in the initialisation of RuleRS with all of the incidents along with all of the predicates for each of the incidents or dynamically add it later.

Incident ID and relevant details of the incidents are also included for each of the predicates and named the predicates with relevant incident information such as “riskForIncident.sql” (for SQL statement) or “riskForIncident.json” (for JSON Statement). The following snippet illustrates the syntax adopted by RuleRS: “riskForIncident” predicate using SQL Statement:

In this example, $\fbox {IncidentDetails}$, $\fbox {IncidentDetails1}$, $\fbox {IncidentDetails2}$ are substituted for the place-holders in the “riskForIncident” predicate from relational databases for the $\fbox {incidentID}$ $\fbox {'XXXXXX}$.

“riskForIncident” predicate using JSON Statement:

(Yu et al. 1992; International Business Machines Corporation 1993, 2001), We could further classify predicates according to the query types⁴ and SQL statement⁵ composition to each kind of constructs such as WHERE, GROUP BY, HAVING and ORDER BY.

In the next step, the records and incidents for which there is a match in the relational database are transformed into predicates to be used by the SPINdle rule engine (Lam and Governatori 2009), and forwarded to SPINdle for further processing using the I/O interface to make the process dynamic.

3.3 Formal rules

One of the features of RuleRS is its ability to perform reasoning based on (legal) requirements. As we alluded to in the introduction such regulatory requirements are represented as formal rules in Defeasible Deontic Logic (Antoniou et al. 2001; Nute 1994); to enable their use with the rule engine used by RuleRS (SPINdle, see the next section) the rules are stored in the DFL format (Lam and Governatori 2009). At this stage the rules are created manually by legal knowledge engineers and stored in a knowledge-base.

3.4 SPINdle reasoner

SPINdle Reasoner⁶ (Lam and Governatori 2009) is a Java-based implementation of DL (Antoniou et al. 2001; Nute 1994) that computes the extension of a defeasible theory. SPINdle supports Modal DL, all types of DL rule, such as facts, strict rules, defeasible rules, defeaters, and superiority. A full description of SPINdle is out of scope for the current paper, we refer the interested readers to Lam and Governatori (2009) for details. In summary, SPINdle is a tool which accepts rules, facts, monotonic and non-monotonic (modal) rules for reasoning with inconsistent and incomplete information. In RuleRS, SPINdle Reasoner receives the formal facts, contexts as predicates from predicate file generated for data stored in the associated relational databases and computes definite or defeasible inferences which are then displayed by the I/O interface.

4 Case studies

The fitness of Defeasible Deontic Logic for modelling norms has been demonstrated elsewhere (Antoniou et al. 2001; Governatori 2005; Governatori and Shek 2013; Governatori 2015a; Islam and Governatori 2015) (refer to Sect. 2). Accordingly, the aims of this section is to evaluate whether RuleRS offers a practical solution to the issue of providing a viable combination of norms and databases. Specifically, the focus is to determine whether the response time is appropriate for practical applications. In general, the response time consists of two components: the time required to execute a SQL query and the time needed to compute the extension of a theory using SPINdle.

In this section, we illustrate RuleRS with the help of two case studies to demonstrate possible applications of the RuleRS architecture.

4.1 Case study: sample online system

In this section, we introduce Case Study 1 a ChildSafe Care Online Management Systems (ChildSafeOMS) and its relational database. ChildSafeOMS maintains a relational database to record child enrolment, attendance, abuse and/or neglect data and relevant system information.

The integration of RuleRS and the underlying relational database extend the ChilSafeOMS’s functionality. In particular, ChildSafeOMS introduces new functionalities including the automation of early identification of children at risk of abuse and neglect.

More specifically, the system contains three sources of information: (1) a child care database ChildSafeOMS with the data described above; (2) a set of defeasible rules encoding the decision trees, definitions and (normative) guidelines of New South Wales Mandatory Reporter Guidelines (NSW Government 2016). We have translated the rules manually from the source document; and (3) a set of bridging statements relating the terms of the regulations and the fields (and data) in the ChildSafeDB. These statements are obtained from cross-analysis of the terms and fields in the two related components.

One of the aims of the system is the early identification of children at risk, in particular, a risk of abuse and neglect. In the rest of the section, we describe the component outlined above.

4.1.1 Sample database

We start by describing the relational database we used in this experiment. Sample RuleRS is a large dataset with more than 11 million records in total, and takes up 2017 MB. To make sure that the readers are not overwhelmed by the size of the data, we have provided brief introductions to RuleRS dataset given the high sensitivity and the nature of the data and application (and legal implications of analysis such data), it is not possible to use real-life databases. However, the aim of this paper is not to build such an application but to validate the methodology of integrating defeasible (legal) reasoning with database technology. As we have already discussed, a vital component of the method is the definition of the predicates/literals to be passed to the reasoners as the facts of a Defeasible theory obtained as an SQL query. In other words, SQL queries define the meaning of literals used in a defeasible theory regarding data and the schema of a database. As we have explained our focus is the response time; thus we can ignore the actual content of the undergoing database. The time taken to answer an SQL query depends on the structure and the size of the database and the structure of the SQL query itself (Osman and Knottenbelt 2012).

For this case study, we have created sample RuleRS database based on dataset available from stat-computing.⁷ Most of the data comes initially from RITA.⁸ These data have derivable variables removed, are packaged in yearly chunks and have been more heavily compressed than the originals. We have downloaded data from 1998 to April 2008. Along with other columns in the tables, each of the tables contain various fields such as “Year”, “Quarter” (1-4), “Month”, “DayofMonth” (Day of Month), “DayOfWeek” (Day of Week Analysis). There are more than 1 million records in each of the tables, and we stored the data in a separate table for each of the years. The data consists of flight arrival and departure details for all commercial flights within the USA. We have inserted to a new column “child_crn” (Child’s Child Reference Number) in each of the year’s data to map data with “children” table. The idea of column insertion using a “sample scenario” is that the child visited an airport during that year. For this experiment, we skipped other details of the year’s columns such as if the children stuck in an airport because their flight was delayed or canceled.

Sample RuleRS database also contains many tables with information about children in a child care centre, types of abuses and their descriptions, and records for possible instances of abuses. The typical work-flow to populate the information is that educators of child care insert data about a child in the database to report day-to-day activities. In case they have concerns for some forms of abuse or neglect, the system offers an interactive questionnaire where the educator answers a set of ‘Health and Welfare’ emotional and physical questions of the type “In your knowledge, do any of the following environments apply to the child?”. Here the questions pertain to any of the abuse types and ask if they were aware of any child was abused or if they have any concern about a particular child. The systems populate the database according to answers provided by the educators/carers. Additionally, the educator/child carer also fills up five additional information along with the abuses types: (i) Child Abuse ID (auto-incremental); (ii) Child crn (Child Reference Number); (iii) Abuse ID; (iv) Abuse type id (Type of child abuse); and Abuse date.

4.1.2 From database records to predicates

As we discussed above to aim of RuleRS is to draw inferences from a database supplemented by a rule systems encoding legal knowledge. In this setting, the information stored in the database is queried to provide the facts for the SPINdle reasoner in the form of predicates. After the facts are queried from the ChildSafeDB, the SPINdle reasoner draws a set of conclusions using the facts and the defeasible rules encoding the decision trees of the NSW mandatory reporter guidelines and a set of additional rules, eventually provided by domain experts, giving further indicators for risks of neglect or abuse.

For the extraction of facts, RuleRS is equipped with SQL/JSON files containing the definitions of the predicates used by the SPINdle component regarding either simple attribute names and values or by SQL queries. For example, according to the guideline, a person is considered a child if her/his age is less than fifteen years. Several steps in the decision trees depend on whether a person at risk of abuse is a child or not. RuleRS In RuleRS, this is represented by the predicate beingChild and contains the following (JSON) definition:

where $id is a parameter given as input by the end-user, and ${today - 15y} is a computation using the system variable $today.

The I/O interface takes the user input, examines the definitions, transforms the descriptions into appropriate SQL queries, and executes the queries to return facts and context as output. For example, for the predicate beingChild and $id=123456789A ChildsafeOMS produces the following JSON snippet, encoding the predicate and the context that it holds.

The predicate is then used in the successive reasoning phase, while the context is used in case of reasoning phase determines that a report has to be submitted to a relevant authority; the RuleRS is used to populate the required reports.

4.1.3 Sample rules

This section includes sample some rules in SPINdle format. Some of the formal rules are converted from the NSW mandatory reporter guidelines (NSW Government 2016). The guidelines are formulated as decision trees, and they specify (i) when an educator of child carer has to report an abuse or potential abuse (ii) under what conditions one has to inform. Also it contains definitions of types of violation or possible abuses and indicators of these.

Consider for example the provision (as part of a decision tree) prescribing to report immediately to CS if there is a situation where you are aware that a child has been sexually abused even though he/she hasn’t told you or the child made a clear, unambiguous statement of sexual abuse (NSW Government 2016). We can represent the rule as If “ChildAbusedSexually” has occurred, then there is an obligation (non-persistent, pre-emptive, achievement) to do “reportToCSImmediately” [refer to Governatori and Rotolo (2010), Governatori (2013) and Hashmi et al. (2015)) for other types of obligations]. Its formal representation in SPINdle is as follows:

$$\begin{aligned} ChildAbusedSexually \Rightarrow \, \hbox {[OANP]}\, reportToCSImmediately \end{aligned}$$

Furthermore, the rule set contains more rules obtained from the definitions of what counts as a child being sexually abused and possible indicators for different types of abuse, where such definitions are given as decision trees themselves.

The next example concerns the situation where a child or young person exhibits a (persistent) problematic sexual behaviour versus other. In this case, an educator has different options to whom report. In case the parent of carers are aware of the problem and they have responded appropriately, the educator has to report to the Child Well fare Unit (consultWithCWU). However, if the initiating child/young person has continuing or imminent contact with the victim and there where some coercion or the victim is in a situation of inferiority, then the situation has to be reported immediately to Community Services (CS). Otherwise, the normal procedure is to file a formal report to CS (NSW Government 2016, p. 19). In case of a problematic behaviour without further conditions, the educator has to continue to monitor the situation.The following rules can represent this part of the decision tree:

$$\begin{aligned}&r_1: PersistentSexualBehaviourVsOther,\\&\quad ParentRespondedAppropriately, \lnot \,\hbox {[OANP]}\, reportCS,\\&\quad \lnot \,\hbox {[OANP]}\, reportCSImmediately \Rightarrow \,\hbox {[OANP]}\,consultWithCWU. \end{aligned}$$

The rule above describes the decision when the parents or carers are aware of the situation and responded appropriately. However, to ensure a single outcome (behaviour) we have to derive the failure of other obligations, namely report to CS or report to CS immediately.

$$\begin{aligned}&r_2: SexualBehaviourVsOther, CoercionOrInferior,\qquad \\&ContactWithVictim \Rightarrow {\hbox {[OANP]}} reportToCSImmediately \end{aligned}$$

Above is the most specific and its antecedent subsumes all other antecedents thus there is no need to include the failure of the other obligations.

$$\begin{aligned}&r_3: SexualBehaviourVsOther, CoercionOrInferior,\\&\quad \lnot \,\hbox {[OANP]}\,reportToCSImmediately \Rightarrow \, \hbox {[OANP]}\, reportToCS \end{aligned}$$

Above is the second most specific rule; thus its antecedent has to include the failure of the rule that is more specific that it, that is the obligation of the conclusion of $r_2$.

Finally, the following rule, whose conclusion is a maintenance obligation, is triggered when the other obligations do not apply.

$$\begin{aligned}&r_4: SexualBehaviourVsOther, \lnot \,\hbox {[OANP]}\, reportCSImmediately,\\&\quad \lnot \,\hbox {[OANP]} \,consultCWU, \lnot \,\hbox {[OANP]}\, reportCS \Rightarrow \,\hbox {[OM]}\, monitor \end{aligned}$$

The SPINdle rules are derived directly from the decision trees provided in the NSW Mandatory Reporter Guidelines (NSW Government 2013). A decision tree can be represented as a set of classical nested if-then-else rules, where for every node (binary decision, question) there is a predicate/proposition corresponding to it. For the representation of a decision tree in Defeasible Deontic Logic we needed a rule for each “positive” outcome (in our case a decision to report a case to a relevant authority or to the determination that a particular type of situation occurred). The conclusion of the rule is the proposition labelling the leaf node, and in the antecedent or body of the rule we have the proposition labelling the nodes from the root of the tree to the particular leaf node, negated or not depending on the conditions on the path. Arguably, the advantage of using Defeasible Deontic Logic as formal representation and implementation of a decision tree is the reduced complexity (no need to create nested if-then-else structures), and reduced number of rules [since, conditions leading to no outcome, i.e., no report in our case, are not needed, and are such situations are handled directly by the defeasibility of the rules, see Governatori et al. (2009)]. Furthermore, the adoption of Defeasible Deontic Logic allows us to use negated deontic literals in the body of rules. This corresponds to use a form of negation of failure, which requires either the duplication of the conditions leading to the failure of other reporting conditions, or the explicit representation of all else conditions in the if-then-else.

Notice that is it possible to formally show that the rules we created are equivalent⁹ to the decision trees provided in the NSW Mandatory Reporter Guidelines (NSW Government 2013). However, since this is to investigate the combination of rules and database, we refrain from proving the equivalence.

4.1.4 Sample predicates

As we have already alluded to the database used in this case study has been chosen for it being available and it size. The subject matter of the database is irrelevant to the content of the rules and the intended application of the rules. The queries are not relevant for content, but just to test the performance of RuleRS with different types of SQL queries (Yu et al. 1992; The PostgreSQL Global Development Group 1996–2015; International Business Machines Corporation 1993, 2001). Alternatively, we could use JSON format too. Each of the queries represents a single predicate, which passes as an input parameter to Spindle (see Sect. 3.4). The following snippet illustrates sample test predicates in SQL format used for this case¹⁰:

Predicate 1 beingChild

Predicate 2 riskForIncident6

Predicate 3 riskForIncident8

Predicate 4 riskForIncident13

Predicate 5 riskForIncident16

4.2 Case study: FAERS

In this section, we illustrate RuleRS as decision support system with the help of a case study to demonstrate possible applications of the RuleRS architecture. Accordingly, we introduce the sample database based on FAERS (US Food and Drug Administration 2016) records.

4.2.1 Sample database

The sample database based on the FAERS records that we have downloaded and converted from into PostgreSQL database format. For the experiment reported in this paper, we use the records for the first quarter 2014. The selected sample FAERS is a large dataset with more than three million forty thousand records in total and makes up 342 MB. To make sure that the readers are not overwhelmed by the size of the data, we have provided brief introductions to the dataset based on FAERS records.

The structure and format of the seven tables of FAERS from the first quarter of 2014 are explained in Table 1. This table also includes the sample examples, FAERS tables organisation, and content details.

Table 1 FAERS tables

4.2.2 From database records to predicates

As we discussed above the aim of RuleRS is to draw inferences from a database supplemented by a rule systems encoding legal knowledge. In this setting, the information stored in the database is queried to provide the facts for the SPINdle reasoner. After the facts are queried from the FAERS database, the SPINdle reasoner draws a set of conclusions using the facts and the defeasible rules encoding the regulations specify the records and reports concerning adverse drug experiences on marketed prescription drugs for human use without approved new drug applications (US Government 2014).

For the extraction of facts, RuleRS is equipped with SQL/JSON files containing the definitions of the predicates used by the SPINdle component regarding either simple attribute names and values or by SQL queries.

4.2.3 Sample predicates

We have used several predicates and queries for testing scenarios which are written in SQL. Alternatively, we could use JSON format too. Each of the queries represents a single predicate, which passes as an input parameter to SPINdle (see Sect. 3.4). The following snippet illustrates sample test predicates: $ICSRs\_contain\_Patient\_age$ in SQL format:

Predicate 6 report_Patient_age_to_FDA

Predicate 7 report_Suspect_medical_product_name_to_FDA

The predicates are then used in the successive reasoning phase (we may process several predicates by joining them in a single run), while the context is used in case of reasoning phase determines that a report has to be submitted to a relevant authority; the context is used to populate the required decision and reports.

4.2.4 Sample rules

This section includes sample rules which are manually converted from U.S. ELECTRONIC CODE OF FEDERAL REGULATIONS–Title 21: Food and Drugs–PART 310–NEW DRUGS–Subpart D–Records and Reports US Government (2014) in SPINdle format. The regulations specify the records and reports concerning adverse drug experiences on marketed prescription drugs for human use without approved new drug applications. In particular, the regulations list the reporting requirements for Manufacturers, Packers, and Distributors (MPD) and information reported on various life-threatening serious and unexpected adverse drug experience for Individual Case Safety report (ICSR). Additionally, the regulations contain definitions of various adverse drug experiences.

Various examples can be made as the rule set (US Government 2014) contains more rules obtained from information reported on ICSRs. Consider for an example the provision (as part of informed on ICSRs) prescribing to report electronically to FDA as ICSRs include “Patient age” while report to FDA. We can represent the rule as While an obligation (persistent, non-pre-emptive, achievement) report on ICSRs to FDA, there is an obligation (persistent, non-pre-emptive, achievement) to include “Patient age” . Its formal representation in SPINdle is given in rule $r_1$:

$$\begin{aligned} r_1: \,\hbox {[OAPN]} \, {report\ on\ ICSRs\ to\ FDA} \Rightarrow \,\hbox {[OAPN]}\, {report\ Patient\ age\ to\ FDA} \end{aligned}$$

Similarly, ICSRs include “Suspect medical product name” while report to FDA. We can represent the rule as While an obligation (persistent, non-pre-emptive, achievement) to report on ICSRs to FDA, there is an obligation (persistent, non-pre-emptive, achievement) to include “Suspect medical product name”. Its formal representation in SPINdle is as $r_2$:

$$\begin{aligned} r_2&: \,\hbox {[OAPN]} \, {report\ on\ ICSRs\ to\ FDA}\\&\Rightarrow \, \hbox {[OAPN]}\, {report\ Suspect\ medical\ product\ name\ to\ FDA} \end{aligned}$$

The rules $r_1$ and $r_2$ describe the decision when voluntary (aka direct) and mandatory report to FDA by companies are compliant or non-compliant. However, to ensure a single outcome (behaviour) we have to derive the failure of other obligations too. The rules $r_1$ and $r_2$, thus their antecedent (i.e., Patient age and Suspect medical product name) has to include the failure of the rules that is more specific, that is the obligation of the conclusion of $r_1$ and $r_2$ respectively for ICSRs. Finally, whose conclusion is an achievement preemptive obligation non-persistent (OAPN) obligations, is triggered when the other obligations do not apply. Figure 4 includes sample RuleRS output, where the predicates with “-” sign were non-compliant and have not been included in the ICSRs while reported to FDA.

5 The evaluation of RuleRS

As we have already pointed out, we are interested in evaluating whether the methodology underlying provides a sound solution but we want to identify if the solution offered by RuleRS is a viable solution for practical applications. To this end, we provide an empirical evaluation of the response time. As we remarked in Sect. 4.1.1 the database for ChildSafeOMS is used only to evaluate the response time, and it does not provide any insight about the domain application.

The database used for the second case study presents data specific to the application. Accordingly, we evaluated the application for correctness, this means to evaluate whether the rules we have encoded properly translate the existing FDA regulation. To this end, we randomly selected twenty events from the database, and we manually check them for compliance with the regulation. In the second phase, the same twenty events were automatically analysed using RuleRS. We have a perfect match between the manual analysis and the automated analysis. The principal evaluation, however, concerned the response time.

5.1 Evaluation methodology

See Algorithm 1 that is used for the evaluation process. In this algorithm, the evaluation process measures the predicate response time which is measured based on the process used in a database. We use a list of predicates and inputs and receive Defeasible Conclusion (a set of literals) as output. Since there is a caching time involved, each query is executed for 10 times consecutively and the average time is computed. We limit the query for top 20 events from FAERS database for each test case. In this process, predicates are executed under six groupings (randomly selected): 1 predicate, 2 predicates, 5 predicates, 10 predicates, 15 predicates and 20 predicates. The operating systems provide various system calls that return the current time (Currim et al. 2013). We use Java method currentTimeMillis(),¹¹ which returns the operating system current time in milliseconds (ms). We measure the time required to execute each of the predicates (i.e., SQL query). We record query response time into several steps using currentTimeMillis() method, examining (a) Predicate response time (b) Total response time in millisecond for 10 executions and generate inferences using RuleRS. that is used for the evaluation process.

5.2 Experimental set-up

After deciding to experiment, RuleRS, our next task was to set-up the experiment using sample database. We have performed the test on a Mac computer. The environment is as follows:

• Model Name: MacBook Pro

• Model Identifier: MacBookPro11,2

• Processor Name: Intel Core i7

• Processor Speed: 2.2 GHz

• Number of Processors: 1

• Total Number of Cores: 4

• L2 Cache (per Core): 256 KB

• L3 Cache: 6 MB

• Memory: 16 GB

5.3 The evaluation results

We now present some of the information obtained after applying RuleRS to study the system evaluation mentioned above. In the particular environment we studied, the following observation can be made for the execution behaviour of predicates.

In the evaluation process using Case Study 1 (Sect. 4.1), 30 predicates were executed and Fig. 3 sketches response times for each of the predicates. The average response time per predicate is 171.19 ms, while that “riskForIncident13”¹² has highest average response time 1696.1 ms and “beingChildAged11To12” (return 0 response with BETWEEN predicate) has lowest average response time 0.4 ms. On average (per run), total response time for 20 predicates is 5142 ms, while that Spindle response time is 42.2 ms and total response Time 5184.2 ms. Figure 5 sketches the response time for each of the predicates used for the evaluation. Furthermore, 60% of the predicates response within a small amount of time (between 0.4 and 7.6 ms). There are 12 predicates; each takes a significant amount of time to the response which is listed in Table 2. These predicates used large transactions, and there is a presence of cross joins with other large tables.

Table 2 Sample Predicate Response Time and analysis

Fig. 3

Case Study1: predicate response time

We now present some of the information obtained for the second case study.

Fig. 4

Sample RuleRS output

Fig. 5

Predicate response time

In this evaluation process, SQL response time (mean) was calculated under six groupings (randomly selected): 1 predicate, 2 predicates, 5 predicates, 10 predicates, 15 predicates and 20 predicates which can be found in Fig. 5. For example, using “10 predicates”, the average total SQL response time was 302.695 ms and SQL response time per predicate is 30.2695 ms and for “15 predicates”, the total response time is 311.89 ms and per predicate is 20.791 ms. Figure 5 also reveals that there has been a gradual increase in the total SQL response time while steady decrease in per predicate response time. From the experiment data we have extrapolated that the query for each predicate would take approximately 5ms. Furthermore, the FDA regulation for FAERS requires approximately 200 rules, we estimate that the time need to compute a report for each single incident would take about 1.5 s. Thus we believe that the proposed approach offers a viable solution for an online system to check whether a new record complies with the data collection requirements mandated by the underlying FDA regulation. Furthermore, it would take approximately 8 h on a laptop to audit the whole data that makes up 342 MB with more than three million forty thousand records in total for a quarter.

6 Related work

We began the RuleRS architecture development by describing our analysis of the fact that various businesses and agencies may have to generate decisions and reports (along with other obligation) based on regulations and policies align with their existing database records. Whereas traditional database management systems (DBMSs) are passive in the sense that their general focus has been to execute commands (e.g., query, update, delete) as and when requested by the user or application program (Paton and Díaz 1999), rule engines cooperate with the rule repository and the underlying database.

Why we need rule engine? Prior studies have established various advantages of improving systems using rule engines.¹³ The key aspects of using rule-based system can be listed as follows: natural knowledge representation, uniform structure, separation of knowledge from its processing and dealing with incomplete and uncertain knowledge.¹⁴

Prior approaches for rule engines can be classified into one of five types as Prolog-based, deductive databases, production and reactive rules, triple engines, and knowledge bases (Liang et al. 2009). The prolog-based system is defined as a top-down inference engine. While various applications have been used prolog-based system (e.g., XSB¹⁵; Yap¹⁶), their common focus has been to provide a complete environment for building applications (Liang et al. 2009). Deductive databases, which are the extension of the database management system, typically structure query language and design storage around a logical model of data (Kozák 2011). These approaches can be seen in various systems: DLV¹⁷; IRIS¹⁸; and Ontobroker.¹⁹ Production and reactive rules-based systems are the bottom-up engines where rules have actions in the consequent (Liang et al. 2009). All of these systems (e.g., Drools²⁰; Jess²¹ Prova²²) are based on (a version of) the Rete algorithm (Liang et al. 2009). Triple engines have been used two rule engines: a bottom-up engine and a top-down one for Semantic Web applications (Liang et al. 2009). A few prior systems that used rule engines based on triples are Jena,²³ SwiftOWLIM²⁴ and BigOWLIM.²⁵ Knowledgebase engine such as CYC²⁶ is an integration of domain knowledge for solving classical and common sense reasoning based complex problems. According to Liang et al. (Liang et al. 2009), although Ontobroker was not the best system in all of these rule-based system, Ontobreaker (and partly DLV) was assigned to be the best performing systems.

The architecture of RuleRS is inspired by the PLIS+ application which is a rule-based Personalised Location Information System (Viktoratos et al. 2012). Similar to PLIS+, RuleRS collects and evaluates rules on-the-fly and delivers specialised and contextualised information/inferences according to rule-based policies.

Indeed, another critical issue of the RuleRS architecture is to search and extract relevant information from a relational database and represent it as defeasible facts. RuleRS employs SQL/JSON syntax for describing facts and contextualised information as an intermediate step to bridge relational database information with a rule engine/reasoner.

To summarise, there is a need for a more fine-grained architecture to integrate relational databases, with a logic-based reasoner and rule engine for assisting to take decisions or create reports according to guidelines, policies or regulations that, in addition to being developed using appropriate research methods, should be readily evaluated. A researcher should be able to ensure designed artifacts are reliable and valid by checking for evaluation with other evidence. The evidence is not limited but we can include as experimental models for validating technology could be comprehensively classified as project monitoring, assertion, field study, literature search, legacy, lessons learned, static analysis, replicated, synthetic analysis, dynamic analysis, and simulation (Zelkowitz and Wallace 1998). Experiments could also be classified as “in vivo” (when it is run at a development location) and “in vitro” (when it is run in an isolated, controlled setting) (Basili 1996). Also, experiments could be classified as nine types which could further divide into three general categories: a quantitative experiment (identify measurable benefits of using a method or tool), a qualitative experiment (assess the features provided by a method or tool, and a benchmarking experiment to determine performance (Kitchenham 1996).

In this paper, we also argued that DL is suitable for making decisions based on regulations, policies and relational database information. Recent research has shown that DL is suitable for modelling and inferring from evidence and reasoning in real-world applications where the conflicting situation may appear simultaneously. In the introduction section, we included some of these claims.

None of the prior studies found met all of these requirements, as their focus had largely been on establishing the rule engine by restricting only human behaviour, without direct connection with any software system (Hu et al. 2009) or database systems. In the remainder of the paper, we examine the premise that combining databases with logic reasoner and rule engine into fine-grained report according to regulations and policies.

7 Conclusion

Our preliminary step was to describe an ongoing work on the integration of existing Online Management system with compliance and regulations, which also combines Defeasible Logic and SPINdle rule reasoner. In particular, we extend the SPINdle rule reasoner to generate reports for a possible consequence of the risk involving business processes. Classically, compliance and regulations perform a significant role in society, and business processes have to comply with such policies to avoid future risks. It turned out that this type of motivations is defeasibly captured by a rule-based approach to static decision trees for Online Mandatory Reporter Guide (NSW Government 2016).

As a next step, a rule-based architecture, RuleRS has been introduced. It collects information from the relational database of an online management system and represents the data into an intermediate file format using SQL/JSON syntax which is further processed by SPINdle rule reasoner.

Thirdly, we demonstrated that external systems such as as reporting to meet legal requirement could integrate rule reasoner, SPINdle. Typically, these reporting systems contain compliance and regulatory requirements which were converted into a rule knowledge-base. We used a pre-defined DL format which can also be represented using predefined plain text or XML file format. We argued that using this technique external stakeholders are can also be connected with the RuleRS.

On the next, we demonstrated a case study based on an Online Child Care Management, ChildSafeOMS to automate the early identification of children at risk, in particular of abuse and neglect. Preliminary results appear to be very promising, showing a close correspondence between compliance, regulations and online management system. RuleRS.

Finally, on account of motivations of the RuleRS as expert system, we demonstrated a case study based on an Adverse Event Reporting System, FAERS to automate the decision-making for compliant and non-complaint information, in particular on adverse event and medication error reports submitted to FDA. Preliminary results appear to be very promising, showing a close correspondence between compliance, regulations and online management system. Last but not least, RuleRS allows users or agents to produce conclusions and consequences automatically, based on a given compliance and regulatory knowledge base or generated on the fly by other applications and existing relational database information of an online management system.