1 Introduction

Inherited Retinal Diseases (IRDs) are a wide set of ocular diseases, which lead to photoreceptor death or dysfunction and blindness. They are genetically complex and are characterised by a significant clinical overlap between the different types. Retinitis Pigmentosa (RP) is the most common IRD.

It primarily affects the rod photoreceptors, with mutations in over 100 genes. One of the earliest symptoms of RP is night blindness, then it brings to the reduction in the peripheral visual field, [1].

The gold standard for the diagnosis of these diseases consists of an instrumental and biomolecular investigation, with subsequent genotyping, [2]. The widely adopted instruments for IRDs diagnosis, such as Ophthalmic Computerised Tomography (OCT), Electro-Retinography (ERG) and Electro-Oculography (EOG), can be too invasive for investigating IRDs in children. For these reasons, an alternative and cutting-edge approach to IRDs diagnosis can be the Chromatic Pupillometry (CP). It represents a pioneering method for the objective and quantitative evaluation of IRDs in paediatric population, [3] and [4]. CP consists in stimulating the patient’s eyes with different wavelengths and light intensities. The various stimulations generate a different pupil response, and physicians can evaluate the status of rods and cones by measuring the variation of the pupil diameter.

The Italian Ministry of Education, Research and University funded the project Toward new methods for early diagnosis and screening of genetic ocular diseases in childhood (PRIN 2015). The purpose of the project is to consider if CP is an efficient approach for IRDs diagnosis in paediatric patients. A pilot study has been designed by the physicians involved in the study, who belong to the University centres of Naples and Milan. It establishes the recruitment of 40 children, with an age-range from 8 to 16 years. 20 of those patients are IRDs- affected subjects (cases). The remaining 20 ones are healthy and they are included in the study as controls. Each participant in the pilot study must undergo a baseline visit, during which his/her personal data are collected and a classical ophthalmologic clinical and instrumental examination and pupillometry is performed. After the baseline visit, the patient is asked to undergo a follow-up visit after 28 ± 7 days, [5].

Pupillometry is performed according to the pupillometric protocol. It establishes the type of stimulations that must be performed on the subject: the colour and intensity of light stimulus and the background colour. Initially, the patient undergoes a 10-min period of adaptation to dark. After this period, the patient’s eyes are stimulated under three different conditions:

  1. 1.

    Low intensity with dark background stimulation

  2. 2.

    High intensity with dark background stimulation

  3. 3.

    High intensity with blue background stimulation

The last stimulation is performed after a 3-min period of adaptation to the blue background. Each stimulus is repeated three times both for the red and blue coloured lights (See Table 1). The different types of stimulation cause a different response by rods and cones. Indeed, cones detect fine details and colours and require high brightness of light, while rods detect a grayscale of black and white and have a lower threshold of activation than cones. The physicians study the velocity of variation of the pupil diameter and the amplitude of the variation itself. These parameters allow them to evaluate the functionality of the photoreceptors of the subject’s eyes.

Table 1 Pupillometric protocol
Full size table

As described above, the pilot study involves physicians and patients of different parts of Italy. Thus, the necessity of an Information Technology (IT) platform, reachable via web, has arisen, to gather data collected from scholars. One of the roles of the Department of Information Engineering, University of Florence (DINFO) in the project is to realise the above-mentioned platform. In this paper ORÁO,Footnote 1 the Electronic Medical Record (EMR) for chromatic pupillometry, is presented. It has been designed to match with the workflow of the examinations expected by the pilot study, and it is based on the specifications given by the physicians involved in the project.

2 Materials and methods

2.1 The pupillometer: Hardware and software

The pupillometry is performed by a specified instrument: the multi-chromatic binocular Pupillometer DP-2000 by NeurOptics. It is an optical scanner which captures and analyses a digital video of a subject’s pupil responding to light stimulation.

The DP-2000 Human Laboratory Pupillometer is a multichromatic binocular/dual camera system that tracks both pupils and stimulates both eyes. It is specifically designed for measuring the pupils of human subjects. The video stream is captured at 30 Hz. The video frame is digitised into 640 × 480 pixels with 8-bit grey level resolution. The profile of the light stimulation can be customised using a Graphic User Interface (GUI). Pupil variables are automatically analysed, reported graphically in a special window and saved into a results file, with. DAT extension.

Light stimulation has four different chromatic options: Red (622 nm), Green (528 nm), Blue (463 nm) and White. Light is emitted through a diffusing screen (approx. 50° × 35° of visual angle). Camera height can be adjusted by pressing the button on the arm and lifting the arm to the desired position. Pupillary distance can be adjusted to fit the subject’s eyes by turning the knob located below the cameras, see Figs. 1 and 2.

Fig. 1
figure 1

This figure shows the whole system of acquisition of chromatic pupillometry. It is possible to see the patient and the binocular camera of the pupillometer, connected to the computer where the software to control the instrument is running

Full size image
Fig. 2
figure 2

The binocular camera of the DP-2000 multi-chromatic pupillometer by NeurOptics

Full size image

2.2 ORÁO: development of the platform

The necessity of an EMR reachable via web to gather data collected by the two clinical centres involved in the project has arisen with the design of the pilot study itself, as previously stated. This task has been assigned to the DINFO because of its previous experience in the development of RESTful web applications [6,7,8] (Tables 1, 2, 3, 4 and 5).

2.2.1 A cloud-based solution

ORÁO has been designed as a cloud-based application, to meet the need of a common platform for the physicians to collect the data of the subjects involved in the study. A Virtual Machine (VM) has been created on the DINFO server farm, see [9] and Fig. 3, to host the web server and the database engine of ORÁO. The VM is running Windows Server 2012 Datacenter, by Microsoft. It is provided with two Intel Xeon E5–4620 processors, 2,20 GHz CPU, 4,00 GB RAM and with a 64 bit operative system.

Fig. 3
figure 3

The figure shows the architecture of the cloud farm provided by the DINFO, [9]

Full size image

In this way, the EMR is remotely reachable via web and accessible by authentication with username and password, after the administrator’s agreement.

2.2.2 RESTful application

The EMR has been developed with the REST architecture (see Fig. 4), to reach a better data exchange between server and client and to permit an easy integration of third-party applications, [10].

Fig. 4
figure 4

The figure explains the REpresentational State Transfer Architecture of the platform ORÁO

Full size image

The Representational State Transfer (REST) architectural style was introduced by T. Fielding in his PhD Dissertation Architectural Styles and the Design of Network-based Software Architectures, [11]. The REST architecture’s foundation is resources. REST architecture is based on four principles:

  1. 1.

    Resource identification through URI: resources are exposed by the Web Service through an URI (Uniform Resource Identifier);

  2. 2.

    Uniform interface: the operations that can be executed on the resources are a fixed set of four. They are expressed by the HTTP methods PUT, GET, POST and DELETE;

  3. 3.

    Self-descriptive messages: each message contains the information required for its management. Each resource is different from its representation given to client. In this way, the content of the resources is available in a lot of different formats;

  4. 4.

    Stateless interactions: the interaction with resources is stateless. The session status is kept only by the client: no client data are stored in the server. The server stores the status of the resources. There is no exchange of resource status between client and server.

The RESTful architecture is widely adopted because it is based on the well-known HTTP standard, [12].

2.2.3 Three-tier architecture

The EMR has been realised in MS Visual Studio 2017 using Visual Basic. NET (VB.NET) and ASP.NET framework 4.5.2. It has been created as a three-tier application, see Fig. 5. The presentation tier has been implemented using Twitter Bootstrap v.3.3.7 framework, an open source toolkit for web development with HTML 5.0, CSS 3, and JavaScript (JS). The JS library JQuery v.3.2.0 has been adopted for front-end coding. The JS libraries D3.js (Data-Driven Document) and Moment.js have been used too. The first one is employed for the creation of dynamic visualizations, while the second one is a library for the manipulation of dates formats and values in JS. The Business Logic Layer (BLL) is the application core: it controls and manages the functionalities and data elaborations. In the BLL, there are four modules for the main application sections: authentication, patient data management, ophthalmology and pupillometry. The Data Access Layer is realised with the Entity Framework (EF). This is a technology that allows the. NET application to work with data from relational DBs, using objects of the application domain without the need of a direct access to the DB. The EF creates the Entity Data Model (EDM), a conceptual model of the DB that the BLL can use to manage data from and to the DB. Moreover, the code has been organised according to the Model-View-Controller (MVC) pattern, [13]. The database has been realised using the Relational Database Management System (RDBMS) engine Microsoft SQL Server Express 2008 R2.

Fig. 5
figure 5

Three-tier architecture of ORÁO. For each tier the employed software instruments are shown

Full size image

2.3 Usability evaluation

According to standards, ORÁO’s interfaces have been designed giving high attention to their usability. Usability is the characteristic of the interface of a Medical Device (MD) that makes its use easier and safer for the end user. A MD must be designed with ergonomic characteristics, including its interface, that reduce the risk of error as much as possible. Usability is defined by the international standard ISO 924111:2018 as “The extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”, [14]. In a medical software usability is strictly related to safety: an interface poorly designed can induce the user to commit errors. Moreover, the international standard IEC 62366–1:2015 (Application of usability engineering to medical devices) has been considered, [15]. It establishes a process for manufacturers to evaluate the usability of a MD. The study and the evaluation of a MD usability must be performed during its design. The results of this evaluation must be considered to introduce changes in the interface with the aim to address the safety issues related to the identified usability issues. The usability of a MD can enhance or reduce its safety. Moreover, usability is a critical parameter that must be assessed in order to evaluate the quality of a software product, [16]. The usability of the EMR developed in this project has been evaluated with the Heuristic Evaluation.

2.3.1 Heuristic evaluation

Heuristic evaluation is an informal method to study the usability of a device interface. It represents a widely adopted method because it requires few resources and it permits an early identification of usability issues. The heuristic evaluation consists in the inspection of the interface by one or more evaluators, comparing it with a set of empirical rules, called heuristics. The most known and widely adopted heuristics for evaluating the usability of software interfaces are the Nielsen’s heuristics, [17]. For the heuristic evaluation of ORÁO interfaces, the Zhang’s heuristics have been adopted. These consist of a set of 14 heuristics developed for the usability evaluation of MDs: consistency and standards, visibility of system state, match between system and world, minimalism, minimization of memory load, informative feedback, flexibility and efficiency, good error messages, preventing errors, clear closure, reversible actions, use of understandable language, users in control and help and documentation, see Table 2.

Table 2 The table lists and explains the 14 Heuristics for medical devices by Zhang, [18]
Full size table

During the inspection of the device interface, evaluators are expected to find all the violations to the heuristics and to consider, for each of them, the potential impact on the safe use of the MD. Each violation found by evaluators is associated to a severity index. This allows to evaluate where the priority of intervention is, in order to address the safety issues associated the use of the MD. If heuristic evaluation is performed during the design of the MD, it is possible to early address the potential safety issues. Thus, the solution is optimised versus safety standards and also versus the user expectations of the device outcome.

The product of an heuristic evaluation is a report, containing all the violations to heuristics, identified by the evaluators. A report of an heuristic evaluation must contain the following data:

  1. 1.

    Where the violation is

  2. 2.

    A description of the problem and of the potential consequences it could cause

  3. 3.

    The heuristic that has been violated

  4. 4.

    The severity rate associated to the issue

Once the evaluation has been carried out and the report has been created, the designer of the software interface is provided with a powerful instrument that makes him capable to modify the design in order to address the most dangerous violations for a safe use of the MD. This is why heuristic evaluation should be performed during the early phase of design of the MD interface.

As stated above, the heuristic evaluation is widely adopted for the study of the safety of the MDs interface. One of the main benefits is its scarce necessity of resources. It also makes the designer aware of the usability issues, since the first phases of the interface design process. On the other hand, the heuristic evaluation has also some limitations, due to the fact that it is an informal method of usability evaluation. All the usability issues the evaluator can find are strictly related, and thus limited, by the set of heuristics employed in the study of the interface. Moreover, the evaluation is performed with the tested device out of its context of use. This can prevent the detection of usability and safety issues that arise with the use of the MD in its proper context of use. Indeed, the effect of external factors influence is not considered in the evaluation. Despite these limitations, the heuristic evaluation represents an important and powerful instrument for the designer of the MD interface, with a very good cost-benefit ratio.

3 Results

In the following sections, the results of the development of ORÁO are presented. The DB Pupil for the application has been deployed with the RDBMS engine Microsoft SQL Server Express, as stated above. The DB is organised in 26 tables, each of them dedicated to store data corresponding to a specific functionality or part of the EMR. There is a dedicated table to manage data relating to the authentication of the users, one for the personal data of the patients, one for each ophthalmologic examination (in particular there is a dedicated table for each instrumental monocular examination) and one for the pupillometry. The tables containing the data of ophthalmologic and pupillometric examinations are related to a specific follow up and in the DB they are identified by the follow up ID. This allows to gather all data recorded during the same examination, but saved in different tables in the DB, and to manage them through the application. Regarding the pupillometry table, each record of this table stores the name of the .dat file from the pupillometer as well as the follow up ID.

The EDM has been created with EF (that in our case represents the DAL) in Visual Studio, with a Database first approach (create the DB first and subsequently develop the model). The EDM is a map of the DB and of the data types each table must contain. The presence of the EDM avoids the direct interaction of the BLL with the DB.

The BLL is the core of the application. It manages the main functionalities of the EMR: authentication, management of patient data, ophthalmology and pupillometry, see Fig. 6.

Fig. 6
figure 6

The figure shows the structure of the BLL of ORÁO. It is possible to notice the encapsulation of the four functionalities provided by the EMR. All of them required the authentication of the user. Once the user is logged in, it is possible to access to all the patients data and, in particular, it is possible to reach the ophthalmologic and pupillometric functionalities, the actual core of the platform

Full size image

3.1 Authentication

The first time each physician uses the EMR, he/she must register with an email address and a password. After the registration, the user access to the platform must be granted by the administrator (DINFO). Figure 7 shows the homepage of ORÁO, after the user login. The platform name and the partners’ logos are shown in the middle of the page. The upper menu and buttons in the page provide user with a direct access to the functionalities related to the patients data management.

Fig. 7
figure 7

Homepage of ORÁO. At the top level of the page, it is possible to observe the main navigation menu, constantly shown in each page of the platform. This menu allows the user to rapidly change page, choosing one of the main sections: New patient, Patients List, Search Patient and Download data

Full size image

The authentication is managed by a dedicated RESTful Application Programming Interface (API). The input parameters of the API are the user email address and password encrypted with the Simple Triple Data Encryption Standard (Simple3DES), for the encryption and decryption of strings. The API searches for a correspondent record in the table of the DB dedicated to the authentication data storage (i.e. the TB_USERS table in the Pupil DB), through an HTTP GET request. If such correspondence is found and the authorization field of the record is set to true, then the API output is a structured data containing the following information: record ID, email address, user name and the associated token. The token is an alphanumeric string that gives the user access to the required resources for a limited time-period. Indeed, the token associated with the access expires when the user logs out the platform.

3.2 Patient data management

The patient management involves registration of new patient personal data, modification of pre-existing personal data and addition of new follow up examinations. The form for the registration of a new patient is composed by a list of fields regarding the patient personal data (name, last name, date and place of birth, gender, fiscal code), residence, date of the first visit and phone contacts. The user must also mark the patient as Control or Case, as stated by the design of the pilot study (see section??). The label for Control is blue while the one for Case is red. This association red-to-Case and blue-to-Control is used consistently throughout each page of the platform.

All the patients registered in the Pupil DB are included in the Patients List, see Fig. 8. Their personal details, as described above, are shown in this table. The user, after selecting a patient from the list, will press one of the two buttons Edit patient data or Follow up to access the corresponding functionalities. In both cases, a dedicated API manages the query and data exchange with the server DB. The API queries the specified table (i.e. HTTP GET request), searching for a record corresponding to the ID of the selected patient and follow up, which is the input data of the API. The data stored in this record are passed as API output to the presentation layer. The same API takes care of creating and updating records via HTTP POST. The API output is used by the presentation layer to provide an informative feedback to the user about the performed action.

Fig. 8
figure 8

The Patients List page. For each patient registered in the Pupil DB, the main personal data are shown. Also the Type, Case or Control, is shown to the end-user, associated with the proper colour, blue or red

Full size image

3.3 Ophthalmology

The Follow Up page, gives the possibility to register a new examination or to open an existing one, identified by its date. If the user clicks on the button Show follow up, a dropdown menu allows to choose the date of the follow up the physician wants to open. The dates in the menu have a white or green background colour. Green background indicates that the follow up is consolidated: it is no more editable. In the page of the follow up, the ophthalmologic examination forms can be opened and compiled. The examinations belong to two different types: binocular and monocular. During a binocular examination, the physician analyses and gathers data about the two eyes together: the form of binocular examinations is organised in a one-column page. The binocular examinations are: anamnesis (medical history), binocular investigation and genetic report. The monocular examination regards the single eye. The examination form is divided in two columns. The right column in the form is intended to collect data about the left eye, the left column gathers data about the right eye. This represents a standard visualization, widely used by physicians. The monocular examination is clinical and instrumental.

Once the patient has been selected from the Patients List, in the subsequent pages (follow up and examination forms) a menu containing the data of the patient is always shown. It is placed on the left hand side of the page, see Fig. 9. It shows the name, last name and type of subject (case or control) in bold characters and other personal data: date of birth, fiscal code, gender and birthplace of the examined patient.

Fig. 9
figure 9

The figure shows the patient personal data menu, highlighted in red color on the left hand of the page,the name of the patient which is repeated in the top menu, highlighted in red color too, and a label with the date of the chosen follow up. highlighted in green color

Full size image

The navigation of the user throughout ORÁO is helped by two menus:

  1. 1.

    A general navigation menu is positioned on top of each page. It gives the user the possibility to rapidly change page, accessing the main functionalities of the platform (Homepage, New Patient form, Patients List, Search Patients), and to download a file with the patient’s data (Download Data).

  2. 2.

    A local menu of navigation: it provides a direct link to the previous page and the current position of the user in ORÁO. In particular, this menu repeats the name and last name of the patient and the date of the follow up examination the user has selected to study or to compile.

3.4 Pupillometry

In the follow up page, in addition to the previous-mentioned ophthalmologic examinations, the user can access the part of ORÁO dedicated to the pupillometric examination. This page has been specifically developed to match the workflow established by the protocol of the pilot study, [19].

After the performance of the pupillometric examination, the physician loads the .dat file coming from the pupillometer (as described in section 2.1). The file is then saved in a dedicated folder (Upload) on the VM and its file path is written in a DB record. Once the file is loaded and saved in the DB, the elaboration of the pupillometric data is performed.

The API developed for the pupillometry functionality of ORÁO queries the appropriate DB table an gets back the file path for the .dat file associated to the selected patient ID and the follow up ID. The API accesses and reads the content of the .dat file using the System.IO (i.e. Input Output) namespace of the. NET framework. This consists of IO related classes that can be used for reading and writing data to files and for file and directory support. In particular, the class StreamReader has been used to create a.

TextReader object that reads characters from the .dat file. The file is scanned up to the section starting with the line <MERGED-PUPIL-DATA>. This section contains the acquired samples of the pupil diameter during the pupillometric protocol. Each line of the section is stored in a list of strings in the API, until the line <\MERGED-PUPIL-DATA> is reached, indicating the end of the section. The total amount of these data is a list containing 21.290 strings, each made up of 14 parameters. Only three of these parameters are needed for the elaboration of the graph of the pupil diameter trend: time, right diameter and left diameter. They are extracted from the list and saved in a matrix for the following elaborations.

Also the section <RIGHT-PUPIL-ANALYSIS> must be read. It associates each index of stimulation (from 0 to 19) to the corresponding start/end samples of the pupil diameter trend. The data read from this section are stored in a matrix with 479 rows and 3 columns corresponding to index of stimulation, start/end sample.

The physician must choose the index of the stimulation (from 1 to 20) he/she wants to analyse from a drop-down menu in the pupillometry page. The index is then used as an input for the API, together with the ID of the follow up. The elaborations described above are performed, and then the API sends to the presentation tier the values of time, right and left pupil diameter. These values feed the input of the JS function responsible for the graph creation, through the D3 library. In the graph, time is expressed in seconds while the pupil diameter in millimetres. If the physician moves the mouse pointer over the graph, the x-value (time) and y-value (pupil diameter) are shown by two dynamic labels, to make the graph analysis as easy and immediate as possible (see Fig. 10).

Fig. 10
figure 10

The graph of the pupil diameter trend over time and the two tables, one showing the technical characteristics of the selected stimulation and the other one summarising the parameters of the pupil measured during the pupillometric examination

Full size image

A summary table of the technical characteristics of the stimulus is also presented. It contains the stimulated eye (right, left or both), the colour of light (red or blue), the intensity (in Lux) and the background colour (black or blue) for each stimulation. The physicians also need to study some more parameters of medical interest, measured during the pupillometric examination and contained in a section of the .dat file. The parameters are reported in Table 3. For a better understanding of these parameters, please refer to Fig. 11. These parameters are gathered in a table, from which physicians can clearly read them.

Table 3 The table makes a list of the parameters measured during the pupillometry, with the meaning and the unit of measure of each of them
Full size table
Fig. 11
figure 11

The graph shows the response of the pupil to the light stimulation. The first red point highlights the maximum pupil diameter, before the contraction. On the other hand, the second red point identifies the minimum value at the peak of the contraction. Region 1 represents the latency between the stimulation and the pupil response. The slopes of the curve in region 2 and 3 correspond to the velocity of contraction and dilation of the pupil respectively

Full size image

3.5 Usability evaluation

In the following sections the results of the usability evaluation of ORÁO are introduced. The method adopted to evaluate the interface is the heuristic evaluation, as stated in section 2.3.

3.5.1 Heuristic evaluation

During the design of ORÁO, an heuristic evaluation has been performed to find out the usability issues of the interface and to address them. The evaluation has been carried out according to the method introduced in section 2.3.1. A three-level severity rating scale has been adopted, as reported in Table 4. The Task List has been composed considering all the possible application functionalities, giving high attention to those operations concerning the registration of the follow up examinations, considered the operations that introduce the major safety risk. A Reference Sheet (RS) has been prepared, containing the task list and the severity rating scale. Then the evaluator has been provided with the RS and a Data Collection Template.

Table 4 Severity Rating Scale adopted for the heuristic evaluation of ORÁO
Full size table

The evaluation of ORÁO interface has identified thirteen usability problems. Among these, six have the lowest severity index (0), five have the mean severity index (1) and the remaining two have the highest severity index (2) (see Table 5). The latter have been addressed immediately. The first of these two more dangerous problems was related to the monocular examination forms (both clinical and instrumental). These pages resulted not coherent with the general design of the interfaces, violating the heuristic Consistency and standards, and in particular the Spatial consistency, [18].

Table 5 The table lists the violations identified with the heuristic evaluation of ORÁO
Full size table

They didn’t have the data patient menu on the left side, moreover the fields to be filled where arranged over the whole page width. The second usability issue with the highest severity index was related to the follow up page. Once the patient is selected from the Patient List, the user can open and edit a follow up examination. This is selected from a drop down menu containing the follow up dates. Some follow ups are consolidated, while others are editable. These two different states of the follow up were not distinguished by anything in the drop down menu.

So, the user was expected to remember the state of each follow up of a patient or to open them one by one to verify if it was consolidated or not. This is a violation of the heuristic Minimize memory load.

Some of the middle severity usability issues concerned the error messages and feedback. For example, in the pupillometry page the user was not notified if the file of pupillometry (.dat file) had been loaded or not within that follow up examination. This represents a violation of the heuristic Informative feedback.

4 Discussion

The heuristic evaluation has constituted the basis for the final stage of the design of ORÁO. The identified usability issues have been addressed according to their severity, as stated in section 3.5.1. The first interventions have addressed the two issues with the highest severity index.

The design of the monocular examination pages has been redefined. The patient menu has been introduced on the left side of the page to reach a coherent structure with the rest of the examination pages. Moreover the input fields have been rearranged in a more compact appearance (see Fig. 14), making the monocular pages similar to the sheets used by the ophthalmologists for the manual registration of the examinations.

On the other hand, the issue concerning the follow up page has been addressed by introducing a green background for the dates of the consolidated follow up in the drop down menu, see Fig. 12. This intervention makes the user immediately aware of the status of the follow up examinations of the patient and avoids time-waste for checking the examinations one by one.

Fig. 12
figure 12

This figure shows the follow up page modified after the heuristic evaluation. As it is possible to see, a green background has been added to the dates of the consolidated follow up. This change allows the immediate distinction between editable and consolidated follow up for the end user

Full size image

High attention has been given to the feedback and error messages. They have been made as clear as possible for the end user, in order to give a complete awareness of the consequences of the actions and to allow him/her to recover from errors, when possible. For example, in the follow up page the user can consolidate an examination, making it no more editable. When the button Consolidate follow up is pressed, a message asks the user to confirm the will to consolidate that follow up, see Fig. 13, reminding the irreversibility of the action.

Fig. 13
figure 13

In this figure, the message showed when the user press the button Consolidate follow up is presented. It gives the user the information about the consequences of the operation, asking for a further confirm of the will to consolidate the follow up

Full size image
Fig. 14
figure 14

The monocular examination form after the heuristic evaluation. It is possible to notice the presence of the data patient menu on the left of the page and the distribution of the input fields, organized into two columns one for the left and one for the right eye. The examined eye is clearly indicated in the top of each column

Full size image

Since ORÁO is available via web, the physicians involved in the project have constantly used it and they have expressed the need for a new functionality. The possibility to download a .csv file containing all the pupillometric parameters of all the patients registered in the Pupil DB was needed in order to evaluate the progression of the pilot study overall. The .csv file can be downloaded by pressing the button Download data, in the top menu of the platform (see Fig. 7). A dedicated query has been written in SQL. It creates a view which gathers data from different tables of the DB. Also for this functionality a specific API has been developed. It makes an HTTP GET request to the view and gets back all the required data for all patients, including patient data, data about the user who performed the input, follow up details as well as parameters measured during the pupillometric examination. These data are written in a .csv file, saved in the Download folder on the server, made accessible for download through an API.

A Clinical Decision Support System (CDSS) for pupillometry based on Machine Learning has been developed. CDSSs are introduced to improve the patient care as well as the hospitals’ structure and organization, [20,21,22]. It is a classification system that gets some parameters from the .dat file of the pupillometer as input and gives back a label of Affected by Retinitis Pigmentosa or Not affected by Retinitis Pigmentosa as output for the selected patient. The CDSS is in the process of being integrated within ORÁO, thanks to the RESTful architecture. An additional VM, running the Apache Web Server, hosts the CDSS engine.

5 Conclusion

The design of ORÁO and all the phases of its development have been carried out in close collaboration with the whole project team. The platform matches with the needs they have expressed and with the workflow of the pupillometric protocol and of the ophthalmologic examinations. So, ORÁO turns out to be a customized IT solution for this project. Moreover, the high attention given to the usability of its interfaces made it very usable, according to all the subjects who have been using it for months.

This work shows how, given the growing complexity of healthcare systems and processes, the daily collaboration between biomedical engineers and physicians and the increasing demand for new tools and methods [23,24,25], there is a constant need for rigorous methodology in designing and realising collaboration tools that can facilitate these processes.