Introduction

In the year 2013, about 7000 mitral valve interventions (including 1400 in combination with aortic valve treatment) were performed in Germany [1]. According to the database of the European Association of Cardio-Thoracic Surgery (EACTS), more than 70,000 valves are replaced EU-wide every year. This number will rise due to an ageing population.

The various treatment options can be classified into two major categories: valve replacement and valve reconstruction. The common reconstruction options are annulus correction, implantation of artificial chords, partial valve resection and clipping [2]. If applicable, repairing the valve instead of replacing it with a prosthesis is beneficial: it is less invasive, has a higher success rate, lower costs and a shorter patient recovery time [3].

A clinician decides on which treatment option will be optimal for the patient, based on own experience and guidelines [4]. Those guidelines mainly consider basic parameters, such as presence of clinical symptoms or heart size. A prediction on how a specific treatment affects the valve function and the cardiac system can be hardly made.

This issue is addressed by decision support systems (DSS). The currently developed DSSMitral system for mitral valve therapy planning comprises a complete pipeline: segmentation of the actual state from time-resolved CT, MR and echo data, virtual treatment based on a dynamic valve model and simulation of the altered cardiac function on different scales (see Fig. 1).

Fig. 1
figure 1

DSSMitral: pipeline of the decision support system. The parts relevant to virtual downsizing are emphasized (green)

Full size image

Our virtual downsizing approach is one of the virtual intervention tools that are part of DSSMitral. Cases of mitral valve regurgitation are often accompanied by some degree of annular dilatation. A ring implant is sewed onto the annulus to reconstruct the original diameter. This reduces the mechanical stress acting on the valve and supports complete closure during the ventricular contraction phase [5, 6]. Depending on the patient-specific flow situation and anatomical variations, a sub-optimal ring diameter choice can lead to mitral stenosis and left-ventricular diastolic dysfunction [7, 8].

The DSS can provide the necessary flow-related data to reduce the risk of sub-optimal ring implantation. The clinician implants different types of annuloplasty rings in a virtual, patient-specific ventricle model. Flow simulations are executed on the altered geometry and provide pressure difference and flow patterns as additional data for the treatment planning.

In the following we describe our approach on the geometric alteration of the ventricular lumen that is induced by the ring implantation. Feature aspects are:

  • Modelling vendor-specific ring implants

  • Patient-specific parameterization of the annular area in atrium and ventricle

  • Automatic ring positioning

  • Fast lumen deformation

The general goal of our approach is to provide a reasonably deformed lumen that can be used to derive a geometric computational domain for a subsequent simulation step. The intended application within a clinical treatment decision workflow leads to additional requirements.

Processing time is a critical factor, if the DSS should be considered beneficial to the treatment planning process. Additionally, an optimization loop between different treatment options and simulation outcomes should be possible. As the simulation will take a significant amount of time, additional components of this loop should be as fast as possible. Thus, we aim for a deformation approach with an almost interactive processing time.

As the DSS should be compatible to standard clinical treatment procedures, we can only rely on a restricted set of available image data. Standard imaging focuses on providing insight into the current anatomical state of the pathological valve (echo, (4D) CT) with additional functional flow data (echo). As none of these modalities can directly provide patient-specific input for a physiological ventricle model, we decided to implement a geometry-driven approach for our deformation model.

While this will reduce patient-specific accuracy, we consider it still feasible for providing a geometric domain for subsequent simulation.

Related work

Systems for decision support can be applied on different levels. Basic classifier systems support assessment of the disease severity, e.g. by classifying valve Doppler signals into normal and abnormal flow [9]. However, to provide treatment-specific decision support, e.g. for mitral valve repair, a DSS also has to provide means for virtual intervention, result modelling and outcome assessment. Virtual valve interventions are based on alterable models of relevant anatomical structures and implants. Those models are often simplified to allow for interactive manipulation [10].

In the context of reconstructive mitral valve treatment, there were several approaches presented, addressing the different surgical options. In [11] an integrated system for virtual mitral clipping is presented. They derive a finite elements (FE) valve model from 3DTEE (3D transoesophageal echocardiography). Clips are represented as vertex connections of the polygonal leaflets. In [12] an echo-derived FEM valve model is basis for virtual resection planning. Their system allows for virtual plication and suturing. Labrosse et al. [13] present a FE valve model for virtual edge-to-edge repair.

With respect to virtual annuloplasty, the ring implants are of additional interest. In [14] FE simulations were performed to compare structural stress on different ring designs and its relevance to mitral regurgitation. Ender et al. [15] concluded that augmenting 3DTEE with virtual ring implant models is valuable to support the choice of ring size. In [16] a system is for virtual ring sizing is described. They model strain and suture forces for different ring sizes and link those parameters with potential complications such as dehiscence.

In [17] the change of valve leaflet dynamics after ring implantation is modelled. An FE model of the valve was deformed with respect to the changed diameter and simulated over a complete cardiac cycle. Stevanella et al. [18] also employ an FE model, but derive the geometry from 18 long-axis MRI planes instead of 3DTEE. In [19], a spring-mass model is used to represent shape and dynamic behaviour of the valve. This simplified model allows for fast simulation and real-time manipulation of the valve geometry and makes it suitable for interactive virtual interventions.

Ventricle deformation in the context of virtual interventions is sparsely documented. Augustin et al. [20] present a dynamic, patient-specific left ventricle model that focusses on the ventricular electromechanics. However, valve leaflet geometry is not considered and no implants are virtually added to the model. Gao et al. [21] present a coupled valve-left-ventricle model with focus on fluid–structure interaction. While this forms the basis for whole-heart simulations that also include valvular dynamics, the application to virtual interventions is not considered. Virtual intervention applications are described for the right ventricle. In [22] a statistical shape model is used to predict RV remodelling after Tetralogy of Fallot repair to determine the right timing for pulmonary valve re-implantation. Mansi et al. also employ personalized electromechanical models of the heart. It allows for virtual ventricle reduction surgery by means of interactive cutting, moving and joining sections of the right ventricle [23].

In conclusion, there are various approaches for virtual mitral valve manipulation (e.g. clipping, resection, annuloplasty). Our downsizing approach can be added to this group. As with some of these approaches, we also reduce the complexity of patient-specific anatomy to allow for a fast application and therefore make it usable within a clinical workflow. When it comes to virtual annuloplasty, the available approaches focus on implant-specific aspects such as the stresses and dynamic behaviour of the implant. Although the changed diameter is quantified, the induced deformation is not considered. While there are physiological left ventricle models, they are not considering changes that are induced by interventions. However, some examples could be found for the right ventricle and general ventricle surgery.

Our approach aims at deriving a reasonably deformed lumen after virtual annuloplasty. It is integrated well within the currently available approaches and derives it novelty from linking virtual implant-based treatment to the alteration of the ventricle lumen.

Methods

Our approach models the local geometric changes that are introduced to the lumen when implanting a ring. While already being relevant for simulation-based estimation of mitral stenosis, it will be used as part of a virtual intervention toolchain in most of the cases. Thus, we focus on automation and a purely geometric approach with a small set of parameters. This is also driven by the fact that an exact physical model of a patient-specific ventricle system is hard to create in a reasonable amount of time and effort, especially for pathological cases.

A lumen mask that contains ventricle, atrium and left-ventricular outflow tract (LVOT), points representing the annulus shape and the size of the implanted ring are necessary inputs (see Fig. 2). They are created in the preceding segmentation and annulus definition steps of the DSSMitral pipeline. We will provide a brief characterization on how the input data are generated in “Segmentation and annulus definition” section; for details we refer to Tautz et al. [24]. The input mask is further processed to generate a mesh and to extract additional landmarks. While our approach is fully automatic, we provide a small set of preset parameters and a widget for manual alteration of the implant placement. The output of our approach is a deformed lumen mesh.

Fig. 2
figure 2

Overview of the virtual downsizing pipeline

Full size image

Segmentation and annulus definition

The DSSMitral pipeline is currently focused on datasets of patients with degenerative mitral valve disease that underwent an annuloplasty. As we concentrate on the application within standard clinical procedures, the available modalities are CT (4D or an end-diastolic phase) and 3DTEE. The CT data are used for full lumen segmentation, as those images contain the left ventricle, the atrium and the ascending aorta completely. In contrast, depending on the location of the echo device, 3DTEE only acquires the lower part of the atrium. The lower temporal resolution of 4DCT compared to 3DTEE is not problematic. We select a single time point with maximum left-ventricular volume during the end-diastole. The 4DCT imaging protocol is defined accordingly.

The automatic lumen segmentation is an iterative process in a preceding DSSMitral pipeline step. Starting with an aorta segmentation, the left ventricle and finally the left atrium are segmented, using previously segmented structures as context. This improves stability with respect to varying image quality and contrast. The resulting mask is further enhanced by morphological operators to account for inhomogeneous contrast due to blood distribution. Papillary muscles and trabeculae often lead to an underestimation of the ventricle mask. Therefore, its shape is approximated by slice-wise convex hulls along the longitudinal axis of the ventricle.

The DSSMitral pipeline also includes an interactive annulus definition step. The annulus contour is defined manually by placing markers in the CT volume. The volume is reformatted with respect to the longitudinal axis of the ventricle, which is computed from a principal component analysis of the ventricle mask. The user is provided with 18 rotational image slices equally distributed around this axis. On each slice, two markers are placed at the annulus locations. From the resulting 36 markers, a 3D spline contour will be computed, representing the annulus (see Fig. 3).

Fig. 3
figure 3

From left to right: segmented 4DCT slice, annulus markers on rotational slice and resulting 3D annulus contour

Full size image

Lumen segmentation and interactive annulus definition are prerequisites to the virtual downsizing approach. For details we refer to Tautz et al. [24].

Mask processing

The segmentation mask contains the lumen of the atrium, the ventricle and the LVOT. We remove the LVOT, so only structures that are directly affected by the subsequent deformation pipeline remain in the mask.

The LVOT removal is based on automatic, size-based component filtering. We compute a distance field from the mask (positive—inside, negative—outside) and threshold it with half of the maximum distance (dfeature). Thus, only the major structures atrium and ventricle remain, while structures with a smaller diameter, such as LVOT and atrium vessels, are removed. By computing an inverted distance field from this mask, and reapplying the same threshold dfeature, we reconstruct a smoothed version of the original mask without small-diameter structures.

By subtracting this mask from the original input mask, we get a set of components that represent atrium vessels, local lumen surface features and the LVOT. We assume the LVOT to belong to the larger structures. Thus, we iteratively apply multiple volume thresholds to the components (25 ml, 20 ml, 5 ml, 2 ml). Filtering the bigger structures can also lead to the removal of atrium vessels, which has no effect on our approach, as they are located outside the deformation influence zone. If no structures are found, the biggest structure is selected. In some cases, this might not be the LVOT, but this also means that the segmentation mask already exhibits severe artefacts, e.g. no clear separation of LVOT and atrium. In those cases, the preceding segmentation step of the DSSMitral pipeline provides manual tools (e.g. a scalable 3D brush) to correct the faulty segmentation mask. The manually corrected mask is then used as new input for the downsizing step.

When subtracting the LVOT mask from the original mask, the separation zone follows the general ventricle shape, as we used the distance field as basis for the filtering. By subtracting a morphologically dilated LVOT mask, we obtain a voxel representation of the separation zone. We define the centre of the voxel cluster as paorta. This landmark is technically not located within the aorta, but provides information about the aorta location with respect to the annulus and will be used for orientation purposes later on.

From the mask, the polygonal surface is created by means of marching cubes (see Fig. 4). To make the mesh suitable for subsequent deformation, the triangle quality is improved. Elongated triangles resulting from the regular space partition of the marching cubes are replaced with more equilateral ones through established local topological operations: edge collapse, edge split and edge flip [25]. Finally, the annulus markers are converted into a densely sampled spline contour, to obtain a smooth annulus contour representation.

Fig. 4
figure 4

Input mask containing atrium, ventricle and LVOT with parts of the aorta, input annulus markers (left), segmented LVOT/aorta part (middle), polygonal lumen surface mesh and smooth annulus spline (right)

Full size image

Implant modelling

The annulus is represented as a contour. To allow for an adequate geometric linking with the annulus, the virtual ring implant will also be represented as contour. We focus on the Carpentier–Edwards Physio II ring model [26], as this type of ring is used for all patients in our database. To create a geometric representation, CT scans of 8 packaged rings were performed, with implant sizes ranging from 26 to 40 mm (2 mm steps).

For each image, the ring is segmented by means of intensity-based thresholding. We select a threshold that allows for separation of the ring and the holding device attached to it. A surface mesh is created with marching cubes. Skeletonization is used to compute the initial centreline. The resulting contour points are used as spline control points, to create a continuous, smooth spline representation of the ring centreline.

The contour points are also used as input for a principal component analysis (PCA). The resulting eigenvectors represent an oriented, local 3D coordinate system of the contour. Based on this local coordinate system, the ring geometry (mesh and contour) is linearly transformed to the origin. It is also oriented in such a way that the z-axis points towards the aorta-saddle section of the implant, the y-axis towards the section related to the posterior commissure, and the x-axis upwards, towards the atrium location after implantation (see Fig. 5).

Fig. 5
figure 5

Reconstruction of a 32-mm Physio II ring: from a CT scan (upper left) the centreline and surface mesh (lower left) are computed from thresholding-based segmentation. A standardized orientation is applied, and the inner (orange) and outer (cyan) contours are computed from the centreline (right)

Full size image

All implant geometries are oriented in this manner to create a comparable ring database. Additionally, the contour point psaddle closest to the positive z-axis, coinciding with the saddle horn after implantation, is marked. This allows for an automatic orientation of the implant with respect to patient-specific landmarks during the virtual downsizing.

From the centreline contour, an inner contour and an outer contour are computed. For each centreline contour point, a ray that is orthogonal to the x-axis is computed. This ray intersects the ring mesh at two points. The point closer to the x-axis is assigned to the inner contour, the other one to the outer contour. The outer contour defines the actual contact zone between annulus tissue and ring after the virtual downsizing. The distance between outer and inner contour is used to approximate the ring diameter during the automatic ring positioning.

Downsizing model

The geometry-driven downsizing model has to address two aspects: the spatial relation of the original annulus and the ring after implantation, and the local deformation of the inner surface of atrium and left ventricle induced by the ring.

Ring-to-annulus relation observation

Independent from vendor-specific designs, e.g. to ease suturing or provide different grades of ring flexibility, the narrowing of the septal-free wall diameter is a major factor for improving functional regurgitation [27]. In order to derive a geometric description of this aspect, we conducted interviews with clinical experts (personal communication 11/2017) and observed interventions.

Additionally, we compared pre- and post-interventional 3DTEE images. An affine, feature-based registration was applied, in order to preserve local shape variation while creating a comparable overlay visualization (see Fig. 6). We choose the contact point of the aortic valve leaflets and the apex as feature points for image alignment. The commissure locations were used to fix the rotational component.

Fig. 6
figure 6

Pre-interventional 3DTEE (blue image/orange contour) superimposed with post-interventional 3DTEE (yellow image/yellow contour) (left), the derived model scheme for ring placement and deformation (right), the s-shape is highlighted (yellow)

Full size image

The observations confirmed the expert comments and interventional observations. The annulus has a stronger in-place fixation towards the inner heart, due to connected, stabilizing structures such as the septum, right ventricle and the aorta. The septal-free, posterior annulus segment is more flexible; thus, the saddle horn works as a fixture point during the sewing process. The ring is pulled towards the original anterior annulus segment, relocating the posterior annulus segment and subsequently reducing the septal-free wall diameter. This shift can be observed in the 3DTEE comparison.

The ring stays within the plane of the original annulus. The new annulus shape strictly follows the shape of the closed ring that was implanted. From intervention observation, expert interviews and the knot distribution visible in the post-interventional 3DTEE, a uniform loop distribution was confirmed.

Lumen deformation observation

The lumen deformation is strongly influenced by the ring location after implantation. From the visual comparison of pre- and post-intentional 3DTEE, we observed that the general deformation of inner atrium and ventricle wall naturally points towards to the sewing seam. Bigger sections of the atrium follow the induced deformation, while the ventricle shows a tendency to stronger local deformation, regaining its original shape within a smaller surface distance to the ring.

This difference could result from a stabilizing effect of the muscular myocardial tissues. Also, the inner ventricular surface deformation showed a more complex pattern. Following the deformation from the seam towards the apex, we first observe a strong, linear connection close to the ring, and while the diameter returns to its original size with increasing distance to the ring, an s-shape can be observed (see Fig. 6). This indicates that a nonlinear geometric deformation description will be necessary for the ventricular part.

Derived geometric model

From the aforementioned observations we deduce that a surface distance-based, nonlinear shifting approach is sufficient to emulate the changes induced by the ring. The ring will be placed within the plane of the annulus and shifted towards the anterior annulus segment.

As ring placement and lumen deformation will be relative to the annulus, we first define a local coordinate system that will aid both. An adapted PCA with all annulus contour points as input is used to compute three orthogonal vectors that are oriented relative to the annulus ring. Two vectors are parallel to the plane defined by the annulus, while the third one points upwards, towards the atrium. This third vector defines the central annulus axis Aa.

Ring placement Each ring geometry is initially positioned with respect to the global origin (cf. “Mask processing” section). Through a set of rotation and transformation operations, each point pr of the model is positioned with respect to patient-specific landmarks:

$$ p_{\text{r}}^{\prime } = p_{\text{r}} \cdot M_{\text{shift}} \cdot M_{\text{rot}} \cdot M_{\text{orient}} \cdot M_{\text{trans}} $$

First the ring is translated to the PCA centre of the annulus (Mtrans). Then, its oriented with respect to the local vectors Vai of the annulus (Morient). We ensure that the former local x-axis of the ring coincides with the annulus axis Aa. Now, the ring is already correctly oriented with respect to atrium and annulus. In the next step, the ring is rotated towards the anterior annulus section. For this we align the landmark psaddle on the ring with the landmark paorta on the lumen (recall “Segmentation and annulus definition”, “Mask processing” sections).

Both points are projected onto Aa in order to define the two vectors \( \vec{v}_{\text{saddle}} \) and \( \vec{v}_{\text{aorta}} \) that are orthogonal to Aa and point to the respective landmarks. The ring is rotated with respect to the angle between \( \vec{v}_{\text{saddle}} \) and \( \vec{v}_{\text{aorta}} \) (Mrot). As last step, the ring is shifted to account for the anterior annulus stability (Mshift).

As the annulus exhibits distinctive anterior bending, the PCA tends to provide an origin with an anterior shift, thus the ring is also shifted. To evaluate the current relative position of the ring, all annulus points a projected onto \( \vec{v}_{\text{aorta}} \). The maximal posterior (dp) and anterior (da) distances are identified, and the anterior pre-shift amount sa = da − dp is computed. The final shift vector is defined as

$$ \vec{s} = \frac{{\vec{v}_{\text{aorta}} }}{{\vec{v}_{\text{aorta}} }} \cdot \left( {\left( {\frac{{d_{\text{a}} + d_{\text{p}} - d_{\text{ring}} }}{2}} \right) \cdot s_{\text{r}} - s_{\text{a}} } \right), $$

with dring equalling the outer, anterior–posterior diameter of the ring and sr acting as a parameter for the amount of shifting (0 = position ring at centre, 1 = shift ring to attach to anterior segment).

Lumen deformation Each annulus point aj on the lumen surface is shifted completely towards its related point on the ring rj. The relation between both points is given via uniform projection from the annulus contour onto the ring contour. This reflects the uniform placement of loops (see Fig. 5).

All other surface points are displaced relatively to this initial displacement. Following the observations, the atrium and the ventricle part are handled differently. Also, the amount of displacement is derived from the relative surface distance of each surface point.

Generally, the displacement can be described as shifting surface points towards the middle of the lumen. For the atrium part, we employ a linear displacement scheme for each surface point pj:

$$ p_{j}^{\prime} = p_{j} + \left( {\left( {1.0 - \frac{{d_{{p_{j} }} }}{{d_{{\rm{max}} } }}} \right) \cdot v_{rj} + \frac{{d_{{p_{j} }} }}{{d_{{\rm{max}} } }} \cdot v_{oj} } \right) \cdot \frac{{d_{{p_{j} }} }}{{d_{{{ {\rm{max}} } . {\text{atrium}}}} }}, $$

where \( v_{oj} \) is the vector from the surface point to its projection on the central annulus axis Aa and \( v_{rj} \) is the vector from the surface point to the related ring contour point \( r_{j} \).

Using a randomly chosen annulus point as origin, the orthogonal angle difference with respect to Aa is computed for every surface point pj (see Fig. 7). Through this angle difference, each surface point can be directly linked to an annulus point and indirectly to a ring point rj. The influence of the annulus point’s displacement is derived from the surface distance \( d_{{p_{j} }} \). In case of the atrium, we employ a linear scheme: \( d_{{p_{j} }} /d_{{{ {\rm{max}} } . {\text{atrium}}}} \).

Fig. 7
figure 7

Displacement mapping from patient-specific annulus to ring (most left), the three scalar surface parameters for lumen deformation (left to right): surface distance to annulus (white), annulus angle difference with respect to Aa, tagging of atrium and ventricle for domain-specific deformation

Full size image

The displacement direction is a linear interpolation between \( v_{oj} \) and \( v_{rj} \) with respect to the relative distance of the surface point \( p_{j} \). If the point is close to the annulus, it is shifted directly towards the ring. If a point is further away, it is displaced more towards the annulus axis. Thus, we avoid an unnatural tangential stretching of the surface.

For the ventricle part, a slightly more complex displacement scheme based on a logistic function is employed. Thus, we can mimic the observed s-shape:

$$ p_{j}^{\prime} = p_{j} + \left( {\left( {1.0 - \frac{{d_{{p_{j} }} }}{{d_{{\rm{max}} } }}} \right) \cdot v_{rj} + \frac{{d_{{p_{j} }} }}{{d_{{\rm{max}} } }} \cdot v_{oj} } \right) \cdot \left( {1 - \frac{1}{{1 + {\text{e}}^{{ - 1.5\left( {2 \cdot \frac{{d_{{p_{j} }} }}{{d_{{{ {\rm{max}} } . {\text{ventricle}}}} }} - 1} \right)}} }}} \right) $$

Additionally, ventricle surface points close to the annulus, within a preset surface distance range rv, are shifted with the same amount as their related annulus points. Thus, a sharp bending of the surface is avoided, resembling the muscular stability of the tissue.

Parameters

Our approaches on ring placement and lumen deformation introduce four external parameters. The ring placement can be parameterized using sr. It defines how much the ring is shifted towards the anterior annulus section. This can be interpreted as a parameter for the anterior annulus stability and is therefore anatomy-related.

Same is true for the lumen deformation parameters dmax.atrium and dmax.ventricle. They define the maximal surface distance of the deformation influence in atrium and ventricle. As only surface points within this range are shifted, it reflects the material flexibility. Lower distance values represent a lower deformability. The parameter rv corrects curvature discontinuities between the atrium and ventricle part and can be interpreted as a suture representation.

In Table 1 we present a parameter set-up that was heuristically derived from observations in registered pre- and post-interventional 3DTEE images and from iterative expert feedback on initial deformation test cases. The set-up generally represents the observations stated in “Ring-to-annulus relation observation” and “Lumen deformation observation” sections, that is, stronger anterior stability of the annulus and a lower deformability of the inner ventricle contour compared to the atrium.

Table 1 Parameter set-up
Full size table

Implementation

Our approach on ring modelling and virtual downsizing is embedded in the DSSMitral prototype, as part of the virtual intervention workflow step (see Fig. 1). The prototype implementation is based on the medical image processing and visualization environment MeVisLab [28].

The lumen is represented as a discrete triangle mesh. All surface meshes (ring, lumen) are generated from the binary masks by means of marching cubes. Subsequent linear interpolation and volume-conserving smoothing remove staircase artefacts that result from the voxel size-related space partition of the binary mask. To represent the mesh for analysis and processing, we use standard VTK data structures [29].

The lumen deformation is applied per-vertex; thus, all deformation-relevant parameters are saved per-vertex for all relevant surface vertices. A central parameter is the annulus contour. In the DSSMitral workflow, the annulus contour is represented as a smooth spline contour that is defined through a set of markers (recall “Segmentation and annulus definition” section). To create an annulus representation on the lumen mesh, the marker points are mapped to vertices that are closest. Between each mapped surface vertex, a Dijkstra shortest path is computed. All vertices on the path are marked with a specific ID. Thus, a consistent, closed annulus path representation on the mesh is ensured.

The marked annulus vertex path is the starting front for the fast marching algorithm that is used for an efficient surface distance computation. Once the two fronts towards atrium and ventricle have completely traversed the mesh, the individual surface distance to the annulus is saved at each vertex.

For each annulus vertex, the correspondence to the outer ring contour markers is computed. From each vertex, a vector to the PCA centre of the ring is defined. All ring contour points are projected onto this vector. The ring contour point that is closest to the vector and has a projection that is closest to the annulus vertex is selected as its correspondent ring point.

The orthogonal angle difference with respect to Aa is computed for each annulus contour vertex and all other lumen vertices. Thus, each lumen vertex can be linked to the annulus vertex with the same angle offset. Through the correspondence, each vertex can also be linked to a ring point. The shift of each surface vertex towards a linked ring point is scaled with respect to the surface distance. For atrium and ventricle, the two different displacement schemes described in “Derived geometric model” section are applied.

With the current, prototypical, Python-based implementation of our approach, the computation of the lumen mesh vertex data (distance, angle, region) takes approximately 8 s on a consumer-class laptop (Intel Core i7, 8 GB RAM, NVIDIA GeForce 750 m) for a mesh with ~ 150.000 triangles. Each subsequent deformation (e.g. different ring position or sizes) takes approximately 0.5 s. The virtual downsizing is completely automatic; no user interaction is required. However, if for some reason the ring placement or orientation is not sufficient (e.g. the aorta landmark was misplaced in a previous workflow step), we additionally provide an interactive widget for correction (see Fig. 8). All results shown here were created without manually altering the ring placement.

Fig. 8
figure 8

Implementation details with an intentionally under-sized and non-shifted ring for visual clarity (left side): deformed mesh (dark grey), annulus marker (white), vertex-mapped annulus contour (orange), mapping to virtual ring (orange arrows), general shifting direction (dark orange), on the right side: the interactive correction widget in translation (upper image) and rotation mode (lower image)

Full size image

Evaluation

In order to evaluate whether the proposed geometric scheme leads to plausible deformation, we compared the virtually deformed lumen with post-interventional 3DTEE image data. While providing a good temporal resolution and having reasonable contrast close to the mitral valve, the echo images also exhibit a low signal–noise ratio and contain artefacts that prevent a complete post-interventional lumen reconstruction. The ring implant is causing an echo shadow that occludes significant portions of the central and apical ventricle tissue. As the intention was to depict the valvular function rather than the complete ventricular and atrial anatomy, broad sections of the atrium are excluded from the field of view.

Thus, we organize the evaluation two-fold. We perform a quantitative evaluation that is focused on the lumen distances close to the annulus. In this area the echo image quality allows for at least a rough segmentation of the lumen. However, this is also the area where most of the model-driven deformation occurs. To get feedback about the plausibility of the overall segmentation and applicability of our approach, we additionally gather qualitative feedback from two domain experts.

Datasets

Eleven datasets were prepared for the quantitative and qualitative evaluation. For each dataset, the model was applied to geometry from a pre-interventional 4DCT (Siemens—SOMATOM Definition Flash) and compared to post-interventional 3DTEE (GE Vingmed Ultrasound—Vivid E9) images, as no post-interventional 4DCT was available. The 4DCT images were reconstructed with a B30f kernel, providing a 0.65 × 0.65 × 0.5 m field of view and a voxel size of 1.0 × 1.0 × 0.5 mm, with 10 phases within the heart cycle. The 3DTEE images were acquired via transoesophageal echocardiography, oriented towards the mitral annulus with a voxel size of 1.0 × 0.7 × 1.0 mm and 18–20 phases.

For each of the datasets, the virtual downsizing was applied (Fig. 9). The ring sizes (30–34 mm) were obtained from the clinical treatment reports. The ring was automatically placed as described in “Derived geometric model” section. The ring was positioned in the centre of the annulus, its saddle point was rotated towards the aorta point, and it was shifted towards the aorta point. The ring placement was superimposed on a volume rendering of the 4DCT dataset, and a domain expert visually reviewed the placement and stated it to be plausible.

Fig. 9
figure 9

Four example results. For each image: pre-interventional 4DCT reconstructed lumen mesh (left), mesh after virtual intervention (right), the virtual ring implant is superimposed (orange)

Full size image

Registration

To be comparable to the deformed lumen, we select the time point with maximum left-ventricular volume during the end-diastole from the 3DTEE image data. The 4DCTA image, and therefore also the derived lumen mesh, exhibits different world orientation and voxel scaling compared to the 3DTEE image. Thus, we apply an affine transformation to align the deformed lumen with the 3DTEE. The transformation matrix is derived from an iterative closest point (ICP) alignment of landmarks on the model-deformed lumen mesh/virtual ring and the 3DTEE image.

As the ring implant is the most visible structure in the echo images, we use it as major structure for defining the registration transformation. For each mesh of the used ring types, nine points of the inner ring contour are manually selected (recall “Mask processing” section). They are systematically selected: four points at the anterior, posterior, left and right side of the inner contour. Four additional points are equally distributed in-between them. The one anterior point is replaced by the two closest inner contour points. This is done to remove ambiguities with respect to rotation during the registration process. These nine points provide a characteristic description of the ring shape and orientation (see Fig. 10).

Fig. 10
figure 10

ICP registration for qualitative evaluation: 9 markers are placed on the virtual ring, one at the apex (left), same marker layout in 3DTEE (middle), ring in 3DTEE superimposed with virtual ring after registration (right)

Full size image

As the ring is fixed with respect to the model-deformed lumen, the nine points also describe the orientation of the deformed lumen. An additional point is selected at the apex, to add the longitudinal ventricle axis to the orientation representation and further stabilize the registration. For each dataset, these 10 characteristic landmarks are manually selected in a volume rendering of the 3DTEE dataset (via ray picking).

The transformation between the two point sets is computed via ICP. The resulting transformation is applied and only small, manual adjustments are performed, to account for small inaccuracies with respect to the landmark placement in 3DTEE (e.g. marker is positioned on knot and not on the inner ring contour).

Quantitative evaluation

As the 3DTEE images were acquired to depict the annular region and the valves, a reasonable explicit lumen segmentation is only possible in this region. Other areas are excluded from the field of view or suffer from a low signal–noise ratio and echo shadows. As this is also the area, where major deformation is induced by our model, a direct comparison in this region of interest (ROI) still provided meaningful insight. Above the annulus, the ROI includes the complete 3DTEE image to obtain all sections of the atriums that might visible. Towards the ventricle the ROI restricted with respect to dmax.ventricle, as this marks the area of deformation influence.

As the intensity distribution even within the ROI exhibits strong variations, we apply a marker-based, semiautomatic watershed-segmentation (pre-flood height: 12, 3D-connectivity), to separate lumen from tissue. The tissue mask is inverted and surface noise is reduced via a rolling ball closing (2 mm diameter) and a connected components analysis which removes disconnected noise. To allow for a direct surface distance computation, the inverted mask is transformed into a polygonal mesh (3DTEE mesh) via marching cubes. For each mesh vertex, the closest distance to the registered deformed lumen mesh is computed (see Fig. 11). As the 3DTEE mesh encloses the inverted lumen, it also includes vertices located at the ROI borders and the pyramidal field of view of the 3DTEE image.

Fig. 11
figure 11

Quantitative evaluation set-up (Dataset ID 39): the inverted 3DTEE lumen mesh is generated for a ROI around the annulus (middle). For each surface vertex the closest distance to the deformed ventricle mesh is computed (left). Only the vertices pointing inwards are used for quantitative analysis (right)

Full size image

The distances of those vertices have no relation to the actual alignment of patient-specific anatomy and the deformed model and need to be omitted from the distance calculation. We compute the geometric centre of the annulus contour Ca. For each 3DTEE mesh vertex, the dot product of the normalized vector towards Ca and the central annulus axis Aa is used as threshold. A dot product > 0.17 is interpreted as pointing away from the lumen and therefore part of the boundary. Due to surface noise and the inclusion of parts of LVOT, the remaining surface vertices might still belong to part other than the tissue-lumen boundary within the ventricle. Nevertheless, this vertex filtering allows at least for an initial analysis of the quantitative surface distance.

The distance histograms for all cases exhibit a characteristic, left-skewed distribution as exemplarily shown in Fig. 12. For quantitative characterization, we computed median, mean and variance of the distance for all remaining vertices as well as the 90% and 70% percentiles. The results are shown in Table 2.

Fig. 12
figure 12

Example of a characteristic surface distance histogram

Full size image
Table 2 Quantitative overview of the surface distances between 3DTEE mesh and deformed lumen mesh
Full size table

Qualitative evaluation

An arbitrary subset of the datasets was presented to the experts using a 2D viewer with the option to interactively navigate through the slices. The 3DTEE and the deformed lumen were reformatted to provide a pseudo-long-axis view, as this provides a good overview with respect to the local deformation at the annulus.

During each interview, the experts first explored the post-interventional 3DTEE to form an impression of the anatomical situation. Then the image data were superimposed with the semitransparent deformed lumen and the virtual ring. To improve visual distinction, a complementary colour scheme (echo: blue—lumen: yellow) was applied (see Fig. 13). The experts were asked to explore the deformed lumen, rate the plausibility of deformation on a scale from 0 to 5 and give general commentary on the contour and potential artefacts. The results are shown in Table 3.

Fig. 13
figure 13

Four of the cases used for qualitative evaluation. Post-interventional 3DTEE overlaid with deformed lumen (yellow). While the deformation close to the annulus is deemed plausible, ventricle over- and under-segmentation can be observed (most left/right images)

Full size image
Table 3 Qualitative rating of contour plausibility
Full size table

Discussion and outlook

The results of the quantitative evaluation on 11 patient-specific datasets indicate a good geometric agreement between the artificially deformed lumen and the post-operative echo images. In the ROI around the annulus, we achieve a mean distance to the post-operative image lumen surface of approximately 1.59 mm. With respect to the voxel size of the post-operative echo images of 1.0 × 0.7 × 1.0 mm, this deviation is almost within the resolution level. However, we also need to consider that the deviation includes errors from various sources that cannot be directly quantified: a slight temporal mismatch between echo and CT due to different sampling of the heart cycle, segmentation artefacts resulting from the low echo image quality, especially in the ventricle due to implant-induced echo shadows and a possible registration mismatch between echo and CT.

As a complete lumen segmentation was not achievable in the echo images, the quantitative results reflect the geometric agreement close to the annulus. Despite this spatial restriction, the quantitative results are still relevant, as the ROI coincides with the lumen surface section that was primarily affected by the geometry-driven deformation. However, the echo FOV during imaging was selected to primarily focus on the valves. Thus, for most of the cases, only a small section of the atrium is visible and therefore quantifiable.

Despite the potential additional influence factors and the spatial restriction, the quantitative evaluation at least supports the hypothesis that our geometry-driven deformation approach generates reasonable results. The qualitative inspection of the distance field mapped on the echo segmentation has shown no signs that indicate a systematic deformation error, e.g. connected elongated regions of distance deviations parallel to the annulus. The deviations were local with no anatomically reasonable explanation and could mainly be attributed to the overall low signal-to-noise ratio of the underlying echo images. Characteristic local deviations on the ring could be ascribed to knots that were visible in the echo image and that are not considered by our approach.

A robust quantitative evaluation over the complete lumen would be possible, if post-treatment 4DCT images would be available. Then, we could apply a reconstruction pipeline similar to the one that created the input for our deformation approach. However, due to aspects such as radiation hygiene, it is to be expected that this kind of data is will not be available in the foreseeable future. Follow-up images can be available for some cases, but the expected remodelling of atrium and ventricle due to the altered workload limit their usefulness for a geometric comparison.

To compensate the aforementioned restrictions to some degree, we also performed an expert-driven, qualitative evaluation on a subset of the datasets. The overall feedback was positive (~ 4.01/5). All deformations were deemed to be plausible. The ring orientation and shift were assessed as correct with respect to aorta and ventricular outflow tract.

While the local deformation close to the annulus was deemed plausible, the experts criticized slight segmentation mismatches in the apical region as well as under-segmentation, e.g. missing vessels, in the atrium. Also the exclusion of the left ventricle outflow tract was deemed slightly irritating but overall not problematic for the intended purpose.

The central scope of our approach on virtual downsizing, the local deformation close to the implanted ring was deemed to be plausible. Critical points considered mainly aspects from previous DSSMitral pipeline steps that are relevant for later pipeline steps. For instance, the observed segmentation offset results from simulation-specific requirements, where convex smoothness is more crucial than a representation of trabecular details and papillary muscles. This is not directly related to our virtual downsizing approach. Additionally, our approach is still applicable if the overall pipeline requirements or even the input modality change, as it only relies on a lumen mesh and an annulus contour.

A similar parameter preset (sr = 90%, dmax.atrium = 20 mm, dmax.ventricle = 15 mm, rv = 2 mm) was used for all case in the quantitative and qualitative evaluation. The values were determined heuristically from post-treatment observations. While our approach is purely geometry-driven, these parameters can be interpreted as global material properties in terms of stability and flexibility of the atrium–ventricle complex.

A ring shift of sr = 100% would mean that no deformation is induced at the anterior section of the annulus and corresponds to a very high septal stability, while sr = 0% would represent a stability equilibrium between posterior and anterior annulus regions. The parameters dmax.atrium and dmax.ventricle reflect the flexibility of ventricle and atrium. Higher values would correspond to a higher stiffness and therefore a wider propagation of the deformation, whereas lower values reflect the tissues ability to stronger local deformations. The parameter rv has a similar effect but is locally restricted to the upper ventricle area.

The alteration of this small parameter set allows for adaption to different patient groups, e.g. younger patients with an expected higher tissue flexibility. While currently not included as no corresponding effect could be observed, the processing pipeline of our approach allows for an easy integration of additional, location-specific parameters. We are currently only evaluating geometric surface parameters (e.g. distance or angle with respect to the annulus). However, additional parameter fields that might for example represent altered local stiffness due to structural tissue deviations could be easily integrated in a similar fashion.

Given a segmentation mask and annulus definition as input, our approach on virtual downsizing can be applied as a fully automatic step in the DSSMitral pipeline. Deformation can be obtained within seconds, and it provides a small set of parameters and an intuitive implant transformation widget to alter those results if necessary. Thus, it can be added to the DSSMitral system as one of the virtual intervention tools without unnecessarily increasing the pipeline complexity or runtime. While it already can be used as standalone tool to assess potential post-treatment stenosis from flow simulations, it will mainly be used as part of extended virtual treatment procedures, such as virtual valve resection, clipping or chord manipulation. To achieve this, the virtual downsizing will be coupled with a dynamic mitral valve model.

If included in the clinical treatment planning, we expect an impact on the sizing strategies. In current clinical applications, the successful implant sizing, e.g. during surgery with help of vendor-specific templates, has proven to be prone to inconsistency and dependent on the clinician’s experience [30]. We assume that a geometric assessment during the virtual downsizing in synergy with predictive flow simulations can improve the current situation. An evaluation that includes all the steps of the DSSMitral pipeline will provide insight.

To better quantify the impact of the local lumen deformation on this assessment, the next step is to perform a parameter dependency analysis. Thus, we can quantify to which degree the currently implemented parameter set influences the flow results and therefore the overall assessment. This will also include a comparison with a naive, linear lumen reconstruction between virtual implant and patient-specific lumen. Thus, we can characterize the impact of the lumen shape close to the annulus on the overall flow.