1 Introduction

Biological cells are constantly exposed to different fluidic environments both in-vivo (in their natural environment) as well as during studies performed to understand their physical properties in-vitro [1,2,3,4]. These include red blood cells in whole blood, cell manipulation in microfluidic devices, propulsion of bacteria and bacteriophages within a fluid medium, and multicellular assemblies under biological and environmental flows. An intricate coupling between cellular behavior and fluid flow is pivotal to understanding the response of cell to external stimuli. Cells in an ambient fluid flow are subjected to stresses that can affect their individual cell dynamics as well as impact multicellular organization. Further, a fluid-flow environment can induce cell deformation, facilitate cell motility, and can be used for probing bio-mechanical properties of cells [1,2,3,4,5,6,7,8,9]. Despite numerous researches in this area, a considerable scientific attention is desirable in view of several unresolved fundamental questions as well as a potential for possible practical applications. The confluence of a number of scientific fields, including soft condensed matter physics, cell biomechanics, biophysics, fluid-dynamics, microrobotics and tissue engineering makes investigations involving cells in an external fluid-flow environment both challenging as well as a rewarding enterprise. A number of interesting open issues include: (a) deformation of cell and multicellular aggregates in an ambient fluid flow, (b) cell manipulation and sorting, (c) cell motility in a fluidic environment, (d) dynamics of multicellular suspension, (e) cellular adhesion in fluidic environment.

Manipulation of microscopic target objects finds wide applications in various biomedical fields such as cell positioning [10], cell sorting [11], cell trapping [12], microassembly [13], microsurgery [14], targeted drug delivery [15], and estimation of cell properties [16]. Micromanipulation employing robotics is in vogue due to improved automation, accuracy, repeatability, safety, and throughput. Magnetic micromanipulation has gained much traction in recent years due to its all-encompassing untethered nature, higher force generation capability, and enhanced post manipulation cell viability as opposed to other available micromanipulation techniques such as atomic force microscopy [17], micropipette aspiration [18], electrophoresis [19], and optical micromanipulation [10]. The incorporation of magnetic microrobots has further armed magnetic micromanipulation with selective manipulation capabilities vis-a-vis microfluidics where positioning of a specific target cell present amongst other cells to a commanded goal location is difficult to perform [20]. Hence, magnetic micromanipulation emerges as an ideal candidate for the selective manipulation of target objects especially large microscopic biological objects (having sizes in the order of hundreds of microns) requiring higher manipulation forces. Often ambient fluid flow is found to be present micromanipulation environments, be it in-vivo such as cardiovascular systems or in-vitro where the target object may be required to be pushed to a certain goal or anchored at a place in the presence of a flow field.

Apart from positioning of target objects, another important and highly developed application of micromanipulation is the sorting of target objects. Fluorescence-activated cell sorting (FACS) employing laser-based interrogation and electrostatics-based sorting has emerged as a pioneering sorting technique with commendable throughput [21]. However, FACS requires a large number of cells as its recovery rate lies between 50 - 70 per cent [22] and, thus, is disadvantageous while dealing with target objects having low population. The bulky nature and generation of hazardous aerosols during the sorting process [23, 24] associated with FACS has led to the development of other microfluidic-based sorting techniques with lab on a chip kind of environments. These microfluidics-based active cell sorting techniques such as acoustophoresis [25], dielectrophoresis [26], magnetophoresis [27], and optical tweezers-based sorting [28] require an external stimulus to sort the cells. Whereas FACS and electrophoresis may compromise cell viability due to strong electric fields [25], optical tweezers are known to cause photo-damage to the cells [29]. Furthermore, FACS, acoustophoresis, electrophoresis, and optical tweezers cannot manipulate large cells that require higher magnitude of actuation forces for manipulation. Magnetophoresis on the other hand demands labelling with magnetic markers for sorting.

Image-based or vision-guided microfluidic approaches have gained much prominence in the preceding years owing to different advantages such as easy classification of a wide range of target objects. Upon classification, the cells are then sorted using diverse sorting techniques employing optical tweezers [30], acoustophoresis [31], electrostatics [32], dual membrane push-pull sorter [33], and piezoelectric actuator integrated with the microfluidic chip [34], etc. The push-pull sorter and piezoelectric actuator-based sorter are capable of sorting large cells but require sheath/focussing flow and cannot segregate more than two or three types of particles due to limitations in their actuating mechanisms.

In this paper, banking on the advantages of magnetic manipulation, we present an automated untethered nonprehensile approach for positioning large microscopic target objects in the presence of a fluid flow. We also perform image-guided benign and label/marker-free sorting of large microscopic target objects having small population in a user-defined sequence. Few examples of large microscopic biological objects are polyploid giant cancer cells (PGCCs) [35, 36], multicellular spheroids [37, 38], and selective autophagy adaptor p62 [39, 40]. As these cells/aggregates are used in stem cell research, cancer research, autophagy, etc., positioning and/or sorting them is of fundamental significance in pursuance of these researches.

In the past, several researchers have used prehensile magnetic microrobots [41,42,43,44] for cell manipulation which required complex microfabrication techniques and turned out to be expensive. Though these microrobots provided excellent control, some of them either could not effectively handle the variation in cell sizes [44] or required an extra stimulus (such as heat) for actuation [42]. We, in our previous work [45], presented an inexpensive and easy to fabricate convex-shaped ferromagnetic microbot for nonprehensile (without form- or force-closure grasp) magnetic micromanipulation of large microscopic target biological objects. Seif et al. presented a magnetic bilateral telemanipulation system for accurate positioning of non-magnetic microbeads by direct contact or by pushing or pulling [46]. The feasibility of the propulsion of a ferromagnetic core in the cardiovascular system with fluid flow has been demonstrated using a magnetic resonance imaging (MRI) system by Mathieu et al. [47]. Erni et al. evaluated the forces required to manipulate magnetic objects inside bodily fluids. The research also presented the optimisation of the magnetic actuation systems based on the working volume [48]. Trapping of contrast agents-linked polystyrene beads mimicking circulating tumour cells (CTCs) in human peripheral vessels using permanent magnets was performed by Wei et al. [49].

Several cell sorting works employing FACS have been reported in the literature. Separation of fetal cells from the blood of pregnant women was carried out by Herzenberg et al. for potential prenatal diagnosis [50]. Isolation of normal and cancer-associated fibroblasts from fresh mouse tissues was performed by Sharon et al. [51]. The decontamination potential of FACS and MACS (magnetic-activated cell sorting) for murine and human testicular cell suspensions has been evaluated. The study aims at developing a method a priori for screening testicular tissue for malignant cells [52]. Literature reviews pertaining to application of FACS in microalgal analysis [21] and plant development and environmental responses [53] have also been reported.

Multiple image/marker-based researches for selective sorting of single-cells have also been carried out. Active cell sorting systems that sort micron-sized particles and cells by surface acoustic waves upon fluorescence investigation have been reported [25, 31]. Kovac and Voldman developed an image-based optofluidic technique for cell sorting. The target objects were levitated from a microwell by the scattering force of a focused infrared laser and the cells were subsequently drifted by the fluid flow for collection [54]. Wang et al. reported a highly accurate cell sorting approach for handling a small cell population. Optical tweezers and microfluidics were integrated to sort yeast and human embryonic stem cells [30]. An optical travelator for transporting and sorting colloidal microspheres using an asymmetrical line optical tweezers was proposed by Cheong et al. Multiple silica beads and polystyrene beads used as target objects were simultaneously manipulated along a single line by the developed travelator [28]. Microfluidics-based cell transport/cell sorting employing different kinds of optical tweezers such as holographic optical tweezers [55] and line optical tweezers [56] have also been reported in the literature. Due to the relatively small actuation forces, the aforementioned selective sorting approaches are not suitable for manipulating large microscopic target objects.

Numerous works in the area of microfluidics-based active cell sorting involving magnetic stimulus have been reported. Separation of circulating tumour cells (CTCs) from blood samples using magnetophoresis caused by permanent magnets have been carried out [11, 27]. Mizuno et al. developed a magnetophoresis-integrated hydrodynamic filtration system for sorting of cells based on their size and magnetic markers [57]. Most of these magnetophoretic cell sorting techniques require labelling with magnetic microbeads. A novel magnetically driven microtool (MMT) has been developed for sorting of unlabelled polystyrene beads having sizes \(\sim\)100 μm. The sorting algorithm uses image-processing for real-time identification of the target object [58]. Literature surveys on microfluidic cell sorting works have also been carried out and reported [59, 60].

Based on the above, the research gaps identified are: i) research works pertaining to automated micromanipulation of target objects to commanded goal locations in the presence of ambient fluid flows are rare, and ii) researches reporting closed-loop feedback-controlled sorting of large microscopic target objects in a benign and marker/label-free manner are still in the early stages. This paper presents an automated image-guided nonprehensile approach that is capable of selectively manipulating large microscopic target objects to any commanded goal location present along any general direction in the presence of a fluidic flow field.

The developed system comprises electromagnetic coils kept in a quadrupole configuration. The magnetic field generated by these coils actuates the ferromagnetic microbot which in turn pushes the target object in the nonprehensile manner towards the commanded goal location. We utilise silica beads (with size range of 150 - 350 μm) as surrogate biological target objects for this purpose. This paper reports the motion model and image-based localisation technique of the microbot and the target object. A feedback planner comprising of five motion manoeuvres, namely, Arrest, Approach, Align, Push, and Home has been developed. Based on the locations of the microbot, target object, and goal location the manoeuvres are called. We also developed a proportional controller whose job is to execute the manoeuvre called by the manoeuvre planner by passing requisite currents to the four coils. We conducted several physical experiments utilising the pushing action of the microbot and present the obtained results in this paper.

The prominent advantages offered by our system are: (a) employing microbots that are inexpensive and easy to fabricate, (b) no prior orientation of these microbots is required as in the case of prehensile robots [41], (c) target objects of varying sizes may be handled, (d) facilitates further phenotyping and recultivation [61] of the marker/label-free target objects post manipulation, and (e) no buffer/focussing flow is required for sorting target objects.

This paper builds on the previous work [62] done by our research group, wherein we developed a robotic tool for automated magnetic micromanipulation in the presence of an ambient fluid flow. The new developments reported by this paper are:

  • In the previous work, a robotic tool was reported where only the microbot was manipulated in the presence of a flow field. In the present work, a comprehensive approach for nonprehensile magnetic micromanipulation of target objects in the presence of an ambient fluid flow has been developed.

  • A new motion manoeuvre namely, Home, has been generated to perform more complicated manipulation tasks. The developed manoeuvre-based approach builds upon the works in [10, 45]. However, positioning of a target object at a specified goal in the presence of a fluid flow, to the best of our knowledge, is being reported here for the first time.

  • Image-based sorting of target objects as per the user-specified sequence has been performed. In the microfluidics-based active sorting domain, image-based sorting techniques have been widely reported [59, 60]. Nonetheless, selective sorting of target objects using a microbot in an ambient fluid flow, has not been reported before.

  • Several physical experiments with varying target object sizes (to mimic a variety of large microscopic biological objects) have been performed. These include positioning, position holding, and sorting of target objects to establish the validity and effectiveness of the proposed micromanipulation approach.

2 Problem statement

Let xr, xc, and \(x_{g} \in \text {I\!R}^{2}\) be the locations of the microbot R, target object C, and the desired goal location G, respectively. Let us suppose I(i,j,t) represents the workspace image at time t with (i,j) being the pixel coordinates. We express the coordinates of the locations of the particles in the inertial reference frame Ω and the coordinates of the pixel in the image frame Λ. Also, let vf be the velocity of the ambient fluid flow (see Fig. 1).

Fig. 1
figure 1

Schematic description of the micromanipulation environment containing an ambient fluid flow. Ω and Λ are the inertial frame and image frame, respectively

Full size image

The problem statement is to determine:

  1. 1.

    xr(t) and xc(t) from I(i,j,t),

  2. 2.

    manoeuvre M(t), depending on xr(t), xc(t), and xg, and

  3. 3.

    the required currents \(I_{x^{+}}(t)\), \(I_{x^{-}}(t)\), \(I_{y^{+}}(t)\), and \(I_{y^{-}}(t)\) to be passed through the coils \(C_{x^{+}}\), \(C_{x^{-}}\), \(C_{y^{+}}\), and \(C_{y^{-}}\), respectively, so that the magnetic field thus generated would actuate R to push C towards G.

3 Solution approach overview

The steps employed to address the problem formulated in the preceding section are as follows (see Fig. 2):

Fig. 2
figure 2

Solution approach overview

Full size image
  1. 1.

    Development of a motion model of the nonprehensile magnetic micromanipulation.

  2. 2.

    Generation of a repository, L, of motion manoeuvres, namely, Arrest, Approach, Align, Push, and Home.

  3. 3.

    Development of a manoeuvre planner which calls one of the five motion manoeuvres from L based on xr(t), xc(t), and xg.

  4. 4.

    Development of a proportional controller to execute the manoeuvre called by the manoeuvre planner by passing requisite currents through the coils.

  5. 5.

    Conducting physical experiments to demonstrate the effectiveness of the proposed micromanipulation technique.

4 Experimental setup

The experimental setup (see Online Resource 1) comprises a set of electromagnetic coils, stereo microscope, syringe pump, DC power supply, controller board and a personal computer.

A setup consisting of four coils kept in a quadrupole configuration has been fabricated. The coils having circular cross-sections are fabricated in-situ using copper wires wound on 3D-printed bobbins. Ferromagnetic cylindrical cores are inserted in the coils to amplify their magnetic field strengths. The magnetic flux density and its gradient produced at the centre of the setup by a coil carrying 3 A current and placed 43 mm away, are found to be 9.42 mT and 0.8 T/m. The coil design parameters are provided in Table 1.

Table 1 Coil design parameters
Full size table

The microbot and the target objects are suspended in the fluid medium passing through an open fluidic channel. The flow inside the channel is generated by a syringe pump (Make: Cole-Parmer; Model: 74905-04). The fluidic channel is placed at the centre of the testbed and observed under a stereo microscope (Make: Magnus; Model: SMZ Bi). The field of view obtained from the 5 MP 30 FPS CMOS camera mounted on the eyepiece of the microscope is (6.38 × 4.78) mm2. The raw image obtained from the camera is acquired and processed to localise the microbot and the target object. The particles are localised using circular Hough transform making use of the MATLAB’s image-acquisition and image-processing toolboxes [45].

The currents carried by the coils are supplied by a DC power supply (Make: GW Instek; Model: GPS-4303) via solid state relays. An Arduino-Uno microcontroller board attached to a computer passes PWM-based control signals to the relays. Based on the duty-factor of the PWM signals received from the microcontroller, the relays continuously modulate the value of the control currents to be routed to the coils in order to generate the desired resultant magnetic field actuating the microbot in the required direction.

5 Motion model

We present a motion model which is developed to simulate the nonprehensile pushing of the target object by the magnetic microbot in the presence of an ambient fluid flow. We make the following assumptions so as to simplify the motion model:

  • The micro-objects are spherical in shape.

  • The target object is rigid in nature.

  • The ambient fluid flow has reached the steady state.

  • The micro-objects have gained terminal velocity which is equal to the velocity of the fluid flow (vf).

Let mr, xr, rr, Vr, and Mr be the mass, position, radius, volume, and magnetisation of the microbot, respectively. The equation of motion of the ferromagnetic microbot actuated by the magnetic force (Fm) generated by the coils and resisted by the drag force (Fd) offered by the fluid medium is given by:

$$m_{r} \ddot{x}_{r} + F_{d} = F_{m}$$
(1)

The magnetic force (Fm) acting on the microbot placed in a global magnetic field having flux density B is as:

$$F_{m} = V_{r} \nabla(M_{r} \cdot B)$$
(2)

We use numerical simulations to evaluate the resultant magnetic field generated by the coil set. COMSOL Multiphysics software is used to simulate the magnetic flux density over the workspace. The simulation results are validated by comparing the obtained values with actual measurements. We use a handheld Hall-effect gaussmeter (Make: SES Instruments; Model: DGM-HH-01) to measure the magnetic flux density, Bmeasured, generated by the coils. A comparison is made between the two flux density values at nine specific locations for the coil, \(C_{x^{-}}\), which is kept along the negative x-axis for five distinct current values (see Online Resource 2). The simulated values are found to be in compliance with the measured values. It is also observed that the gradient of the magnetic flux density, i.e., ∇B, does not incur much fluctuations and, hence, is assumed to be locally linear. Furthermore, as per Biot-Savart law [63], the magnetic flux density gradient (∇B) is proportional with coil current (I) and the same has also been experimentally validated by us (see Online Resource 3).

Since the microbot size and average velocity are \(\sim\) 100 μm and \(\sim\) 100 μm/s, respectively, we approximate the micromanipulation environment to be a low Reynolds number regime. Thence, the drag force (Fd) acting on the microbot turns out to be the Stokes’ drag [63, 64] and is given as:

$$F_{d} = 6 \pi \eta r_{r} (\dot{x}_{r} - v_{f})$$
(3)

where, η is the dynamic viscosity of the ambient fluid. Moreover, due to the low Reynolds number regime where the viscous force dominates over the inertial force, the inertial term in Eq. 1 is neglected [41] simplifying the equation to:

$$6 \pi \eta r_{r} (\dot{x}_{r} - v_{f}) = V_{r} \nabla(M_{r} \cdot B)$$
(4)

Equation 4 may be used to determine the velocity (\(\dot {x}_{r}\)) and position (x) of the microbot. The microbot pushes the target object in a nonprehensile manner and the velocity gained by the target object (\(\dot {x}_{rc}\)) due to this pushing action (see Fig. 3) is given by:

$$\dot{x}_{rc} = \alpha \xi_{m} max((\dot{x}_{r} \cdot \hat{d}_{RC}),0)$$
(5)

where, \(\hat {d}_{RC} = \frac {x_{c}-x_{r}}{\parallel x_{c}-x_{r} \parallel }\) is the unit vector along the line joining the centres of R and C, \(\xi _{m} = \frac {m_{r}}{m_{c}}\) represents the ratio of the microbot and target object masses, and α is the damping factor to account for the losses because of the neighbouring fluid. This term depends on the type of surrounding fluid (having the maximum value of 1 for vacuum state) and may be determined from the physical experiments. Also, as α depends on the viscosity of the ambient fluid, it does not change with evaporation or change in flow rate as long as the viscosity remains unchanged and the flow is maintained in the channel. In the experiments performed by us, it is observed that the target object does not break contact with the microbot while being pushed by it. Therefore, for the sake of simplicity, we consider the value of the term α to be \(\frac {1}{\xi _{m}}\) for all practical purposes of the reported experiments. The resultant velocity of the freely drifting target object pushed by the microbot is defined by the following equation:

$$\dot{x}_{c} = \begin{cases} \dot{x}_{rc} + v_{f}, & \text{if} \parallel x_{r}-x_{c}\parallel \leq r_{r}+r_{c} \\ v_{f}, & \text{otherwise} \end{cases}$$
(6)
Fig. 3
figure 3

Micro-object velocities: (a) velocities of the target object before and after coming in contact with the microbot, and variation of the microbot velocity with coil current for: (b) particle having size 320 μm and flow rate and flow velocity of 2 μl/s and 52.2 μm/s along the positive x-axis, respectively and (c) particle having size 216 μm and flow rate and flow velocity of 3 μl/s and 78.3 μm/s along the positive x-axis, respectively

Full size image

It may be concluded from Eq. 4 and the proportionality between the magnetic flux density gradient (∇B) and coil current (I) that the average microbot velocity (\(\dot {x}_{r}\)) is proportional to the coil current (I) within the operating region of the microbot. We conducted physical experiments to validate this proportionality. Experiments were performed in three different test environments with varying microbot size, flow rate, and flow direction to observe the variation of the average microbot velocity at the centre of the workspace with five different current values. For each test environment, three separate experiments were conducted and the mean of the obtained average microbot velocities was calculated to give the required average microbot velocity. Figure 3 shows the results for two test environments where the microbots of sizes 320 and 216 μm are actuated against and along two different flow fields, respectively. It may be observed from the figure that the average microbot velocity is approximately proportional to the coil current. We use this result in developing the proportional controller-based control strategy.

It is to be noted here that we have not modelled surface-based forces such as adhesion. The proportional controller-based closed-loop feedback control scheme developed on the basis of the presented motion model has been found to successfully manipulate the target objects and its effectiveness has been demonstrated by a number of physical experiments reported in Section 8. Therefore, the surface properties of the particles having negligible effect are not considered during the design of the motion control system. Nonetheless, the enhancement of the controllability of the system would warrant the use of more sophisticated control schemes such as model predictive control (MPC) and the unmodelled forces such as adhesion or interaction forces between the microparticles and the environment may then be modelled on the lines of Seif et al. [46].

6 Manoeuvre planner

During the nonprehensile pushing of the target object, the microbot tends to break contact with it. This is by virtue of several factors such as lack of perfect collinearity between the microbot, target object, and goal location, irregular shapes of the micro-objects, and drag forces. Therefore, we develop and present a manoeuvre planner which depending on the positions of the microbot, target object and goal location, invokes specific motion manoeuvres to ensure sustained nonprehensile manipulation. The manoeuvre planner also facilitates the micromanipulation of multiple target objects in the presence of fluid flow.

In order to implement the manoeuvre planning algorithm, we discretize the analog time t and represent the discretized time-step by the superscript k. For example, we use the variable \({x_{c}^{k}}\) to represent the position of the target object at the k-th time-step. We now introduce two new terms, namely, ensemble state and action to aid the description of the feedback-based manoeuvre planning and define them as follows:

  • Ensemble state (\(x^{k} = [{x_{r}^{k}}, {x_{c}^{k}}, x_{g}]^{T}\)): It is a vector xk ∈ I\!R6, where \({x_{r}^{k}}\), \({x_{c}^{k}}\), and xg are the positions of R, C, and G, respectively.

  • Action (\(u^{k} = [I_{x^{+}}^{k}, I_{x^{-}}^{k}, I_{y^{+}}^{k}, I_{y^{-}}^{k}]^{T}\)): The action is defined as a vector uk ∈ I\!R4, where \(I_{x^{+}}^{k}\), \(I_{x^{-}}^{k}\), \(I_{y^{+}}^{k}\), and \(I_{y^{+}}^{k}\) are the currents passed through the coils \(C_{x^{+}}\), \(C_{x^{-}}\), \(C_{y^{+}}\), and \(C_{y^{-}}\), respectively.

The manoeuvre planner calls one of the five motion manoeuvres, namely, (1) Arrest, (2) Approach, (3) Align, (4) Push, and (5) Home, depending on the ensemble state. These motion manoeuvres which direct the controller to actuate the microbot along a specific direction during the course of manipulation are described as follows (see Fig. 4).

Fig. 4
figure 4

Motion manoeuvres for manoeuvre planning: (a) Threshold parameter for determining the alignment between the microbot, target object, and goal location, (b) Arrest manoeuvre, (c) Approach manoeuvre, (d) Align manoeuvre, (e) Push manoeuvre, and (f) Home manoeuvre

Full size image
Arrest::

This manoeuvre is invoked to arrest/hold the position of the microbot at its initial location (A) against the ambient flow when the target object is farther away. This allows the target object to come closer to the microbot before the next manoeuvre is called. The Arrest manoeuvre is executed when the distance between the microbot and the target object is larger than the arrest threshold (ϕ) by actuating the microbot along the vector \(\hat {d}_{RA}\).

Approach::

This manoeuvre makes the microbot approach the target object along the direction \(\hat {d}_{RC}\) and reach its vicinity. It is executed when the distance between the two micro-objects is lesser than the arrest threshold (ϕ) and larger than the approach threshold (ψ).

Align::

The microbot, if not collinear with the target object and goal location, must align and become collinear first so as to successfully push the target object towards the commanded goal. However, due to factors such as noise and dynamic interaction between the particles, a perfect alignment does not happen. We assume the alignment to take place if the angle subtended between \(\hat {d}_{RC}\) and \(\hat {d}_{CG}\) is less than the threshold parameter 𝜃th. The align manoeuvre aligns the microbot by commanding it to move along the tangential vector, \(\hat {d}_{P}\), until the condition \((1 - \hat {d}_{RC} \cdot \hat {d}_{CG}) < \upbeta\) is obtained. Here, β is 1 − cos(𝜃th) is introduced.

Push::

Due to this manoeuvre, the microbot moves along the vector \(\hat {d}_{RC}\) so as to push the target object towards the goal location. The manoeuvre is called when the microbot, target object and the goal location are found to be aligned, i.e, \((1 - \hat {d}_{RC} \cdot \hat {d}_{CG}) < \upbeta\).

Home::

This manoeuvre is invoked when the microbot successfully places the target object at the commanded goal location. Home manoeuvre brings the microbot back to its home location (H) so that the microbot is able to receive and manipulate the next incoming target object in an efficient manner.

The stated manoeuvre generation scheme is summarised in Algorithm 1.

figure a

7 Controller

The manoeuvre called by the manoeuvre planner is executed by the controller. The task of the control algorithm is to determine the suitable action, \(u^{k} = [I_{x^{+}}^{k}, I_{x^{-}}^{k}, I_{y^{+}}^{k}, I_{y^{-}}^{k}]^{T}\), for a given manoeuvre, M (see Fig. 5). In the present scheme, we employ a proportional controller to control the currents passed through the coils.

Fig. 5
figure 5

Controller scheme

Full size image

At first, we determine the error as per the following relation:

$$e = \begin{cases} \hat{d}_{RA}, & \text{if \textit{M = Arrest}} \\ \hat{d}_{RC}, & \text{if \textit{M = Approach/Push}} \\ \hat{d}_{P}, & \text{if \textit{M = Align}} \\ \hat{d}_{RH}, & \text{if \textit{M = Home}} \end{cases}$$
(7)

where, \(\hat {d}_{P} = \frac {(\hat {d}_{CG} \times \hat {d}_{RC}) \times \hat {d}_{RC}}{\parallel (\hat {d}_{CG} \times \hat {d}_{RC}) \times \hat {d}_{RC}\parallel }\) is a vector perpendicular to \(\hat {d}_{RC}\) along which the microbot is required to traverse in order to get aligned with the target object and the goal position.

The value of the error so obtained is multiplied with the gain matrix (Gp) to give the required output, i.e, action (uk). The control scheme is summarised in Algorithm 2.

figure b

The control currents (uk) determined by the control scheme is passed through the four coils using the solid-state relays. As the output of the controller which also acts as an input to the relays is a PWM signal, the analog control current values were first mapped to the PWM values. This was done by recording the values of the current passing through each coil at different PWM duty-factors. Based on the dataset, a three-degree polynomial was fitted to obtain the required PWM duty-factor versus coil current map.

The proportional controller sends the control signals to the relays which energise the coils generating requisite magnetic force which further actuates the microbot eventually manipulating the target object in the desired manner.

8 Experimental results

In this section, we report the various physical experiments conducted by us to test the validity and effectiveness of the developed approach. The reported manoeuvre planning and proportional control schemes were implemented so as to perform four sets of physical experiments. The microbots used in these experiments are iron-filings (Make: Educational Innovations Inc., Model: M-600) with saturation magnetisation (Ms) of 1.7 × 10 6 A/m. The particles having near-spherical shapes are directly chosen as microbots without any prior treatment. Silica beads acting as cell surrogates are used as target objects. The size ranges of microbots and target objects over which the experiments have been conducted are 200 - 350 μm and 150 - 350 μm, respectively. The maximum value of current passed through the coils is 3 A. Two open fluidic channels made of poly(methyl methacrylate) (PMMA) are designed and fabricated to perform the experiments (see Fig. 6). The first channel is a prismatic channel with a single inlet and single outlet and is used in conducting the first three sets of experiments. The second channel being Y-shaped has one inlet and two outlets and is used for the fourth set of experiments. The channels are placed at the centre of the testbed along the x-axis and a flow having flow rate of 3 μl/s is continuously maintained during all the experiments. The ambient flow rate value is chosen based on the size range of the particles to be manipulated and the magnetic field and its gradient generated by the coils. The value of 3 μl/s was the maximum flow rate found suitable for manipulating the entire size range of the particles over the workspace. Up to this flow rate, owing to the closed-loop feedback control, the target objects were always found to be successfully manipulated irrespective of the manipulation time or the number of the executed motion manoeuvres. The corresponding flow velocity values in the two fluidic channels obtained after the division of the flow rate (3 μl/s) by the respective cross-sectional areas of the two channels (i.e, 38.3 mm2 and 37.2 mm2) are 78.3 μm/s and 80.6 μm/s, respectively. These flow velocity values are comparable to the other microfluidics-based active sorting works reported by Chung et al. [65] (0.09 – 0.25 μm/s), Qi et al. [66] (20 μm/s), and Ma et al. [56] (20 – 60 μm/s).

Fig. 6
figure 6

Schematic of the fluidic channels with: (a) one inlet and one outlet, and (b) one inlet and two outlets. Channel depth is 2.66 mm. All dimensions are in mm

Full size image

8.1 Microbot position holding

In the first set of experiments, we perform the microbot holding experiment. In this, the microbot and target objects of varying shapes and sizes are placed in the workspace. The fluid flow would tend to drift the microparticles along with it. The algorithm using Arrest manoeuvre is required to hold the microbot at a goal location (which is the initial location in this case) notwithstanding the effect of the fluid drag. A total of 4 experiments were conducted where the microbot was found to successfully hold its position in close vicinity of the goal position against the fluid flow. Figure 7 shows one such case for a particle of size 304 μm whose position was arrested at goal position (2626, 2249) μm for 28 s. It may be seen from the figure that the ferromagnetic microbot actuated by the magnetic field is able to hold its position against the fluid drag whereas the non-magnetic target objects are drifted away with the flow. The Root Mean Square Error (RMSE) for this case is calculated to be 46.1 μm. It was observed that the RMS errors were less than the radii of the corresponding microbots and the goal locations were always found to be within the periphery of the microbots for all the four experiments. The duration for which the experiments are run is more than sufficient to demonstrate the capability of the system to execute Arrest manoeuvre during the target object positioning and sorting experiments described in the later sections. Online Resource 4 illustrates the aforementioned experiments.

Fig. 7
figure 7

Microbot position holding: snapshot of the micromanipulation environment at (a) the beginning (t = 0 s), (b) t = 12 s, and (c) and at the end of the experiment (t = 28 s). Plots of the microbot’s: (d) location in x-y plane, (e) x-coordinate of location varying with time, and (f) y-coordinate of location varying with time. Flow rate and flow speed are 3 μl/s and 78.3 μm/s, respectively

Full size image

8.2 Target object positioning

In this set of experiments, we performed the nonprehensile manipulation of the target object to the commanded goal locations. The target object is manipulated by the pushing action of the microbot. A total of 9 experiments were conducted with different microbot and target object sizes. The goal locations were varied across the four quadrants such that the manipulation capability along different directions in the presence of ambient flow could be gauged. The target objects were found to be successfully pushed to their corresponding goal locations. Figure 8 shows one such case where a target object of size 272 μm is manipulated by a microbot of size 268 μm to a user-specified goal location. The target object is displaced by 1886 μm upon impact by the microbot with an average velocity of 235.8 μm/s. The manipulation accuracy by which the target object was positioned at the goal, in this case, was evaluated to be 90 μm. It goes without saying that the positioning accuracy for any experiment will always be lesser than the radius of acceptance (ε), which is taken as 100 μm. This value is half of the lowest size of the microbots (i.e., 200 μm) used in our experiments and provides reasonable accuracy for target object positioning. The positioning accuracies obtained in manipulating a 300 μm target object by non-contact pushing and pulling by El-Gazzar et al. are 177 and 100 μm, respectively [67]. A positioning accuracy of 75 μm in manipulating a triangular target object having an edge length of 190 μm has been reported by Khalil et al. [68]. The experiments in both these works were conducted sans a flow field. The nine experiments are illustrated in Online Resource 5.

Fig. 8
figure 8

Target object positioning: (a) image of the micromanipulation environment after the target object reached the goal. Plots of: (b) trajectories traced by the microrobot and the target object, (c) x-coordinates of microrobot and the target object varying with time, and (d) y-coordinates of microrobot and the target object varying with time. The goal location is represented using the symbol xg while the initial and final locations of the particles are represented by xini and xfin, respectively. Flow rate and flow speed are 3 μl/s and 78.3 μm/s, respectively

Full size image

8.3 Target object position holding

In the next set of experiments, we perform the target object holding experiment. The objective here is to prevent the drift of the target object and confine it within a region, say, a containment circle. The target object is first brought inside the containment circle having a radius of 600 μm and thereafter is continuously pushed towards the goal location for 28 s. Figure 9 shows one such case where a target object of size 338 μm is successfully contained inside the containment circle against the fluid flow. The Root Mean Square Error (RMSE) for this case is found to be 235.3 μm. It was observed that the RMS errors were less than the diameters of the corresponding target objects and the centres of the target objects did not cross the containment circle for all the experiments. We conducted 3 target object position holding experiments which are illustrated in Online Resource 6.

Fig. 9
figure 9

Target object position holding: snapshot of the micromanipulation environment at (a) the beginning (t = 0 s), (b) t = 24 s, and (c) at the end of the experiment (t = 35 s). Plots of the target object’s: (d) location, (e) x-coordinates of locations varying with time, and (f) y-coordinates of locations varying with time. Flow rate and flow speed are 3 μl/s and 78.3 μm/s, respectively

Full size image

8.4 Target object sorting

In the fourth set of physical experiments, we demonstrated the possibility of performing the highly accurate selective sorting of target objects having a small population. The target objects entering the workspace are to be sorted to one of the two outlets. As ours is a selective manipulation technique, any desired sorting sequence may be executed using our image-guided automated manipulation approach. We conducted a total of 5 different experiments where the target objects were successfully sorted to the desired outlets as per the user-specified sequence fed to the algorithm. For sorting purposes, the target object instead of being pushed to a specific goal location is continuously nudged along the y-axis till it reaches a specified distance from the centre of the workspace, i.e., nudge threshold (χ). The particles are then carried away by the ambient flow towards the outlets. Figure 10 shows one such case where the particles are sorted in an alternate manner through the two outlets. All these five sorting experiments are illustrated in Online Resource 7.

Fig. 10
figure 10

Target object sorting: snapshots of the micromanipulation environment at (a) t = 0 s at the beginning, (b) t = 5 s when the first target object is sorted, (c) t = 6 s while the microbot is executing the Home manoeuvre to reach its home position, (d) t = 9 s while the microbot is executing the Arrest manoeuvre to wait for the next incoming target, (e) at t = 18 s after the second target object is sorted, (f) at t = 33 s after the third target object is sorted, (g) at t = 46 s after the fourth target object is sorted, and (h) at t = 87 s after all the sorted particles exit the workspace. Flow rate and flow speed in the main channel are 3 μl/s and 80.6 μm/s, respectively

Full size image

In all of the above-reported experiments, the microbots were found to manipulate target objects around 0.4 to 1.3 times their own sizes. The parameter β is set as 0.07 and the value of the nudge threshold (χ) is 1300 μm. The value of the arrest threshold (ϕ) is varied in the range of 1300 - 1500 μm whereas the approach threshold (ψ) is taken in the range of 450 - 550 μm for different sets of experiments. The control commands were issued after image acquisition, image processing, determination of motion manoeuvre and its associated action. The mean control time-step is computed as 72 ms. The corresponding control frequency then turns out to be 13.9 Hz.

It is worth mentioning here that in order to ensure sustained pushing towards the goal, the microbot has to be continuously realigned. This could be reduced to a certain extent by careful selection the threshold parameter, 𝜃th. In addition, the velocity of the microbot should not lag behind the fluid flow velocity for successful micromanipulation similar to other microfluidics-based active sorting techniques. Moreover, in the target object sorting experiments, the population of the incoming particles should be appropriate enough to allow the microbot to sort the particles one by one.

The flow velocity and throughput of the presented system may be enhanced by the use of coils generating higher field and field gradient, optimising the planning and control parameters, and exercising better control using advanced control schemes [69,70,71,72]. The throughput may further be enhanced by an independent manipulation of different target objects by multiple microbots in a simultaneous manner. The microbots are to be independently actuated by local magnetic fields generated by microcoils [73].

During the course of manipulation, the motion of the microbot is seen to pose a negligible influence on the movement of the surrounding particle(s). This is overcome using the image-guided feedback planning and control approach, and the manipulation tasks are successfully performed.

Our approach provides image-based closed-loop feedback control which enables robotic sorting in a controlled manner based on any desired sequence. This comes with further potential to carry image-based sorting on the basis of colour and shape and sorting of more than two types of target objects. Courtesy of the advantages of a benign and label/marker-free approach having large microscopic particles sorting ability, the developed system may be augmented with other conventional sorting systems (either in tandem or parallel) to increase the throughput and sort a wide range of target objects. The proposed robotic sorting system is envisaged to sort rare and/or unknown target objects [74] with high purity to desired outlets having different physio-chemical environments for further investigation.

9 Conclusions

In this paper, we presented an image-guided automated feedback-based nonprehensile approach for magnetic micromanipulation of large microscopic surrogate biological objects in the presence of an ambient fluid flow, employing inexpensive and easy to fabricate ferromagnetic microbots. We developed a feedback-based manoeuvre planner comprising five motion manoeuvres, namely, Arrest, Approach, Align, Push, and Home, for sustained and successful manipulation in the nonprehensile manner. Different physical experiments were performed to test the effectiveness of the developed approach and the microbot was successfully found to perform the tasks of pushing, holding, and sorting of the target objects in the presence of a fluid flow.

In the future, we would like to investigate the performance of the proportional controller under circumstances having non-uniform and transient flow with higher magnetic forces. We would further like to perform the manipulation task having non-uniform and transient flow with higher flow rate by employing a higher field gradient coil set and improving the controller employing control techniques like sliding mode control [69], linear quadratic Gaussian control [70], backstepping control [71], and H-infinity control [72]. We would also like to incorporate obstacle avoidance in micromanipulation by the generation of a collision-free optimal path using graph search techniques [75, 76].

The microbot material’s inherent biocompatibility makes it possible to functionalise it with living cells and/or suitable chemicals, making it feasible to conduct biomedical operations [77]. To conclude, our proposed nonprehensile micromanipulation approach holds tremendous promise involving experiments with large microscopic target cells and cell aggregates, and may open new portals for in-vitro and in-vivo applications involving fluid flow.