Introduction

On January 1, 2019, the New Horizons spacecraft performed a close encounter with the farthest object ever visited by a spacecraft, (486958) 2014 MU69, unofficially nicknamed Ultima Thule (in this text, simply Ultima).Footnote 1 In order to deliver the spacecraft to the target for successful science observation acquisition, thorough and accurate navigation and maneuver execution was performed throughout the months leading up to Ultima closest approach (C/A). Acquiring and processing optical measurements to determine the spacecraft state relative to the target of interest, generally referred to as optical navigation (OpNav), was a fundamental component of the navigation and orbit determination of New Horizons’ Ultima flyby.

Navigating a spacecraft to planetary targets with relatively large ephemeris uncertainties is challenging in many ways. Ultima was only recently discovered during a Hubble Space Telescope (HST) imaging campaign in 2014 in search of a new target for New Horizons to explore after its reconnaissance of the Pluto system [6]. Since Ultima had not been observed over a significant portion of its orbit period around the Solar System barycenter (294 years, [4]), the ephemeris uncertainties of Ultima were larger than the level of uncertainty required for navigation. Additionally, while radio metric data provided accurate Earth-relative spacecraft range and velocity information, it did not provide information about the target-relative state of the spacecraft for small body encounters, since the gravitational acceleration imparted by small bodies is not observable at the level of measurement precision. These spacecraft state uncertainties coupled with the large Ultima state uncertainties made necessary the use of on-board optical measurements to estimate the spacecraft state relative to Ultima to adequate accuracy and knowledge. These challenges are inherent to most interplanetary missions that require optical navigation.

However, there are several challenges specific to New Horizons’ exploration of Ultima that were characterized and mitigated. These include but are not limited to the relatively small size and faint brightness of Ultima relative to other interplanetary mission targets, the density of the background star field (approximately 2° × 2° area of sky in the direction of the spacecraft to Ultima position vector), and the relatively slow apparent motion of Ultima across the sky. These challenges and others are presented and discussed in further detail.

One method of determining the target-relative spacecraft state is using “star-based” optical navigation. Calculating star-based optical navigation measurements begins by using the spacecraft camera(s) to take images of the foreground target(s) against a background of catalogued stars. Comparing the observed star centers in the images to the catalogued reference stars provides precise information about the inertial orientation or pose of the camera, and distinguishes “star-based” OpNav from “’landmark-based” methods or otherwise (not covered in this text). For a flyby scenario like New Horizons’ Ultima encounter, OpNav data helped improve the knowledge of both the B-plane target and time of closest approach (TCA).

The New Horizons OpNav approach imaging scheme consisted of extensive long- and short-exposure imaging over the course of the 5 months preceding the Ultima flyby, and primarily utilized New Horizons’ LOng Range Reconnaissance Imager (LORRI). These images were processed by the navigation team to estimate the target-relative spacecraft state throughout the duration of the approach. Similar to Pluto flyby operations, an independent navigation team (INAV) from NASA Jet Propulsion Laboratory (JPL) also performed orbit determination during the last few weeks of operations to provide a verification of the primary navigation team’s results.

Details regarding the image planning process, optical navigation system design, trajectory design, imaging scheme, and robustness to the detection of multiple bodies on approach to Ultima are discussed. In addition, the optical centerfinding techniques and operational readiness tests that have been designed and executed in preparation for Ultima image processing are introduced. Lastly, a first look at Ultima as viewed from the New Horizons LORRI imager is presented.

Optical Navigation System Overview and Mission Planning

The successful acquisition and processing of on-board images for optical navigation required the design of a robust and reliable image processing pipeline and imaging schedule. The New Horizons optical navigation image processing function was originally designed and implemented for the approach and flyby of the Pluto system in 2015. As such, most of the optical navigation system that was used for the Kuiper-Belt Extended Mission (KEM) is a heritage design that had already been successfully flight-proven. However, a variety of new features were added to the OpNav system design due to several key challenges specific to the Ultima flyby. Similarly, a unique observation schedule was constructed, primarily driven by downlink data volume restrictions and astrometric challenges inherent to very faint and distant targets such as Ultima. This section highlights the OpNav imagers on-board the spacecraft as well as the imaging schedule, trajectory design and astrometric considerations that are unique to the Ultima flyby.

Optical Navigation Imagers and Data Flow

The New Horizons spacecraft is equipped with several scientific instruments, including two optical imagers used for both scientific and optical navigation observations. The LOng Range Reconnaissance Imager (LORRI) [1] is the highest resolution imager on the New Horizons spacecraft, and is the primary optical navigation imager since it provides the highest resolution and thus most precise measurement of the target(s). LORRI has two modes which will be used for optical navigation: the highest resolution “1 × 1” mode and the lower resolution “4 × 4” mode. The 1 × 1 mode is the standard operating mode, in which the optical point spread function (PSF) is well matched to the pixel size and thus well sampled by the detector array. When 4 × 4 mode is used, the photonic charge for each 4 × 4 array of pixels is aggregated on the detector before the charge is read out by the detector. This decreases the effective resolution of the image by a factor of 4 in each dimension and thus under-samples the PSF, but increases the effective signal to noise ratio (SNR). Due to the fact that Ultima is very faint, LORRI 4 × 4 mode was used extensively in the months leading up to the final encounter in order to maximize SNR. LORRI 1 × 1 mode was only used when Ultima was expected to be visible in short (< 1 s) exposure images, starting 8 days before the closest approach. The second optical navigation imager [5], was only to be used for optical navigation in the event that LORRI fails before closest approach, which was not the case.

The optical navigation images of Ultima that were acquired on the spacecraft follow a well-established image downlink process that resulted in an optical data type that was processed by the broader orbit determination filter to determine the target-relative spacecraft state. The OpNav data flow was originally designed and tested during the Pluto flyby and the several preceding years. The system architecture for the KEM flyby of Ultima remained mostly identical, with the exception that the Science Operations Center (SOC) pipeline further included the ability to package “windowed” LORRI 4 × 4 image data into a cohesive and readable FITS image file. The term “windowed” describes an image for which only a subset of the pixel array has been downlinked from the spacecraft, primarily to reduce data downlink volume. Although there were no plans to window optical navigation images, the science operations pipeline added the feature to process windowed LORRI 4 × 4 images in the event of a contingency scenario where significant downlink data volume is unavailable, which was not required.

Trajectory Design Considerations

One particular challenge of optical navigation to Ultima was the densely populated star field behind the apparent position of Ultima relative to the New Horizons spacecraft. Since the approach asymptote to Ultima placed the target in a particularly crowded region of Sagittarius, near the galactic center, and Ultima maintained a relatively low apparent magnitude during the entire OpNav imaging campaign, the degradation of astrometric quality due to star signal corruption was a challenge that had to be mitigated. One of the techniques used to decrease the chances of Ultima overlapping relatively bright stars during the approach was to choose Prime (3,500 km altitude) and Alternate (10,000 km altitude) flyby trajectories that mitigated the likelihood of such an event. This was done by finding a trajectory that best avoided bright star conjunctions during critical observation times while still achieving all of the scientific objectives. An example of what the Ultima star background looks like using long LORRI 4 × 4 exposures, as well as the nominal apparent position of Ultima over the optical navigation imaging campaign is shown in Fig. 1. Since the true apparent position of Ultima was unknown (given the underlying necessity to use optical navigation), this design strategy was only able to decrease the likelihood of Ultima colliding with a relatively bright star, not guarantee such. Other methods for mitigating star overlap are described in more detail in further sections.

Fig. 1
figure 1

Example LORRI 4 × 4 coadded image of the Ultima background star field, with the predicted nominal position of Ultima as observed along the Prime trajectory overlain in yellow. The position of Ultima at the beginning of the imaging campaign (August 16, 2018) is identified by the yellow ×, and the position of Ultima at the last OpNav (December 30, 2018) is identified by the ∘. This and all images in this text are presented in units of Data Number (DN), the raw unit of measure that is recorded by the imager detector

Full size image

Observation Schedule and Design

To ensure robust spacecraft-relative astrometric measurements of Ultima’s position were acquired for optical navigation, an extensive imaging schedule was designed and planned for the months preceding closest approach. To maximize SNR, LORRI 4 × 4 mode was used for many observations from initial detection on August 15, 2018 through the last OpNav image on December 30, 2018, two days before the closest approach. Due to the larger pixel size, using LORRI 4 × 4 mode along with the “Relative Control Mode (RCM)” of the attitude control system allows for very long exposures to be taken without significant image degradation caused by smearing or similar. During a flight test in September 2017, it was determined that up to 30-s exposure images could be acquired without significant degradation in image quality. Thus, 30-s exposure LORRI 4 × 4 images were used for all of the optical navigation imaging from initial detection through December 7, at which point the exposure times gradually stepped down to 20, 10, 4, and 2 s. These shorter exposure times were chosen to optimize SNR but ensure that Ultima was not overexposed in the event that it was much brighter than predicted. To determine how many images at which exposure time should be acquired, the worst-case Ultima SNR at each observation epoch was required to be > 8, with the exception of the very earliest epochs, in which case the SNR was > 3.

Due to the fact that Ultima is very faint, an image processing technique called “coadding” or “stacking,” explained in detail in “Simulated Target Implantation”, was implemented to ensure the SNR of the target was high enough for reliable astrometric processing. Many images of the target were acquired per observation epoch or “visit,” and then coadded together to produce “coadded images.” This process was used extensively during the earliest observations when the apparent magnitude of Ultima was the most dim.

Once the apparent magnitude of Ultima was bright enough, much shorter exposure LORRI 1 × 1 images were acquired to provide the highest resolution and thus most precise measurement of Ultima’s position. The first attempt at observing Ultima in LORRI 1 × 1 images was on December 24, eight days before closest approach. From that point until the last optical navigation observation on December 30, multiple LORRI 1 × 1 images were taken once per day, at a cadence that facilitated timely downlink and processing within the broader New Horizons navigation and mission operations schedule.

To facilitate the image processing technique of star subtraction, which is also explained in detail in “Image Coadding”, several sets of LORRI 4 × 4 and 1 × 1 images of the Ultima background star field were acquired at different exposure times in January, September, and December 2017. This provides a set of images in which only background stars are present, which was used to subtract from optical navigation images of Ultima, in an attempt to remove stars and isolate Ultima in the image. The schedule of optical navigation observations for the KEM flyby of Ultima, including the background star field images already acquired, is provided in Table 1.

Table 1 Approach Optical Navigation Schedule Summary for New Horizons’ Ultima flyby. A “visit” is defined as a set of images taken at nearly identical times. ⋆Daily imaging except on days on which a Trajectory Correction Maneuver (TCM) is planned
Full size table

The image attitude design for the Ultima flyby is relatively simple compared to that of New Horizons’ Pluto flyby, due to the number of visible targets in the image scene. Since Ultima was the only planetary target observed in the scene (aside from any natural satellites that may be detected), the attitude commanding design for all Ultima images was to point the camera boresight (center of image) at Ultima’s predicted location (nadir-pointing). The camera roll angle was commanded to always align the spacecraft +Z axis with the North Celestial Pole. Special care did not need to be taken to command different attitudes or roll angles for different observations, as there are no bright targets in the scene that would cause significant diffraction spikes, optical ghosting, or readout smear. In addition, the exposure times of both the LORRI 4 × 4 and 1 × 1 images are long enough such that nadir-pointing with any roll angle provides enough stars in the scene to determine the inertial attitude for every image, as described in “Image Processing Techniques”. Even though Ultima itself is very faint, the other stars in the scene were not bright enough to cause significant charge bleeding or runoff. This was not the case for the Pluto flyby, where very precise attitude commanding was necessary to ensure artifacts from overexposed signals of Pluto and its largest moon Charon did not corrupt the signals of its smaller moons Nix and Hydra, and to ensure there were enough stars in the shorter exposure LORRI 1 × 1 images [3].

Considerations Regarding Detection and Astrometry of Multiple Bodies

Given various limitations of the existing astrometric, photometric, and occultation data collected for Ultima to date, there was a non-negligible chance that Ultima may have in fact been a system of more than one body. Earth-based measurements taken before the flyby had constrained the size, brightness, and other physical parameters of potential satellites, but did not entirely eliminate the chance that one or more satellites may be discovered using spacecraft-relative imaging. Thus, the optical navigation system was scrutinized and enhanced in order to prepare for the timely detection and estimation of multiple target ephemerides.

Given the original design of the optical navigation system for the Pluto flyby, it is relatively straightforward to identify, associate, and process multiple bodies in a given image when there is adequate a priori knowledge of the target ephemerides [3]. This process had been designed and proven during the Pluto flyby and was exercised in the event of Ultima multiplicity. However, the challenge arises when no a priori knowledge is provided, as could have been the case for the Ultima flyby. In this case, detection and ephemeris estimation would have needed to occur in near real-time. However, this was not the case for the Ultima flyby.

In order to address this challenge, a variety of image processing techniques were developed and exercised to optimize the ability to detect natural satellites and produce astrometric results that could be used to estimate the satellite ephemerides. Image coadding and star subtraction, described in detail below, were used to maximize the SNR of both primary and secondary bodies. To monitor a potential detection, the astrometric and photometric (PSF) residuals of the (assumed) primary, single body were analyzed along with photometric lightcurve data. Manual image comparison (blinking) across multiple image epochs was also performed by users to visually monitor any potential new satellite detections. In the event that multiple bodies were detected, new centroiding algorithms were implemented to ensure reliable astrometric measurements of each target. These are described in detail in “Multi-Gaussian Least Squares Fitting”.

To provide the orbit determination filter with a priori knowledge of a binary orbit, astrometric and photometric data is used to predict the state of the system at each observation time. Since the images only provide two-dimensional knowledge of the body positions at a given time, the target separation in the line-of-sight direction must be assumed based on known physical constraints of the system like the gravitational parameter and period. After constructing relative position estimates over multiple epochs, a rough estimation of the orbital elements can be performed. These rough predictions provide an initial condition for more precise ephemeris estimation using the broader orbit determination filter.

In addition to detection and centerfinding enhancements developed to provide robustness against multiplicity, the broader New Horizons navigation and operations teams designed and tested the operational process which would be executed in this scenario. While some of this is outside the scope of optical navigation preparation, it is useful to highlight that the successful orbit estimation and timely operational response to the detection of multiple bodies required close coordination between the navigation, science operations, and mission operations teams in order to ensure a productive reconnaissance of such a system.

Image Processing Techniques

New Horizons optical navigation is performed by acquiring and processing images of the target(s) in front of a background field of catalogued stars. In this process, often referred to as “star-based” optical navigation, the centers of the stars are calculated and their differences minimized in a least-squares sense against the predicted star centers, provided by a reference catalogue, to determine the inertial attitude of the camera at each image epoch. The observed center of the target is then calculated using an appropriate centerfinding technique. The inertial camera attitude and observed target center(s) are then passed to the orbit determination filter and used to simultaneously estimate target and spacecraft states, using a combination of optical and radio metric measurements. The error of both the attitude and target center estimation are used to inform and assign a weight to the target center measurements in the orbit determination filter.

The KinetX Image Processing (KXIMP) software suite was used to perform all New Horizons star-based OpNav operations. KXIMP consists of a suite of MATLAB functions and scripts that are used to calculate star-based OpNav observables using on-board images of the target. From a high-level, KXIMP ingests images of the target and background stars, identifies targets in the images using a priori knowledge, determines the centers of visible stars and targets in the field, calculates the inertial camera attitude, then packages the relevant data into an output file that is passed to the primary orbit determination (OD) filter program. In the case that a priori knowledge of the target state is not available, the user can identify the target using various techniques including manual image blinking. KXIMP was successfully used for optical navigation processing during the New Horizons Pluto flyby [3], and was used for both the New Horizons KEM and OSIRIS-REx exploration of asteroid Bennu. While a high-level overview of the KXIMP functionality relevant to New Horizons is presented here, a more detailed description of KXIMP and its capabilities can be found in a previous publication [2].

Star and target center calculations within KXIMP can be split into two regimes: point-source centerfinding and extended-body centerfinding. Point-source centerfinding algorithms are used to calculate the centers of objects that are \(\sim \)3 pixels in diameter or less. In this case, a number of two-dimensional Gaussian functions may be fit to the point-source image data in a least-squares sense in order to determine the best estimate for the center(s). Since the diameter of Ultima as observed in the images was expected to be < 3 pixels from initial detection until the last optical navigation measurement 2 days before closest approach, only the point-source algorithms were exercised for processing Ultima approach images.

In order to address certain challenges posed by Ultima during the KEM, a variety of functional enhancements were added to KXIMP since its use during the Pluto flyby. In addition to the most important enhancements described in the following subsections, several smaller capabilities were developed, including the ability to process windowed images, defined above. Since windowed images are not planned to be used for routine optical navigation, this capability was only to be exercised in a contingency scenario, which did not occur.

In this section, the image simulation, target implantation, image coadding, star subtraction, and centerfinding techniques unique to the Ultima flyby will be presented and explored.

Image Simulation

In addition to processing images to produce astrometric results, KXIMP also has the ability to simulate images in order to test expected performance and prepare for flight operations. Simulated images from KXIMP were used extensively to prepare for the Pluto flyby and were an integral source of simulated data for prior successful Pluto-based Operation Readiness Tests (ORTs). These simulated images from KXIMP were again a useful source of test images for testing the optical navigation system for the New Horizons KEM. Although a detailed explanation of the image simulation process can be found in previous publications [2], a new simulation capability has been added to more accurately simulate KEM images of Ultima in order to ensure realistic images are used for various KEM navigation- and project-level tests, described in detail below. Although the simulated images from KXIMP are used for internal testing and the first couple of ORTs, it was decided to use simulated images generated by the Southwest Research Institute (SWRI) for later ORTs to provide robustness against issues that may arise due to using the same software to simulate and process the images.

Simulated Target Implantation

In order to increase the fidelity of simulated images for KEM, the ability to implant simulated signals of targets onto a real background star image was developed. The process ingests a real background image, determines the inertial attitude using the stars, then uses the resulting inertial to camera frame rotation matrix to project the simulated inertial position of the target onto the image plane. If the target is unresolved, it is modeled as a 2D Gaussian point spread function. Otherwise, the target is simulated by ray-tracing from the detector into the scene assuming a tri-axial ellipsoid or arbitrary target shape as defined using a SPICE digital shape kernel (DSK). This simulation technique produces images that more closely resemble real-world imaging conditions, including a realistic characterization of the stars’ point spread functions, noise, stray light, and background levels. Additionally, this simulation method allows for more accurate assessment of KXIMP’s coadding and star-subtraction tools, due to the presence of real stars and attitude estimation uncertainties. In September 2017, the LORRI and MVIC instruments were used to acquire images of the Ultima star field that was observed during optical navigation imaging on approach to the target. These images were used to generate simulated Ultima images with the full, real background star field present. An example image with and without target implantation is provided in Fig. 2.

Fig. 2
figure 2

Left: LORRI 4 × 4 image without implanted target. Right: Same LORRI 4 × 4 image with Ultima implanted into image. The yellow arrow points at the Ultima signal that has been implanted into the image. The simulation epoch for these images is December 26th, 2018, six days before Ultima closest approach

Full size image

Image Coadding

Due to its small diameter and faint optical properties, Ultima was the faintest object ever targeted using interplanetary optical navigation. As such, there are several image processing techniques that were implemented in order to produce reliable astrometric results that could be used for spacecraft navigation. One of these is the technique of “coadding” or “stacking” images to increase the SNR of the observed targets. Generically, coadding is the technique of registering a set of images to a uniform reference frame, then taking an average of the stack at each pixel location to produce one final “coadded” image. Since each resulting pixel is an average of all the pixels in the stack, the noise in the final coadded image is reduced by approximately the square root of the number of images in the stack and the resulting SNR of the targets in the image (SNRf) is increased as \(\text {SNR}_{f} = \sqrt {n}~\text {SNR}_{0}\) where n is the number of images and SNR0 is the SNR in a single image. The image coadding technique implemented in KXIMP is described in detail below:

  1. 1.

    Calculate the inertial attitude of all images in the set to be coadded by minimizing the star center residuals in a least-squares sense. This determines the inertial coordinate frame for each image in the set.

  2. 2.

    Define a “master” inertial coordinate grid onto which all images will be projected. This is typically defined as the coordinate grid of the median or mean observation (in time).

  3. 3.

    For each image in the set, project each inertial coordinate of the master grid onto the local coordinate frame to determine a grid of projected image pixel coordinates.

  4. 4.

    Interpolate each image at the reprojected master image coordinates using bilinear interpolation. The result is a set or “stack” of images that have all been registered to the same master grid.

  5. 5.

    Find the robust average of the stack of images at each pixel location to create one final “coadded” image. The signal intensity of the scene remains constant, but the noise of the coadded image decreases by \(\sqrt {n}\) relative to a single image, where n is the number of images in the stack. The robust average refers to the average after removing a few of the highest outliers (to account for image artifacts such as cosmic rays).

An example of a single and coadded LORRI 4 × 4 image is provided in Fig. 3.

Fig. 3
figure 3

Left: Single 30-s LORRI 4 × 4 image. Right: Coadded 30-s LORRI 4 × 4 image, using ninety-six images. The effect of the coadding process can be seen visually by the lower effective noise in the coadded image relative to the single image

Full size image

While image coadding is an effective method for increasing SNR, it also may introduce small errors and biases, some based on the symmetry of each image’s effective PSF. These small errors are generally mitigated by coadding many images together to average out non-symmetry, or when this is not possible, ensure only images with similar PSFs are used to generate the coadded image.

Star Subtraction

The fact that Ultima as observed from New Horizons was fainter than and overlapping many nearby field stars meant that the astrometric measurement of Ultima’s position would be corrupted due to star signal(s) biasing the result. Thus, a method for removing the background star signal was implemented to isolate Ultima’s signal from nearby stars. This method of star subtraction requires two primary sets of images; one set of star background images without the target and one set of images with the target visible against the same background star field. The star background images must have the same exposure time as the target images such that the stars in the scene are approximately the same DN in each set. Once these two data sets have been selected, the following steps are followed:

  1. 1.

    Construct a coadded image of the target by following Steps #1–5 in “Simulated Target Implantation”. The master inertial coordinate grid is typically defined as the coordinate grid of the median or mean target observation (in time).

  2. 2.

    Construct a coadded image of the background star field by following Steps #1–5 in “Simulated Target Implantation”, using the same master coordinate grid as the coadded target image.

  3. 3.

    Subtract each coadded star background image pixel from the corresponding coadded target image pixel to produce one final “star-subtracted” image. This is a simple element-by-element subtraction since both coadded images have been registered to the same inertial grid.

The final product is a “star-subtracted” image, where in theory only the target should remain visible. In practice, due to various noise sources (most notably Poisson noise on the star signals) and non-zero precision of the inertial attitude solutions and image registration, the background stars are never perfectly subtracted and thus artifacts are still present in the subtracted image. Bright stars in the images typically are the most challenging to subtract due to the presence of higher Poisson noise, and thus provide the greatest threat to astrometric quality if a star is in close proximity to Ultima. In this case, either the resulting center is deweighted in the orbit determination filter and/or a multi-Gaussian centerfinding technique (explained in “Multi-Gaussian Least Squares Fitting”) is used to simultaneously solve for the unsubtracted star and target centers. As described in “Trajectory Design Considerations”, this was avoided as best as possible by designing the flyby trajectory to avoid bright star conjunctions during critical observation epochs. An example coadded image before and after star subtraction is provided in Fig. 4. Note that this process of star subtraction may be used without image coadding if desired.

Fig. 4
figure 4

Left: Coadded set of twenty-four 30-s LORRI 4 × 4 star background images. Center: Coadded set of twenty-four 30-s LORRI 4 × 4 simulated OpNav images, with Ultima present (although nearly invisible by eye). The yellow arrow points to the signal from Ultima and the red arrow points to a nearby bright star mostly obscuring Ultima’s signal. Right: Simulated coadded image after subtracting the coadded star background. The yellow arrow points to the signal from Ultima that remains after the subtraction, which is now clearly visible. The outer ring of star-subtraction artifacts near the edges of the figure is an example of the star subtraction process performing less reliably for bright stars as it does for dim stars located in the center of the ring. The simulation epoch for these images is September 15th, 2018, K-108 days before Ultima closest approach

Full size image

Centroiding Techniques

In addition to the star and target centerfinding techniques already developed for the New Horizons Pluto flyby [2], a variety of functional enhancements were made in preparation for the Kuiper-Belt Extended Mission. Perhaps one of the most important is described in detail below, which would have been exercised in the event that the Ultima system contained one or more satellites, which was not the case.

Multi-Gaussian Least Squares Fitting

Multi-body systems with small orbital separations present challenges for optical navigation due to the increased likelihood of blended point spread functions and combined signal confusion. For images where multiple bodies’ point spread functions overlap, processing each target signal independently would likely introduce a bias in each solution. Signal intensity from nearby bodies tends to bias the solution towards those other bodies, causing observed target separations to be underestimated. For multi-body systems in which the separations are relatively large (> 7–10 pixels in the image plane), this is not a problem. However, when the targets are closer, it is necessary to use other centerfinding techniques to distinguish and calculate each target’s center. This simultaneous multi-body estimation is only exercised when there are a known number of bodies within a relatively small area of the image, to contain the estimation problem.

As such, a multi-Gaussian least-squares centerfinding algorithm has been developed to solve for the Gaussian parameters of all overlapping bodies simultaneously. The algorithm is designed such that if any unresolved bodies fall within a user-defined pixel distance of any other bodies, all bodies within that region are solved simultaneously. In addition, by modeling the complete overlapping system using a multi-modal Gaussian solution, photometric estimates for each target can also be determined to higher accuracy.

This strategy helps to enable early detection and characterization of multi-body systems, since astrometric and photometric residuals can be calculated before each body in the system can be explicitly distinguished by eye. However, if reliable a priori knowledge of the system’s orbital and photometric properties is not available, the centerfinding algorithm is not able to perform target association, and thus it remains a task of the user to manage the target assignments during early characterization of the system.

This algorithm has also been expanded to simultaneously solve for a set of both bodies and stars in the region of interest. The same process is followed, with every target and star position estimated in the same least squares solution, except the star center solution is discarded, and only the relevant target solutions are carried forward. This technique is most needed when star subtraction can not be performed reliably on a given image, as described in “Image Coadding”.

Operational Readiness Tests

From September 2017 through July 2018, the New Horizons navigation team conducted nine Operational Readiness Tests (ORTs) in preparation for Ultima approach. These tests were designed to test the navigation and operations methods and capabilities under a variety of mission scenarios. In this section, the objectives, optical navigation methods, and optical measurement results for a select number of tests are presented.

ORT-3b

Objectives and Methodology

ORT-3b was designed to simulate operations between the first official optical navigation image at K-108 days before Ultima encounter through the last nominal OpNav image at K-3 days. The objective was to focus on operational processing in the “Knowledge Update (KU)” phase of the mission between K-7 and K-3 days, where the project was to decide whether or not to uplink an updated set of spacecraft and target ephemerides to ensure the science observations were commanded to the correct position during encounter. KXIMP was used to simulate LORRI 4 × 4 and 1 × 1 images of Ultima using target implantation with real background star fields. Image coadding was exercised for most observations to generate stacked image sets in order to maximize SNR. Background star fields were subtracted from early 1 × 1 and 4 × 4 images, regardless of whether coadding was utilized or not. Target centerfinding was then performed on the final star-subtracted images to generate optical observables. After processing these results in the orbit determination filter, the resulting ephemeris and B-plane solutions were assessed against the knowledge requirements in order to inform a final KU Go/No-Go decision.

Results

The pre-fit OpNav residuals, as shown in Fig. 5, which are calculated using the a priori spacecraft and Ultima ephemerides, exhibit an offset of approximately 300 km primarily in the right ascension (RA) direction, as well as a growing offset in declination. The residuals derived from the earliest images exhibit a larger standard deviation when Ultima is still dim with low SNR, requiring large coadded sets. However, as New Horizons approaches Ultima and the SNR increases, the standard deviations begin to decrease. The pre-fit residual means do not converge to zero, since the a priori spacecraft and Ultima ephemerides are not a good representation of the truth ephemerides used to simulate the test data, as planned.

Fig. 5
figure 5

Left: ORT-3b OpNav pre-fit residuals in the RA and Dec directions in units of kilometers vs. 2018 day of year (DOY). Right: ORT-3b OpNav residuals with respect to truth in the RA and Dec directions in units of pixels. Data epochs span from K-108 through K-3 days before Ultima encounter

Full size image

The truth residuals exhibit a standard deviation of \(\sim \)0.3 pixels in RA and \(\sim \)0.1 pixels in declination (Dec), but show a small bias in RA in the earliest LORRI 4 × 4 images due to imperfect star subtraction. The residuals decrease dramatically to \(\sim \)0.03 pixels in both RA and Dec in the later LORRI 4 × 4 images and \(\sim \)0.1 pixels in both RA and Dec in the LORRI 1 × 1 images, as the SNR increases and star confusion is less problematic. These results are encouraging, however, this performance was much better than was expected in flight operations due to the fact that the centerfinding model (2D Gaussian) was the same as was used to simulate the data. Even though simulated noise is present, the centerfinding performance was overoptimistic. Future tests in which the images are simulated by the science team at Southwest Research Institute (SWRI) tended to be more realistic.

These results illustrate the efficacy and necessity of KXIMP’s coadding tool in increasing the SNR (particularly for the earliest images), the usefulness of the star-subtraction tool in enabling detection and centerfinding amongst a dense background star field, and the ability to produce astrometric results with accuracies on the order of tenths of a pixel.

ORT-4b

Objectives and Methodology

ORT-4b was designed to test and refine the navigation team’s ability to detect and react to an Ultima system consisting of multiple bodies. The simulated system included a primary body (labeled Ultima B1) and a small moon (labeled Ultima B2). As a result of the large mass ratio between the two bodies, the relative motion of the primary about the system barycenter was not large enough to endanger scientific measurements of the primary. In other words, the offset between the barycenter (imaging target) and primary body (B1) was small enough such that a specific KU for B1 (and B2) was not required to successfully capture fruitful scientific data.

The primary optical navigation goal of this ORT was to test the ability to perform initial detection and target centerfinding of a binary system given no a priori knowledge of the two bodies’ relative sizes, brightnesses, separation, or orbit geometry about the system barycenter. The image processing techniques used in this test included attitude determination, coadding, star field subtraction, and target centerfinding of subtracted images.

Unlike previous ORTs, image simulation was performed independently by science team members at Southwest Research Institute (SWRI), who provided images with artificial stars and bodies for all OpNav observations from K-108 to K-3 days. The simulated truth Ultima trajectory was defined to be a prior real ephemeris derived from HST and ground-based occultation data, and the truth spacecraft trajectory was generated using a perturbation from the nominal Alternate (10,000 km altitude) flyby trajectory.

Results

From the beginning of the data set at K-47 through K-13, the small moon (B2) was not detectable, and single-body processing was used to determine the photocenter the assumed system barycenter. From K-13 onward, both Ultima B1 and Ultima B2 were processed independently using a two-dimensional Gaussian least squares algorithm. After detection of Ultima B2 at K-13, the prior observations of the barycenter were reassigned as Ultima B1 measurements, since the pixel separation of the two bodies was large enough during the entire imaging period that Ultima B1 was assumed to be separable from the photocenter of the barycenter in pixel-space.

The Ultima B1 pre-fit residuals, as shown in Fig. 6, exhibit large initial Ultima B1 offsets of \(\sim \)9,000 km and \(\sim \)3,000 km in RA and Dec respectively, that decrease over time. The two unmodeled trajectory correction maneuvers (TCMs) in the pre-fit residuals can clearly be seen in the residuals of Ultima B1. The Ultima B2 residuals show an initial offset of \(\sim \)4,800 km in RA and Dec. Ultima B2 is offset from Ultima B1 by \(\sim \)200 km in the RA direction and by \(\sim \)1,800 km in the Dec direction. This constant offset between the Ultima B1 and B2 residuals illustrates the lack of significant orbital motion observed over the imaging span.

The Ultima B1 truth residuals exhibit RA and Dec standard deviations of \(\sim \)0.05 pixels for the 4 × 4 images and \(\sim \)0.2 pixels for the 1 × 1 images. The Ultima B2 residuals exhibit standard deviations of \(\sim \)0.2 pixels for the 4 × 4 images and \(\sim \)0.1 pixels for the 1 × 1 images. After K-8, the B2 residuals lie within ± 0.1 pixels, and the B1 residuals grow slightly. These results provide verification that the optical navigation system is producing astrometric results with accuracies < 0.25 pixels, 1-sigma, for both the primary body and small moon, and provides justification that the tools are well-suited to analyzing high mass ratio systems.

Fig. 6
figure 6

Left: ORT-4b OpNav pre-fit residuals in the RA and Dec directions in units of kilometers. Right: ORT-4b OpNav truth residuals in the RA and Dec directions in units of pixels. Data epochs span from K-47 through K-4 days

Full size image

ORT-7

Objectives and Methodology

In July 2018, ORT-7 was conducted to test the OpNav capabilities and interface with the science team in response to the detection of a binary Ultima system. Unlike ORT-4b, the simulated binary system contained two bodies whose relative orbital motions about the system barycenter were large enough to endanger scientific imaging opportunities during closest approach. This test was also more challenging than ORT-4b since the simulated Ultima system had a shorter semi-major axis and mass ratio closer to unity. As a result, detection of the binary system occurred significantly later, which gave the orbit determination team many fewer data points in which to estimate the target ephemerides.

Another objective of this ORT was to test the ability to ingest astrometric solutions from the science team into the orbit determination pipeline in a contingency scenario. In these situations, using the science team’s astrometry may extend the optical data arc used by the orbit determination team to estimate the binary ephemerides. To prepare for this scenario, the operational interface between the navigation and science astrometry teams were officially exercised for the first time.

Results

Single-body target processing was used to calculate the photocenter of the system barycenter using LORRI 4 × 4 images from K-41 through K-3, as the apparent pixel distance between the two bodies was never large enough to obtain independent centers for each body. From K-6 to K-3, the distance was large enough in LORRI 1 × 1 images to determine independent centers for Ultima B1 and B2 using the multi-gaussian centerfinding technique.

The pre-fit residuals, as shown in Fig. 7, exhibit initial offsets of \(\sim \)11,000 km in the RA direction and \(\sim \)4,000 km in the Dec direction. These offsets decrease to \(\sim \)800 km and \(\sim \)200 km, respectively, by the end of the dataset. The Ultima B1 and Ultima B2 solutions are offset from each other by \(\sim \)60 km and \(\sim \)70 km in the RA and Dec directions, respectively.

The truth residuals exhibit a largely unbiased (zero-mean) solution set for the barycenter. The standard deviation of the barycenter residuals remains \(\sim \)0.1 pixels or less over the entire data span. The standard deviations in the residuals of Ultima B1 and B2 are \(\sim \)0.2 pixels or less, except for one outlier of 0.5 pixels for Ultima B2. This outlier comes from the observation at K-4 where only three 1 × 1 images were coadded, resulting in a coadded image with fairly low SNR. This performance provides evidence that the optical navigation system is producing astrometric results with accuracies < 0.35 pixels, 1-sigma, and provides confidence that KXIMP is able to generate reliable observables of a close binary system with no a priori information.

Fig. 7
figure 7

Left: ORT-7 OpNav pre-fit residuals in the RA and Dec directions in units of kilometers. Right: ORT-7 OpNav residuals with respect to truth in the RA and Dec directions in units of pixels. Data epochs span from K-41 through K-3 days

Full size image

In addition to using KXIMP astrometry, the science team’s astrometric solutions were also ingested into the orbit determination filter and compared to the KXIMP-derived solutions. After performing astrometric and ephemeris comparisons, it was determined that the KXIMP and science team astrometry solutions were consistent within adequate precision.

First Look at Ultima

As presented in Table 1, the first images of Ultima as observed from the New Horizons LORRI imager were acquired on August 16, 2018. This set of 48 images provided a first look at Ultima’s observed brightness and relative position with respect to the spacecraft. The resulting coadded and star-subtracted images for this observation are provided in Fig. 8. The astrometric results derived from this observation along with results from the entire Ultima flyby and trajectory reconstruction will be presented in a future publication.

Fig. 8
figure 8

Left: Coadded image derived from forty-eight 30-s LORRI 4 × 4 exposures acquired on August 16, 2018. The signal from Ultima is identified by the yellow arrow, but is mostly obstructed by signal from the bright nearby star and nearly invisible by eye (similar to Fig. 4). Right: Star-subtracted image derived from subtracting a coadded image of ninety-six LORRI 4 × 4 star field images acquired on September 21, 2017 from the coadded image of Ultima. The yellow arrow identifies the remaining signal from Ultima after subtracting the background star field. The outer ring of star-subtraction artifacts near the edges of the figure is an example of the star subtraction process performing less reliably for bright stars as it does for dim stars located in the center of the ring

Full size image

Conclusion

The successful navigation of NASA’s New Horizons mission to its next target, Kuiper Belt Object (486958) 2014 MU69 (unofficially nicknamed Ultima Thule), required a robust preparation of the optical navigation system to allow for reliable astrometric processing of Ultima and any potential natural satellites. Due to the faint properties of Ultima, image processing and simulation techniques such as coadding, star subtraction, windowing, target implantation, and multi-Gaussian centroiding algorithms were implemented and tested. In addition, the imaging schedule was designed to maximize target SNR while still facilitating reasonable image data downlink volume throughout the approach imaging campaign. Various operational readiness tests using a combination of simulated and real data were executed to analyze and verify optical navigation system performance. The challenges that the Ultima flyby presents were well characterized and mitigated by various functional and operational enhancements, and test performance proved that the optical navigation system was well prepared for the reconnaissance of Ultima and any potential natural satellites.