ASTROMETRY.NET: BLIND ASTROMETRIC CALIBRATION OF ARBITRARY ASTRONOMICAL IMAGES

The system automatically detects compact objects in each input image and centroids them to yield the pixel-space locations of stars. This problem is an old one in astronomy; the challenge here is to perform it robustly, with no human intervention, on the large variety of input images the system faces. Luckily, because the rest of the system is so robust, it can handle detection lists that are missing a few stars or have some contaminants.

The first task is to identify (detect) localized sources of light in the image. Most astronomical images exhibit sky variations, sensitivity variations, and scattering; we need to find peaks on top of such variations. First, we subtract off a median-smoothed version of the image to "flatten" it. Next, to find statistically significant peaks, we need to know the approximate noise level. We find this by choosing a few thousand random pairs of pixels separated by five rows and columns, calculating the difference in the fluxes for each pair, and calculating the variance of those differences, which is approximately twice the variance σ² in each pixel. At this point, we identify pixels which have values in the flattened image that are >8σ, and connect detected pixels into individual detected objects.

The second task is to find the peak or peaks in each detected object. This determination begins by identifying pixels that contain larger values than all of their neighbors. However, keeping all such pixels would retain peaks that are just due to uncorrelated noise in the image and individual peaks within a single "object." To clean the peak list, we look for peaks that are joined to smaller peaks by saddle points within 3σ of the larger peak (or 1% of the larger peak's value, whichever is greater), and trim the smaller peaks out of our list.

Finally, given the list of all peaks, the third task is to centroid the star position at sub-pixel accuracy. Following previous work (Lupton et al. 2001), we take a 3 × 3 grid around each star's peak pixel, effectively fitting a Gaussian model to the nine values, and using the peak of the Gaussian. Occasionally, this procedure produces a Gaussian peak outside the 3 × 3 grid, in which case we default to the peak pixel, although such cases are virtually always caused by image artifacts. This procedure produces a set of x and y positions in pixel coordinates corresponding to the position of objects in the image.

Compared to other star detection systems such as SExtractor (Bertin & Arnouts 1996), our approach is simpler and generally more robust given a wide variety of images and no human intervention. For example, while we do a simple median-filtering to remove the background signal from the image, SExtractor uses sigma-clipping and mode estimation on a grid of sub-images, which are then median-filtered and spline-interpolated.

Hypotheses about the location of an astronomical image live in the continuous four-dimensional space of position on the celestial sphere (pointing of the camera's optical axis), orientation (rotation of the camera around its axis), and field of view (solid angle subtended by the camera image). We want to be able to recognize images that span less than one-millionth the area of the sky, so the effective number of hypotheses is large; exhaustive search will be impractical. We need a fast search heuristic: a method for proposing hypotheses that almost always proposes a correct hypothesis early enough that we have the resources to discover it.

Our fast search heuristic uses a continuous geometric hashing approach. Given a set of stars (a "quad"), we compute a local description of the shape—a geometric hash code—by mapping the relative positions of the stars in the quad into a point in a continuous-valued, four-dimensional vector space ("code space"). Figure 1 shows this process. Of the four stars comprising the quad, the most widely separated pair are used to define a local coordinate system, and the positions of the remaining two stars in this coordinate system serve as the hash code. We label the most widely separated pair of stars "A" and "B." These two stars define a local coordinate system. The remaining two stars are called "C" and "D," and their positions in this local coordinate system are (x_C, y_C) and (x_D, y_D). The geometric hash code is simply the 4-vector (x_C, y_C, x_D, y_D). We require stars C and D to be within the circle that has stars A and B on its diameter. This hash code has some symmetries: swapping A and B converts the code to (1 − x_C, 1 − y_C, 1 − x_D, 1 − y_D) while swapping C and D converts (x_C, y_C, x_D, y_D) into (x_D, y_D, x_C, y_C). In practice, we break this symmetry by demanding that x_C ⩽ x_D and that x_C + x_D ⩽ 1; we consider only the permutation (or relabeling) of stars that satisfies these conditions (within noise tolerance).

Zoom In Zoom Out Reset image size

This mapping has several properties that make it well suited to our indexing application. First, the code vector is invariant to translation, rotation and scaling of the star positions so that it can be computed using only the relative positions of the four stars in any conformal coordinate system (including pixel coordinates in a query image). Second, the mapping is smooth: small changes in the relative positions of any of the stars result in small changes to the components of the code vector; this makes the codes resilient to small amounts of positional noise in star positions. Third, if stars are uniformly distributed on the sky (at the angular scale of the quads being indexed), codes will be uniformly distributed in (and thus make good use of) the four-dimensional code-space volume.

Noise in the image and distortion caused by the atmosphere and telescope optics lead to noise in the measured positions of stars in the image. In general, this noise causes the stars in a quad to move slightly with respect to each other, which yields small changes in the hash code (i.e., position in code space) of the quad. Therefore, we must always match the image hash code with a neighborhood of hash codes in the index.

The standard geometric hashing "recipe" would suggest using triangles rather than quads. However, the positional noise level in typical astronomical images is sufficiently high that triangles are not distinctive enough to yield reasonable performance. An important factor in the performance of geometric hashing system is the "oversubscription factor" of code space. The number of hash codes that must be contained in an index is determined by the effective number of objects that are to be recognized by the system: if the goal is to recognize a million distinct objects, the index must contain at least a million hash codes. Each hash code effectively occupies a volume in code space: since hash codes can vary slightly due to positional noise in the inputs (star positions), we must always search for matching codes within a volume of code space. This volume is determined by the positional noise levels in the input image and the reference catalog. The oversubscription factor of code space, if it is uniformly populated, is simply the number of codes in the index multiplied by the fractional volume of code space occupied by each code. If triangles are used, the fractional volume of code space occupied by a single code is large, so the code space becomes heavily oversubscribed. Any query will match many codes by coincidence, and the system will have to reject all of these false matches, which is computationally expensive. By using quads instead of triangles, we nearly square the distinctiveness of our features: a quad can be thought of as two triangles that share a common edge, so a quad essentially describes the co-occurrence of two triangles. A much smaller fraction of code space is occupied by each quad, so we expect fewer coincidental (false) matches for any given query, and therefore fewer false hypotheses which must be rejected.

We could use quintuples of stars, which are even more distinctive than quads. However, there are two disadvantages to increasing the number of stars in our asterisms. The first is that the probability that all k of the stars in an indexed asterism appear in the image and that all k of the stars in a query asterism appear in the index both decrease with increasing k. For images taken at wavelengths far from the catalog wavelength, or shallow images, this consideration can become severe. The second disadvantage is that near-neighbor lookup, even with a kd-tree, becomes more time consuming with increasing dimensionality. The dimensionality of the code space for quintuples is six-dimensional, compared to the four-dimensional code space of quads. We test triangle- and quintuple-based indices in Section 3.1.7.

When presented with a list of stars from an image to calibrate, the system iterates through groups of four stars, treating each group as a quad and computing its hash code. Using the computed code, we perform a neighborhood lookup in the index, retrieving all the indexed codes that are close to the query code, along with their corresponding locations on the sky. Each retrieved code is effectively a hypothesis, which proposes to identify the four reference catalog stars used to create the code at indexing time with the four stars used to compute the query code. Each such hypothesis is evaluated as described below.

The question of which hypotheses to check and when to check them is a purely heuristic one. One could chose to wait until a hypothesis has two or more "votes" from independent codes before checking it or check every hypothesis as soon as it is proposed, whichever is faster. In our experiments, we find that it is faster, and much less memory-intensive, to simply check every hypothesis rather than accumulate votes.

As with all geometric hashing systems, our system is based around a pre-computed index of known asterisms. Building the index begins with a reference catalog of stars. We typically use an all-sky (or near-all-sky) optical survey such as USNO-B1 (Monet et al. 2003; Barron et al. 2008) as our reference catalog, but we have also used the infrared Two Micron All Sky Survey (2MASS) catalog (Skrutskie et al. 2006) and the ultraviolet catalog from the Galaxy Evolution Explorer (GALEX; Martin & GALEX Science Team 2003), as well as non-all-sky catalogs such as SDSS. From the reference catalog we select a large number of quads (using a process described below). For each quad, we store its hash code and a reference to the four stars of which it is composed. We also store the positions of those four stars. Given a query quad, we compute its hash code and search for nearby codes in the index. For each nearby code, we look up the corresponding four stars in the index, and create the hypothesis that the four stars in the query quad correspond to the four stars in the index. By looking up the positions of the query stars in image coordinates and the index stars in celestial coordinates, we can express the hypothesis as a pointing, scale, and rotation of the image on the sky.

In order for our approach to be successful, our index must balance several properties. We want to be able to recognize images from any part of the sky, so we want to choose quads uniformly over the sky. We want to be able to recognize images with a wide range of angular sizes, so we want to choose quads of a variety of sizes. We expect that brighter stars will be more likely to be found in our query images, so we want to build quads out of bright stars preferentially. However, we also expect that some stars, even the brightest stars, will be missing or mis-detected in the query image (or the reference catalog), so we want to avoid over-using any particular star to build quads.

We handle the wide range of angular sizes by building a series of sub-indices, each of which contains quads whose quads have scales within a small range (for example, a factor of 2). At some level this is simply an implementation detail: we could recombine the sub-indices into a single index, but in what follows it is helpful to be able to assume that the sub-index will be asked to recognize query images whose angular sizes are similar to the size of the quads it contains. Since we have a set of sub-indices, each of which is tuned to an overlapping range of scales, we know that at least one will be tuned to the scale of the query image.

We begin by selecting a spatially uniform and bright subset of stars from our reference catalog. We do this by placing a grid of equal-area patches ("HEALPixels;"; Górski et al. 2002) over the sky and selecting a fixed number of stars, ordered by brightness, from each grid cell. The grid size is chosen so that grid cells are a small factor smaller than the query images. Typically we choose the grid cells to be about a third of the size of the query images, and select 10 stars from each grid cell, so that most query images will contain about a hundred query stars. Figure 2 illustrates this process on a small patch of sky.

Zoom In Zoom Out Reset image size

Next, we visit each grid cell and attempt to find a quad within the acceptable range of angular sizes and whose center lies within the grid cell. We search for quads starting with the brightest stars, but for each star we track the number of times it has already been used to build a quad, and we skip stars that have been used too many times already. We repeat this process, sweeping through the grid cells and attempting to build a quad in each one, a number of times. In some grid cells we will be unable to find an acceptable quad, so after this process has finished we make further passes through the grid cells, removing the restriction on the number of times a star can be used, since it is better to have a quad comprised of over-used stars than no quad at all. Typically, we make a total of 16 passes over the grid cells, and allow each star to be used in up to 8 quads. Figures 3 and 4 show the quads built during the first two rounds of quad-building in our running example.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In principle, an index is simply a list of quads, where for each quad we store its geometric hash code, and the identities of the four stars of which it is composed (from which we can look up their positions on the sky). However, we want to be able to search quickly for all hash codes near a given query hash code. We therefore organize the hash codes into a kd-tree data structure, which allows rapid retrieval of all quads whose hash codes are in the neighborhood of any given query hash code. In order to carry out the verification step we also keep the star positions in a kd-tree, since for each matched quad we need to find other stars that should appear in the image if the match is true. Since none of the available kd-tree implementations were satisfactory for our purposes, we created a fast, memory-efficient, pointer-free kd-tree implementation (Lang 2009).

Given a query image, we detect stars as discussed above, and sort the stars by brightness. Next, we begin looking at quads of stars in the image. For each quad, we compute its geometric hash code and search for nearby codes in the index. For each matching code, we retrieve the positions of the stars that compose the quad in the index, and compute the hypothesized alignment—the WCS—of the match. We then retrieve other stars in the index that are within the bounds of the image, and run the verification procedure to determine whether the match is true or false. We stop searching when we find a true match in which we are sufficiently confident, or we exhaust all the possible quads in the query image, or we run out of time. See Figures 5 and 6.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The indexing system generates a large number of hypothesized alignments of the image on the sky. The task of the verification procedure is to reject the large number of false matches that are generated, and accept true matches when they are found. Essentially, we ask, "if this proposed alignment were correct, where else in the image would we expect to find stars?" and if the alignment has very good predictive power, we accept it.

We have framed the verification procedure as a Bayesian decision process. The system can either accept a hypothesized match—in which case the hypothesized alignment is returned to the user—or the system can reject the hypothesis, in which case the indexing system continues searching for matches and generating more hypotheses. In effect, for each hypothesis we are choosing between two models: a "foreground" model, in which the alignment is true, and a "background" model, in which the alignment is false. In Bayesian decision-making, three factors contribute to this decision: the relative abilities of the models to explain the observations, the relative proportions of true and false alignments we expect to see, and the relative costs or utilities of the outcomes resulting from our decision.

The Bayes factor is a quantitative assessment of the relative abilities of the two models—the foreground model F and the background model B—to produce or explain the observations. In this case the observations, or data, D, are the stars observed in the query image. The Bayes factor

$$\begin{equation} K = \frac{p(D\,\vert \,F)}{p(D\,\vert \,B)} \end{equation} \tag{ 1 }$$

is the ratio of the marginal likelihoods. We must also include in our decision-making the prior p(F)/p(B), which is our a priori belief, expressed as a ratio of probabilities, that a proposed alignment is correct. Since we typically examine many more false alignments than true alignments (because we stop after the first true alignment is found), this ratio will be small. We typically set it, conservatively, to 10⁻⁶.

The final component of Bayesian decision theory is the utility table, which expresses the subjective value of each outcome. It is good to accept correctly a true match or reject correctly a false match ("true positive" and "true negative" outcomes, respectively), and it is bad to reject a true match or accept a false match ("false negative" and "false positive" outcomes, respectively). In the Astrometry.net setting, we feel it is very bad to produce a false positive: we would much rather fail to produce a result rather than produce a false result, because we want the system to be able to run on large data sets without human intervention, and we want to be confident in the results. Our utility table is shown as Table 1.

Applying Bayesian decision theory, we make our decision to accept or reject the hypothesized alignment by computing the expected utility $\mathbb{E}[u]$ of each decision. The expected utility of accepting the alignment is

$$\begin{eqnarray} \mathbb {E}[ u \,\vert \,\mathrm{Accept}, D ] &= u(\mathrm{TP}) \, p(\mathrm{TP}\,\vert \,D) + u(\mathrm{FP}) \, p(\mathrm{FP}\,\vert \,D)\quad \end{eqnarray} \tag{ 2 }$$

$$\begin{eqnarray} \qquad \qquad \ \ &= u(\mathrm{TP}) \, p(F\,\vert \,D) + u(\mathrm{FP}) \, p(B\,\vert \,D), \end{eqnarray} \tag{ 3 }$$

while the expected utility of rejecting the alignment is

$$\begin{eqnarray} \mathbb {E}\left[ \, u \,\vert \,\mathrm{Reject}, D \, \right] &= u(\mathrm{FN}) \, p(\mathrm{FN}\,\vert \,D) + u(\mathrm{TN}) \, p(\mathrm{TN}\,\vert \,D)\quad \end{eqnarray} \tag{ 4 }$$

$$\begin{eqnarray} &&\hspace{4.5pc}= u(\mathrm{FN}) \, p(F\,\vert \,D) + u(\mathrm{TN}) \, p(B\,\vert \,D). \end{eqnarray} \tag{ 5 }$$

We should accept the hypothesized alignment if

$$\begin{eqnarray} &&\mathbb {E}[ u \,\vert \,\mathrm{Accept}, D ] > \mathbb {E}[ u \,\vert \,\mathrm{Reject}, D ] \end{eqnarray} \tag{ 6 }$$

$$\begin{eqnarray} &&u(\mathrm{TP}) \, p(F\,\vert \,D) + u(\mathrm{FP}) \, p(B\,\vert \,D) > \nonumber\\ && \quad u(\mathrm{FN}) \, p(F\,\vert \,D) + u(\mathrm{TN}) \, p(B\,\vert \,D) \end{eqnarray} \tag{ 7 }$$

$$\begin{eqnarray} &&\frac{p(F\,\vert \,D)}{p(B\,\vert \,D)} > \frac{u(\mathrm{TN}) - u(\mathrm{FP})}{u(\mathrm{TP}) - u(\mathrm{FN})} \end{eqnarray} \tag{ 8 }$$

$$\begin{eqnarray} &&K > \frac{p(B)}{p(F)} \ \frac{u(\mathrm{TN}) - u(\mathrm{FP})}{u(\mathrm{TP}) - u(\mathrm{FN})}, \end{eqnarray} \tag{ 9 }$$

where we have applied Bayes' theorem to get

$$\begin{equation} \frac{p(F\,\vert \,D)}{p(B\,\vert \,D)} = K \ \frac{p(F)}{p(B)}. \end{equation} \tag{ 10 }$$

With the prior and utilities given above, we find that we should accept a hypothesis if

$$\begin{eqnarray} &&K > \frac{p(B)}{p(F)} \ \frac{u(\mathrm{TN}) - u(\mathrm{FP})}{u(\mathrm{TP}) - u(\mathrm{FN})} \end{eqnarray} \tag{ 11 }$$

$$\begin{eqnarray} &&K > 10^6 \ \frac{1 - -1999}{1 - -1} \end{eqnarray} \tag{ 12 }$$

$$\begin{eqnarray} &&K > 10^9. \end{eqnarray} \tag{ 13 }$$

That is, we accept a proposed alignment if the Bayes factor of the foreground model to the background model exceeds a threshold that is set based on our desired operating characteristics. In our case, the threshold is large so the foreground model (in which the alignment is true) must be far better at explaining (i.e., predicting) the observed positions of stars in the query image than the background model (in which the alignment is false).

In the foreground model, F, the four stars in the query image and the four stars in the index are aligned. We therefore expect that other stars in the query image will be close to other stars in the index. However, we also know that some fraction of the stars in the query image will have no counterpart in the index, due to occlusions or artifacts in the images, errors in star detection or localization, differences in the spectral bandpass, or because the query image "star" is actually a planet, satellite, comet, or some other non-star, non-galaxy object. True stars can be lost, and false stars can be added. Our foreground model is therefore a mixture of a uniform probability that a star will be found anywhere in the image—a query star that has no counterpart in the index—plus a blob of probability around each star in the index, where the size of the blob is determined by the combined positional variances of the index and query stars.

Under the background model, B, the proposed alignment is false, so the query image is from some unknown part of the sky; the index is not useful for predicting the positions of stars in the image. Our simple model therefore places uniform probability of finding stars anywhere in the test image. We have experimented with a more sophisticated background model that adapts to the observed distribution of image stars, but we do not discuss that work here.

The verification procedure evaluates stars in the query image, in order of brightness, under the foreground and background models. The product of the foreground-to-background ratios is the Bayes factor. We continue adding query stars until the Bayes factor exceeds our threshold for accepting the match, or we run out of query stars.

We explored the potential for automatically organizing and annotating a large real-world data set by taking a sample of images generated by the SDSS and considering them as an unstructured set of independent queries to our system. For each SDSS image, we discarded all meta-data, including all positional and rotational information and the date on which the exposure was taken. We allowed ourselves to look only at the two-dimensional positions of detected "stars" (most of which were in fact stars but some of which were galaxies or detection errors) in the image. Normally, our system would take images as input, running a series of image-processing steps to detect stars and localize their positions. The SDSS data reduction pipeline already includes such a process, so for these experiments we used these detected star positions rather than processing all the raw images ourselves. Further experiments have shown that we would likely have achieved similar, if not better, results by using our own image-processing software.

Each SDSS image has 2048 × 1489 pixels and covers 9 × 13 arcmin², slightly less than one-millionth the area of the sky. Each image measures one of five bandpasses, called u, g, r, i, and z, spanning the near-infrared through optical to near-ultraviolet range. Each band receives a 54 s exposure on a 2.5 m telescope. A typical image is shown in Figure 7.

Zoom In Zoom Out Reset image size

The SDSS image-processing pipeline assigns to each image a quality rating: "excellent," "good," "acceptable," or "bad." We retrieved the source positions (i.e., the list of objects detected by the SDSS image-processing pipeline) in every image within the main survey (Legacy and SEGUE footprints), using the public Catalog Archive Server (CAS) interface to Data Release 7 (Abazajian et al. 2009). We retrieved only sources categorized as "primary" or "secondary" detections of stars and galaxies, and required that each image contained at least 300 objects. The images that are excluded by these cuts contain either very bright stars or peculiarities that cause the SDSS image-processing pipeline to balk. The number of images in Data Release 7 and our cut is given in Table 2.

3.1.1. Performance on Excellent Images

In order to show the best performance achievable with our system, we built an index that is well matched to SDSS r-band images. Starting with the red bands from a cleaned version of the USNO-B catalog, we built an index containing stars drawn from a HEALPix grid with cell sizes about 4 × 4 arcmin², and 10 stars per cell. We then built quads with diameters of 4–5.6 arcmin. For each grid cell, we searched for a quad whose center was within the grid cell, starting with the brightest stars but allowing each star to be used at most 8 times. We repeated this process 16 times. The index contains a total of about 100 million stars and 150 million quads. Figure 8 shows the spatial distribution of the stars and quads in the index.

Zoom In Zoom Out Reset image size

We randomized the order of the excellent-quality r-band images, discarded all astrometric meta-data—leaving only the pixel positions of the brightest 300 stars—and asked our system to recognize each one. We allowed the system to create quads from only the brightest 50 stars. All 300 stars were used during the hypothesis-checking step, but since the Bayes factors tend to be overwhelmingly large, we would have found similar results if we had kept only the 50 brightest stars. We also told our system the angular size of the images to within about 1%, though we emphasize that this was merely a means of reducing the computational burden of this experiment: we would have achieved exactly the same results (after more compute time) had we provided no information about the scale whatsoever; we show this in Section 3.1.5.

The results, shown in Table 3 and in Figures 9, 10, 11, and 12, are that we can successfully recognize over 99.97% of the images, spending up to 10 s of CPU time per image. The majority of images are recognized after only 0.1 s of CPU time per image.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Table 3. Number of SDSS Images Recognized After Several Stages of Processing

Phase	Images Recognized	Unrecognized	Percent Recognized
USNO-B: 0.1 s	172,882	9,339	94.87
USNO-B: 1 s	181,826	395	99.78
USNO-B: 10 s	182,158	63	99.97
USNO-B: 15 s	182,160	61	99.97
2MASS	182,211	10	99.99
Original images	182,221	0	100.00

Notes. The stages are as follows. First, using an index based on the USNO-B reference catalog, after increasing amounts of CPU time. Then using a 2MASS-based index. Finally, retrieving the original images and reprocessing them to find stars. After these steps, all the excellent-quality images in our sample were recognized by our system.

Download table as:  ASCIITypeset image

Next, we examined the USNO-B reference catalog at the true locations of the images that were unrecognized. Some of these locations contained unusual artifacts. For example, Figure 13 shows a region where the USNO-B catalog contains "worm" features. The cause of these artifacts is unknown (D. Monet 2009, private communication), but they affect several square degrees of one of the photographic plates that were scanned to create the USNO-B catalog.

Zoom In Zoom Out Reset image size

In order to determine the extent to which our failure to recognize images is due to problems with the USNO-B reference catalog, we built an index from the 2MASS catalog, using the same process as for the USNO-B index, using the 2MASS J-band rather than the USNO-B red bands. We then asked our system to recognize each of the SDSS images that were unrecognized using the USNO-B-based index. Of the 61 images, 51 were recognized correctly, leaving only 10 images unrecognized. Examining these images, we found that some contained bright, saturated stars which had been flagged as unreliable by the SDSS image-reduction pipeline. We retrieved the original data frames and asked our system to recognize them. All 10 were recognized correctly: our source extraction procedure was able to localize the bright stars correctly, and with these the indexing system found a correct hypothesis. With these three processing steps, we achieve an overall performance of 100% correct recognition of all 182, 221 excellent images, with no false positives. This took a total of about 80 minutes of CPU time. The index is about 5 gigabytes in size, and once it is loaded into memory, multiple CPU cores can use it in parallel, so the wall-clock time can be a fraction of the total CPU time. During this experiment, a total of over 180 million quads were tried, resulting in about 77 million matches to quads in the index. Many of these matches were found to result in image scales that were outside the range we provided to the system, so the verification procedure was run only 6 million times.

For completeness, we also checked the images that were rated as excellent but failed our selection cut. We retrieved the original images and used our source extraction routine and both the USNO-B- and 2MASS-based indexes. Our system was able to recognize correctly all 1138 such images.

3.1.2. Performance on Images of Varying Bandpass

In order to investigate the performance of our system when the bandpass of the query image is different than that of the index, we asked the system to recognize images taken through the SDSS filters u, g, r, i, and z. We used only the images rated "excellent."

The results, shown in Table 4 and in Figure 14, demonstrate that as the difference between the query image bandpass and the index bandpass increases, the amount of CPU time required to recognize the same fraction of images increases. This performance drop is more pronounced on the blue (u) side than the red (z) side. After looking at the brightest 50 stars, the system is able to recognize essentially the same fraction of images. As shown by the first experiment in this section, this asymptotic recognition rate is largely due to defects in the reference catalog from which the index is built.

Zoom In Zoom Out Reset image size

Table 4. Percentage of SDSS Images Recognized by Our System After Increasing Amounts of CPU Time, for Images Taken Through Five Different SDSS Filters

CPU time	Percentage of Images Recognized
(per image)	u	g	r	i	z
0.1 s	87.80	93.88	94.87	93.59	94.36
1 s	98.58	99.73	99.78	99.73	99.75
10 s	99.82	99.96	99.97	99.96	99.96
60 s	99.84	99.96	99.97	99.96	99.96

Note. The performance is nearly identical for all but u-band images.

Download table as:  ASCIITypeset image

3.1.3. Performance on Images of Varying Quality

In order to characterize the performance of the system as image quality degrades, we asked the system to recognize r-band images that were classified as "excellent," "good," "acceptable," or "bad" by the SDSS image-reduction pipeline.

The results, shown in Table 5 and in Figure 15, show almost no difference in performance between excellent, good, and acceptable images. Bad images show a significant drop in performance, though we are still able to recognize over 99% of them.

Zoom In Zoom Out Reset image size

Table 5. Percentage of SDSS Images Recognized by Our System After Increasing Amounts of CPU Time, for Images of Varying Quality Rating

CPU time	Percentage of Images Recognized
(per image)	Excellent	Good	Acceptable	Bad
0.1 s	94.87	94.85	94.57	84.11
1 s	99.78	99.74	99.64	96.58
10 s	99.97	99.94	99.94	99.11
60 s	99.97	99.94	99.95	99.18

Note. The performance is similar for all but "bad-" quality images.

Download table as:  ASCIITypeset image

3.1.4. Performance on Images of Varying Angular Size

We investigated the performance of our system with respect to the angular size of the images by cropping out all but the central region of the excellent-quality r-band images and running our system on the sub-images. Recall that the original image size is 13 × 9 arcmin², and that the index we are using contains quads with diameters between 4 and 5.6 arcmin. We cut the SDSS images down to sizes 9 × 9, 8 × 8, 7 × 7, and 6 × 6 arcmin². The 7 × 7 and 6 × 6 images required much more CPU time, so we ran only small random subsamples of the images of these sizes.

The results, presented in Table 6 and in Figure 16, show that performance degrades slowly for images down to 8 × 8 arcmin², and then degrades sharply. This is not surprising given the size of quads in the index used in this experiment: in the smaller images, only stars near the edges of the image can possibly be 4–5.6 arcmin away from another star, so the set of stars that can form the "backbone" of a quad is small. Observe that images below 2.8 × 2.8 arcmin² cannot possibly be recognized by this index, since no pair of stars can be 4 arcmin or more away from each other.

Zoom In Zoom Out Reset image size

Table 6. Percentage of SDSS Sub-images Recognized by Our System Using an Index Tuned to 13 × 9 arcmin² Images

Image size (arcmin2)	Percentage of Images Recognized
13 × 9	99.97
9 × 9	99.88
8 × 8	99.52
7 × 7	86.53
6 × 6	24.75

Note. The performance is acceptable for a range of scales but drops quickly as the images become too small.

Download table as:  ASCIITypeset image

This does not imply that small images cannot be recognized by our system: given an index containing smaller quads, we may still be able to recognize them. The point is simply that for any given index there is some threshold of angular size below which the image recognition rate will drop, and another threshold below which the recognition rate will be exactly zero.

3.1.5. Performance with Varying Image Scale Hints

In all the experiments above, we told our system the angular scale (in arcseconds per pixel) to within ±1.25% of the true value, and we used an index containing only quads within a small range of diameters. In this experiment, we show that these hints merely make the recognition process faster without affecting the general results.

We created a set of sub-indices, each covering a range of $\sqrt{2}$ in quad diameters. The smallest-scale sub-index contains quads of 2–2.8 arcmin, and the largest contains quads of about 20–30 degrees in diameter. Each sub-index is built using the same methodology as outlined above for the 4–5.6 arcmin index, with the scale adjusted appropriately. The smallest-scale sub-index contains only six stars per HEALPix grid cell rather than 10 as in the other sub-indices, because the USNO-B catalog does not contain enough stars: a large fraction of the smallest cells contain fewer than 10 stars. In the smallest-scale sub-index we then do nine rounds of quad-building, reusing each star at most five times, as opposed to 16 rounds reusing each star at most eight times as in the rest of the sub-indices. The results are shown in Figure 17.

Zoom In Zoom Out Reset image size

Each time our system examines a quad in the image, it searches each sub-index in turn for matching quads, and evaluates each hypothesized alignment generated by this process. The system proceeds in lock-step, testing each quad in the image against each sub-index in turn. The first quad match that generates an acceptably good alignment is taken as the result and the process stops. Note that several of the sub-indices may be able to recognize any given image. Indeed, Figure 18 shows that every sub-index that contains quads that can possibly be found in SDSS images is first to recognize some of the images in this experiment. Different strategies for ordering the computation—for example, spending equal amounts of CPU time in each sub-index rather than proceeding in lock-step—might result in better overall performance.

Zoom In Zoom Out Reset image size

3.1.6. Performance with Varying Index Quad Density

The fiducial index we have been using in these experiments contains about 16 quads per HEALPix grid cell. Since each SDSS image has an area of about seven cells, we expect each image to contain about 100 quad centers. The number of complete quads (i.e., quads for which all four stars are contained in the image) in the image will be smaller, but we still expect each image to contain many quads. This gives us many chances of finding a correct match to a quad in the image. This redundancy comes at a cost: the total number of quads in the index determines the rate of false matches—since the code-space volume is fixed, packing more quads into the space results in a larger number of matches to any given query point—which directly affects the speed of each query. By building indices with fewer quads, we can reduce the redundancy but increase the speed of each query. This does not necessarily increase the overall speed, however: an index containing fewer quads may require more quads from the image to be checked before a correct match is found. In this experiment, we vary the quad density and measure the overall performance.

As shown in Table 7 and Figure 19, reducing the density from 16 to 9 quads per HEALPix grid cell has almost no effect on the recognition rate but takes only two-thirds as much CPU time. Reducing the density further begins to have a significant effect on the recognition rate, and actually takes more CPU time overall.

Zoom In Zoom Out Reset image size

Table 7. Percentage of SDSS Images Recognized by Our System Using Indices with Varying Number of Quad Features

CPU time	Percentage of Images Recognized
(per image)	16 quads/cell	9 quads/cell	4 quads/cell	3 quads/cell	2 quads/cell
0.1 s	94.44	96.32	95.89	94.30	90.94
1 s	99.76	99.84	99.61	99.36	97.35
10 s	99.96	99.95	99.79	99.65	97.92

Notes. Reducing the number of quads from 16 to 9 makes the system slightly faster with only a small loss of asymptotic recognition rate. Reducing the number of quads further makes the system both slower and less successful.

Download table as:  ASCIITypeset image

3.1.7. Performance on Indices Built from Triangles and Quints

We tested the performance of our quad-based index against a triangle-based index and a quintuple-based ("quint") index. The index-building processes were exactly as in our quad-based indices.

In the experiments above, we searched for all matches within a distance of 0.01 in the quad feature space. Since the triangle- and quint-based indices have feature spaces of different dimensionalities (2 for triangles, 6 for quints), we first ran our system with the matching tolerance set to 0.02 in order to measure the distribution of distances of correct matches in the three feature spaces. We then set the matching tolerance to include 95% of each distribution. For the triangle-based index, we found this matching tolerance to be 0.0064, for the quad-based index it was 0.0095, and for the quint-based index, 0.011. These experiments were performed on a random subset of 4000 images, because they are quite time consuming.

The results are given in Table 8 and in Figure 20. After looking at the brightest 50 stars in each image, the triangle-based index is able to recognize the largest number of images, but both the triangle- and quint-based indices take significantly more time than the quad-based index. It seems that quad features strike the right balance between being distinctive enough that any given query does not generate too many coincidental (false) matches—as the triangle-based index does—but containing few enough stars that it does not take long to find a feature that is in both the image and the index—as the quint-based index does.

Zoom In Zoom Out Reset image size

Table 8. Percentage of SDSS Images Recognized by Our System Using Indices Containing Three-star, Four-star, and Five-star Features ("Triangles," "Quads," and "Quints," Respectively)

CPU time	Percentage of Images Recognized
(per image)	Triangles	Quads	Quints
0.1 s	0.20	57.45	23.35
1 s	28.07	92.20	36.67
10 s	78.58	99.28	72.75
100 s	99.15	99.33	95.08
1000 s	99.97	99.33	96.25

Note. Quads are significantly faster.

Download table as:  ASCIITypeset image

We expect that the relative performance of triangle-, quad-, and quint-based indices depends strongly on the angular size of the images to be recognized. An index designed to recognize images of large angular size requires fewer features, so the code space is less densely filled and fewer false matches are generated. For this reason, we expect that above some angular size, a triangle-based index will recognize images more quickly than a quad-based index.

To show the performance of our system on significantly larger images, at a bandpass quite far from that of our index, we experimented with data from the GALEX. GALEX is a space telescope that observes the sky through near- and far-UV bandpass filters. Although it is fundamentally a photon-counting device, the GALEX-processing pipeline renders images (by collapsing the time dimension and histogramming the photon positions), and these are the data products used by most researchers. The images are circular, with a diameter of about 1.2 degrees. See Figure 21. In this experiment, rather than using the images themselves, we use the catalogs (lists of sources found in each image) that are released along with the images. These catalogs are produced by running a standard SExtractor on the images. We retrieved the near-UV catalogs for all 28,182 images in GALEX Release 4/5 that have near-UV exposure.

Zoom In Zoom Out Reset image size

We built a series of indices of various scales, each spanning roughly $\sqrt{2}$ in scale. The smallest contained quads with diameters from 11 to 16 arcmin, the next contained quads between 16 and 22 arcmin, followed by 22 to 30, 30 to 42, and 42 to 60 arcmin. Each index was built according to the "standard recipe" given above, using stars from the red bands of USNO-B as before. In total these indices contain about 30 million stars and 36 million quads.

In this experiment, we told our system that the images were between 1 and 2 degrees wide, and we allowed it to build quads from the first 100 sources in each image. The results are shown in Table 9 and Figure 22. The recognition rate is quite similar to that of the excellent-quality SDSS r-band fields, suggesting that even though the near-UV bandpass of these images is quite far from the bandpass of the index, which we would expect to make the system less successful at recognizing these images, their larger angular size seems to compensate. The system is significantly slower at recognizing GALEX images, but this is partly because we gave it a fairly wide range of angular scales, and because we used several indices rather than the single one whose scale is best tuned to these images.

Zoom In Zoom Out Reset image size

In order to demonstrate that there is nothing intrinsic in our method that limits us to images of a particular scale, we retrieved a set of HST images and built a custom index to recognize them. We chose the All-wavelength Extended Groth strip International Survey (AEGIS; Davis et al. 2007) footprint because it has many HST exposures and is within the SDSS footprint.

To build the custom index for this experiment, we retrieved stars and galaxies from SDSS within a 2 × 2 deg² centered on R.A. = 215 degree, decl. = 52.7 degree with measured r-band brightness between 15 and 22.2 mag, yielding about 57,000 sources. We created the index as usual, using a HEALPix grid with cells of size 0.5 arcmin, and building quads with diameters between 0.5 and 2 arcmin. This yielded just over 100,000 quads. Since the density of stars and galaxies in our index is only about four sources per square arcmin, the system tended to produce quads with sizes strongly skewed toward the larger end of the allowed range of scales. See Figure 23.

Zoom In Zoom Out Reset image size

We queried the Hubble Legacy Archive (Jenkner et al. 2006) for images taken by the Hubble Advanced Camera for Surveys (ACS; Ford et al. 1998) within the AEGIS footprint. Since individual ACS exposures contain many cosmic rays, we requested only "level 2" images, which are created from multiple exposures and have cosmic rays removed. A total of 191 such images were found. We retrieved 600 × 600-pixel JPEG previews—see Figure 24 for an example—and gave these images to our system as inputs.

Zoom In Zoom Out Reset image size

Our system successfully recognized 100% of the 191 input images, taking an average of 0.3 s of CPU time per image (not including the time required to perform source extraction). The footprints of the input images are shown in Figure 24. Although there are 191 images, there are only 64 unique footprints, because images were taken through several different bandpass filters for many of the footprints. Although the index contained quads with diameters from 0.5 to 2 arcmin, the smallest quad that was used to recognize a field was about 0.9 arcmin in diameter.

The Harvard Observatory archives contain over 500,000 photographic glass plates exposed between 1880 and 1985. See Figure 25 for an example. The Digital Access to a Sky-Century at Harvard (DASCH) project is in the process of scanning these plates at high resolution (Mink et al. 2006; Grindlay et al. 2009). Since the original astrometric calibration for these plates consists of handwritten entries in log books, a blind astrometric calibration system was required to add calibration information to the digitized images. DASCH has been using the Astrometry.net system for the past year to create an initial astrometric calibration which is then refined by WCSTools (Mink 2006).

Zoom In Zoom Out Reset image size

The DeepSky project (Nugent et al. 2009) is reprocessing the data taken as part of the Palomar-QUEST sky survey and Nearby Supernova Factory (Djorgovski et al. 2009; Aldering et al. 2002). Since many of the images have incorrect astrometric meta-data, they are using Astrometry.net to do the astrometric calibration. Over 14 million images have been successfully processed thus far (P. Nugent 2009, private communication).

We have also had excellent success using the same system for blindly calibrating a wide class of other astronomical images, including amateur telescope shots and photographs from consumer digital SLR cameras, some of which span tens of degrees. We have also calibrated videos posted to YouTube. These images have very different exposure properties, capture light in the optical, infrared, and ultraviolet bands, and often have significant distortions away from the pure tangent-plane projection of an ideal camera. Part of the remarkable robustness of our algorithm, which allows it to calibrate all such images using the same parameter settings, comes from the fact that the hash function is scale invariant so that even if the center of an image and the edges have a significantly different pixel scale (solid angle per pixel), quads in both locations will match properly into the index (although our verification criterion may conservatively decide that a true solution with substantial distortion is not correct). Furthermore, no individual quad or star, either in the query or the index is essential to success. If we miss some evidence in one part of the image or the sky we have many more chances to find it elsewhere.

Although we set our operating thresholds to be very conservative in order to avoid false positive matches, images that do not conform to the assumptions in our model can yield false positive matches at much higher rates than predicted by our analysis. In particular, we have found that images containing linear features that result in lines of detected sources are often matched to linear flaws in the USNO-B reference catalog. We removed linear flaws resulting from diffraction spikes (Barron et al. 2008), but many other linear flaws remain. An example is shown in Figures 26 and 27.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size