Skip to: Content | Navigation | Downloads | FAQ | Footer

bUnwarpJ
ImageJ plugin for bidirectional elastic registration of two images...

Introduction
This free code is an ImageJ plugin that allows the simultaneous registration of two images based on elastic deformations represented by B-splines. The invertibility of the deformations is forced through a consistency restriction. bUnwarpJ is also integrated in Fiji. Contact author: Ignacio Arganda-Carreras. For a quick start and introduction, you can watch a video tutorial (v2.0) here (awarded at the Second ImageJ User & Developer Conference).

General description
bUnwarpJ is an algorithm for elastic and consistent image registration developed as an ImageJ plugin. It performs a simultaneous registration of two images, A and B. Image A is elastically deformed in order to look as similar as possible to image B, and, at the same time, the "inverse" transformation (from B to A) is also calculated so a pseudo-invertibility of the final deformation could be guaranteed. Two images are given as a result: the deformed versions of A and B images.

Figure 1: bUnwarpJ scheme: bidirectional Unwarping in Java.

Downloads
The latest Beta distribution of bUnwarpJ plugin is dated September 17, 2009. If you detect any bug, please feel free to contact the author. Any feedback will be very appreciated. Both, the JAVA source code and the class files can be downloaded with the following JAR file: bUnwarpJ_.jar [278 KB] . Older versions of bUnwarpJ and their related documentation can be found here. You can also browse the source since October 6, 2008 in the source repository. For Fiji users, just upgrade your program to the last Fiji version.

API documentation
The API documentation can be reached online here or downloaded from here: ZIP file [255 KB]. TAR.GZ file [133 KB].

Installation
You must simply download bUnwarpJ_.jar to the Plugins folder of ImageJ, restart ImageJ and there will be a new "Registration > bUnwarpJ" command in the Plugins menu. To execute bUnwarpJ as a macro or from the command line, see the description in the User Manual.

Related work
The algorithm implemented on bUnwarpJ and its technical explanations are detailed on the paper: Ignacio Arganda-Carreras, Carlos O. S. Sorzano, Roberto Marabini, Jose M. Carazo, Carlos Ortiz de Solorzano, and Jan Kybic, Consistent and Elastic Registration of Histological Sections using Vector-Spline Regularization, Lecture Notes in Computer Science, Springer Berlin / Heidelberg, volume 4241/2006, CVAMIA: Computer Vision Approaches to Medical Image Analysis, pages 85-95, 2006. The previous work (UnwarpJ) is reachable here, and its related paper is: "Elastic Registration of Biological Images Using Vector-Spline Regularization", C.. Snchez Sorzano, P. Thvenaz, M. Unser, IEEE Transactions on Biomedical Engineering, vol. 52, no. 4, pp. 652-663, April 2005.

Technical Explanations
This image registration algorithm is based on the minimization of an energy functional that includes the dissimilarity between the source and target images -in both directions- E img, an optional landmark constraint E , a regularization term (E div + E rot ), and an energy term E cons that accounts for the geometrical consistency between the elastic deformation in both directions. Namely, the energy function is given by E = w i E img + w E + (wdE div + w rE rot ) + w cE cons Where the weights of every term are set by the user in the main window of the plugin. The optimization process is a Levenberg-Marquardt minimization enhanced by a BroydenFletcher-Goldfarb-Shanno (BFGS) estimate of the local Hessian of the goal function, and both, images and deformations are represented by cubic B-splines.

User manual
Generalities
First of all, the plugin must be installed as described on Installation (for further information

go to ImageJ). Once it is properly installed, the plugin should appear in the main menu under Plugins > Registration > bUnwarpJ. Two images (8, 16, 32-bit grayscale or RGB Color) need to be opened in order to be able to use the plugin. If so, the following window will open:

Figure 2: Main plugin window (v2.6) Both selected images will work simultaneously as source and target, their tags are there only for the sake of clarification. The registration mode can be "Accurate", "Fast" and "Mono". The registration mode "Mono" (included since version 2.5) makes the program to perform only unidirectional registration, i.e. from source to target. The two registration modes "Accurate" and "Fast" involve performing bidirectional registration and affect the stopping criteria internally used by the program. More internal options can be modified in the "Advanced Options" panel. This panel gives you access to most of the internal parameters of the algorithm. The "Initial" and "Final" deformation lists allow you to select the coarsest and finest scale of the spline deformation field. "Very coarse" corresponds to 4 splines (one in each corner of the image). As you increase the deformation level, the

number of splines is doubled in each direction (horizontal and vertical). Since bUnwarpJ 2.5 there is a new parameter on the main window to allow subsampling the input images. The registration will be then calculated using the subsampled versions of the images but the results will be applied to the original ones. The image subsampling parameter can be chosen between 0 and 7, i.e. the image dimensions can be reduced by a factor of 2 0 = 1 to 2 7 = 128. This is very useful when registering large images. The different weights of the goal function control the relative weight of each one of the terms. These weights are not restricted to be between 0 and 1, and they may take any value as long as it is non-negative=2E. You can see a description of the different function weights in the presentation "bUnwarpJ: Consistent and Elastic Registration in ImageJ. Methods and Applications.", given at the Second ImageJ User & Developer Conference (2008) or have a look at the FAQ page. The stop threshold is used by the algorithm to stop the optimization process at each multiresolution level when the error relative change is not larger than this threshold. RGB Color images will be converted to grayscale during the registration process but the resulting transformations will be applied to the original color images. If you want to exit the plugin, press "Cancel". When you want the plugin to perform the registration press "OK". After running the plugin (on "Accurate" or "Fast" mode), the results are two stacks with the following three images: 1. One image (warping image) registered as to fit the other image (fixed image); 2. The fixed image; 3. The warping mask with the same deformation as the warping image. The final registration values appear in a separate ("Results") window. Figure 3 shows one of the resulting stacks from registering a source Lena image to a target warped version of the same image:

Figure 3: Result stack (basic view) The verbose mode produces more information: 1. The elastic deformation as a vector field. Each point in the fixed image must be deformed according to this field to fit into the warping image; 2. The grid obtained after deforming the fixed image with the vector field described above; 3. The step values of the optimization process in a separate ("Results") window.

Figure 4: Additional result images (verbose mode), deformation field and grid

Figure 5: Results window with values of the goal function f during the optimization process and final optimal values Since both, source and target images work as warping and fixed images, there are two sets (stacks) of results: from source to target and from target to source.

The "Mono" mode produces only results from the source to the target image. During the registration process, the current difference images and a mapping of the grid from the fixed images onto the moving images are shown.

Figure 6: Examples of output images during the registration process (current target-source difference image and original source image with current grid) During the registration process the toolbar will be changed to

Figure 7. Appearance of the window while registration is in progress Click on the stop button to stop the process. The output at the current state of the optimization will be returned in the normal way. Landmarks When the plugin is called and before pressing "OK" in the main window, the toolbar changes its appearance and it is possible to manually add landmarks to the selected images:

Figure 8: Adding landmarks activated The depressed button indicates that you may ADD A LANDMARK now. Landmarks are added in either image. The landmark will be automatically placed in the same position on both images. The new landmark becomes the "current landmark" (indicated by a thicker [+] sign in the current image and a [] sign in the other image, while all the rest are represented by [+] signs). To move any landmark, press on the MOVE LANDMARK button.

Figure 9: Moving landmarks activated Click and drag on any landmark to make it correspond to the same position in both images. Here goes an example of the two Lena images with corresponding landmarks:

Figure 10: Corresponding landmarks Landmarks can be removed through the REMOVE LANDMARK (crosses) button.

Figure 11: Removing landmarks activated This is the way of manually adding landmarks to the registration process. However, since bUnwarpJ v2.0 there is the option as well of using automatic landmarks as explained in the section SIFT and MOPS support, or manually adding point selections in both images before calling the plugin. If the number of point selections is the same in both images, they will be transformed into landmarks. When exiting bUnwarpJ all the images are restored to their previous state, i.e. the original regions of interest and point selections are restored. Masks This program allows you using masks in two mutually exclusive ways. In the first way, masks are introduced together with the input images. In this mode, input images must be a stack of images (first slice: the image itself, second slice: the mask). In this way, the mask can have any shape. In the second way, the input images must not be stacks and simple polygonal masks can be used. These masks are defined using the two buttons (INNER MASK and OUTER MASK) shown below:

Figure 12: Inner mask activated

Figure 13: Outer mask activated The inner mask keeps the information in the interior of the polygon, while the outer mask keeps the information in the exterior of the polygon. The thrown-out information is grayed. Here goes an example of an inner mask:

Figure 14: Inner mask example Masks can be used for one of the images, both, or none. You can put a mask in one of the images and not in the other, you can put a mask (with different shapes) in both images, or you may not use masks at all. After calling the plugin, the masks are erased and the initial images are restored. Input/Output options When using the "I/O Menu" from the toolbar, we have the possibility of processing different input and output files that will affect the registration.

Figure 15: ImageJ toolbar when using bUnwarpJ plugin. I/O menu option is selected.

Figure 16: I/O Menu options. In the last release, the plugin presents the following Input/Output options: Load Landmarks: it allows loading landmarks files. Save Landmarks As: it allows saving the image landmarks into a file. Show Landmarks: it forces the landmarks to be displayed in a separate table. Load Elastic Transformation: it loads an elastic transformation from a file and applies it to the so-called source image. The transformation file must be in the format of the plugin B-spline transformations, i.e. the same format as the files created with the "Save Transformation" option. Load Raw Transformation: it loads a raw transformation from a file and applies it to the so-called source image. The transformation file must be in "raw" format, i.e. the same format as the files created with the "Convert Transformation To Raw" option. See raw_transformation_template.txt for a format description. Compare Opposite Elastic Transformations: it calculates the warping index of two elastic transformations, i.e. the average of the geometrical distance between every pixel and its version after applying both transformations (direct and inverse). This value

is given as a result. The transformation files must be in the format of the plugin Bspline transformations (same format as the files created with the "Save Transformation" option). Compare Raw Transformations: it calculates the warping index of two raw transformations (same direction). The transformation file must be in "raw" format, i.e. the same format as the files created with the "Convert Transformation To Raw" option. Convert Transformation To Raw: it converts a B-spline transformation file into a raw transformation file. Compose Elastic Transformations: it composes two elastic transformations into a raw transformation. The input files must be in the B-spline transformation format, and the output file will be in "raw" format. Compose Raw Transformations: it composes two raw transformations into another raw transformation. Both input and output files will be in "raw" format. Compose Raw and Elastic Transformations: it composes one raw transformation and one elastic transformation into a raw transformation. Evaluate Image Similarity: it calculates de current similarity error between the source and target images. The results is displayed in the "Results" window. Adapt Coefficients: it transforms the coefficients of an specific elastic transformation according to a real image factor. Very useful for example when we have very large images. We can register subsampled versions of our images (let us say 4 times smaller) and then adapt the result transformations (image factor = 4) so we can apply them to the high resolution images. Load Source Mask: it loads a mask from file (binary image) in the source image. Load Source Initial Affine Matrix: it loads an affine matrix as starting point for the source-target transformation from a file. See affine_transformation_matrix.txt for a description of the format. These new options (since version 1.1) allow the user comparing the results of our program with any other registration method. You can also test bUnwarpJ with SplineDeformationGenerator, an ImageJ plugin that allows the user to generate five different image deformations: elastic, fisheye, perspective, barrel/pincushion and smile effect. SplineDeformationGenerator uses the same raw transformation file format as bUnwarpJ, so they are completely compatible.

Macro call
Since bUnwarpJ v2.0 the plugin allows using ImageJ macro language to be called. Therefore, the user can launch the plugin from a macro setting all the parameters of the plugin main window, for instance: run("bUnwarpJ", "source_image=A target_image=B registration=Accurate image_subsample_factor=0 initial_deformation=[Very Coarse] final_deformation=Fine divergence_weight=0 curl_weight=0 landmark_weight=0 image_weight=1 consistency_weight=10 stop_threshold=0.01");

When calling the plugin like this, if the "save_transformations" parameter is set to true, the transformations files will be saved in the following predefined format (not save dialog is shown): source image name + "_direct_transf.txt" and target image name + "_inverse_transf.txt". Only the main registration method can be called like this. To use the main Input/Output options from a macro, the call function must be used. There is a corresponding static method defined in the plugin for any of this options: elasticTransformImageMacro, rawTransformImageMacro, composeRawElasticTransformationsMacro, composeRawTransformationsMacro, composeElasticTransformationsMacro, compareRawElasticTransformationsMacro, compareElasticTransformationsMacro, compareRawTransformationsMacro, convertToRawTransformationMacro, adaptCoefficientsMacro. Notice here that the input and output file names must include the path, since they are not taken from the list of images. For instance, if we want to apply an elastic deformation stored in the file A_direct_transf.txt to the source image A.jpg with target image B.jpg and save the result in output.tif, we call: call("bunwarpj.bUnwarpJ_.elasticTransformImageMacro", "My_path/A.jpg", "My_path/B.jpg", "My_path/A_direct_transf.txt", "My_path/output.tif");

Command line call

bUnwarpJ might be called as well as an ImageJ from the command line. In the command line, the program offers the following options: -help: shows the syntax of the program -align: launches the registration of two input images -elastic_transform: transforms the source image with a given elastic deformation (previously calculated) -raw_transform: transforms the source image with a given raw deformation (previously calculated) -compare_elastic: compares two previously calculated opposite elastic deformations through the warping index -compare_elastic_raw: compares an elastic deformation with a raw deformation (both direct transformations) through the warping index -compare_raw: compares two previously calculated and direct raw deformations through the warping index -convert_to_raw: converts an elastic transformation into raw format -compose_elastic: composes two elastic deformations, the result will be in raw format -compose_raw: composes two raw deformations, the result will be too in raw format -compose_raw_elastic: composes a raw deformation and an elastic one. Result in raw format

-adapt_transform: adapts an specific elastic transformation given a resolution image factor For instance, to see the program help we can call the program from the command line (where $IJDIR is the directory where ImageJ is installed) like this: java -Xmx1000m -cp $IJDIR/ij.jar:$IJDIR/plugins/bUnwarpJ_.jar bunwarpj.bUnwarpJ_ -help For the rest of the options, follow the help instructions.

Consistency weight
The main reason to create bUnwarpJ was the idea of enforcing deformations' consistency through the consistency weight. This number forces the algorithm to move into solutions that ensure the invertibility of the resulting deformations. Therefore, higher is this number, more strictly is one deformation the inverse of the other. Due to the different units, there is no rule for selecting the right parameters. For instance, to perform the registration of the tumor images above, we recommend consistency weight values around 20.0. One important advantage of bUnwarpJ over the previous method lies in the fact that many registration problems can be solved without using the landmarks and regularization terms of the energy function (that means setting their corresponding parameters to 0.0). Therefore, no user interaction is needed and the computational complexity of the algorithm is reduced.

SIFT and MOPS plugin support

The last release of bUnwarpJ has compatibility with the Stephan Saalfeld's plugin for automatic feature extraction: SIFT and MOPS algorithms. You only need to download the following files to your ImageJ plugins folder: Stephan Saalfeld's plugin: mpicbg_.jar (browse source here) imagescience.jar from Erik Meijering Jama-1.0.2.jar An explanation of the parameters is here. This plugin is also integrated in Fiji. After applying SIFT or MOPS methods, you will get two sets of corresponding points in both images. If you call then bUnwarpJ, then the corresponding points will appear as source and target landmarks.

Figure 17: SIFT/MOPS - bUnwarpJ compatibility. Top row: MOPS results. Bottom row: landmarks after calling bUnwarpJ.

Conditions of use
This program is free software; you can redistribute it and/or modify it under the terms of

the GNU General Public License as published by the Free Software Foundation (http://www.gnu.org/licenses/gpl.txt). This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. We just expect you to include a citation or acknowledgment whenever you present or publish results that are based on it. Enjoy it!

Acknowledgments
bUnwarpJ has been developed during several years already and many people need to be acknowledged for: It started as an extension of UnwarpJ from Carlos O. S. Sorzano and thanks to him and Jan Kybic at the Center for Machine Perception in Prague, bUnwarpJ was born in the summer 2005 and published online in 2006. Many of the plugin updates and improvements would have never been possible without the hackathons that took place in Janelia Farm Research Campus (Virginia, summer 2008) and the Institute of Neuroinformatics (Zrich, winter 2008). Albert Cardona organized the hackathons and is responsible for the code parallelization and javascripting support. Stephan Saalfeld proposed and helped to make the SIFT/MOPS support possible. Marta Rivera-Alba helped with the macro support and code debugging. Johannes Schindelin is responsible for the Fiji integration and he is simply there whenever a Java problem comes up!

Welcome!
bUnwarpJ is an open source software for consistent and elastic image registration in Java. It is part of ImageJ and Fiji.

Menu:
Introduction General Description Downloads

API Documentation Installation Related work User manual Generalities Macro call Command line call Consistency weight SIFT and MOPS support Conditions of use Acknowledgments FAQ

Site news:
September 17, 2009 Removed limitation in the number of landmarks. June 6, 2009 New stable version release! bUnwarpJ v2.6! March 10, 2009 New Beta version release! bUnwarpJ v2.5! November 10, 2008 Plugin update: changed "divergency" to "divergence" in all files. November 3, 2008 Plugin update: fixed bug to use masks on color images and set initial image pyramid depth. October 31, 2008 Basic tutorial release! (awarded at the Second ImageJ User & Developer Conference) October 24, 2008 Last release: fixed different bugs (evaluate similarity, adapt coefficients...). October 14, 2008 Plugin update: fixed bugs to align from command line. October 10, 2008 Plugin update: fixed bugs to load masks from stack. October 7, 2008 Plugin update: color images support! September 12, 2008 New release: fixed bugs to restore initial images.

September 8, 2008 Fixed bugs and macro calls for I/O options. August 29, 2008 New Beta version release! bUnwarpJ v2.0! June 12, 2008 Last version release: more options, SIFT/MOPS support... May 17, 2007 New version release: more command-line possibilities! April 27, 2007 SplineDeformationGenerator version 1.0 available! A perfect tool to test bUnwarpJ. February 5, 2007 User manual for version 1.1 available! bUnwarpJ v1.1 is now stable. December 1, 2006 Beta version release! bUnwarpJ v1.1! July 28, 2006 JAR version available! July 20, 2006 First version on the net: bUnwarpJ v1.0!

Links:
ImageJ (main project) SplineDeformation Generator (plugin to generate test cases) UnwarpJ (first project: unidirectional registration) Visualizing Divergence and Curl Contact the author! Contact us | Back to top 2009 Ignacio Arganda-Carreras | | Design by Andreas Viklund