MARIO: Modélisation De L’Anatomie Normale et Pathologique Pour le Recalage Non Linéaire entre Images TDM et TEP en Oncologie

MARIO: Modélisation De L’Anatomie Normale et Pathologique Pour le Recalage Non Linéaire entre Images TDM et TEP en Oncologie

Sylvie Chambon Antonio Moreno  Anand Santhanam  Jannick Rolland  Isabelle Bloch 

IFSTTAR, Nantes, France

INSERM, U992, Unité de Neuroimagerie Cognitive, CEA/NeuroSpin Gif-sur-Yvette, France

Department of radiation oncology, University of California Los Angeles, USA

Rochester University, USA

Télécom ParisTech - CNRS LTCI - Paris, France

30 June 2011
| Citation



The MARIO project deals with the problem of 3D registration of Computed Tomography (CT) images (at two different instants of the breathing cycle) and Positon Emission Tomography (PET) images of thoracic regions. In order to guarantee physiologically plausible deformations, we present a novel method to incorporate a breathing model in a non-linear registration procedure. Our registration method is based on the segmentation of anatomical structures and potential tumors, and on an automatic selection of landmark points based on the curvature of the lung surface. The rigidity of the tumors is preserved during the registration and constraints on the heart are included, while guaranteeing a continuous deformation.

Extended Abstract

Positron Emission Tomography (PET) is a widely used imaging modality for oncological applications. It provides good sensitivity in tumor detection, but does not provide a precise localization of the pathology. However, radiotherapy treatment planning requires such a precise localization and a good knowledge of the spatial extent of the tumors in order to monitor and to control the dose delivered inside the body to both pathological and healthy tissues. Therefore it is often useful to also use Computerized Tomography (CT) images, which provide precise information on the size and shape of the lesion and surrounding anatomical structures, but only reduced information concerning malignancy. Joint exploitation of these two imaging modalities has thus a significant impact on improving medical decision-making for diagnosis and therapy, while requiring registration of the images. The registration is important for radiotherapy, in addition to segmentation, given that neither of the two modalities provide all the necessary information. Finally, to visualize the overall pathology in the lungs, it is necessary to register the whole volume and not just regions of interest such as tumor or heart regions.

The aim of the MARIO project was to develop methods and algorithms addressing the main issues raised by the fusion of PET and CT imaging data in the case of lung tumors. Even with combined PET/CT scanners, non-linear registration remains necessary to compensate for cardiac and respiratory motion, in particular when dealing with highly deformable regions as in thoracic imaging. In the particular case of lungs and lung tumors, the difficulty of the problem is increased as a result of the patient’s breathing and the induced displacement of the tumor, which does not undergo the same type of deformation as the normal lung tissues. For example, the tumor is not dilated during the inspiration phase. As a first approximation, its movement can be considered as rigid. Unfortunately, most of the existing non-linear registration methods do not take into account any knowledge of the physiology of the human body or of the tumors, and may lead to artificially deformed tumors and to a loss of valuable information in the area of the pathology.

In this paper, we propose to overcome these limitations by developing a nonlinear registration method with two key features: a breathing model is used to ensure physiologically plausible deformations during the registration, and the specific deformations of the tumors are taken into account while preserving the continuity ofthe deformations around them. In the context of radiotherapy treatment planning, precision requirements for registration and delineation of lung and tumor borders are somewhat alleviated by the use of a security margin around the tumor. As a consequence, millimetric precision is not required, and it is possible to work on the PET data without having to cope specifically with its limited resolution and induced partial volume effects. Other foreseen applications concern aid to diagnosis and patient’s follow-up. In this context, it is also important to assess how pathologies and organs are spatially organized. This is of prime importance for organs at risk such as the heart, which is highly sensitive to radiotherapy.

Based on these considerations, we propose a method to introduce a 3D model of lung deformation during the breathing cycle, as well as information on pathologies and on the heart, in a novel registration method. Moreover, an automatic method for landmark points detection is proposed, based on curvature information. The input data for the method consist of two CT volumes acquired at two different instants of the breathing cycle, and one PET volume, which is acquired while the patient is breathing normally (it is hence averaged over the breathing cycle).

The proposed method involves first a series of surface registrations and then image volume registration, based on initial segmentation and landmark detection steps. Its main components can be summarized as follows:

1) Segmentation of anatomical structures and tumors in CT and PET images, exploiting anatomical knowledge.

2) Estimation of a rigid transformation between tumors detected in CT and PET images.

3) Definition of landmark points on the anatomical structures in an automatic way, optimizing a criterion on curvature (high curvature regions should be represented by landmark points) and on the uniformity of the point distribution on the surface.

4) A physiologically driven breathing model is introduced into a 3D non-linear surface registration process. This model computes realistic deformations of the lung surface. For this purpose, once the breathing model has been adapted to the patient’s data, it is used to compute artificial (synthetic) CT volumes at different instants during the breathing cycle. Registration with the PET image volume is then performed by using the closest CT volume.

5) Physiology is further taken into account with a landmark-based surface registration, using the previously selected anatomical points of interest and forcing homologous points to match.

6) Volume registration is based on the displacement field identified during surface registration, combined with rigidity constraints that help preserving the size and shape of the tumors. Constraints on the heart are also introduced. The transformation on the whole image volume involves a weighting function depending on the distance to the tumors and to the heart, to guarantee a smooth deformation field.

Results have been evaluated on one normal case and on four pathological ones. They show the improvements in accuracy when using a breathing model andlandmark points based on both curvature and uniformity criteria. In particular the physiological deformations of the lungs are well taken into account, while preserving tumors and organs at risk such as the heart. The method appears therefore suitable for applications in radiotherapy.


Le projet ANR MARIO se situe dans le cadre du recalage 3D entre images tomodensitométriques (TDM), acquises à deux instants du cycle respiratoire, et images de tomographie par émission de positons (TEP), en imagerie thoracique. Pour garantir des déformations physiologiquement réalistes, nous proposons une nouvelle approche qui permet d’introduire un modèle de respiration dans une méthode de recalage non linéaire. Le recalage mis en oeuvre s’appuie sur une segmentation des structures anatomiques et des tumeurs éventuelles, puis sur une détection automatique de points d’intérêt en exploitant la courbure de la surface du poumon. La rigidité des tumeurs est préservée pendant le recalage, et des contraintes sur le coeur sont introduites, tout en garantissant la continuité de la déformation. Les résultats obtenus sur un cas normal et quatre cas pathologiques montrent que cette nouvelle technique de recalage prend mieux en compte les déformations physiologiques du poumon et préserve correctement les tumeuret les organes à risque tels que le coeur, pour des applications en radiothérapie par exemple.


medical imaging, CT, PET, lungs, segmentation, landmarks, breathing model, nonlinear registration.


imagerie médicale, TDM, TEP, poumons, segmentation, points d’intérêt, modèle de respiration, recalage non linéaire

1. Introduction
2. Segmentation Intégrant des Connaissances Anatomiques
3. Modèle de Respiration et Synthèse d’images TDM Intermédiaires
4. Sélection de Points de Repère
5. Recalage non Linéaire Adaptatif
6. Évaluation
7. Conclusion

Beil W., Rohr K., Stiehl H. (1997). « Investigation of Approaches for the Localization of Anatomical Landmarks in 3D Medical Images », Computer Assisted Radiology and Surgery, CARS, p. 265-270.

Besl P., McKay N. (1992). « A Method for Registration of 3-D Shapes », IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, n° 2, p. 239-256.

Betke M., Hong H., Thomasa D., Princea C., Kob J. (2003). « Landmark detection in the chest and registration of lung surfaces with an application to nodule registration », Medical Image Analysis, vol. 7, n° 3, p. 265-281.

Bloch I., Colliot O., César R. (2006). « On the Ternary Spatial Relation “Between” », IEEE International Conference on Systems, Man, and Cybernetics SMC-B, vol. 36, n° 2, p. 312-327.

Bookstein F. (1989). « Principal warps: thin-plate splines and the decomposition of deformations », IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, n° 6, p. 567-585.

Chambon S., Moreno A., Santhanam A., Brocardo R., Rolland J., Angelini E., Bloch I. (2008). « Introduction d’un modèle de respiration dans une méthode de recalage à partir de points d’intérêt d’images TEP et TDM du poumon – Using a 3D breathing model for landmarkbased CT-PET lung registration », 16e Conférence en Reconnaissance des Formes et Intelligence Artificielle, RFIA, Amiens, France, janvier.

Colliot O., Camara O., Bloch I. (2006). « Integration of Fuzzy Spatial Relations in Deformable Models - Application to Brain MRI Segmentation », Pattern Recognition, vol. 39, n° 8, p. 1401-1414.

Crawford C., King K., Ritchie C., Godwin J. (1996). « Respiratory Compensation in Projection Imaging Using a Magnification and Displacement Model », IEEE Transactions on Medical Imaging, vol. 15, n° 3, p. 327-332.

Ecabert O., Peters J., Lorenz C., Von Berg J., Vembar M., Subramanyan K., Lavi G.,Weese J. (2005). « Towards Automatic Full Heart Segmentation in Computed-Tomography Images », Computers in Cardiology, vol. 32, Lyon, France, septembre, p. 223-226.

Ecabert O., Peters J., Walker M., Berg J., Lorenz C., Vembar M., Olszewski M., Weese J. (2007). « Automatic Whole Heart Segmentation in CT Images: Method and Validation », Pluim J., Reinhardt J., Eds., SPIE Medical Imaging, vol. 6512, San Diego, California, Etats-Unis, mars.

Ehrhardt J., Schmidt-Richberg A., Handels H. (2007). « A Variational Approach for Combined Segmentation and Estimation of Respiratory Motion in Temporal Image Sequences », International Conference on Computer Vision, Rio de Janeiro, Brésil.

Funka-Lea G., Boykov Y., Florin C., Jolly M.-P., Moreau-Gobard R., Ramaraj R., Rinck D. (2006). « Automatic Heart Isolation for CT Coronary Visualization Using Graph-Cuts », IEEE International Symposium on Biomedical Imaging (ISBI), Arlington, Virginia, Etats-Unis, avril, p. 614-617.

Gregson P. H. (1994). « Automatic Segmentation of the Heart in 3D MR Images », Canadian Conference on Electrical and Computer Engineering, vol. 2, septembre, p. 584-587.

Guerrero T., Zhang G., Segars W., Huang T., Bilton S., Ibbott G., Dong L., Forster K., Lin K. (2005). « Elastic image mapping for 4-D dose estimation in thoracic radiotherapy », Radiation Protection Dosimetry, vol. 115, n° 1-4, p. 497-502.

Hartkens T., Hill D., Castellano-Smith A., Hawkes D., Maurer Jr. C., Martin A., Hall W., Liu H., Truwitt C. (2002). « Using Points and Surfaces to Improve Voxel-Based Non-Rigid Registration », International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. LNCS 2489, p. 565-572.

Jolly M. (2006). « Automatic Segmentation of the Left Ventricle in Cardiac MR and CT Images », International Journal of Computer Vision, vol. 70, n° 2, p. 151-163.

Klette R., Rosenfeld A. (2004). Digital Geometry: Geometric Methods for Digital Picture Analysis, Morgan Kaufmann.

Lelieveldt B. P. F., van der Geest R. J., Rezaee M. R., Bosch J. G., Reiber J. H. C. (1999). « Anatomical Model Matching with Fuzzy Implicit Surfaces for Segmentation of Thoracic Volume Scans », IEEE Transactions on Medical Imaging, vol. 18, n° 3, p. 218-230.

Lynch M., Ghita O., Whelan P. (2006). « Automatic Segmentation of the Left Ventricle Cavity and Myocardium in MRI Data », Computers in Biology and Medicine, vol. 36, n° 4, p. 389-407.

Maintz J., Viergever M. (1998). « A Survey of Medical Image Registration », Medical Image Analysis, vol. 2, n° 1, p. 1-36.

McClelland J., Blackall J., Tarte S., Chandler A., Hughes S., Ahmad S., Landau D., Hawkes D. (2006). « A Continuous 4D Motion Model from Multiple Respiratory Cycles for Use in Lung Radiotherapy », Medical Physics, vol. 33, n° 9, p. 3348-3358.

Moreno A., Delso G., Camara O., Bloch I. (2006). « Non-linear Registration Between 3D Images Including Rigid Objects: Application to CT and PET Lung Images With Tumors », Workshop on Image Registration in Deformable Environments (DEFORM), Edimbourg, Royaume-Uni, p. 31-40.

Moreno A. (2007). « Recalage non-linéaire d’images TEP et CT du thorax pour la caractérisation des tumeurs : application à la radiothérapie », PhD thesis, École Nationale Supérieure des Télécommunications, Paris, France.

Moreno A., Chambon S., Santhanam A., Rolland J., Angelini E., Bloch I. (2008). « Combining a Breathing Model and Tumor-Specific Rigidity Constraints for Registration of CT-PET Thoracic Data », Computer Aided Surgery (CAS), vol. 13, n° 5, p. 281-298.

Moreno A., Takemura C., Colliot O., Camara O., Bloch I. (2008). « Using Anatomical Knowledge Expressed as Fuzzy Constraints to Segment the Heart in CT Images », Pattern Recognition, vol. 41, n° 8, p. 2525-2540.

Narusawa U. (2001). « General characteristics of the sigmoidal model equation representing quasi-static pulmonary P-V curves », Journal of Applied Physiology, vol. 92, n° 1, p. 201-210.

Nehmeh S., Erdi Y., Pan T., Pevsner A., Rosenzweig K., Yorke E., Mageras G., Schoder H., Vernon P., Squire O., Mostafavi H., Larson S., Humm J. (2004). « Four-dimensional (4D) PET/CT imaging of the thorax », Physics in Medecine and Biology, vol. 31, n° 12, p. 3179-3186.

Noble N., Hill D., Breeuwer M., Schnabel J., Hawkes D., Gerritsen F., Razavi R. (2002). « Myocardial Delineation Via Registration in a Polar Coordinate System », 5th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Part I, vol. LNCS 2488, Tokyo, Japon, Springer-Verlag Berlin/Heidelberg, p. 651-658.

Pluim J., Fitzpatrick J. (2003). « Image Registration », IEEE Transactions on Medical Imaging, vol. 22, n° 11, p. 1341-1343.

Pollari M., Lotjonen J., Makela T., Pauna N., Reilhac A., Clarysse P. (2004). « Evaluation of cardiac PET-MRI registration methods using a numerical breathing phantom », IEEE International Symposium on Biomedical Imaging, Arlington, Virginie, Etats-Unis, p. 1447-1450.

Promayon E., Baconnier P., Puech C. (1997). « Physically-Based Model for Simulating the Human Trunk Respiration Movements », Joint Conference Computer Vision, Virtual Reality and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery (CVRMed-MRCAS), vol. LNCS 1205, Grenoble, France, p. 379-388.

Rit S., Sarrut D., Boldea V., Ginestet C. (2006). « Extraction du signal respiratoire à partir de projections cone-beam pour l imagerie TDM4D », Traitement du signal, vol. 23, n° 3-4, p. 307-319.

Rohlfing T., Maurer C.R., Jr., O’Dell W.G., Zhong J. (2004). « Modeling Liver Motion and Deformation During the Respiratory Cycle Using Intensity-Based Free-Form Registration of Gated MR Images », Medical Physics, vol. 31, n° 3, p. 427-432.

Rohlfing T., Maurer C., Zhong J. (2001). « Modeling liver motion and deformation during the respiratory cycle using intensity-based free-form registration of gated MR images », SPIE, Medical Imaging 2004: Visualization, Display, and Image-Guided, vol. 4319, p. 337-348.

Rohr K., Stiehl H., Sprengel R., Buzug T., Weese J., Kuhn M. (2001). « Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines », IEEE Transactions on Medical Imaging, vol. 20, n° 6, p. 526-534.

Santhanam A., Imielinska C., Davenport P., Kupelian P., Rolland J. (2008). « Modeling Real-Time 3-D Lung Deformations for Medical Visualization », IEEE Transactions on Information Technology in Biomedicine, vol. 12, n° 2, p. 257-270.

Sarrut D., Boldea V., Ayadi M., Badel J., Ginestet C., Clippe S., Carrie C. (2005). « Non-rigid registration method to assess reproducibility of breath-holding with ABC in lung cancer », International Journal of Radiation Oncology-Biology-Physis, vol. 61, n° 2, p. 594-607.

Sarrut D., Boldea V., Miguet S., Ginestet C. (2006). « Simulation of four-dimensional CT images from deformable registration between inhale and exhale breath-hold CT scans », Medical Physics, vol. 33, n° 3, p. 605-617.

Schweikard A., Glosser G., Bodduluri M., Murphy M., Adler J. (2000). « Robotic motion compensation for respiratory movement during radiosurgery », Computer Aided Surgery (CAS), vol. 5, n° 4, p. 263-277.

Segars W. P. et al. (2002). « Study of the Efficacy of Respiratory Gating in Myocardial SPECT Using the New 4-D NCAT Phantom », IEEE Transactions on Nuclear Science, vol. 49, n° 3, p. 675-679.

Shekhar R., Walimbe V., Raja S., Zagrodsky V., Kanvinde M., Wu G., Bybel B. (2005). « Automated 3-Dimensional Elastic Registration of Whole-Body PET and CT from Separate or Combined Scanners », The Journal of Nuclear Medicine, vol. 46, n° 9, p. 1488-1496.

Sundaram T., Gee J. (2005). « Towards a Model of Lung Biomechanics: Pulmonary Kinematics Via Registration of Serial Lung Images », Medical Image Analysis, vol. 9, n° 6, p. 524-537.

Venegas J., Harris R., Simon B. (1998). « A comprehensive equation for the pulmonary pressure-volume curve », Journal of Applied Physiology, vol. 84, n° 1, p. 389-395.

Vogel W., van Dalen J., Schinagl D., Kaanders J., Huisman H., Cortens F., Oyen W. (2006). « Correction of an image size difference between positron emission tomography (PET) and computed tomography (CT) improves image fusion of dedicated PET and CT », Physics in Medecine and Biology, vol. 27, n° 6, p. 515-519.

West J. B., Maurer C. R., Dooley J. R. (2005). « Hybrid Point-and-Intensity-Based Deformable Registration for Abdominal CT Images », SPIE Medical Imaging 2005: Image Processing, vol. 5747, p. 204-211.

Wojak J., Angelini E. D., Bloch I. (2010). « Introducing Shape Constraint via Legendre Moments in a Variational Framework for Cardiac Segmentation on non-Contrast CT Images », International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP). Angers, France, mai, p. 209-214.

Zitovà B., Flusser J. (2003). « Image Registration Methods: A Survey », Image and Vision Computing, vol. 21, p. 977-1000.

Zordan V., Celly B., Chiu B., DiLorenzo P. (2006). « Breathe Easy: Model and Control of Human Respiration for Computer Animation », Graphical Models, vol. 68, n° 2, p. 113-132.