template<typename TFixedImage, typename TMovingImage>
class itk::ImpactImageToImageMetric< TFixedImage, TMovingImage >
A semantic similarity metric for multimodal image registration based on deep learning features.
This class define a loss by compares the fixed and moving images using high-level semantic representations extracted from pretrained deep learning models, rather than relying on raw pixel intensities or handcrafted features.
- Author
- V. Boussot, Univ. Rennes, INSERM, LTSI- UMR 1099, F-35000 Rennes, France
- Note
- This work was funded by the French National Research Agency as part of the VATSop project (ANR-20-CE19-0015).
-
If you use the Impact anywhere we would appreciate if you cite the following article:
V. Boussot et al., IMPACT: A Generic Semantic Loss for Multimodal Medical Image Registration, arXiv preprint arXiv:2503.24121 (2025). https://doi.org/10.48550/arXiv.2503.24121
Definition at line 58 of file itkImpactImageToImageMetric.h.
|
| using | ConstPointer = SmartPointer<const Self> |
| using | DerivativeValueType |
| using | FixedImageLimiterOutputType |
| using | FixedImageLimiterType |
| using | FixedImageMaskPointer |
| using | FixedImageMaskType |
| using | FixedImagePixelType |
| using | ImageSampleContainerPointer |
| using | ImageSampleContainerType |
| using | ImageSamplerPointer |
| using | ImageSamplerType |
| using | MovingImageDerivativeScalesType |
| using | MovingImageLimiterOutputType |
| using | MovingImageLimiterType |
| using | MovingImageMaskPointer |
| using | MovingImageMaskType |
| using | MovingImageRegionType |
| using | NumberOfParametersType |
| using | Pointer = SmartPointer<Self> |
| using | Self = ImpactImageToImageMetric |
| using | Superclass = AdvancedImageToImageMetric<TFixedImage, TMovingImage> |
| using | ThreadInfoType |
| using | AdvancedTransformType = AdvancedTransform<ScalarType, FixedImageDimension, MovingImageDimension> |
| using | BSplineOrder1TransformPointer = typename BSplineOrder1TransformType::Pointer |
| using | BSplineOrder1TransformType = AdvancedBSplineDeformableTransform<ScalarType, FixedImageDimension, 1> |
| using | BSplineOrder2TransformPointer = typename BSplineOrder2TransformType::Pointer |
| using | BSplineOrder2TransformType = AdvancedBSplineDeformableTransform<ScalarType, FixedImageDimension, 2> |
| using | BSplineOrder3TransformPointer = typename BSplineOrder3TransformType::Pointer |
| using | BSplineOrder3TransformType = AdvancedBSplineDeformableTransform<ScalarType, FixedImageDimension, 3> |
| using | CombinationTransformType = AdvancedCombinationTransform<ScalarType, FixedImageDimension> |
| using | ConstPointer = SmartPointer<const Self> |
| using | DerivativeValueType = typename DerivativeType::ValueType |
| using | FixedImageLimiterOutputType = typename FixedImageLimiterType::OutputType |
| using | FixedImageLimiterPointer = typename FixedImageLimiterType::Pointer |
| using | FixedImageLimiterType = LimiterFunctionBase<RealType, FixedImageDimension> |
| using | FixedImageMaskConstPointer = SmartPointer<const FixedImageMaskType> |
| using | FixedImageMaskPointer = SmartPointer<FixedImageMaskType> |
| using | FixedImageMaskType = ImageMaskSpatialObject<Self::FixedImageDimension> |
| using | FixedImagePixelType = typename FixedImageType::PixelType |
| using | FixedImagePointer = typename FixedImageType::Pointer |
| using | ImageSampleContainerPointer = typename ImageSamplerType::OutputVectorContainerPointer |
| using | ImageSampleContainerType = typename ImageSamplerType::OutputVectorContainerType |
| using | ImageSamplerPointer = typename ImageSamplerType::Pointer |
| using | ImageSamplerType = ImageSamplerBase<FixedImageType> |
| using | MovingImageDerivativeScalesType = FixedArray<double, Self::MovingImageDimension> |
| using | MovingImageLimiterOutputType = typename MovingImageLimiterType::OutputType |
| using | MovingImageLimiterPointer = typename MovingImageLimiterType::Pointer |
| using | MovingImageLimiterType = LimiterFunctionBase<RealType, MovingImageDimension> |
| using | MovingImageMaskConstPointer = SmartPointer<const MovingImageMaskType> |
| using | MovingImageMaskPointer = SmartPointer<MovingImageMaskType> |
| using | MovingImageMaskType = ImageMaskSpatialObject<Self::MovingImageDimension> |
| using | MovingImagePointer = typename MovingImageType::Pointer |
| using | MovingImageRegionType = typename MovingImageType::RegionType |
| using | NumberOfParametersType = typename AdvancedTransformType::NumberOfParametersType |
| using | Pointer = SmartPointer<Self> |
| using | ScalarType = typename TransformType::ScalarType |
| using | Self = AdvancedImageToImageMetric |
| using | Superclass = ImageToImageMetric<TFixedImage, TMovingImage> |
| using | ThreadInfoType = MultiThreaderBase::WorkUnitInfo |
|
| virtual const char * | GetClassName () const |
| virtual unsigned int | GetCurrentLevel () const |
| void | GetDerivative (const ParametersType ¶meters, DerivativeType &derivative) const override |
| virtual torch::Device | GetDevice () const |
| virtual std::vector< std::string > | GetDistance () const |
| virtual std::string | GetFeatureMapsPath () const |
| virtual int | GetFeaturesMapUpdateInterval () const |
| virtual const std::vector< ImpactModelConfiguration > & | GetFixedModelsConfiguration () |
| virtual std::vector< float > | GetLayersWeight () const |
| virtual std::string | GetMode () const |
| virtual const std::vector< ImpactModelConfiguration > & | GetMovingModelsConfiguration () |
| virtual std::vector< unsigned int > | GetPCA () const |
| virtual unsigned int | GetSeed () const |
| virtual std::vector< unsigned int > | GetSubsetFeatures () const |
| virtual bool | GetUseMixedPrecision () const |
| MeasureType | GetValue (const ParametersType ¶meters) const override |
| void | GetValueAndDerivative (const ParametersType ¶meters, MeasureType &value, DerivativeType &derivative) const override |
| void | GetValueAndDerivativeSingleThreaded (const ParametersType ¶meters, MeasureType &value, DerivativeType &derivative) const |
| virtual MeasureType | GetValueSingleThreaded (const ParametersType ¶meters) const |
| virtual bool | GetWriteFeatureMaps () const |
| void | Initialize () override |
| | ITK_DISALLOW_COPY_AND_MOVE (ImpactImageToImageMetric) |
| | itkStaticConstMacro (FixedImageDimension, unsigned int, FixedImageType::ImageDimension) |
| | itkStaticConstMacro (MovingImageDimension, unsigned int, MovingImageType::ImageDimension) |
| virtual void | SetCurrentLevel (unsigned int _arg) |
| virtual void | SetDevice (torch::Device _arg) |
| virtual void | SetDistance (std::vector< std::string > _arg) |
| virtual void | SetFeatureMapsPath (std::string _arg) |
| virtual void | SetFeaturesMapUpdateInterval (int _arg) |
| virtual void | SetFixedModelsConfiguration (std::vector< ImpactModelConfiguration > _arg) |
| virtual void | SetLayersWeight (std::vector< float > _arg) |
| virtual void | SetMode (std::string _arg) |
| virtual void | SetMovingModelsConfiguration (std::vector< ImpactModelConfiguration > _arg) |
| virtual void | SetPCA (std::vector< unsigned int > _arg) |
| virtual void | SetSeed (unsigned int _arg) |
| virtual void | SetSubsetFeatures (std::vector< unsigned int > _arg) |
| virtual void | SetUseMixedPrecision (bool _arg) |
| virtual void | SetWriteFeatureMaps (bool _arg) |
| virtual void | BeforeThreadedGetValueAndDerivative (const TransformParametersType ¶meters) const |
| virtual const FixedImageLimiterType * | GetFixedImageLimiter () |
| const FixedImageMaskType * | GetFixedImageMask () const override |
| virtual double | GetFixedLimitRangeRatio () const |
| ImageSamplerType * | GetImageSampler () const |
| virtual const MovingImageDerivativeScalesType & | GetMovingImageDerivativeScales () |
| virtual const MovingImageLimiterType * | GetMovingImageLimiter () |
| const MovingImageMaskType * | GetMovingImageMask () const override |
| virtual double | GetMovingLimitRangeRatio () const |
| virtual double | GetRequiredRatioOfValidSamples () const |
| virtual bool | GetScaleGradientWithRespectToMovingImageOrientation () const |
| const AdvancedTransformType * | GetTransform () const override |
| AdvancedTransformType * | GetTransform () override |
| virtual bool | GetUseFixedImageLimiter () const |
| virtual bool | GetUseImageSampler () const |
| virtual const bool & | GetUseMetricSingleThreaded () |
| virtual bool | GetUseMovingImageDerivativeScales () const |
| virtual bool | GetUseMovingImageLimiter () const |
| virtual const bool & | GetUseMultiThread () |
| void | Initialize () override |
| | ITK_DISALLOW_COPY_AND_MOVE (AdvancedImageToImageMetric) |
| | itkOverrideGetNameOfClassMacro (AdvancedImageToImageMetric) |
| | itkStaticConstMacro (FixedImageDimension, unsigned int, TFixedImage::ImageDimension) |
| | itkStaticConstMacro (MovingImageDimension, unsigned int, TMovingImage::ImageDimension) |
| virtual void | SetFixedImageLimiter (FixedImageLimiterType *_arg) |
| virtual void | SetFixedImageMask (const FixedImageMaskType *const arg) |
| virtual void | SetFixedLimitRangeRatio (double _arg) |
| virtual void | SetImageSampler (ImageSamplerType *_arg) |
| virtual void | SetMovingImageDerivativeScales (MovingImageDerivativeScalesType _arg) |
| virtual void | SetMovingImageLimiter (MovingImageLimiterType *_arg) |
| virtual void | SetMovingImageMask (const MovingImageMaskType *const arg) |
| virtual void | SetMovingLimitRangeRatio (double _arg) |
| void | SetRandomVariateGenerator (Statistics::MersenneTwisterRandomVariateGenerator &randomVariateGenerator) |
| virtual void | SetRequiredRatioOfValidSamples (double _arg) |
| virtual void | SetScaleGradientWithRespectToMovingImageOrientation (bool _arg) |
| virtual void | SetTransform (AdvancedTransformType *arg) |
| virtual void | SetUseMetricSingleThreaded (bool _arg) |
| virtual void | SetUseMovingImageDerivativeScales (bool _arg) |
| virtual void | SetUseMultiThread (bool _arg) |
| virtual void | UseMetricSingleThreadedOff () |
| virtual void | UseMetricSingleThreadedOn () |
| virtual void | UseMultiThreadOff () |
| virtual void | UseMultiThreadOn () |
|
| void | AfterThreadedGetValue (MeasureType &value) const override |
| void | AfterThreadedGetValueAndDerivative (MeasureType &value, DerivativeType &derivative) const override |
| unsigned int | ComputeValue (const std::vector< FixedImagePointType > &fixedPoints, LossPerThreadStruct &loss) const |
| unsigned int | ComputeValueAndDerivativeJacobian (const std::vector< FixedImagePointType > &fixedPoints, LossPerThreadStruct &loss) const |
| unsigned int | ComputeValueAndDerivativeStatic (const std::vector< FixedImagePointType > &fixedPoints, LossPerThreadStruct &loss) const |
| unsigned int | ComputeValueStatic (const std::vector< FixedImagePointType > &fixedPoints, LossPerThreadStruct &loss) const |
| | ImpactImageToImageMetric () |
| void | InitializeThreadingParameters () const override |
| bool | SampleCheck (const FixedImagePointType &fixedImageCenterCoordinate) const |
| bool | SampleCheck (const FixedImagePointType &fixedImageCenterCoordinate, const std::vector< std::vector< float > > &patchIndex) const |
| void | ThreadedGetValue (ThreadIdType threadID) const override |
| void | ThreadedGetValueAndDerivative (ThreadIdType threadID) const override |
| void | UpdateFixedFeaturesMaps () |
| void | UpdateMovingFeaturesMaps () |
| | ~ImpactImageToImageMetric () override=default |
| | AdvancedImageToImageMetric () |
| void | CheckForAdvancedTransform () |
| void | CheckForBSplineInterpolator () |
| void | CheckForBSplineTransform () const |
| void | CheckNumberOfSamples () const |
| virtual bool | EvaluateMovingImageValueAndDerivative (const MovingImagePointType &mappedPoint, RealType &movingImageValue, MovingImageDerivativeType *gradient) const |
| bool | EvaluateTransformJacobian (const FixedImagePointType &fixedImagePoint, TransformJacobianType &jacobian, NonZeroJacobianIndicesType &nzji) const |
| virtual void | EvaluateTransformJacobianInnerProduct (const TransformJacobianType &jacobian, const MovingImageDerivativeType &movingImageDerivative, DerivativeType &imageJacobian) const |
| bool | FastEvaluateMovingImageValueAndDerivative (const MovingImagePointType &mappedPoint, RealType &movingImageValue, MovingImageDerivativeType *gradient, const ThreadIdType threadId) const |
| Statistics::MersenneTwisterRandomVariateGenerator & | GetMutableRandomVariateGenerator () const |
| Statistics::MersenneTwisterRandomVariateGenerator & | GetRandomVariateGenerator () |
| virtual void | InitializeImageSampler () |
| void | InitializeLimiters () |
| virtual bool | IsInsideMovingMask (const MovingImagePointType &point) const |
| | itkAlignedTypedef (ITK_CACHE_LINE_ALIGNMENT, PaddedGetValueAndDerivativePerThreadStruct, AlignedGetValueAndDerivativePerThreadStruct) |
| | itkPadStruct (ITK_CACHE_LINE_ALIGNMENT, GetValueAndDerivativePerThreadStruct, PaddedGetValueAndDerivativePerThreadStruct) |
| void | LaunchGetValueAndDerivativeThreaderCallback () const |
| void | LaunchGetValueThreaderCallback () const |
| void | PrintSelf (std::ostream &os, Indent indent) const override |
| void | SetFixedImageMask (const typename Superclass::FixedImageMaskType *) final |
| void | SetFixedImageMask (typename Superclass::FixedImageMaskType *) final |
| void | SetMovingImageMask (const typename Superclass::MovingImageMaskType *) final |
| void | SetMovingImageMask (typename Superclass::MovingImageMaskType *) final |
| virtual void | SetUseFixedImageLimiter (bool _arg) |
| virtual void | SetUseImageSampler (bool _arg) |
| virtual void | SetUseMovingImageLimiter (bool _arg) |
| MovingImagePointType | TransformPoint (const FixedImagePointType &fixedImagePoint) const |
| | ~AdvancedImageToImageMetric () override=default |
|
| torch::Tensor | EvaluateFixedImagesPatchValue (const FixedImagePointType &fixedImageCenterCoordinate, const std::vector< std::vector< float > > &patchIndex, const std::vector< int64_t > &patchSize) const |
| torch::Tensor | EvaluateMovingImagesPatchValue (const FixedImagePointType &fixedImageCenterCoordinate, const std::vector< std::vector< float > > &patchIndex, const std::vector< int64_t > &patchSize) const |
| torch::Tensor | EvaluateMovingImagesPatchValuesAndJacobians (const FixedImagePointType &fixedImageCenterCoordinate, torch::Tensor &movingImagesPatchesJacobians, const std::vector< std::vector< float > > &patchIndex, const std::vector< int64_t > &patchSize, int s) const |
| template<typename ImagePointType> |
| std::vector< ImagePointType > | GeneratePatchIndex (const std::vector< ImpactModelConfiguration > &modelConfig, std::mt19937 &randomGenerator, const std::vector< ImagePointType > &fixedPointsTmp, std::vector< std::vector< std::vector< std::vector< float > > > > &patchIndex) const |
| std::vector< unsigned int > | GetSubsetOfFeatures (const std::vector< unsigned int > &featuresIndex, std::mt19937 &randomGenerator, int n) const |
| | itkAlignedTypedef (ITK_CACHE_LINE_ALIGNMENT, PaddedLossPerThreadStruct, AlignedLossPerThreadStruct) |
| | itkPadStruct (ITK_CACHE_LINE_ALIGNMENT, LossPerThreadStruct, PaddedLossPerThreadStruct) |
template<typename TFixedImage, typename TMovingImage>
template<typename ImagePointType>
| std::vector< ImagePointType > itk::ImpactImageToImageMetric< TFixedImage, TMovingImage >::GeneratePatchIndex |
( |
const std::vector< ImpactModelConfiguration > & | modelConfig, |
|
|
std::mt19937 & | randomGenerator, |
|
|
const std::vector< ImagePointType > & | fixedPointsTmp, |
|
|
std::vector< std::vector< std::vector< std::vector< float > > > > & | patchIndex ) const |
|
private |
Generates valid patch indices and filters out invalid points.
This method takes a list of fixed points and model configurations to generate patch indices for each point. It filters out points that lie outside the mask or image boundaries, returning only the valid fixed points. The valid patch indices are stored in patchIndex.
- Parameters
-
| modelConfig | A vector of model configurations, each specifying the properties of the models used. |
| randomGenerator | A random number generator used for random sampling in the patch generation process. |
| fixedPointsTmp | A vector of fixed points to generate patch indices for. |
| patchIndex | A reference to a 4D vector to store the generated patch indices for each valid fixed point. |
- Returns
- A vector of valid fixed points that are inside the mask or boundaries of the image.
template<typename TFixedImage, typename TMovingImage>
| std::vector< unsigned int > itk::ImpactImageToImageMetric< TFixedImage, TMovingImage >::GetSubsetOfFeatures |
( |
const std::vector< unsigned int > & | featuresIndex, |
|
|
std::mt19937 & | randomGenerator, |
|
|
int | n ) const |
|
private |
Retrieves a subset of features based on the provided indices.
This method selects a subset of features from the full set of features, based on the indices provided in featuresIndex. It uses a random number generator for sampling a subset of the features, ensuring that the selected features are randomly chosen.
- Parameters
-
| featuresIndex | A vector of indices representing the features to be considered. |
| randomGenerator | A random number generator used for sampling the features. |
| n | The number of features to select from the provided list of indices. |
- Returns
- A vector containing the indices of the randomly selected features.
template<typename TFixedImage, typename TMovingImage>
Checks if a patch centered at the given fixed image point is valid for sampling.
This function verifies that a patch, defined by the patchIndex, centered at the given fixed image point, is valid for sampling. It ensures that all transformed points of the patch remain inside the domain of the moving image, or within the moving mask if one is defined. It is used to validate patches before they are processed in the similarity metric.
- Parameters
-
| fixedImageCenterCoordinate | The center coordinate of the patch in the fixed image. |
| patchIndex | A layout of the patch, defining the indices of the region to sample around the fixed image point. |
- Returns
- true if the patch is valid for sampling, meaning all transformed points are inside the moving image domain or mask; false otherwise.