Transformations Meeting 1

Minutes of meeting 1

Date: 28th April 2020 Membership: M. Jorge Cardoso (Lead), 
Michael Baumgartner, Tom Varsavsky, 
Fernando Perez-Garcia, Richard Shaw, Mark Graham, Mauricio Arteaga, Stephen Aylward, 
Rahul Choudhury, Wenqi Li

Define the scope of the group

Focus on imaging “tensors” as inputs and outputs Input-to-output transformation and data types

Segmentation: Images/Labels as joint pieces of information Classification: Images/classes Semantic Regression: Image/image pairs Object Detection: Image/Object-descriptor Registration: Image/transformation AutoML: Requirements regarding differentiability Including post-processing? Cache intermediate representations

Performance Algorithmic purposes Differentiability

Types of Transformations

Voxel/physical-coordinates consideration

All transformation should have a mm equivalent Different orientations Different images of the same subject with different orientation Should allow the user to bypass all of this Spatial Transforms

Rotation, Translation, Scale Scale factor Target resolution Affine, Flips, Non-linear transformations B-splines Velocity fields Shooting Integration Exponentiation Composability of transformations Interpolation (linked with data type) Nearest neighbor, linear, b-spline, sinc Crop Centre crop Localised Crop Padding Intensity transforms

Noise Whitening (zero=mean, std=1) Min-max Robust Min-Max Outlier removal Log Equalisation Histogram matching Learned? Gamma Window-Level Clipping Quantisation Filters: CLAHE / AHE / VCD / Curvature / ... Blurring Voxel Physical Spaces Represent NANs Acquisition Physics transforms

MRI Bias-field correction K-space artefacts Motion Ultrasound Speckle Occlusions/shadows Poor acoustic coupling Motion CT Simulating changes in acquisition (keV) Metal artifacts Motion Projections CT -> XRAY CT -> tomosynthesis Simulating other modalities Simulating image artefacts Simulating results from different scanners/manufacturers Segmentation specific

Categorical to probabilistic (vice versa) Categorical-mapping (forward and backwards) Temperature scaling Patient transforms

Simulating body habitus variations Simulating aging Simulation in general and what an API would look like Sampling strategies

Sampling 2D images from a 3D volume (maybe part of the “normal” 3d patch pathway) Sampling multiple contiguous 2d images (pseudo3d) from 3d volume Sampling 2D images with multiple orientations (2.5D) Patches from a Volume/Subject from dataset Random Segmentation-aware Masked sampling (only in a subregion) Hard-negative mining Grid sampling Grid-sampling with overlap Aggregation of grid sampling Mapping values Weighted aggregation Interaction with other packages?

BatchGenerator - numpy/SimpleITK TorchIO - Torch/numpy/SimpleITK/nibabel Phoenix Rising - torch ITK - itk Nvidia DALI - 2d, separate ecosystem Kornia - torch .Considerations

Torch Tensor? => CPU/GPU Manage Batches? Points of discussion for the next meeting

Differentiability Data type transformation and performance Looking into Transformation (code) API 1 slide per method Geometric transforms and composability Linear vs DAG execution Export/import augmentation (random sample) parameters