Document new capabilities, add changelog, update contact info

Documents three-tabular-modality support, main_modality/attention_modality
selection, and image transforms - all previously undocumented. Adds
CHANGELOG.md covering everything unreleased since v1.2.3. Updates the
README bio to past tense and points contact queries to GitHub issues
instead of a UCL email address.

Author: florencejt

CHANGELOG.md

@@ -0,0 +1,63 @@
+# Changelog
+
+## v2.0.0
+
+Everything below covers all unreleased work since the last published version, v1.2.3.
+
+### Breaking changes
+
+- `data_dims` and data `sources` are now dictionaries (e.g. `{"mod1_dim": ..., "mod2_dim": ...}`) instead of lists,
+  to support a variable number of tabular modalities. Anything calling the lower-level classes directly with the
+  old list format will need updating; `prepare_fusion_data` users are unaffected.
+- Fusion model `forward()` methods now take modalities as separate tensor arguments and return a single tensor,
+  instead of taking/returning lists. This makes model outputs compatible with libraries like SHAP that expect a
+  single tensor in, single tensor out.
+- The binary classification final layer no longer applies `Sigmoid` internally; the loss function now uses
+  `BCEWithLogitsLoss` on raw logits directly (more numerically stable). `preds` are still thresholded probabilities
+  as before, computed via `sigmoid(logits) > 0.5`.
+
+### New features
+
+- **Three-tabular-modality support**: most tabular-tabular fusion methods (`TabularDecision`, `ConcatTabularData`,
+  `ConcatTabularFeatureMaps`, `TabularChannelWiseMultiAttention`, `TabularCrossmodalMultiheadAttention`,
+  `ActivationFusion`, `AttentionAndSelfActivation`) now accept a third tabular modality via a `tabular3` data source.
+  Attention-based methods gained a configurable `main_modality`/`attention_modality` for choosing which modality
+  gets special treatment.
+- **Image transforms**: `prepare_fusion_data` accepts a `transforms` argument (`torchio.transforms`-style) for
+  image augmentation/preprocessing, for 2D and 3D images.
+- **GPU/device configuration**: a `training_modifications` argument (accelerator, number of devices) is now
+  supported on `prepare_fusion_data` and `train_and_save_models`; metric calculations follow the data's device.
+- MCVAE early-stopping patience and tolerance are now configurable via the layer modifications dictionary.
+
+### Robustness fixes
+
+- Fixed the binary classification activation function (see "Breaking changes" above) - models were previously
+  double-squashing predictions through a `Sigmoid`.
+- Fixed a bug where MCVAE's early-stopping patience attribute was misnamed internally.
+- Fixed the custom early-stopping callback.
+- Image-based methods no longer error out when the image is too small for the default network architecture -
+  they now require `layer_mods` to fix the architecture instead of crashing outright.
+- The attention reduction ratio for channel-attention methods now auto-corrects to a working value instead of
+  raising when it doesn't evenly divide the modality's feature dimension.
+- `torch.load` calls updated for current PyTorch versions (`weights_only`).
+
+### Output / logging improvements
+
+- Training and validation logits are now recorded on trained models, in addition to reals and predictions.
+- `ModelComparison` now also returns the underlying reals/preds for further analysis.
+- Multiclass dimensions and image downsample size can now be passed to more of the `from_new_data` evaluation
+  functions.
+- WandB's local log directory is now configurable via an environment variable.
+- Loading CSVs with a stray `Unnamed: 0` index column now raises a clear warning instead of silently including it.
+
+### Documentation and packaging
+
+- Documented the three-tabular-modality feature, `main_modality`/`attention_modality` selection, and image
+  transforms (previously undocumented).
+- Fixed a broken link and a class-name typo in the docs.
+- Bumped a vulnerable `urllib3` dependency pin.
+- Fixed CI so the vendored `mcvae` git submodule is actually fetched during test runs - this had been silently
+  giving the MCVAE fusion model 0% test coverage.
+- Fixed a ReadTheDocs build timeout caused by an earlier docs cleanup accidentally discarding the sphinx-gallery
+  example cache.
+- General branch cleanup and consolidation; test coverage raised from 89% to 93% (304/304 tests passing).

README.md

@@ -102,8 +102,8 @@ the [contributions documentation](https://fusilli.readthedocs.io/en/latest/contr
 
 `fusilli` is authored by Florence J Townend, James Chapman, and James H Cole.
 
-Florence J Townend is supported by a UCL UKRI Centre for Doctoral Training in AI-enabled Healthcare studentship (
-EP/S021612/1).
+Florence J Townend was supported by a UCL UKRI Centre for Doctoral Training in AI-enabled Healthcare studentship (
+EP/S021612/1) during the initial development of this software.
 
 ## License
 

docs/choosing_model.rst

@@ -1,3 +1,5 @@
+.. _choosing-a-model:
+
 Trait-Driven Model Imports
 ============================
 
@@ -67,4 +69,38 @@ Examples of Criteria
 
     criteria = {
         "class_name": ["Tabular1Unimodal", "Tabular2Unimodal", "ConcatTabularData"],
+    }
+
+- **Models That Support Three Tabular Modalities**:
+
+.. code-block:: python
+
+    criteria = {
+        "three_tabular_modalities": True,
+        "modality_type": "tabular_tabular",  # exclude the tabular unimodal models
+    }
+
+Choosing the "Main" Modality for Attention-Based Models
+---------------------------------------------------------
+
+Some attention-based tabular-tabular models (:class:`.ActivationFusion` and :class:`.AttentionAndSelfActivation`)
+let you choose which modality is treated as the "main" one - i.e. whose feature maps get concatenated with the
+fused output, and (for :class:`.AttentionAndSelfActivation`) which modality the channel attention is applied to.
+By default this is modality 1. You can change it either directly on an instantiated model:
+
+.. code-block:: python
+
+    model = ActivationFusion(prediction_task=..., data_dims=..., multiclass_dimensions=...)
+    model.main_modality = 2  # 1, 2, or 3
+
+or via the layer modification dictionary passed to :func:`fusilli.data.prepare_fusion_data` and
+:func:`fusilli.train.train_and_save_models` (see :ref:`modifying-models`):
+
+.. code-block:: python
+
+    layer_mods = {
+        "AttentionAndSelfActivation": {
+            "main_modality": 2,
+            "attention_modality": 3,
+        },
     }

docs/data_loading.rst

@@ -3,7 +3,7 @@
 Loading your Data
 ==================
 
-``fusilli`` facilitates fusion of **tabular data with tabular data** or **tabular data with images**.
+``fusilli`` facilitates fusion of **tabular data with tabular data** (two or three modalities) or **tabular data with images**.
 
 Data Format Requirements
 ----------------------------
@@ -17,6 +17,7 @@ The paths to the data source files must be in a dictionary before being passed t
     data_paths = {
         "tabular1": "path/to/tabular1_data.csv",
         "tabular2": "path/to/tabular2_data.csv",
+        "tabular3": "path/to/tabular3_data.csv",
         "image": "path/to/image_data.pt",
     }
 
@@ -26,6 +27,12 @@ The paths to the data source files must be in a dictionary before being passed t
 
     For example, if you are not using ``tabular2``, set ``tabular2`` in the dictionary to ``""``.
 
+.. note::
+
+    ``tabular3`` is optional. Most tabular-tabular fusion methods support a third tabular modality, but not all of
+    them do, and it isn't supported for tabular-image fusion. See :ref:`choosing-a-model` for how to filter for
+    models that support three tabular modalities.
+
 Tabular and Tabular Data
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -54,6 +61,27 @@ Columns named ``ID`` and ``prediction_label`` are required:
                                       data_paths=data_paths,
                                       output_paths=...)
 
+**Example of loading three tabular modalities:**
+
+Most tabular-tabular fusion methods also accept a third tabular modality. Just add a ``tabular3`` key with
+the same CSV format (``ID`` and ``prediction_label`` columns) as the other tabular sources.
+
+.. code-block:: python
+
+    from fusilli.data import prepare_fusion_data
+
+    data_paths = {
+        "tabular1": "path/to/tabular1_data.csv",
+        "tabular2": "path/to/tabular2_data.csv",
+        "tabular3": "path/to/tabular3_data.csv",
+        "image": "",
+    }
+
+    data_module = prepare_fusion_data(prediction_task=...,
+                                      fusion_model=some_example_model_with_3_modalities,
+                                      data_paths=data_paths,
+                                      output_paths=...)
+
 Tabular and Image Data
 ~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -97,6 +125,32 @@ To downsample images before model input, use the ``image_downsample_size`` param
                                       output_paths=...,
                                       image_downsample_size=(16, 16))
 
+Applying Image Transforms
+*********************************
+
+To apply data augmentation or preprocessing transforms to your images, use the ``transforms`` parameter in the
+:func:`fusilli.data.prepare_fusion_data` function. Pass a list of ``torchio.transforms`` functions with their
+arguments already filled in - each one is applied to every image in turn. (For 2D images, ``fusilli`` handles adding
+and removing the extra depth dimension that ``torchio`` expects internally, so you don't need to worry about that
+yourself.)
+
+**Example of applying a random flip and a random affine transform:**
+
+.. code-block:: python
+
+    import torchio.transforms as T
+
+    my_transforms = [
+        T.RandomFlip(axes=(0,), flip_probability=0.5),
+        T.RandomAffine(degrees=10),
+    ]
+
+    data_module = prepare_fusion_data(prediction_task=...,
+                                      fusion_model=some_example_model,
+                                      data_paths=data_paths,
+                                      output_paths=...,
+                                      transforms=my_transforms)
+
 
 -----
 

docs/developers_guide.rst

@@ -50,4 +50,4 @@ Additional Notes
 
 Thank you for your interest in contributing to Fusilli! Your contributions are highly appreciated.
 
-For any further assistance or queries, feel free to reach out to florence.townend.21@ucl.ac.uk
+For any further assistance or queries, please open an issue on the `GitHub issues page <https://github.com/florencejt/fusilli/issues>`_.

docs/index.rst

@@ -25,7 +25,7 @@ Why would you want to use fusilli?
     * - Problem
       - Solution
     * - You have a dataset that contains multiple modalities. 🩻 📈
-      - Either two types of tabular data or one type of tabular data and one type of image data. Ever thought that maybe they'd be more powerful together? Fusilli can help you find out if multimodal fusion is right for you! ✨
+      - Two or three types of tabular data, or one type of tabular data and one type of image data. Ever thought that maybe they'd be more powerful together? Fusilli can help you find out if multimodal fusion is right for you! ✨
     * - You've looked at methods for multimodal fusion and thought "wow, that's a lot of code" and "wow, there are so many names for the same concept". 🤔 🆘
       - *So* relatable. Fusilli provides a simple way for comparing multimodal fusion models without having to trawl through Google Scholar! ✨
     * - You've found a multimodal fusion method that you want to try out, but you're not sure how to implement it or it's not quite right for your data. 😵‍💫 🙌

docs/introduction.rst

@@ -33,8 +33,7 @@ Templates and specific guidance on how to do this are in :ref:`contributing`.
 Contact
 --------
 
-If you have any questions, please feel free to contact me at florence.townend.21@ucl.ac.uk or on Twitter:
-`@FlorenceTownend <https://twitter.com/florencetownend>`_.
+If you have any questions, please open an issue on the `GitHub issues page <https://github.com/florencejt/fusilli/issues>`_.
 
 
 

docs/modifying_models.rst

@@ -77,8 +77,12 @@ Modifiable Attributes
     - ``nn.ModuleDict``
   * - :attr:`~.ActivationFusion.mod2_layers`
     - ``nn.ModuleDict``
+  * - :attr:`~.ActivationFusion.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
   * - :attr:`~.ActivationFusion.fused_layers`
     - ``nn.Sequential``
+  * - :attr:`~.ActivationFusion.main_modality`
+    - int (1, 2, or 3). Which modality's feature maps get concatenated with the fused feature map.
 
 :class:`.AttentionAndSelfActivation`
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -94,10 +98,16 @@ Modifiable Attributes
     - ``nn.ModuleDict``
   * - :attr:`~.AttentionAndSelfActivation.mod2_layers`
     - ``nn.ModuleDict``
+  * - :attr:`~.AttentionAndSelfActivation.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
   * - :attr:`~.AttentionAndSelfActivation.fused_layers`
     - ``nn.Sequential``
   * - :attr:`~.AttentionAndSelfActivation.attention_reduction_ratio`
     - int
+  * - :attr:`~.AttentionAndSelfActivation.main_modality`
+    - int (1, 2, or 3). Which modality's feature maps get concatenated with the fused feature map.
+  * - :attr:`~.AttentionAndSelfActivation.attention_modality`
+    - int (1, 2, or 3). Which modality the channel attention is applied to.
 
 
 :class:`.AttentionWeightedGNN`
@@ -268,6 +278,8 @@ Modifiable Attributes
     - ``nn.ModuleDict``
   * - :attr:`~.ConcatTabularFeatureMaps.mod2_layers`
     - ``nn.ModuleDict``
+  * - :attr:`~.ConcatTabularFeatureMaps.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
   * - :attr:`~.ConcatTabularFeatureMaps.fused_layers`
     - ``nn.Sequential``
 
@@ -498,10 +510,12 @@ Modifiable Attributes
       * Overrides modification of ``mod1_layers`` made to "all"
       * Must have same number of layers as :attr:`~.TabularCrossmodalMultiheadAttention.mod2_layers`
   * - :attr:`~.TabularCrossmodalMultiheadAttention.mod2_layers`
-    - 
+    -
       * ``nn.ModuleDict``
       * Overrides modification of ``mod2_layers`` made to "all"
       * Must have same number of layers as :attr:`.TabularCrossmodalMultiheadAttention.mod1_layers`
+  * - :attr:`~.TabularCrossmodalMultiheadAttention.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
 
 ------
 
@@ -562,10 +576,12 @@ Modifiable Attributes
       * Overrides modification of ``mod1_layers`` made to "all"
       * Must have same number of layers as :attr:`~.TabularChannelWiseMultiheadAttention.mod2_layers`
   * - :attr:`~.TabularChannelWiseMultiAttention.mod2_layers`
-    - 
+    -
       * ``nn.ModuleDict``
       * Overrides modification of ``mod1_layers`` made to "all"
       * Must have same number of layers as :attr:`~.TabularChannelWiseMultiheadAttention.mod1_layers`
+  * - :attr:`~.TabularChannelWiseMultiAttention.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
   * - :attr:`~.TabularChannelWiseMultiAttention.fused_layers`
     - ``nn.Sequential``
 
@@ -587,8 +603,10 @@ Modifiable Attributes
       * ``nn.ModuleDict``
       * Overrides modification of ``mod1_layers`` made to "all"
   * - :attr:`~.TabularDecision.mod2_layers`
-    - 
+    -
       * ``nn.ModuleDict``
       * Overrides modification of ``mod2_layers`` made to "all"
+  * - :attr:`~.TabularDecision.mod3_layers`
+    - ``nn.ModuleDict``. Only used if a third tabular modality is provided.
   * - :attr:`~.TabularDecision.fusion_operation`
     - Function (such as mean, median, etc.). Should act on the 1st dimension.