Add a ConceptLocalizer via Black-box Attributions - #186
Draft
Agustin-Picard wants to merge 9 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces input-to-concept localization for
HolisticCraft,answering the question "which regions of the input are responsible for
activating concept k?" by exposing the fitted concept extractor as a
K-output black-box model and delegating to any compatible Xpliqueattribution method (initially validated with
RiseandSobolAttributionMethod).It complements — but does not replace — the existing coefficient-based
concept visualization, and reuses the standard Xplique attribution
machinery instead of introducing a parallel concept-specific explanation
stack.
Motivation
HolisticCraftcurrently visualizes concept coefficientsU_k(x)as spatialheatmaps by resizing them to the input resolution. This is a direct
visualization of the latent coefficients — not an input attribution — and can
be misleading for deep architectures, large receptive fields, attention-based
models, complex latent cuts, multi-scale models, and object-detection
backbones.
This PR adds a second, complementary interpretation:
flowchart LR X["input x"] --> LOC["ConceptLocalizer"] subgraph LOC_INTERNALS ["ConceptLocalizer internals"] direction LR E["latent extractor"] --> F["factorizer.encode(...)"] F --> R["spatial reduction<br/>per concept"] R --> S["[s_1(x), ..., s_K(x)]"] end LOC --> LOC_INTERNALS LOC_INTERNALS --> BB["black-box attribution<br/>method targeting concept k"] BB --> H["input-space<br/>localization heatmap"]The
ConceptLocalizerbehaves like aK-output black-box model, soconcept
kcan be targeted with an ordinary one-hot vector, without anycustom attribution operator.
The proposed abstraction is symmetric with the existing
ConceptDecoder:flowchart LR subgraph EXISTING ["Existing"] direction LR C1["concept coefficients"] --> D1["ConceptDecoder"] --> P1["final prediction"] end subgraph NEW ["New"] direction LR I2["input image"] --> L2["ConceptLocalizer"] --> S2["concept scores"] endConcept vocabulary
The PR makes the three distinct quantities explicit throughout docstrings:
kis present in the latentkcontributes to the task predictionkWhat this PR adds
New abstractions
ConceptLocalizer(xplique/concepts/holistic_craft.py)scalar score per concept.
"mean","sum","max"and arbitrarycallables. Default is
"mean"(consistent with the existingtop-image ranking).
[B, H, W, K], token[B, T, K], and already-global[B, K]coefficient layouts; validates output shape and finiteness.NotImplementedErrorfrom non-inductive factorizers into aclear
RuntimeErrorexplaining thatfactorizer.encode()isrequired because attribution methods evaluate perturbed inputs.
(e.g. mean absolute activation) when magnitude is intended.
ConceptLocalizerTf(xplique/concepts/tf/holistic_craft.py)tf.Tensorwith shape(B, K).ConceptLocalizerTorch(xplique/concepts/torch/holistic_craft.py)nn.Modulethat accepts native NCHW tensors.HolisticCraftTorch.make_concept_localizer()returns it wrapped inXplique's existing
TorchWrapperwithis_channel_first=Trueandrequires_grad=False, so Xplique's standard NHWC-to-NCHW conversionand device handling apply and no gradient path is used.
New
HolisticCraftAPIHolisticCraft.make_concept_localizer(concept_reducer="mean")(abstract,implemented per framework). Mirrors the existing
make_concept_decoder(latent_data)symmetry:HolisticCraft.compute_concept_attributions(images, partial_explainer, ...)PartialExplainerconfigured with a compatibleblack-box explainer (validated for
RiseandSobolAttributionMethod; compatible withOcclusion,Lime,KernelShap, and callable explainer factories).WhiteBoxExplainersubclasses and any explicitoperatorargument with an actionable error.
concept_ids(integers, no booleans, no duplicates,in range, non-empty) before any expensive work; enforces
non-empty batches with positive spatial dimensions.
(NHWC), including PyTorch
Tensor/ list inputs.them across all requested concepts.
float32maps of shape(N, H, W, number_of_concepts).Uncomputed channels are
NaNso that "not computed" isunambiguously distinguishable from a valid zero attribution.
concept_ids=Nonereturns a fully finite tensor.(N, H, W)or(N, H, W, 1);unexpected shapes / channel counts / non-finite values are rejected
with informative errors.
Visualization integration
display_concept_heatmap()resolution (avoids needlessly resampling RISE / Sobol maps).
(H, W, 1)maps to(H, W)and validates finiteness.display_images_per_concept(..., concept_maps=None)concept_maps=None, behavior is identical to v2.0.0 (usescoeffs_u).concept_mapsis provided, it drives the overlays;coeffs_ukeeps its original meaning.
display_top_images_per_concept(..., concept_maps=None)coeffs_u);attribution maps only drive the overlays. This preserves the
semantic that concept-coefficient magnitude represents presence
while attribution maps represent input sensitivity of the concept
score.
_normalize_image_batch_to_nhwc,_validate_concept_ids,_prepare_concept_maps, and_resolve_concept_map_sourceback both display paths and the newattribution API, eliminating the previous duplicated NHWC / order
validation and concentrating localization-specific logic in one
place.
Backward compatibility
display_images_per_concept(images[, coeffs_u=...])and
display_top_images_per_concept(images[, coeffs_u=...])retain theexact v2.0.0 rendering and ordering behavior.
ConceptDecoder,compute_explanation_per_concept(),estimate_importance(), andget_topk_images_per_concept()arenot modified.
Documentation and tutorials
WIP, coming up soon!