Skip to content

Commit 634bd87

Browse files
Merge pull request #58 from UTAustin-SwarmLab/attentionvig-log-2026
Add AttentionViG LoG 2026 blog, news, and publication
2 parents e78d3c7 + 3354ddf commit 634bd87

9 files changed

Lines changed: 105 additions & 0 deletions

File tree

‎_bibliography/references.bib‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22
---
33
references
44
==========
5+
@inproceedings{gedik2026attentionvig,
6+
title={{AttentionViG}: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision {GNNs}},
7+
author={Gedik, Hakan Emre and Martin, Andrew and Munir, Mustafa and Baser, Oguzhan and Marculescu, Radu and Chinchali, Sandeep P. and Bovik, Alan C.},
8+
booktitle={Proceedings of the Fifth Learning on Graphs Conference (LoG)},
9+
year={2026},
10+
url={https://arxiv.org/abs/2509.25570},
11+
pdf={https://arxiv.org/pdf/2509.25570},
12+
website={/2026/10/01/AttentionViG.html}
13+
}
14+
515
@article{bhatt2025uncap,
616
title={{UNCAP}: Uncertainty-Guided Planning Using Natural Language Communication for Cooperative Autonomous Vehicles},
717
author={Neel P. Bhatt and Po-han Li and Kushagra Gupta and Rohan Siva and Daniel Milan and Alexander Todd Hogue and Sandeep P. Chinchali and David Fridovich-Keil and Zhangyang Wang and ufuk topcu},

‎_data/news.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
- date: 2026-10-01
2+
details: >-
3+
Our paper <a href="https://arxiv.org/abs/2509.25570">AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs</a> was accepted to the Fifth Learning on Graphs Conference (LoG 2026)! <a href="/2026/10/01/AttentionViG.html">Read the blog post</a>.
14
- date: 2026-05-21
25
details: >-
36
Po-han passed his PhD defense.

‎_posts/2026-10-01-AttentionViG.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
title: 'AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs'
3+
description: LoG 2026 paper
4+
categories: blog
5+
---
6+
7+
*By Oguzhan Baser*
8+
9+
# AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs
10+
11+
Accepted to the Fifth Learning on Graphs Conference (LoG 2026).
12+
13+
**Authors:** Hakan Emre Gedik, Andrew Martin, Mustafa Munir, Oguzhan Baser, Radu Marculescu, Sandeep P. Chinchali, and Alan C. Bovik.
14+
15+
**TL;DR:** AttentionViG teaches Vision GNNs which neighbors matter. Cross-attention dynamically weights neighboring nodes, helping sparse graphs suppress irrelevant features while reaching up to **83.9% ImageNet-1K top-1 accuracy**.
16+
17+
[Read the paper](https://arxiv.org/abs/2509.25570) | [PDF](https://arxiv.org/pdf/2509.25570)
18+
19+
<figure style="text-align: center;">
20+
<img src="{{site.baseurl}}/images/post/AttentionViG/02_architecture.png" alt="AttentionViG multi-scale architecture combining inverted residual blocks and Grapher layers" style="max-width: 100%; height: auto; margin: auto; display: block;">
21+
<figcaption>Figure 1. Overview of the AttentionViG architecture across scales.</figcaption>
22+
</figure>
23+
24+
## Motivation
25+
26+
Vision Graph Neural Networks (ViGs) represent image patches as nodes and exchange information through graph edges. Their performance depends on graph construction and on how each node aggregates its neighbors.
27+
28+
Dynamic k-nearest-neighbor graphs can capture semantic relationships, but neighbor search is expensive. Fixed sparse graphs such as SVGA are more efficient, yet may connect semantically unrelated nodes. Common aggregation operators also lack an explicit mechanism for learning how important each proposed neighbor should be.
29+
30+
We ask:
31+
32+
- **Q1 (Adaptive aggregation):** Can a ViG learn which proposed neighbors are relevant instead of treating every connection equally?
33+
- **Q2 (Efficiency and transfer):** Can it retain sparse graph efficiency while improving classification and dense prediction?
34+
35+
## Contributions
36+
37+
- **Cross-attention aggregation.** Queries come from each node, while keys and values come from its neighbors. Cosine similarity is transformed through an exponential affinity function, giving neighbors independent relevance weights rather than forcing them to compete through softmax.
38+
- **AttentionViG.** We integrate this aggregation into a multi-scale hybrid CNN-GNN architecture. Inverted residual blocks perform local processing, while Grapher layers use efficient SVGA connectivity for non-local message passing.
39+
- **Broad evaluation.** We evaluate AttentionViG on ImageNet-1K, MS-COCO, and ADE20K.
40+
41+
<figure style="text-align: center;">
42+
<img src="{{site.baseurl}}/images/post/AttentionViG/01_cross_attention.png" alt="Cross-attention aggregation uses node queries and neighbor keys and values to learn independent neighbor relevance weights" style="max-width: 100%; height: auto; margin: auto; display: block;" loading="lazy">
43+
<figcaption>Figure 2. Cross-attention aggregation: neighbor weighting and aggregation pipeline.</figcaption>
44+
</figure>
45+
46+
## How AttentionViG Works
47+
48+
The key idea is to separate graph construction from neighbor importance. SVGA supplies candidate neighbors; cross-attention learns which should contribute strongly and which should be suppressed. This makes message passing adaptive to image content without expensive dynamic neighbor search. The parallelized Grapher implementation retains linear scaling with input resolution.
49+
50+
## Results
51+
52+
On ImageNet-1K, AttentionViG-S/M/B achieve **81.3% / 83.1% / 83.9% top-1 accuracy** at **1.6 / 3.2 / 4.8 GFLOPs**. AttentionViG-B matches GreedyViG-B at 83.9% while using 4.8 rather than 5.2 GFLOPs. These results use the paper's training setup with knowledge distillation from a RegNetY-16GF teacher.
53+
54+
<figure style="text-align: center;">
55+
<img src="{{site.baseurl}}/images/post/AttentionViG/03_imagenet_results.png" alt="ImageNet-1K results: AttentionViG-S 81.3 percent at 1.6 GFLOPs, M 83.1 percent at 3.2 GFLOPs, and B 83.9 percent at 4.8 GFLOPs" style="max-width: 100%; height: auto; margin: auto; display: block;" loading="lazy">
56+
<figcaption>ImageNet-1K performance summary. Values are reported in the paper.</figcaption>
57+
</figure>
58+
59+
AttentionViG-B also reaches **46.4 box AP and 42.3 mask AP on MS-COCO**, and **47.8 mIoU on ADE20K**, using Mask R-CNN and Semantic FPN, respectively.
60+
61+
<figure style="text-align: center;">
62+
<img src="{{site.baseurl}}/images/post/AttentionViG/04_dense_prediction.png" alt="AttentionViG-B dense prediction results: 46.4 box AP and 42.3 mask AP on COCO, and 47.8 mIoU on ADE20K" style="max-width: 100%; height: auto; margin: auto; display: block;" loading="lazy">
63+
<figcaption>COCO and ADE20K transfer performance summary. Values are reported in the paper.</figcaption>
64+
</figure>
65+
66+
In vanilla ViG, cross-attention reaches 74.3% top-1 at 1.6 GFLOPs, matching EdgeConv's accuracy at about two-thirds of its FLOPs. In the affinity-function ablation, exponential affinity reaches 81.3% top-1 versus 80.8% with softmax.
67+
68+
Similarity visualizations show the model emphasizing semantically related regions while suppressing unrelated ones.
69+
70+
<figure style="text-align: center;">
71+
<img src="{{site.baseurl}}/images/post/AttentionViG/05_neighbor_visualization.png" alt="Query-key similarity maps for fish and bird images highlight semantically related regions" style="max-width: 100%; height: auto; margin: auto; display: block;" loading="lazy">
72+
<figcaption>Figure 3. Query-key similarity visualizations highlighting semantically related regions.</figcaption>
73+
</figure>
74+
75+
## Impact
76+
77+
AttentionViG shows that efficient graph construction does not require every proposed edge to be equally useful. Learning neighbor relevance allows Vision GNNs to remain sparse and efficient while becoming more selective about the information they propagate. Extending the aggregation idea to video, point clouds, and other graph-based domains is a promising direction for future work.
78+
79+
For the complete experiments, ablations, and implementation details, [read the paper](https://arxiv.org/abs/2509.25570).
122 KB
Loading
59.7 KB
Loading
76.8 KB
Loading
72.5 KB
Loading
1020 KB
Loading

‎images/post/AttentionViG/README.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# AttentionViG blog figures
2+
3+
Assets for the LoG 2026 AttentionViG blog post.
4+
5+
Source: Hakan Emre Gedik et al., [AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs](https://arxiv.org/abs/2509.25570).
6+
7+
- `01_cross_attention.png`: cross-attention aggregation diagram from the paper.
8+
- `02_architecture.png`: multi-scale architecture diagram from the paper.
9+
- `03_imagenet_results.png`: visual summary of the paper's ImageNet-1K results, with the knowledge-distillation setting noted.
10+
- `04_dense_prediction.png`: visual summary of the paper's COCO and ADE20K results.
11+
- `05_neighbor_visualization.png`: query-key similarity visualizations from the paper.
12+
13+
The performance summaries are presentation graphics, not reproductions of the paper's tables.

0 commit comments

Comments
 (0)