Skip to content

Commit 9b642ab

Browse files
committed
Build LowEntropyAI research directory and publish with GitHub Pages
1 parent 1b8b5de commit 9b642ab

45 files changed

Lines changed: 1328 additions & 23 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/pages.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Build and deploy Proactive Atlas
2+
on:
3+
push:
4+
branches: [main]
5+
pull_request:
6+
branches: [main]
7+
workflow_dispatch:
8+
permissions:
9+
contents: read
10+
concurrency:
11+
group: pages-${{ github.ref }}
12+
cancel-in-progress: true
13+
jobs:
14+
build:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v6
18+
- uses: actions/setup-node@v6
19+
with:
20+
node-version: '24'
21+
cache: npm
22+
- run: npm ci
23+
- run: npm run check
24+
- name: Configure Pages
25+
if: github.event_name != 'pull_request'
26+
uses: actions/configure-pages@v5
27+
- name: Upload site
28+
if: github.event_name != 'pull_request'
29+
uses: actions/upload-pages-artifact@v4
30+
with:
31+
path: dist
32+
deploy:
33+
if: github.event_name != 'pull_request'
34+
needs: build
35+
runs-on: ubuntu-latest
36+
permissions:
37+
pages: write
38+
id-token: write
39+
environment:
40+
name: github-pages
41+
url: ${{ steps.deployment.outputs.page_url }}
42+
steps:
43+
- name: Deploy
44+
id: deployment
45+
uses: actions/deploy-pages@v4

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,4 @@
11
.DS_Store
2+
node_modules/
3+
dist/
4+
.qa/

‎APPLICATIONS.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Agents & Applications
2+
3+
[README](README.md) · [Personal Agents](PERSONAL_AGENTS.md) · [Projects & Products](PROJECTS.md) · [Streaming Proactivity](STREAMING.md) · [Benchmark Matrix](BENCHMARKS.md)
4+
5+
Browse by **who receives assistance and what the assistant helps with**. A personal assistant may run on a phone, watch, glasses, desktop, or connected service. Streaming describes an interaction capability; it can support any of these applications. These routes overlap and do not assign each work to a single exclusive category.
6+
7+
## Personal Assistance
8+
9+
Persistent goals, routines, preferences, and everyday context make assistance useful to a particular person. Start with the [Personal Agents guide](PERSONAL_AGENTS.md), which connects research methods, implemented assistants, streaming models, and evaluation across devices.
10+
11+
| Setting | Starting resources | Proactive decision to inspect |
12+
|---|---|---|
13+
| Everyday digital workflows | [Proactive Agent](papers/conference/ICLR2025/proactive-agent-shifting-llm.md); [dot, GAIA, MineContext](PROJECTS.md) | Which unstated task, contextual suggestion, or background update is worth surfacing? |
14+
| Mobile preferences and routines | [PersonalAlign](papers/conference/ACL2026/personalalign-long-term-gui-intent.md); [FingerTip 20K](papers/conference/ICLR2026/fingertip-20k.md); [KnowU-Bench](papers/arxiv/2026-04/knowu-bench.md) | When does history justify a suggestion, and when should the assistant ask or respect rejection? |
15+
| Wearable and physical-world assistance | [Satori](papers/conference/CHI2025/satori-proactive-ar-bdi.md); [ProMemAssist](papers/conference/UIST2025/promemassist-working-memory-wearable.md); [WatchGuardian](papers/arxiv/2025-02/watchguardian-personalized-jiti.md) | Does the user's inferred task, cognitive state, or user-defined behavior warrant an intervention? |
16+
17+
## Coding and Knowledge Work
18+
19+
| Starting resources | Proactive decision | Evaluation route |
20+
|---|---|---|
21+
| [Need Help?](papers/conference/CHI2025/need-help-proactive-programming.md), [CodingGenie](papers/arxiv/2025-03/codinggenie-proactive-programming-assistant.md) | Offer programming help at useful moments while preserving user control. | Human studies, interruption cost, acceptance, and task outcomes. |
22+
| [Ask or Assume?](papers/conference/EMNLP2026/ask-or-assume-coding-agents.md), [HiSentinel](papers/arxiv/2026-09/hisentinel-coding-agent-intervention.md) | Ask about missing requirements, redirect a risky action, or continue execution. | Clarification burden and downstream success; distinguish intervention on another agent from assistance initiated for a human. |
23+
| [ProAgentBench](papers/arxiv/2026-02/proagentbench.md), [ProCodeBench](papers/arxiv/2026-05/procodebench-proactive-coding-assistants.md) | Assess assistance opportunities in desktop or coding workflows. | See protocol and data columns in the [Benchmark Matrix](BENCHMARKS.md). |
24+
25+
## Meetings and Collaboration
26+
27+
| Starting resources | Proactive decision | Evaluation route |
28+
|---|---|---|
29+
| [InsightToast](papers/arxiv/2026-08/insighttoast-meeting-side-channel.md) | Retrieve and surface missing context in a peripheral meeting channel. | Information utility and disruption during collaboration. |
30+
| [Speak for Me](papers/arxiv/2026-09/speak-for-me-meeting-delegation.md) | Decide whether, when, and what to contribute on a participant's behalf. | Situational awareness, floor control, and delegation boundaries. |
31+
| [DocuTeam](papers/arxiv/2026-09/docuteam-evolving-documents.md) | Use document changes to initiate or redirect collaboration. | Relevance of contributions to the evolving artifact. |
32+
33+
## Embodied and Robot Assistance
34+
35+
| Starting resources | Proactive decision | Evaluation route |
36+
|---|---|---|
37+
| [PACE](papers/conference/ICRA2025/pace-action-completion-assistance.md) | Use estimated human action completion to time physical assistance. | Idle time and fluency in structured collaboration. |
38+
| [PACT](papers/arxiv/2026-05/pact-continual-robot-assistance.md) | Ask for missing context or act using cross-day interaction history. | Assistance accuracy versus clarification cost. |
39+
| [ProRobo](papers/arxiv/2026-09/prorobo-cognitive-action-reasoning.md) | Anticipate needs in embodied interaction. | Consult the evidence card for task and deployment limits. |
40+
41+
Personalization can overlap with this route, but a collaborative robot is not automatically a personal assistant. Household assistance is an intended application to track; controlled assembly or simulated multi-day routines do not establish an open-ended home deployment.
42+
43+
## Domain-Specific Assistance
44+
45+
| Starting resources | Why it belongs here | Evidence boundary |
46+
|---|---|---|
47+
| [See, Infer, Intervene](papers/arxiv/2026-06/see-infer-intervene-retail.md) | Retail assistance explicitly selects engagement actions or `HOLD`. | Scripted and small-scale settings; inspect the perception-to-policy gap. |
48+
| [AI Assistants Overassist](papers/arxiv/2026-07/ai-assistants-overassist.md) | Tutoring tests whether intervention arrives too early or supplies too much. | Learning outcomes and user independence matter alongside immediate correctness. |
49+
| [Oops, Not Now / PEARL](papers/arxiv/2026-09/pearl-proactive-gameplay-support.md); [Proact-VL](papers/arxiv/2026-03/proact-vl-realtime-companions.md) | Gameplay assistance and commentary expose timing and unwanted-interruption trade-offs. | Commentary, requested guidance, and spontaneous assistance have different participation expectations. |
50+
51+
## How These Routes Connect
52+
53+
Use [Projects & Products](PROJECTS.md) for implementation/access evidence, [Streaming Proactivity](STREAMING.md) for model mechanisms, [Supporting Infrastructure](INFRASTRUCTURE.md) for sensing/memory/runtime roles, and the [Research Map](RESEARCH_MAP.md) for scientific questions. The [full bibliography](README.md#papers) remains the source of publication metadata; these routes select representative entries rather than duplicate that catalog.

‎BENCHMARKS.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,16 @@
22

33
This page compares benchmarks by what they actually test. The goal is to make benchmark selection faster than scanning individual paper summaries.
44

5+
[Layered Must Read](README.md#must-read) · [Streaming models and evaluation routes](STREAMING.md) · [Projects & Products](PROJECTS.md)
6+
57
## Quick Takeaways
68

79
- **Best long-horizon personal assistant benchmarks**: VibeLifeBench, π-Bench, VitaBench 2.0, ChronosBench.
810
- **Best computer-use benchmarks**: Act2Intention Bench, Ambig-SWE, CONFLICTGUI, ProAgentBench, PIRA-Bench, GUIDE, KnowU-Bench, Pare-Bench, ProCodeBench.
911
- **Best dialogue, clarification, and premise-critique benchmarks**: AskBench, CC-Mediation, ClarifyBench, ContextClarify, Drift-Bench++, FinInteract, IdeaAMBIG, IntentFlux, Interactive Visual Grounding, OR-Clarify, ProactBench, ProactiveEval, ProMediConv, ProMISe, RPCBench, Ψ-Bench.
1012
- **Best memory-oriented benchmarks**: APM-Bench, CogEval-Bench, MemEye, TWIST, VitaBench 2.0.
1113
- **Best human-factor / timing benchmarks**: DuplexAct-Bench, Int-Bench, JarvisBench, PASTABench, PROACTIVITY-GYM, RealHumanEval, Pare-Bench, ProAgentBench, ProactiveBench (MLLM).
12-
- **Best streaming multimodal benchmarks**: APM-Bench, DuplexAct-Bench, Full-Duplex Floor Selection, Live Assistant, TRACE, TIMELI, ProReady-QA, OmniAssistBench, OmniPro, EgoPro-Bench, OmniMMI, StreamArena, IPIBench, EgoServe, StreamSoccer.
14+
- **Streaming multimodal benchmarks by task** (see [evaluation routes](STREAMING.md#evaluation-routes)): ESTP-Bench, StreamGaze, SOVBench, Live Gaming Benchmark, APM-Bench, DuplexAct-Bench, Full-Duplex Floor Selection, Live Assistant, TRACE, TIMELI, ProReady-QA, OmniAssistBench, OmniPro, EgoPro-Bench, OmniMMI, StreamArena, IPIBench, EgoServe, StreamSoccer.
1315

1416
## Benchmark Matrix
1517

@@ -22,6 +24,10 @@ This page compares benchmarks by what they actually test. The goal is to make be
2224
| **Ambig-SWE** | Ambig-SWE | Software engineering | Repository, underspecified issue, and interactive user responses | Detect underspecificity, ask targeted questions, and use the answer | Simulator backed by withheld issue information | SWE-bench Verified-derived interactive benchmark | Detection, clarification quality, post-interaction task resolution | [OpenReview](https://openreview.net/forum?id=X2yzXtH4wp) · [GitHub](https://github.com/sani903/InteractiveSWEAgents) |
2325
| **ProactiveBench** | Proactive Agent | Desktop activity | Event streams and desktop context | Predict useful next tasks | Simulated user feedback / reward model | Synthetic + event-derived | Task prediction usefulness, acceptance proxy | [GitHub](https://github.com/thunlp/ProactiveAgent) |
2426
| **OmniMMI** | OmniMMI | Streaming multimodal interaction | Continuous video, audio, speech, and dialogue history | Trigger proactive alerts and turn-taking while supporting reactive QA | Standing user queries | Human-annotated benchmark | Alert timing, turn-taking, streaming QA, multi-turn interaction | [CVPR 2025](https://openaccess.thecvf.com/content/CVPR2025/html/Wang_OmniMMI_A_Comprehensive_Multi-modal_Interaction_Benchmark_in_Streaming_Video_Contexts_CVPR_2025_paper.html) · [Website](https://omnimmi.github.io/) |
27+
| **ESTP-Bench** | Eyes Wide Open | Egocentric streaming video | Past/current first-person frames and evolving questions | Coherent just-in-time responses during perception | Evolving question context | Benchmark with instruction-tuning resources | ESTP-F1; coherence, response timing, efficiency | [Paper](https://arxiv.org/abs/2510.14560) · [Code](https://github.com/SooLab/EyeWO) |
28+
| **StreamGaze** | StreamGaze | Gaze-conditioned egocentric video | Causal video plus gaze trajectories | Anticipation and gaze-triggered alerts, alongside past/present reasoning | Measured gaze as an attention cue | 285 videos; 8,521 QA pairs across 10 tasks | Task-specific reasoning/alert evaluation; proactive tasks reported separately | [Paper](https://arxiv.org/abs/2512.01707) · [Code](https://github.com/daeunni/StreamGaze) |
29+
| **SOVBench-O / SOVBench-T** | StreamOV | Streaming omni-video interaction | Continuous audio/video and multi-turn queries | Contextual inference; trigger on present evidence, stay silent for absent evidence | Standing queries and dialogue history | O: 172 sessions / 1,739 QA turns; T: 226 positive/negative samples | QA accuracy; trigger precision, recall, F1 | [Paper / protocol](https://arxiv.org/html/2605.25621v1) |
30+
| **Live Gaming Benchmark** | Proact-VL | Live companion commentary/guidance | Game video and scenario-conditioned interaction | Choose timely commentary, co-commentary, or guidance | Scenario roles / user guidance request | Game and guidance evaluation data | LLM Score, CC, F1, Time-Diff, PAUC | [Code / protocol](https://github.com/microsoft/AnthropomorphicIntelligence/tree/main/Proact-VL) · [Dataset](https://huggingface.co/datasets/oaaoaa/LiveGamingBenchmark) |
2531
| **Full-Duplex Floor Selection** | Full-Duplex Speech Models Take the Floor When Asked, Not When Needed | Always-on speech interaction | Context-matched monologues with controlled false-fact, hazard, missing-word, address, and silence triggers | Self-select to speak because intervention is needed, not merely because the floor is open | Speaker monologues rather than personalized users | Ten-condition controlled diagnostic | Speech-initiation probability, non-empty response rate, false-fact challenge, hazard warning | [arXiv](https://arxiv.org/abs/2609.19596) |
2632
| **Live Assistant** | Live Assistant | Livestream social assistance | Native audio-video, comments, gifts, viewer dynamics, and room metadata | Choose silence, private memory, or a grounded message to the right recipient | Hosts and viewers reconstructed from real livestreams | 275 clips and 13,812 human-reviewed decision intervals | State, recipient, and task accuracy | [arXiv](https://arxiv.org/abs/2609.27303) |
2733
| **ProAction** | Cognitive Action Reasoning for Proactive Robots | Human-centered embodied assistance | Visual observations, audio, and text without an action instruction | Infer a cognitively grounded high-level robot action from human and environmental cues | Human-refined action judgments informed by affective theory of mind | 10,000 samples across 12 scenarios and five scenes | Action reasoning, modality contribution, subject-disjoint transfer, human evaluation | [arXiv](https://arxiv.org/abs/2609.23486) |
@@ -94,7 +100,7 @@ This page compares benchmarks by what they actually test. The goal is to make be
94100
| How do we evaluate personalization? | RobotEQ 3.0, KnowU-Bench, FingerTip 20K, VitaBench 2.0, UserVille, Ψ-Bench | They include profiles, preferences, or user-specific trajectories. |
95101
| How do we evaluate memory as a proactive substrate? | APM-Bench, CogEval-Bench, MemEye, TWIST, VitaBench 2.0, ProActEval | They test memory formation, retrieval, evidence preparation, intent emergence, missing-evidence awareness, or whether memory should trigger an intervention. |
96102
| How do we evaluate computer-use agents? | Act2Intention Bench, CONFLICTGUI, SWE-Intervene, OverAct, ProAgentBench, GUIDE, PIRA-Bench, KnowU-Bench, Pare-Bench, ProCodeBench | They connect proactive behavior to GUI, mobile, IDE, tool calls, or workflow contexts; CONFLICTGUI and OverAct test restraint, while SWE-Intervene tests pre-execution control. |
97-
| How do we evaluate proactive streaming video or speech? | APM-Bench, DuplexAct-Bench, Live Assistant, TRACE, TIMELI, ProReady-QA, OmniAssistBench, OmniPro, EgoPro-Bench, OmniMMI, StreamArena, IPIBench, EgoServe, StreamSoccer | They require causal processing of an ongoing stream and timely outputs rather than offline clip answering; TRACE audits execution conditions and DuplexAct-Bench adds proactive initiation and active silence. |
103+
| How do we evaluate proactive streaming video or speech? | ESTP-Bench, StreamGaze, SOVBench, Live Gaming Benchmark, APM-Bench, DuplexAct-Bench, Live Assistant, TRACE, TIMELI, ProReady-QA, OmniAssistBench, OmniPro, EgoPro-Bench, OmniMMI, StreamArena, IPIBench, EgoServe, StreamSoccer | They require causal processing of an ongoing stream and timely outputs rather than offline clip answering; TRACE audits execution conditions and DuplexAct-Bench adds proactive initiation and active silence. |
98104
| How do we evaluate proactive critique? | RPCBench, MMPCBench, AskBench | They require the assistant to challenge or repair a faulty premise instead of complying silently; RPCBench additionally tests localization, handling strategy, and evidence faithfulness in recommendation. |
99105
| How do we evaluate ask-versus-stop clarification? | FinInteract, IdeaAMBIG, OR-Clarify, ClarifyBench, AskBench | They evaluate whether missing information justifies another question and whether the agent can stop without silently assuming critical information; FinInteract additionally tests whether the answer is integrated under competing valid interpretations. |
100106
| How do we evaluate social intervention outcomes? | ProMediConv, CC-Mediation, ProMediate, ProACT Collaboration Bench | They evaluate intervention strategies or points and downstream collaborative or interpersonal effects rather than response quality alone. |

‎CONTRIBUTING.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Contributing to Proactive Atlas
2+
3+
Help keep the collection useful, traceable, and easy to maintain. Bibliographic content and the LowEntropyAI website share the same Markdown sources.
4+
5+
## Add or correct a paper
6+
7+
1. Check `README.md` for the title and canonical source URL to avoid duplicates.
8+
2. Confirm a user-facing proactive decision: infer an unstated need, choose an intervention moment, ask, suggest, remind, act, or deliberately stay silent. Describe the actual decision and its evaluation.
9+
3. Add a row under the best-fitting `## Papers` section using the existing five-column schema: date, title, venue/source, tags, resources. Use existing tags from the vocabulary.
10+
4. Preserve publication uncertainty: a submission, preprint, repository claim, and verified conference acceptance are distinct statuses.
11+
5. Add a note under `papers/arxiv/YYYY-MM/` or `papers/conference/VENUE/`. Use the exact headings `Why It Matters`, `Proactivity Signal`, `Evaluation Setup`, `Key Limitations`, and `Use For`. Ground results in primary sources and state evaluation limits.
12+
6. Link the note from the paper's README row using `[Notes](papers/.../note.md)` or the existing Notes badge format.
13+
7. Update `RESEARCH_MAP.md`, `BENCHMARKS.md`, or `STREAMING.md` when the work adds a useful research route or evaluation target.
14+
15+
For a benchmark, keep all nine columns in `BENCHMARKS.md`. Do not compare scores across incompatible datasets, hardware, protocols, or timing boundaries.
16+
17+
## Add a project
18+
19+
Use `PROJECTS.md`. Record what triggers the assistant, what context it sees, what user-facing behavior it chooses, release/access status, evidence boundaries, canonical links, and checked date. Distinguish a memory/capture/serving component from a complete proactive assistant.
20+
21+
## Place a resource in the collection
22+
23+
Use [Applications](APPLICATIONS.md) and [Personal Agents](PERSONAL_AGENTS.md) for selective reading routes. A paper keeps its canonical metadata in the README; projects keep implementation/access facts in PROJECTS; streaming mechanisms stay in STREAMING; benchmark protocols stay in BENCHMARKS. Link between these views instead of copying full records.
24+
25+
Record application, device/modality, decision mechanism, and resource type separately. Do not infer personalization from a camera interface or learned proactivity from a scheduler. Include supporting components through [Infrastructure](INFRASTRUCTURE.md) with their specific role. A model may support personal interaction without being a complete personal assistant. The [restructuring design](docs/RESTRUCTURE_PLAN.md) defines the scope and staged data plan.
26+
27+
## Verify
28+
29+
```sh
30+
npm ci
31+
npm run check
32+
git diff --check
33+
```
34+
35+
Preview website changes with `npm run dev`. Verify desktop and mobile reading layouts and relevant interactions. See [WEBSITE.md](WEBSITE.md) for architecture and deployment.

0 commit comments

Comments
 (0)