You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 823161c
Browse filesBrowse the repository at this point in the historyBrowse files
| Everyday digital workflows |[Proactive Agent](papers/conference/ICLR2025/proactive-agent-shifting-llm.md); [dot, GAIA, MineContext](PROJECTS.md)| Which unstated task, contextual suggestion, or background update is worth surfacing? |
14
+
| Latent needs and GUI intent recommendation |[PASK](papers/arxiv/2026-04/pask-intent-aware-proactive-agent.md); [PIRA-Bench / PIRF](papers/arxiv/2026-03/pira-bench.md)| When should ongoing conversation or passive screen context produce assistance, an intent recommendation, or silence? |
14
15
| Mobile preferences and routines |[PersonalAlign](papers/conference/ACL2026/personalalign-long-term-gui-intent.md); [FingerTip 20K](papers/conference/ICLR2026/fingertip-20k.md); [KnowU-Bench](papers/arxiv/2026-04/knowu-bench.md)| When does history justify a suggestion, and when should the assistant ask or respect rejection? |
15
16
| Wearable and physical-world assistance |[Satori](papers/conference/CHI2025/satori-proactive-ar-bdi.md); [ProMemAssist](papers/conference/UIST2025/promemassist-working-memory-wearable.md); [WatchGuardian](papers/arxiv/2025-02/watchguardian-personalized-jiti.md)| Does the user's inferred task, cognitive state, or user-defined behavior warrant an intervention? |
Copy file name to clipboardExpand all lines: PERSONAL_AGENTS.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,6 +32,8 @@ The canonical release, licensing, and checked-date entries live in [PROJECTS.md]
32
32
|---|---|---|
33
33
| Desktop and cross-app workflows |[Proactive Agent](papers/conference/ICLR2025/proactive-agent-shifting-llm.md), [ContextAgent](papers/conference/NeurIPS2025/context-agent.md)|[ProAgentBench](papers/arxiv/2026-02/proagentbench.md), [π-Bench](papers/arxiv/2026-05/pi-bench-long-horizon-workflows.md), [VibeLifeBench](papers/arxiv/2026-08/vibelifebench-living-world.md). These are complementary protocols, not necessarily each method's reported benchmark. |
34
34
| Mobile |[PersonalAlign / HIM-Agent](papers/conference/ACL2026/personalalign-long-term-gui-intent.md)| AndroidIntent in PersonalAlign; [FingerTip 20K](papers/conference/ICLR2026/fingertip-20k.md) and [KnowU-Bench](papers/arxiv/2026-04/knowu-bench.md) cover additional suggestion, execution, and interaction questions. |
35
+
| Conversation, meetings, and learning |[PASK / IntentFlow](papers/arxiv/2026-04/pask-intent-aware-proactive-agent.md)| LatentNeeds-Bench tests demand detection and silence in speech-transcript sessions; staged user evidence remains preliminary. |
36
+
| Passive GUI context and intent recommendation |[PIRF / PIRA-Bench](papers/arxiv/2026-03/pira-bench.md)| PIRA-Bench evaluates profile-conditioned future-intent recommendation and noise rejection; PIRF is its memory-aware, state-tracking baseline. Recommendation is distinct from executing the suggested task. |
35
37
| AR glasses and wearables |[Satori](papers/conference/CHI2025/satori-proactive-ar-bdi.md), [ProMemAssist](papers/conference/UIST2025/promemassist-working-memory-wearable.md), [WatchGuardian](papers/arxiv/2025-02/watchguardian-personalized-jiti.md)| Task-specific user studies; a watch's user-defined behavior trigger differs from a learned multimodal dialogue policy. |
36
38
| Continuous egocentric video |[Vinci2 / EgoMemo](papers/arxiv/2026-07/vinci2-egoserve-proactive-video-assistance.md)| EgoServe and [EgoPro-Bench](papers/arxiv/2026-05/egopro-bench-personalized-streaming.md); recorded video or simulated profiles do not establish longitudinal wearable deployment. |
37
39
| Continual embodied collaboration |[PACT](https://arxiv.org/abs/2605.24350)| Cross-day ask/act adaptation in controlled embodied scenarios. Also belongs in the [robot assistance route](APPLICATIONS.md#embodied-and-robot-assistance). |
Copy file name to clipboardExpand all lines: PROJECTS.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,6 +29,7 @@ This guide covers implemented resources alongside research papers. Entries are g
29
29
|---|---|---|---|---|
30
30
|**Proactive Agent**| Predicts tasks and assistance opportunities from activity/event streams. | Event collection, data generation, agent/reward checkpoints, and evaluation pipeline. | ICLR 2025 method + ProactiveBench; distinguish generated training scenarios from real-user deployment outcomes. |[Paper](https://arxiv.org/abs/2410.12361) · [Repository](https://github.com/thunlp/ProactiveAgent) · [Notes](papers/conference/ICLR2025/proactive-agent-shifting-llm.md)|
31
31
|**ContextAgent**| Uses open-world sensory context to decide on assistance and tool use. | Inference, SFT, tools, prompts, and sensory-context data layout. | NeurIPS 2025 research system; inspect sensing assumptions and the context-to-assistance policy. |[Paper](https://arxiv.org/abs/2505.14668) · [Repository](https://github.com/openaiotlab/ContextAgent) · [Notes](papers/conference/NeurIPS2025/context-agent.md)|
32
+
|**PASK**| IntentFlow detects latent needs and chooses silence, fast intervention, or deeper assistance with hierarchical user/workspace/global memory. | Official repository, project page, system descriptions, and client download links. | Technical report and preliminary user evidence. Public client/documentation links do not establish that every training component or model weight is released; no installation or evaluation was repeated here. |[Paper](https://arxiv.org/abs/2604.08000) · [Repository](https://github.com/xzf-thu/Pask) · [Project](https://xzf-thu.github.io/Pask/) · [Notes](papers/arxiv/2026-04/pask-intent-aware-proactive-agent.md)|
32
33
33
34
Streaming research implementations have a dedicated [model and framework matrix](STREAMING.md#models-and-frameworks), including JoyAI-VL-Interaction, MOSS-VL-Realtime, OneStreamer, Proact-VL, Response-G1, Eyes Wide Open, MiniCPM-o 4.5, and Gander.
Copy file name to clipboardExpand all lines: README.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -98,6 +98,7 @@ A layered starting path, selected for **coverage of different mechanisms and evi
98
98
<tr><td>Sensory context → assistance</td><td><a href="https://arxiv.org/abs/2505.14668"><b>ContextAgent</b></a> · <a href="https://github.com/openaiotlab/ContextAgent">Code</a><br><sub>NeurIPS 2025 · context-aware agent</sub></td><td>Move from desktop logs to open-world sensory context and tool-mediated help.</td></tr>
99
99
<tr><td>Persistent agent activation</td><td><a href="https://github.com/openclaw/openclaw"><b>OpenClaw</b></a> · <a href="https://docs.openclaw.ai/gateway/heartbeat">Heartbeat docs</a><br><a href="https://github.com/refixai/proactivity-sdk"><b>Proactivity SDK</b></a><br><sub>Runtime projects · see <a href="PROJECTS.md">Projects & Products</a></sub></td><td>Compare periodic attention checks with durable goals and model-selected wake cadence.</td></tr>
100
100
<tr><td>Product-level initiative</td><td><a href="https://help.openai.com/en/articles/20001530-getting-started-with-your-dot"><b>OpenAI dot</b></a><br><a href="https://github.com/volcengine/MineContext"><b>MineContext</b></a><br><sub>Always-on product / ambient desktop project</sub></td><td>Compare background research and connected-app context with proactive desktop summaries and tips.</td></tr>
101
+
<tr><td>Latent needs and persistent memory</td><td><a href="https://arxiv.org/abs/2604.08000"><b>PASK / IntentFlow</b></a> · <a href="https://github.com/xzf-thu/Pask">Code / project</a><br><a href="papers/arxiv/2026-04/pask-intent-aware-proactive-agent.md">Notes</a><br><sub>Technical report · need detection and assistance routing</sub></td><td>Connect silent / fast / full assistance decisions to ongoing conversations and hierarchical personal memory.</td></tr>
101
102
<tr><td>Personal routines across devices</td><td><a href="https://aclanthology.org/2026.acl-long.1669/"><b>PersonalAlign / HIM-Agent</b></a><br><a href="PERSONAL_AGENTS.md">Personal Agents guide</a><br><sub>Mobile user history · preference and routine memory</sub></td><td>Separate completing omitted preferences from anticipating an unrequested routine; personal assistance extends beyond desktop interfaces.</td></tr>
102
103
<tr><td>Wearable and physical-world help</td><td><a href="https://arxiv.org/abs/2410.16668"><b>Satori</b></a><br><a href="papers/conference/UIST2025/promemassist-working-memory-wearable.md"><b>ProMemAssist</b></a><br><sub>AR guidance · working-memory-aware intervention</sub></td><td>Ground assistance in inferred user/task state and weigh the benefit of help against interruption.</td></tr>
103
104
<tr><td>Embodied ask / act decisions</td><td><a href="https://arxiv.org/abs/2605.24350"><b>PACT</b></a><br><a href="https://doi.org/10.1109/ICRA55743.2025.11127399"><b>PACE</b></a><br><sub>Continual robot assistance · action-completion timing</sub></td><td>Compare asking versus acting from interaction history with timing physical help from observed task progress.</td></tr>
@@ -106,6 +107,7 @@ A layered starting path, selected for **coverage of different mechanisms and evi
106
107
<tr><td>Full-duplex interaction → agent work</td><td><a href="https://arxiv.org/abs/2604.27393"><b>MiniCPM-o 4.5</b></a> · <a href="https://github.com/OpenBMB/MiniCPM-o">Code</a><br><a href="https://arxiv.org/abs/2609.08977"><b>Gander</b></a> · <a href="https://github.com/Omni-Interaction-Gander/Omni-Interaction-Agent">Code</a><br><sub>Simultaneous perception, speech, and asynchronous work</sub></td><td>Study speech/listening control, scene-driven comments, and the bridge to a background reasoning agent.</td></tr>
107
108
<tr><th colspan="3" align="left">Tier 3 · Evaluation and boundaries — test the route you chose</th></tr>
108
109
<tr><td>Real workflows and personalization</td><td><a href="https://arxiv.org/abs/2602.04482"><b>ProAgentBench</b></a><br><a href="https://arxiv.org/abs/2604.08455"><b>KnowU-Bench</b></a> · <a href="https://github.com/ZJU-REAL/KnowU-Bench">Code</a></td><td>Pair real desktop traces with personalized, consent-aware mobile evaluation.</td></tr>
110
+
<tr><td>GUI intent recommendation</td><td><a href="https://arxiv.org/abs/2603.08013"><b>PIRA-Bench / PIRF</b></a> · <a href="https://www.pira-bench.top">Project</a><br><a href="https://huggingface.co/datasets/Yuxiang007/PIRA-Bench-data">Data</a><br><sub>Benchmark with a memory-aware recommendation baseline</sub></td><td>Evaluate latent future intents, user profiles, interleaved tasks, and rejection of noisy visual context before instruction-following execution.</td></tr>
109
111
<tr><td>Long-horizon initiative</td><td><a href="https://arxiv.org/abs/2605.14678"><b>π-Bench</b></a> · <a href="https://github.com/Simplified-Reasoning/Pi-Bench">Code</a><br><a href="https://arxiv.org/abs/2608.10875"><b>VibeLifeBench</b></a></td><td>Evaluate hidden intents and act / ask / stay-silent decisions over evolving personal workflows.</td></tr>
110
112
<tr><td>Streaming timing and intent</td><td><a href="https://omnimmi.github.io/"><b>OmniMMI</b></a><br><a href="https://arxiv.org/abs/2512.01707"><b>StreamGaze</b></a> · <a href="https://github.com/daeunni/StreamGaze">Code</a><br><a href="STREAMING.md#evaluation-routes">Streaming evaluation routes</a></td><td>Compare multimodal alert/turn timing with gaze-conditioned intent; inspect proactive subsets separately from general streaming QA.</td></tr>
111
113
<tr><td>Personalized physical-world assistance</td><td><a href="https://arxiv.org/abs/2605.07299"><b>EgoPro-Bench</b></a><br><a href="https://arxiv.org/abs/2607.11523"><b>Vinci2 / EgoServe</b></a><br><sub>Egocentric timing, user context, and silence</sub></td><td>Connect streaming to personalized assistance while keeping recorded-video evaluation separate from live wearable deployment.</td></tr>
@@ -349,7 +351,7 @@ The complete bibliography retains five research areas for archival browsing. Use
| 2026-05 |**MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation**| arXiv 2605 |`Optimization` · `Skill Learning` · `Memory`|[](https://arxiv.org/abs/2605.27366)[](papers/arxiv/2026-05/muse-autoskill.md)|
355
357
| 2026-05 |**Do Proactive Agents Need an LLM to Decide When to Act?**| arXiv 2605 |`Intervention Timing` · `Optimization` · `Tool Use`|[](https://arxiv.org/abs/2605.30152)[](papers/arxiv/2026-05/do-proactive-agents-need-llm-wake-anchor.md)|
-`STREAMING.md`: model/framework mechanisms, release status, capability routes, and evidence boundaries.
31
31
-`PROJECTS.md`: assistants, implementations, products, and supporting components.
32
-
-`website/content/selected.json`: balanced homepage picks (two agents, two streaming models, two benchmarks), short navigation descriptions, original figure URLs, source attribution, and alternate text.
32
+
-`website/content/selected.json`: homepage picks across agents, models, and benchmarks, including PASK and PIRA-Bench / PIRF, short navigation descriptions, original figure URLs, source attribution, and alternate text.
33
33
34
-
Homepage selections use three explicit groups: Proactive Agents, Streaming Proactive Models, and Benchmarks, with two picks each. Official overview images retain their source links; a missing image does not exclude a representative work. Colored SVG icons and the clickable collection map provide visual navigation. The Observe → Decide → Assist / wait diagram is an editorial concept, not a reconstructed paper figure.
34
+
Homepage selections use three explicit groups: Proactive Agents, Models, and Benchmarks. PASK adds a latent-need/personal-memory system; PIRA-Bench / PIRF adds GUI intent recommendation and its reference framework. Official overview images retain their source links; a missing image does not exclude a representative work. SVG icons and the clickable collection map provide visual navigation. All directory pages share the same sidebar, with an icon for each entry and a standard current-page marker. Paper, project, and benchmark tags use small outlined boxes. Streaming uses the same outlined tag and `library/?tag=Streaming` filter as every topic. It has no separate metadata badge, top-level navigation item, sidebar item, or exclusive accent color. The existing `/streaming/` guide remains reachable from documentation links for compatibility. Proactive remains the primary visual emphasis. The Observe → Decide → Assist / wait diagram is an editorial concept, not a reconstructed paper figure.
35
35
36
36
The design adapts the MIT-licensed **Minted Directory Astro** directory template to this repository's static generator. See [website/TEMPLATE.md](website/TEMPLATE.md) for the upstream source, adapted components, and retained license. The website focuses on direct Paper / Repo / Project / Model / Dataset navigation, category browsing, and selected overview figures.
0 commit comments