Skip to content

Commit 51501dc

Browse files
feat: Implement Obsidian-like Canvas Feature in Notes System (#12)
* feat(notes): validate and persist JSON Canvas documents Add the canvas note type, initialize empty documents, preserve canvas filenames, and validate JSON Canvas content before writes. Cover format validation and all notes scopes with backend tests. * build(frontend): add canvas editing and rendering dependencies Add React Flow, Markdown rendering, HTML sanitization, and image export packages. Declare the peer packages needed by the existing chart and testing dependencies. * feat(canvas): add format, reference, and export utilities Validate and preserve JSON Canvas data, resolve stable file references, and support geometry, duplication, arrangement, and portable Markdown exports. Include unit coverage for interchange and invalid input. * feat(canvas): implement the visual canvas editor Add cards, connections, groups, nested previews, linked-note editing, navigation, selection, history, and layout controls. Support media import and canvas, PNG, and file-bundle exports. * feat(canvas): autosave documents and recover unsaved drafts Serialize writes per document, flush pending changes on tab close, and recover browser-backed drafts. Test save failures, retries, concurrent edits, and exhausted browser storage. * feat(notes): integrate canvases into explorers and note links Enable canvas creation and editing across library, project, experiment, and item notes. Add rich-text canvas embeds, wiki-link suggestions, graph references, and backlinks; fix library folder creation and test shared-panel creation. * test(canvas): cover editing and persistence in the browser Exercise directory and project creation, tab switches and reloads, nested references, linked-note editing, connections, grouping, history, media, import validation, read-only behavior, and downloads. * docs(canvas): document workflows and compatibility boundaries Document canvas note APIs, editing controls, persistence, exports, and interoperability limits. Link the canvas guide from the documentation index. * fix(canvas): allow dragging cards from their preview bodies Keep editors, links, and media controls interactive while enabling dragging from both headers and preview content. Add browser regressions for pointer tracking on selected and unselected cards and for text selection, and update the usage guide. * style(canvas): shrink minimap to 120 by 80 pixels
2 parents de2f1f6 + 86decbb commit 51501dc

27 files changed

Lines changed: 3397 additions & 138 deletions

‎backend/models/canvas.py‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
"""JSON Canvas 1.0 validation; retain extension fields for lossless round trips."""
2+
import json
3+
import math
4+
import re
5+
6+
7+
def validate_canvas(content: str) -> str:
8+
try:
9+
value = json.loads(content)
10+
except (ValueError, TypeError) as exc:
11+
raise ValueError("Canvas content must be valid JSON") from exc
12+
if not isinstance(value, dict):
13+
raise ValueError("Canvas must be a JSON object")
14+
nodes, edges = value.get("nodes", []), value.get("edges", [])
15+
if not isinstance(nodes, list) or not isinstance(edges, list):
16+
raise ValueError("Canvas nodes and edges must be arrays")
17+
ids = set()
18+
19+
def text(item, key, required=False):
20+
if required or key in item:
21+
if not isinstance(item.get(key), str):
22+
raise ValueError(f"Canvas {key} must be a string")
23+
24+
def identity(item):
25+
if not isinstance(item, dict) or not isinstance(item.get("id"), str) or not item["id"]:
26+
raise ValueError("Canvas elements require a nonempty string id")
27+
if item["id"] in ids:
28+
raise ValueError("Canvas element ids must be unique")
29+
ids.add(item["id"])
30+
if "color" in item and (not isinstance(item["color"], str) or not re.fullmatch(r"[1-6]|#[0-9a-fA-F]{6}", item["color"])):
31+
raise ValueError("Canvas color must be a preset 1–6 or #RRGGBB")
32+
33+
for node in nodes:
34+
identity(node)
35+
if node.get("type") not in ("text", "file", "link", "group"):
36+
raise ValueError("Unknown canvas node type")
37+
for key in ("x", "y", "width", "height"):
38+
number = node.get(key)
39+
if isinstance(number, bool) or not isinstance(number, (int, float)) or not math.isfinite(number) or number != int(number):
40+
raise ValueError(f"Canvas {key} must be a finite integer")
41+
if key in ("width", "height") and number <= 0:
42+
raise ValueError("Canvas dimensions must be positive")
43+
if node["type"] == "text":
44+
text(node, "text", True)
45+
elif node["type"] == "file":
46+
text(node, "file", True)
47+
text(node, "subpath")
48+
if "subpath" in node and not node["subpath"].startswith("#"):
49+
raise ValueError("Canvas subpath must start with #")
50+
elif node["type"] == "link":
51+
text(node, "url", True)
52+
else:
53+
text(node, "label")
54+
text(node, "background")
55+
if "backgroundStyle" in node and node["backgroundStyle"] not in ("cover", "ratio", "repeat"):
56+
raise ValueError("Invalid canvas background style")
57+
node_ids = set(ids)
58+
for edge in edges:
59+
identity(edge)
60+
for key in ("fromNode", "toNode"):
61+
text(edge, key, True)
62+
if edge[key] not in node_ids:
63+
raise ValueError("Canvas edges must reference existing nodes")
64+
for key in ("fromSide", "toSide"):
65+
if key in edge and edge[key] not in ("top", "right", "bottom", "left"):
66+
raise ValueError("Invalid canvas connection side")
67+
for key in ("fromEnd", "toEnd"):
68+
if key in edge and edge[key] not in ("none", "arrow"):
69+
raise ValueError("Invalid canvas connection endpoint")
70+
text(edge, "label")
71+
return content

‎backend/models/note.py‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
from typing import Optional
1+
from typing import Literal, Optional
2+
from pydantic import model_validator
3+
from .canvas import validate_canvas
24
from .base import CamelModel
35

46

@@ -12,7 +14,7 @@ class Note(CamelModel):
1214
experiment_id: Optional[str] = None
1315
name: str
1416
parent_id: Optional[str] = None
15-
type: str = "file" # file | folder
17+
type: str = "file" # file | folder | canvas
1618
content: str = ""
1719
is_pinned: bool = False
1820
created_at: str
@@ -22,9 +24,17 @@ class Note(CamelModel):
2224
class NoteCreate(CamelModel):
2325
name: str
2426
parent_id: Optional[str] = None
25-
type: str = "file"
27+
type: Literal["file", "folder", "canvas"] = "file"
2628
content: str = ""
2729

30+
@model_validator(mode="after")
31+
def validate_document(self):
32+
if self.type == "canvas":
33+
self.content = validate_canvas(self.content or '{"nodes":[],"edges":[]}')
34+
if not self.name.lower().endswith(".canvas"):
35+
self.name += ".canvas"
36+
return self
37+
2838

2939
class NoteUpdate(CamelModel):
3040
name: Optional[str] = None

‎backend/services/note_service.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@
1010
from services.cost_service import record_openai_usage
1111

1212
from models.note import Note, NoteCreate, NoteUpdate
13+
from models.canvas import validate_canvas
14+
from fastapi import HTTPException
1315
from models.note_batch import ItemNotes, NoteBatchRequest
1416
from services.db import get_client
1517

@@ -124,6 +126,14 @@ def update_note(note_id: str, data: NoteUpdate) -> Optional[Note]:
124126
return None
125127
# exclude_unset (not exclude_none) so callers can explicitly set fields to null
126128
updates = data.model_dump(exclude_unset=True)
129+
if existing.type == "canvas":
130+
if "content" in updates:
131+
try:
132+
validate_canvas(updates["content"])
133+
except ValueError as exc:
134+
raise HTTPException(status_code=422, detail=str(exc)) from exc
135+
if updates.get("name") and not updates["name"].lower().endswith(".canvas"):
136+
updates["name"] += ".canvas"
127137
# When reassigning to a different source, null out all other source fields so
128138
# a note always belongs to exactly one source at a time.
129139
if any(f in updates for f in _SOURCE_FIELDS):

‎backend/tests/test_canvas.py‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
import json
2+
from unittest.mock import Mock
3+
4+
import pytest
5+
from pydantic import ValidationError
6+
7+
from models.canvas import validate_canvas
8+
from models.note import Note, NoteCreate, NoteUpdate
9+
from services import note_service
10+
11+
12+
def node(**updates):
13+
return {"id": "n1", "type": "text", "x": 0, "y": 0, "width": 300, "height": 200, "text": "Hello", **updates}
14+
15+
16+
def test_canvas_creation_defaults_and_extensions():
17+
data = NoteCreate(name="Map", type="canvas")
18+
assert data.name == "Map.canvas"
19+
assert json.loads(data.content) == {"nodes": [], "edges": []}
20+
content = json.dumps({"nodes": [node(researchos={"custom": True})], "edges": [], "vendor": {"future": 1}})
21+
assert NoteCreate(name="Map.canvas", type="canvas", content=content).content == content
22+
23+
24+
@pytest.mark.parametrize("value", [None, [], {"nodes": None}, {"nodes": [node(), node()]}, {"nodes": [node(width=0)]}, {"nodes": [node(x=True)]}, {"nodes": [node(x=0.5)]}, {"nodes": [node(color="red")]}, {"nodes": [node(type="file", file="a.md", subpath="heading")]}, {"nodes": [node()], "edges": [{"id": "e", "fromNode": "n1", "toNode": "missing"}]}])
25+
def test_invalid_canvas_is_rejected(value):
26+
with pytest.raises(ValueError):
27+
validate_canvas(json.dumps(value))
28+
29+
30+
def test_optional_arrays_and_all_node_types():
31+
assert validate_canvas('{}') == '{}'
32+
data = {"nodes": [node(), node(id="f", type="file", file="Nested.canvas", subpath="#Heading"), node(id="l", type="link", url="https://example.com"), node(id="g", type="group", label="Group", background="image.png", backgroundStyle="ratio")], "edges": [{"id": "edge", "fromNode": "f", "toNode": "l", "fromSide": "bottom", "toSide": "top", "fromEnd": "arrow", "toEnd": "none", "color": "#123456", "label": "reference"}]}
33+
assert json.loads(validate_canvas(json.dumps(data))) == data
34+
35+
36+
def test_canvas_create_api_rejects_bad_json_before_write(client, monkeypatch):
37+
create = Mock()
38+
monkeypatch.setattr(note_service, 'create_note', create)
39+
result = client.post('/api/libraries/lib/notes', json={"name": "Bad", "type": "canvas", "content": "invalid"})
40+
assert result.status_code == 422
41+
create.assert_not_called()
42+
43+
44+
def test_update_rejects_invalid_content_before_database_write(monkeypatch):
45+
existing = Note(id="c", name="Map.canvas", type="canvas", content='{}', created_at="now", updated_at="now")
46+
monkeypatch.setattr(note_service, "get_note", lambda _: existing)
47+
db = Mock()
48+
monkeypatch.setattr(note_service, "get_client", lambda: db)
49+
from fastapi import HTTPException
50+
with pytest.raises(HTTPException) as error:
51+
note_service.update_note("c", NoteUpdate(content="<p>Accidental HTML overwrite</p>"))
52+
assert error.value.status_code == 422
53+
db.table.assert_not_called()
54+
55+
56+
@pytest.mark.parametrize('scope,identifier', [('paper_id', 'paper'), ('website_id', 'site'), ('github_repo_id', 'repo'), ('library_id', 'library'), ('project_id', 'project'), ('experiment_id', 'experiment')])
57+
def test_canvas_persists_in_every_notes_scope(monkeypatch, scope, identifier):
58+
db = Mock()
59+
monkeypatch.setattr(note_service, 'get_client', lambda: db)
60+
created = note_service.create_note(NoteCreate(name='Map', type='canvas', parent_id='folder'), **{scope: identifier})
61+
assert created.type == 'canvas'
62+
assert getattr(created, scope) == identifier
63+
saved = db.table.return_value.insert.call_args.args[0]
64+
assert saved['parent_id'] == 'folder'
65+
assert json.loads(saved['content']) == {'nodes': [], 'edges': []}
66+
67+
68+
def test_existing_note_html_is_unchanged():
69+
assert NoteCreate(name='Note', content='<p>Hello</p>').content == '<p>Hello</p>'
70+
with pytest.raises(ValidationError):
71+
NoteCreate(name='Invalid', type='unknown')

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## For Users
44

5+
- [Canvas documents](canvas.md) — Visual notes, nested canvases, file cards, and portable exports
56
- [User Guide](user-guide.md) — Walkthrough: first papers to projects to AI features
67
- [Costs & Privacy](costs-and-privacy.md) — What goes to OpenAI, cost estimates, data ownership
78
- [FAQ & Troubleshooting](faq.md) — Common questions and fixes

‎docs/api/notes.md‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Notes API
22

3-
Notes implement a file-tree system. Each note is either a `file` (with HTML content) or a `folder`. Notes belong to exactly one parent scope.
3+
Notes implement a file-tree system. Each entry is a `file` (with HTML content), a `canvas` (with JSON Canvas content), or a `folder`. Notes belong to exactly one parent scope.
44

55
---
66

@@ -90,12 +90,31 @@ Request body (`NoteCreate`):
9090
}
9191
```
9292

93-
- `type`: `"file"` or `"folder"`. Folders have empty `content`.
93+
- `type`: `"file"`, `"folder"`, or `"canvas"`. Folders have empty `content`.
9494
- `parentId`: ID of parent note (folder) for nested structure; `null` for root-level.
95-
- `content`: HTML string (tiptap editor output). Empty string for folders.
95+
- `content`: HTML string for files, a serialized JSON Canvas 1.0 object for canvases, or an empty string for folders.
9696

9797
Returns the created Note at `201`.
9898

99+
### Canvas documents
100+
101+
The same creation endpoint works in every scope:
102+
103+
```json
104+
{
105+
"name": "Research map.canvas",
106+
"parentId": null,
107+
"type": "canvas",
108+
"content": "{\"nodes\":[],\"edges\":[]}"
109+
}
110+
```
111+
112+
Omitting canvas content initializes empty `nodes` and `edges` arrays. Creation and rename append `.canvas` if absent. Save using `PATCH /api/notes/{id}` with the serialized document in `content`. The type of an existing entry is not changed by PATCH.
113+
114+
Canvas creation and content updates validate JSON Canvas node types, geometry, unique IDs, colors, file subpaths, and connection endpoints. Invalid content returns `422` before any database write. Unknown extension fields are retained. Existing HTML note behavior is unchanged. No database migration is needed: the existing `notes.type` and `notes.content` columns store the document.
115+
116+
ResearchOS-created file nodes retain their portable `file` path and add `researchos.noteId` for stable references across renames and moves. Local attachments may include `researchos.mediaData` or `researchos.textContent`; group backgrounds may include `researchos.backgroundData`. These optional extensions do not alter the JSON Canvas schema. See [Canvas guide](../canvas.md) for exports and interoperability.
117+
99118
---
100119

101120
## Update Note

‎docs/canvas.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Canvas documents
2+
3+
Use **New canvas** in a notes explorer, or right-click a folder and choose **New canvas**. Canvases appear alongside notes and folders in library, paper, website, repository, project, and experiment notes. Names end in `.canvas`; storage uses the existing notes API and database, with no migration or new service required.
4+
5+
The editor stores [JSON Canvas 1.0](https://jsoncanvas.org/spec/1.0/), the open format used by [Obsidian Canvas](https://obsidian.md/help/plugins/canvas). It preserves node ordering, standard optional fields, and extension fields on import and export.
6+
7+
## Cards and connections
8+
9+
- **Text** adds a Markdown card. Double-click to edit. Text supports headings, lists, tasks, links, code, tables, and images. **Convert to note** creates a rich-text note and replaces the card with a reference.
10+
- **File / canvas** adds a note or another canvas. Pick a folder to add its files, or drag entries from the explorer. Referenced notes show live content. **Edit linked note** edits the original through the existing rich-text editor; **Open file** opens its tab; **Swap file** relinks the card. **Heading / block** accepts `#Heading` or `#^block-id`.
11+
- Nested canvases show previews, with a depth limit and circular-reference detection. Open the nested canvas to continue navigating. Missing or ambiguous file paths show an unavailable state and can be relinked.
12+
- **Media** imports images, audio, video, PDF, or text files; local files can also be dropped. Each attachment is limited to 15 MB. Imported `.canvas` attachments become canvas documents in the current notes folder. Remote media can be referenced by an imported file node's HTTP(S) URL.
13+
- **Web page** embeds an HTTP(S) page in a sandboxed frame. The inspector can change or copy its URL and reload it. Some sites block embedding; their external link remains available.
14+
- **Group** creates a visual container around selected cards, or an empty group. Moving a group carries its contained cards. Set its label, color, and image background, with cover, aspect-ratio, or repeat rendering.
15+
- Drag a side handle to another card to connect them. Dropping a new connection on empty space creates a text card. Connections support labels, colors, side selection, and arrows at either or both ends. Drag endpoints to reconnect; drop an existing endpoint on empty space to disconnect. **Follow start/end** navigates to either card.
16+
17+
## Editing and navigation
18+
19+
Drag a card's header or preview body to move it, and selected card borders to resize. Move groups by their headers. Text editors, links, and media controls retain their normal interactions. Drag blank space for a selection box, Shift-click for multiple selection, or use Ctrl/Command+A. Alt-drag leaves a duplicate at the original position. The inspector offers duplication, deletion, layer ordering, grouping, six alignments, row/column/grid layouts, spacing distribution, and horizontal/vertical justification.
20+
21+
Use Space-drag or the middle mouse button to pan, wheel to scroll, Ctrl/Command-wheel or pinch to zoom, and the minimap to navigate. The controls include fit-all, fit-selection, 100% zoom, group navigation, and card text search. Settings expose grid snapping, object snapping, resize aspect ratio, minimap visibility, and read-only mode.
22+
23+
| Shortcut | Action |
24+
| --- | --- |
25+
| Ctrl/Command+S | Save canvas |
26+
| Ctrl/Command+Z | Undo |
27+
| Ctrl/Command+Y or Ctrl/Command+Shift+Z | Redo |
28+
| Ctrl/Command+D | Duplicate selection |
29+
| Ctrl/Command+C / V | Copy/paste cards within the open canvas |
30+
| Delete / Backspace | Remove selection and its connections |
31+
| Arrow keys | Move selected cards |
32+
| Shift+arrow | Move farther |
33+
| Shift+1 / Shift+2 | Fit all / fit selection |
34+
| Escape | Finish text editing or clear selection |
35+
36+
Undo history holds the last 100 canvas operations and resets when the editor closes. Read-only mode prevents document changes while retaining navigation and export.
37+
38+
## Saving and references
39+
40+
Changes autosave after a short debounce. Writes are serialized per document, and switching tabs flushes pending changes. The directory receives each draft immediately. Unsaved content is also backed up in browser storage when available; a failed save shows **Retry save**, and reopening a canvas recovers its draft. Browser storage limits can prevent large media drafts from being backed up; API saving still works. The application warns before leaving with a pending save.
41+
42+
Internal references use the note ID as an optional `researchos.noteId` extension. Renaming or moving a referenced note keeps the reference intact. File paths refresh on export. Imported paths resolve within the current source first, then across loaded sources if unique. A deleted ID does not silently resolve to a different file with the same name. References resolve only within notes loaded by the current library/project/panel.
43+
44+
Canvases participate in directory search, pinning, recent notes, graph links, and backlinks. In a rich-text note, use **Embed canvas** or type `![[Name.canvas]]` to insert a preview. Ordinary wiki links can also target canvases.
45+
46+
## Import and export
47+
48+
- **Import .canvas** validates and replaces the current canvas, asking before replacing a nonempty document. Undo restores the previous document. Invalid imports leave it untouched. The import limit is 60 MB.
49+
- **Download .canvas** downloads the JSON document, including ResearchOS extensions. Other JSON Canvas applications use its standard file paths and ignore extensions they do not understand.
50+
- **Canvas + files (.zip)** packages the canvas, referenced notes and nested canvases, and local attachments/backgrounds. It strips internal note IDs so paths can resolve in another application. Export stops with a clear error if distinct files would overwrite the same archive path. Remote URLs remain references.
51+
- **PNG** exports the full canvas at 1× or 2×, the current viewport, or a privacy image hiding text. Very large exports are capped at 64 megapixels; use the viewport or lower scale. Browser restrictions can prevent remote media or embedded pages from appearing in an image.
52+
53+
Attachments are stored in optional ResearchOS extensions inside the canvas, rather than separate backend uploads. Use ZIP export when moving media to Obsidian; a standalone `.canvas` file cannot supply separate attachment files to applications that ignore those extensions. Importing an Obsidian canvas does not grant this web application access to its local vault: matching notes must already exist in ResearchOS, or be added/relinked. ZIP export is supported; ZIP/vault import is not.
54+
55+
## Compatibility boundaries
56+
57+
This is an implementation of the JSON Canvas format and the core visual Canvas workflow, not the Obsidian application runtime. Obsidian Bases (`.base` databases), community plugin behaviors, vault filesystem services, Obsidian-specific commands/branding, and exact mobile long-press menus are not implemented. Generic file nodes referencing those files are preserved. Browser/touch interaction uses the graph editor's gestures; desktop and touch menus are not identical to Obsidian's.
58+
59+
Local SVG and HTML media are not executed. Web embeds run in sandboxed frames, HTML note previews are sanitized, and Markdown cards do not execute embedded HTML. Cyclic previews stop rather than recursively rendering forever.
60+
61+
## Development checks
62+
63+
From `frontend`, run `npm run test:run`, `npm run test:e2e -- canvas.spec.js`, and `npm run build`. The repository currently has a pre-existing KaTeX/Tiptap peer-version mismatch; dependency installation may require `npm install --legacy-peer-deps`. From `backend`, run `.venv/Scripts/python.exe -m pytest tests/test_canvas.py tests/test_note_batch.py tests/test_chat_and_notes_routes.py -q` on Windows.

0 commit comments

Comments
 (0)