Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions docs/accessibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@

Switchify Remote is designed for VoiceOver, TalkBack, iOS Switch Control, and Android Switch Access.

## Custom button layouts
## Custom section layouts

Mouse, Typing, and Window offer Edit layout when movement repeat, dragging, held modifiers, and live Enter delivery are inactive. The editor supports long-press dragging and cell actions for moving, swapping, adding, and removing buttons without sending PC commands. Select a cell to insert or remove a row or column at that position. Occupied row/column removal asks for confirmation.
Normal remote use hides section edit buttons and editing-only restriction text. The Layout edit mode toggle sits beside Surface in the sticky toolbar on Mouse, Typing, and Window. It starts off, exposes its selected state and a descriptive hint, and changes from Edit layout to Done editing when enabled. The toolbar wraps when width or enlarged text needs another line; all targets remain at least 48 points. Forwarding and disconnected/recovering screens omit the toggle. Edit mode is temporary, shared across customizable surfaces for the connected PC, and resets after disconnection or changing PCs; it is not persisted. Toggling it does not remount typing inputs, send commands, or save/reset layouts.

Layouts retain explicit positions across rotation and text scaling. Labels wrap, rows grow vertically, and targets remain at least 48 points. Empty cells and row wrappers add no scan stops during normal use; buttons scan in row-major order. Unavailable capabilities keep their disabled positions. The typing field, mode selector, status and recovery actions, and Stop movement remain outside the editable grid.
While edit mode is enabled, each visible section exposes an Edit section action. Those actions remain disabled during movement repeat, dragging, held modifiers, or live Enter delivery, with the existing restriction explanation. Sections retain their headings, cards, help text, dynamic status, order, and capability/mode visibility. Draft text, live input, recovery actions, and Stop movement remain outside editable grids.

The editor is the only accessibility context while open. Cell actions restore focus to the edited cell, moves announce their destination once, and dismissing the editor returns focus to Edit layout. All operations must be possible with TalkBack, VoiceOver, Switch Access, and Switch Control without dragging. Save persists locally; Cancel discards only after confirmation when changes exist. Reset to default takes effect on Save.
Each section editor offers the shared remote-action catalog, with one copy of each action per section (copies in other sections are allowed). Draft actions can only be placed on Typing. Tap an empty cell to open Choose action; search descriptive names, categories, and keywords. Pointer movement and arrow keys have distinct names. Unsupported actions remain selectable with an explanation and are disabled at runtime. Selecting immediately assigns the action to the draft cell, closes the popup, announces its position, and restores cell focus; it never executes a command. Close, scrim tap, Android Back, and accessibility escape dismiss without assignment. While moving a button, an empty-cell tap completes the move instead. Button drags swap occupied cells or move into empty cells. Row and column handles move complete tracks, including empty cells, with insertion indicators and edge scrolling. Use row and column handles to insert before/after or remove a row/column. Occupied removal asks for confirmation. Limits are 20 rows and four columns, with at least one of each.

The picker hides the underlying editor from touch and accessibility navigation. Its keyboard-aware dialog scrolls all content, including search and Close, for landscape and large text. Android search disables full-screen keyboard extraction so results remain reachable, and the Remote scroll container lets result taps through on the first press. Stop movement and repeat status are available on every customizable surface.

All operations have tap-based alternatives: select a button or track, choose Move, and select its destination cell or track. The editor is the only accessibility context while open. Labels identify row and column positions, moves announce their destination once, and closing returns focus to the section’s Edit action. Validate TalkBack, VoiceOver, Switch Access, and Switch Control without dragging. Empty runtime cells preserve spacing without becoming scan stops. Controls remain at least 48 points, labels wrap, and horizontal scrolling preserves custom columns at large text sizes and on narrow screens. Rotation, text resizing, backgrounding, and cancelled/outside drops must not commit a drag.

Save persists only the selected section and preserves neighboring sections. An unchanged Save leaves adaptive defaults intact. Cancel asks before discarding edits; Reset restores the section’s original responsive layout only on Save. Failed saves retain the draft and previous saved value for retry. Editing dispatches no PC commands and preserves live typing text.

- Every interactive target is at least 48 by 48 logical points and has a concise accessible name.
- First-run setup explains the Remote before asking for Bluetooth. Its two steps expose headings and "Step 1 of 2"/"Step 2 of 2" announcements in logical reading order, remain scrollable at large text sizes, and never move focus to a permission prompt until Allow Bluetooth is selected.
Expand Down
26 changes: 23 additions & 3 deletions docs/physical-smoke-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,27 @@

Record the app commit, Switchify PC release, phone model/OS, and desktop platform for each run. Never paste pairing credentials or typed personal content into the record.

## Button layout editor checks
## Layout edit mode checks

On Android and iOS, customize Mouse, Typing, and Window at 100%, 150%, and 200% text in both themes and orientations. Move into empty cells, swap occupied cells, drag near scroll edges, and rotate during a drag. Confirm cancelled drags do not change the grid. Insert and remove rows and columns, cancel an occupied deletion, restore a removed button, Save, restart, and verify positions. Confirm Reset returns to the original arrangement only after Save and Cancel preserves the saved layout.
Start a connected remote in normal mode. Confirm section headings, cards, and remote controls are present while section edit buttons and editing-only restrictions are absent. Toggle Edit layout beside Surface, verify selected state/Done editing and section edit actions, then use Done editing to hide them. Scroll and confirm the toggle stays with the pinned selector. Repeat in portrait/landscape and large text on phone/tablet widths, checking wrapping and 48-point targets with TalkBack, VoiceOver and switch navigation. Forwarding and disconnected/recovering screens must omit the toggle. Reconnect or change PCs and confirm edit mode starts off.

Repeat editing without gestures using TalkBack, VoiceOver, Switch Access, and Switch Control. Confirm modal containment, cell labels, row-major scanning, destination announcements, focus return, complete labels, and 48-point targets. Check the last row clears system navigation. Confirm editing sends no PC input, retains live typing text, and is unavailable during active repeat, drag, modifiers, or Enter delivery. Verify Stop movement and typing recovery actions remain available after customization. Reconnect to a PC with fewer capabilities and verify unavailable buttons stay disabled in their saved positions.
Toggle without changing/sending live or draft typing content, executing PC input, or modifying saved layouts/preferences. Verify section editors retain their Save/Cancel/Reset behavior and focus return while the mode stays enabled. During repeat, drag, held modifiers or live Enter delivery, turn the mode on and confirm section edit actions remain disabled with explanations; Stop movement and recovery stay available.

## Searchable action picker checks

Enable Edit layout, open a section editor, then tap an empty cell and confirm Choose action identifies the destination. Search mixed-case names, categories, and keywords; distinguish pointer movement from arrow keys. Select an unsupported action, confirm its explanation, immediate assignment and focus return, then Save and verify the runtime button stays disabled. Confirm no commands execute during selection. Test no matches, all actions placed, Close, scrim, Android Back and accessibility escape. A move-mode destination must move the button without opening the picker.

With the software keyboard open, both orientations, and 100%, 150%, and 200% text, scroll to all results and Close. Check TalkBack/VoiceOver focus containment and assignment announcements, Switch Access/Switch Control navigation, and focus restoration to the filled cell. Place monitor, movement/scroll, modifier, window and key actions on another surface; verify current capabilities/labels after reconnect, normal key commands outside live Typing, live Enter submission, and Stop movement on Typing and Window. Draft actions placed in PC keys must be disabled in live mode. Preserve active user typing/layout drafts by using an isolated simulator or a separate test installation.

## Section layout editor checks

On Android and iOS, edit each visible section of Mouse, Typing, and Window independently at 100%, 150%, and 200% text, in both themes and orientations. Verify fixed section boundaries: Movement, Clicks and scroll, Pointer speed, monitors; Draft actions and PC keys; Modifiers, Windows, Shortcuts, monitors. Opening and saving untouched defaults must not change their responsive grids.

Drag buttons to empty and occupied cells. Drag whole rows and columns forwards and backwards using the handles, including tracks with empty cells. Check insertion indicators and both edge-scroll directions. Release outside, cancel, rotate, resize text, and background during a drag; none may commit the move. Insert rows and columns before/after and at the end, confirm occupied deletion, restore removed buttons, and verify already-placed actions are excluded while actions from other surfaces are offered. Draft actions must be absent on Mouse and Window.

Save one section, restart, and verify its exact grid plus all neighboring sections, cards, headings, help text and dynamic status. Reset restores only that section after Save; Cancel preserves its saved layout. Test saved geometry and horizontal scrolling on narrow screens. Hide/show Draft actions by changing typing mode; reconnect with fewer capabilities and return to the original PC; hidden section layouts must survive.

Repeat all editing without gestures using TalkBack, VoiceOver, Switch Access, and Switch Control. Confirm modal containment, descriptive cell/track labels, logical scanning, destination announcements, focus return, complete labels, and 48-point targets. Confirm editing sends no PC input, retains live typing text, and is unavailable during active repeat, drag, modifiers, or Enter delivery. Stop movement and typing recovery remain outside customization.

Run the matrix on a physical Android phone and iPhone against current Switchify PC on both Windows and macOS:

Expand Down Expand Up @@ -36,3 +52,7 @@ Run the matrix on a physical Android phone and iPhone against current Switchify
10. Export diagnostics and verify that no typed content, token, authentication proof, nonce, or verification code appears.

The development-preview PR remains draft until all four platform pairings are recorded successfully.

### Movement width regression

On Android and iOS, check a saved three-column Movement grid at large text in portrait and landscape. When Mouse sections stack, Movement must use the available content width so its last column is visible when the grid fits. Narrow screens must still allow horizontal scrolling without changing saved rows, columns, or actions. At normal text size on wide screens, retain the two-pane arrangement.
6 changes: 5 additions & 1 deletion docs/protocol-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Switchify Remote is a protocol v1 client. It does not change the Bluetooth service, characteristic UUIDs, framed transport, desktop pairing records, or command schema.

Custom button layouts are local presentation data in the separate `switchify.remote.layouts.v1` storage key. The versioned record contains only surface names, control IDs, columns, and cells. Existing preference and pairing keys remain readable without migration. Missing, malformed, oversized, unknown-version, or unrecognized-control layouts use the default presentation. Capability changes disable saved controls in place; layouts never store command payloads, text, or authentication material. Older app builds ignore the new key.
Custom section layouts use the separate `switchify.remote.layouts.v2` storage key with `{ version: 2, layouts: { [surface]: { [section]: { columns, cells } } } }`. Fixed section names are unchanged. All existing action IDs and v2 grids remain supported; action IDs are now validated against the shared catalog and surface placement rules instead of their original section. Draft actions are accepted only on Typing. An action may appear once in each section, including sections on different surfaces. Missing, malformed, oversized, unknown-version, duplicate-control, or unknown-action or forbidden-placement section data falls back independently to original section presentation. Version 1 flat layouts are deliberately ignored: their format cannot preserve section boundaries. Existing preference and pairing keys are unchanged, and old builds ignore the v2 key. Hidden capability/mode sections retain saved grids; visible controls continue to use current capability checks and dynamic labels. Records contain no command payloads, text, or authentication material.

The TypeScript compatibility suite carries the canonical authentication and pairing-code vectors from Switchify Android. Android requests the established 517-byte MTU, while both platforms adapt the inner frame payload so the encoded GATT value fits the negotiated ATT limit. Transport tests enforce the 160-byte maximum inner payload, 16 KiB message limit, 10-second partial timeout, duplicate and out-of-order handling, UTF-8 reassembly, response correlation, and sanitized failure behavior.

Expand All @@ -18,3 +18,7 @@ The active preview surface supports:
Switch profiles, system-wide Switch Forwarding, media controls, accounts, and subscriptions are outside this preview. Unknown capabilities use safe disabled defaults.

Authenticated `connection.ping` commands may include an optional `deviceName`. Current PC builds use it to refresh the saved display name for that authenticated device. Older PC builds ignore the extra payload field, and older Remote builds remain compatible because an empty ping is still valid. A sanitized `name_update_failed` response does not fail authentication; Remote retains the local name and retries it on a later connection. The name does not change the device ID, token, BLE identity, or pairing authorization.

## Shared remote actions

The catalog contains only stable IDs, descriptive searchable metadata, placement rules, and declarative action definitions. The runtime resolver attaches current platform labels, selected/disabled states, explanations, and callbacks from the active RemoteSession. Both original and customized grids use it. Monitor behavior is defined once; keys in live Typing use its stream and Enter controller, while keys elsewhere use normal PC key commands. Draft actions require draft mode and applicable text/capabilities. Picker selection handles IDs only. Layout storage contains no handlers, search text, typing content, or command payloads. Reconnection refreshes runtime state without rewriting layouts.
13 changes: 8 additions & 5 deletions src/app/(tabs)/remote.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import { MouseSurface } from '@/remote/MouseSurface';
import { DisconnectedRemote } from '@/remote/DisconnectedRemote';
import { RemoteDeviceSwitcher } from '@/remote/RemoteDeviceSwitcher';
import { RemoteSession } from '@/remote/RemoteSession';
import { SurfaceSelector } from '@/remote/SurfaceSelector';
import { RemoteToolbar } from '@/remote/RemoteToolbar';
import { LayoutEditModeProvider } from '@/layouts/LayoutEditMode';
import { TypingSurface } from '@/remote/TypingSurface';
import { WindowSurface } from '@/remote/WindowSurface';
import { profilePresentation } from '@/remote/profilePresentation';
Expand Down Expand Up @@ -48,11 +49,13 @@ export default function RemoteScreen() {
return <Screen title="Remote" bottomAccessory={deviceSwitcher}><EmptyState icon={unavailablePresentation.icon} title={unavailablePresentation.title} body={unavailablePresentation.body} /></Screen>;
}
return (
<Screen title="Remote" headerAccessory={<StatusBadge icon="check-circle" label={`Connected · ${connection.desktop.displayName}`} tone="success" />} bottomAccessory={deviceSwitcher} scrollToTop stickyAccessory={<SurfaceSelector selected={preferences.surface} />}>
{preferences.surface === 'mouse' ? <MouseSurface session={session} state={sessionState} physicalSwitchStopAvailable={bridgeSnapshot.captureAvailable && bridgeSnapshot.externalSwitches.length > 0} /> : null}
{preferences.surface === 'typing' ? <TypingSurface session={session} mode={preferences.typingMode} draft={preferences.draft} /> : null}
{preferences.surface === 'window' ? <WindowSurface session={session} state={sessionState} platform={connection.desktop.platform} /> : null}
<LayoutEditModeProvider key={desktopId}>
<Screen title="Remote" keyboardShouldPersistTaps="handled" headerAccessory={<StatusBadge icon="check-circle" label={`Connected · ${connection.desktop.displayName}`} tone="success" />} bottomAccessory={deviceSwitcher} scrollToTop stickyAccessory={<RemoteToolbar selected={preferences.surface} />}>
{preferences.surface === 'mouse' ? <MouseSurface platform={connection.desktop.platform} session={session} state={sessionState} physicalSwitchStopAvailable={bridgeSnapshot.captureAvailable && bridgeSnapshot.externalSwitches.length > 0} /> : null}
{preferences.surface === 'typing' ? <TypingSurface platform={connection.desktop.platform} physicalSwitchStopAvailable={bridgeSnapshot.captureAvailable && bridgeSnapshot.externalSwitches.length > 0} session={session} mode={preferences.typingMode} draft={preferences.draft} /> : null}
{preferences.surface === 'window' ? <WindowSurface physicalSwitchStopAvailable={bridgeSnapshot.captureAvailable && bridgeSnapshot.externalSwitches.length > 0} session={session} state={sessionState} platform={connection.desktop.platform} /> : null}
{preferences.surface === 'forwarding' ? <ForwardingSurface manager={manager} bridge={bridge} profile={connection.profile} desktopId={connection.desktop.desktopId} preferences={preferences} restore={forwardingRestore} /> : null}
</Screen>
</LayoutEditModeProvider>
);
}
Loading