Skip to content

Add Windows native on-screen keyboard scanning #787

Description

@enaboapps

Goal

Integrate Windows On-Screen Keyboard (osk.exe) with row-then-key switch scanning using the shared foundation delivered by the Mac keyboard issue. Deliver a separate Windows PR.

Behaviour

  • Keyboard menu tile and optional press/hold action open or reuse the keyboard and scan keys, preserving current editor focus without clicking the selected point.
  • Activate each visible enabled key once and return to rows. Back to scanning leaves the keyboard open; Close keyboard hides it; both return to normal armed idle. The Keyboard switch action during keyboard scanning performs Back.
  • Reuse colour, timing, manual/automatic movement, reverse, pause, holds and emergency stop. Native keyboard output must not recursively trigger mapped switches.
  • Discover keys with UI Automation, filter hidden duplicates, refresh geometry/layout and reject stale activation. Use native key invocation, with switch-accessible failures.
  • Preserve existing saved settings and local/Remote parity without BLE wire changes.

Qualification and validation

First prove native open/close, focus-preserving key invocation and generated-event identification using Switchify's signed UIAccess build in the Windows ARM VM. Ordinary-permission probing found no accessible keys. No administrator workaround, driver or security-policy change. Report blockers rather than substituting point scanning or ignoring all injected events.

Fake tests cover navigation, controls, timing, modifiers, repeats, layout changes, cancellation, failures and exactly-once execution. VM tests use disposable text and mapped Space/Enter/letters, re-entry, focus, resizing and displays. Restore settings, permissions, keyboard visibility and prior VM state; leave the signed local Mac app untouched.

Branch from current main, run required frontend/Rust checks, independent latest-head review and ready PR with passing CI. Merge/release require separate instructions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions