Use asyncio (2) by rwols · Pull Request #3021 · sublimelsp/LSP

rwols · 2026-09-04T18:18:28Z

This PR switches the codebase to using async def functions and asyncio. The loop provider is sublime_aio.

should be merged (and released) at the same time as:

The main driver for doing this is to decrease the thread usage of this plugin from O(n) to O(1) threads, where n is the number of language servers running. The secondary driver is syntax sugar.

Why is this PR so large? Please read: What color is your function?

Continuation of: #2880

… resolving

- Typo fixes: various typos fixed - Session starting logic: only partially. Sessions attach to listeners now, but, only one session starts while multiple should start. I think the solution is now to simply start all the `WindowManager.start` coroutines at the same time. They'll wait on each other via the `WindowManager._start_lock`. - Fix folding ranges using `send_request_async` while it should be using `send_request` (because it calls that from the main thread). - Requests seem to be generally working *provided pull diagnostics are not used*. Testing with clangd works, testing with pyright shows requests not working and something fundamental being stuck somewhere. Broken: - didOpen/didClose is sent two times - Workspace/pull diagnostics are broken - There's various `sublime.set_timeout_async` calls throughout the codebase, but we're now at the point where that's "wrong". I hope to find-and-replace these invocations with `sublime_aio.call_soon_threadsafe`.

…o.py

rchl · 2026-09-07T19:50:20Z

-    def purge_changes_async(self) -> None:
+    def purge_changes(self) -> asyncio.Future[list[BaseException | None]]:
        raise NotImplementedError

    @abstractmethod
-    def trigger_on_pre_save_async(self) -> None:
+    def trigger_on_pre_save(self) -> asyncio.Future[list[BaseException | None]]:


Is there specific reason for annotating the return value with asyncio.Future vs. just using async keyword on the method?

Some methods here do it one way and some do another.

The reason for returning asyncio.Future is that this function will unconditionally do that.

In a coroutine, say

async def trigger_on_pre_save(self) -> list[BaseException | None]: ...

you can conditionally return early, or conditionally await something and suspend. You need the function to be a coroutine function in that case.

But unconditionally, always, returning something awaitable, such a function does not have to be a coroutine function but can be a "simpler" regular function that returns the Future.

And why do that? What is the benefit of trigger_on_pre_save implementation returning a Future rather than awaiting it itself?

The benefit is that the typing is more precise...

I've converted the methods anyway.

rchl

With latest changes I'm seeing a ton of errors in the console like this one:

Exception in callback TaskContainer.create_task_and_wrap_in_promise.<locals>.executor_func.<locals>.on_asyncio_thread.<locals>.handle_on_done() at /Users/rafal/Library/Application Support/Sublime Text/Packages/LSP/plugin/core/aio.py:241
handle: <Handle TaskContainer.create_task_and_wrap_in_promise.<locals>.executor_func.<locals>.on_asyncio_thread.<locals>.handle_on_done() at /Users/rafal/Library/Application Support/Sublime Text/Packages/LSP/plugin/core/aio.py:241>
Traceback (most recent call last):
  File "./python3.14/asyncio/events.py", line 94, in _run
  File "/Users/rafal/Library/Application Support/Sublime Text/Packages/LSP/plugin/core/aio.py", line 243, in handle_on_done
    resolve(Error.from_exception(ex))
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rafal/Library/Application Support/Sublime Text/Packages/LSP/plugin/core/promise.py", line 185, in <lambda>
    executor_func(lambda resolve_value=None: self._do_resolve(resolve_value))
                                             ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/rafal/Library/Application Support/Sublime Text/Packages/LSP/plugin/core/promise.py", line 267, in _do_resolve
    raise RuntimeError("cannot set the value of an already resolved promise")
RuntimeError: cannot set the value of an already resolved promise

Not sure when are those triggered as I wasn't paying attention when those were posted but maybe it's clear from the stack trace what the problem is.

rchl · 2026-09-10T16:54:50Z

+        task.add_done_callback(on_done)
+        return task
+
+    def create_task_threadsafe(self, coro: Coroutine[object, object, object], name: str | None = None) -> None:


The name is never used (also in create_task). Is there any use case envisioned for it?

Yes, I'd like to give these tasks more descriptive names in the future.

Futhermore I think create_task should follow the signature of asyncio.create_task exactly. There is an additional keyword-only argument introduced in py 3.14 called eager_start that will allow the task to run the coroutine body eagerly. Which I'm intending to use in a few places.

rchl · 2026-09-10T18:01:31Z


    @abstractmethod
-    def read(self) -> JSONRPCMessage | None:
+    async def read(self) -> JSONRPCMessage | None:


StreamTransport implementation returns JSONRPCMessage so this should probably also be changed to do the same?

A function is covariant in its return type. So an overridden method of a subclass is allowed to return a more specific type than the base class. JSONRPCMessage is more specific than JSONRPCMessage | None.

But if there are no implementations that return None then it doesn't really make sense to define it as such. That just means that the logic has to cater for None even if it can't be returned in practice. If in the future there needs to be an implementation that returns None then than the base definition can be widened.

This should be satisfied now. Transport.read return type is JSONRPCMessage.

rchl · 2026-09-10T18:07:58Z

            try:
-                return socket.create_connection(('localhost', port))
+                reader, writer = await asyncio.wait_for(
+                    asyncio.open_connection(host='127.0.0.1', port=port), timeout=time_left


Binding to 127.0.0.1 instead of localhost (like before) has some functional differences. Did you change that on purpose and if so, why? Server might bind to IPv6 and then this will not work anymore while previously should.

Also equivalent change was done in TcpServerTransportConfig.

This was done for macOS. I can’t remember why exactly, but there is some ancient commit from 2017 that concluded the same thing. I’ll get back to you on this.

But the code before asyncio used localhost, even for MacOS, no?

Yeah it seems it was always localhost. When using localhost, then it might indeed bind to IPv6. If both IPv4 and IPv6 are enabled ("dual-stack") then asyncio does not raise ConnectionRefusedError but rather an OSError. I think we can use localhost and catch OSError.

rchl · 2026-09-12T18:46:34Z

+                    def handle_on_done(f: asyncio.Future[T]) -> None:
+                        if ex := f.exception():
+                            resolve(Error.from_exception(ex))


f.exception() raises when future was canceled. Should there be an additional case like:

if f.cancelled(): resolve(Error(LSPErrorCodes.RequestCancelled, "cancelled"))

?

rchl · 2026-09-12T18:56:32Z

-        inside_workspace = self._workspace.contains(listener.view)
-        scheme = parse_uri(listener.get_uri())[0]
-        for session in self._sessions:
-            if session.can_handle(listener.view, scheme, capability=None, inside_workspace=inside_workspace):


Session.can_handle is now dead code so should be removed.

But also, there is Session.state that is being set at different Session stages and it's no longer checked by anything. I think it's meant to be used so that views don't match session that is already shutting down.

rchl · 2026-09-12T19:03:12Z

+    async def _end_sessions(self, config_names: list[str] | None = None) -> list[Exception]:
+        coros = []
        for session in list(self._sessions):
            if config_names is None or session.config.name in config_names:
-                session.end_async()
+                debug(f"stopping {session.config.name}")
+                coros.append(session.end())
                self._sessions.discard(session)
+        return await gather_and_flatten_exceptions(*coros)


Feels like this should be guarded by _start_lock.
Sessions are discarded before end() coroutines run so concurrent start() could launch duplicate server during gather.

rchl · 2026-09-12T19:06:59Z

+    async def destroy(self) -> list[Exception]:
+        """Destroy everything related to this instance."""
+        result = await self._end_sessions()
        if self.panel_manager:
            self.panel_manager.destroy_output_panels()
            self.panel_manager = None
+        return result


Not sure how I feel about when things return a list of exceptions.

Usually (or always) nothing is done with returned exceptions. For example nothing logs those returned from destroy()/disabled(). I wonder if it would be better to just log the exceptions inside the functions that generate those (for example in _end_sessions) instead of returning...

rchl · 2026-09-12T19:08:38Z

-        config = ClientConfig.from_config(config, {})
-        config.set_view_status_handler(self)
-        file_path = initiating_view.file_name() or ''
-        if not self._can_start_config(config.name, file_path):


_can_start_config is now dead code.

rchl · 2026-09-12T19:22:40Z

+                self._sessions.add(session)
+                # Do not let an exception in listener.on_session_initialized_async cause a failure in this method.
+                asyncio.get_running_loop().call_soon(listener.on_session_initialized_async, session)
+                config.set_view_status(listener.view, "")


Those lines should IMO not be inside the try block. Not likely to throw but if they would then the session and process would not be closed.

rchl · 2026-09-12T19:30:15Z

+                # Do not let an exception in listener.on_session_initialized_async cause a failure in this method.
+                asyncio.get_running_loop().call_soon(listener.on_session_initialized_async, session)
+                config.set_view_status(listener.view, "")
+            except Exception as e:


What if session.initialize raises?
The session won't be ended than?
Also, I guess initialize can technically get cancelled which would have the same issue.
So the except block should likely catch BaseException and end the session.

rchl · 2026-09-12T20:07:25Z

+        try:
+            tasks = self._tasks
+        except AttributeError:
+            # This object already died on *some* thread... Most likely DocumentSyncListener.
+            return None


Use of try/except here might be for completely wrong reason and potentially should be removed as it can (and does currently) mask real issue.

It might have been added due to the fact that currently DocumentSyncListener doesn't initialize TaskContainer due to super.__init__() chain being broken due to ViewListener.__init__() not calling super().__init__(). So we might need explicit initialization in DocumentSyncListener with TaskContainer.__init__(), like in transports.py.

also: sublimehq/sublime_text#6979

rchl · 2026-09-12T20:13:18Z

+            await format_selection(self)
+            sublime.status_message("Paste was formatted")


How about making format_selection return True if something has changed and only then print the status? Currently it's also printed if nothing has changed or even if there was an error.

rchl · 2026-09-12T20:40:29Z

        error_reader: ErrorReader | None,
    ) -> None:
-        self._closed = False
+        TaskContainer.__init__(self)


Would use super().__init__() instead. More future proof.

rchl · 2026-09-12T20:46:22Z

-
-            session.execute_command(params, progress=True, view=self.view).then(handle_response)
+    async def _run(self, session: Session, command_name: str, params: ExecuteCommandParams) -> None:
+        result: LSPAny | Error = await session.run_command(params, progress=True, view=self.view)


No need for annotation.

Suggested change

result: LSPAny | Error = await session.run_command(params, progress=True, view=self.view)

result = await session.run_command(params, progress=True, view=self.view)

rchl · 2026-09-12T20:49:30Z

        session = self.session_by_name(session_name or self.session_name)
        if session and command_name:
            params: ExecuteCommandParams = {"command": command_name}
            if command_args:
                params["arguments"] = self._expand_variables(command_args)
+            run_coroutine(self._run(session, command_name, params))


Would be more correct to also do session lookup on asyncio thread.

I agree. But looking up sessions in main thread happens all over the place, also on main. I don't consider it the job of this PR to fix that for now.

rchl · 2026-09-12T20:51:02Z

        if not prefetch:
+


stray blank line

rchl · 2026-09-12T20:58:37Z

+            text_edits = await format_document(self)
+            if isinstance(text_edits, Error):
+                return text_edits
+            return await self._apply_text_edits(text_edits, label=self.label)


_apply_text_edits doesn't return anything so following would document the intention better

Suggested change

return await self._apply_text_edits(text_edits, label=self.label)

await self._apply_text_edits(text_edits, label=self.label)

return

same on line 176

rchl · 2026-09-12T21:02:39Z


    @override
-    def on_tasks_completed(self, *, select: bool = False, **kwargs: dict[str, Any]) -> None:
+    async def on_tasks_completed(self, *, select: bool = False, **kwargs: dict[str, Any]) -> Error | None:


Nothing handles returned Error. I feel like we should return None and only print/debug the error here. Or just ignore like we did before. The range command does print itself so it's also inconsistent.

Also the base interface declares None Return value

rchl · 2026-09-12T21:09:48Z

+    def __init__(self, text_command: LspTextCommand) -> None:
+        super().__init__(text_command)


This can be removed now.

rchl · 2026-09-12T21:14:30Z

            uri = urlunsplit(uri_parts._replace(fragment=''))
        for session in self.sessions():
-            if session.try_open_uri_async(uri, r) is not None:
+            if isinstance(await session.open_uri(uri, r), sublime.View):


If the return value is View | None then why would we need an isinstance check here?

I think there is some confusion here due to the fact that previously try_open_uri_async would return Promise | None. Promise is what plugin's url_handler returned so this code treated Promise as "plugin handled" and returned from the loop. Currently, this seems no longer possible so this will try next session if plugin returns a sheet without a View. That's a bug.

Is it possible to return a Sheet without a View?

Yes, ImageSheet and HtmlSheet have no View.

For example the Typst server uses a custom URI when hovering over a link to an included image file, and this URI gets handled by the plugin by either opening the image in a new tab (ImageSheet) or with an external program.

Indeed with this change for the return type of session.open_uri we can't fully distinguish anymore whether the URI was handled by the plugin or not.

Perhaps we can make Session.open_uri return Sheet | None?

Went back to making it return the strange mix of sublime.View | Literal[False] | None for now...

rchl · 2026-09-12T21:39:52Z

+        status = 'on' if enable else 'off'
+        sublime.status_message(f'Inlay Hints are {status}')


Feels like those two lines should be moved back to above the loop, like before, so that potential exception when running coroutines doesn't prevent status from showing.

Why should a message that sounds like things went OK be printed when things did not go OK? I think the message should not be printed when an error occurs.

actually you're right, the status message is just about the view setting.

rchl · 2026-09-12T21:42:12Z

+                inlay_hint = result
+
+        if session and (text_edits := inlay_hint.get('textEdits')):
+            for sb in session.session_buffers_async():
+                sb.remove_inlay_hint_phantom(phantom_uuid)
+            await apply_text_edits(self.view, text_edits, label="Insert Inlay Hint")
+
+        if label_part and (command := label_part.get('command')):


blank lines

Suggested change

inlay_hint = result

if session and (text_edits := inlay_hint.get('textEdits')):

for sb in session.session_buffers_async():

sb.remove_inlay_hint_phantom(phantom_uuid)

await apply_text_edits(self.view, text_edits, label="Insert Inlay Hint")

if label_part and (command := label_part.get('command')):

inlay_hint = result

if session and (text_edits := inlay_hint.get('textEdits')):

for sb in session.session_buffers_async():

sb.remove_inlay_hint_phantom(phantom_uuid)

await apply_text_edits(self.view, text_edits, label="Insert Inlay Hint")

if label_part and (command := label_part.get('command')):

rchl · 2026-09-12T21:48:46Z

References open_location_async which no longer exists.

rchl · 2026-09-12T21:52:19Z

            if version == self.view.change_count() and (session := self.session_by_name(session_name)) and \
                    session.has_capability('documentLinkProvider.resolveProvider'):
                request = Request.resolveDocumentLink(link, self.view)
                sublime.set_timeout_async(lambda: session.send_request_async(request, self._on_link_resolved_async))


Why is this case not converted to a coroutine?

hover.py has complex logic, and I would like to defer changing all send_request / send_request_async / send_request_task / send_request_task_2 refactorings to follow-up PRs.

Some relatively simple features like inlay hints and document links I've done in this PR just to make sure that things are working correctly.

rchl · 2026-09-12T22:01:48Z

+        finally:
+            await self.on_tasks_completed(**kwargs)


Will run on_tasks_completed even if the runner was cancelled. Must not happen.

rchl · 2026-09-12T22:04:44Z

+                try:
+                    await asyncio.wait_for(task(self).run(), timeout=userprefs().on_save_task_timeout_ms / 1000)
+                except asyncio.TimeoutError as ex:


Now timeout is per task but one task timing out will also skip all remaining tasks. On main, only the task that time outs is skipped.

rchl · 2026-09-12T22:11:47Z

+            sublime.status_message(str(ex))
+        except Exception as ex:
+            sublime.status_message("Error running save tasks. See the Console for more information.")


The task is per-view so shouldn't we be showing those messages in view's status instead?

rchl · 2026-09-12T22:13:23Z

-    def _on_task_completed_async(self) -> None:
-        self._pending_tasks.pop(0)
-        self._process_next_task()
+        self._text_command.view.erase_status(self._status_key)


_set_view_status was removed so nothing is now setting the view status. Related to my other issue:

The task is per-view so shouldn't we be showing those messages in view's status instead?

rchl · 2026-09-12T22:19:45Z

+            async def purge() -> None:
+                await listener.purge_changes()
+
+            run_coroutine(purge())


Is it guaranteed that the didChange notification is sent before the requests below?

Yes, scheduled coroutines run in the order they are scheduled.

True in general but this case is more complex. I've checked manually and the rename request was indeed sent before "purge changes" (I had to modify "purge_changes slightly to make it trigger didChange every time, even if there were no changes).

Here is what AI has to say about it:

Both dispatches land on the loop's ready queue, but send_request runs send_request_async inline on the loop thread and creates its send_payload task right away, whereas run_coroutine(purge()) costs one step to create the purge task and another for asyncio.gather to wrap each sv.purge_changes() in its own task. The request write therefore always gets there first.

When running "rename symbol" this shouldn't really matter because manual action is usually slow enough that purge changes will be triggered from a "timeout" earlier than the command but I guess there are cases when rename can be triggered programmatically right after some changes were made.

Hmm right. I'll put these changes into a separate commit.

rchl · 2026-09-12T22:28:25Z

            label = f"Rename {Path(old_path).name} -> {new_name}"
-            sublime.set_timeout_async(lambda: self.prompt_rename_async(file_rename, label, rename_command_args))
+
+            run_on_asyncio_thread(self.prompt_rename_async, file_rename, label, rename_command_args)


Marking this to check later what happens when triggering sublime's dialog on asyncio thread. It probably blocks all communications but maybe that's OK. On main, the communication threads should still be running with dialog open.

Good point. A prompt dialog on the asyncio thread indeed blocks all other tasks. You can test it with this self-contained test plugin (run the "test" command):

import asyncio import sublime import sublime_aio echo_task: asyncio.Task | None = None async def echo_forever() -> None: i = 0 while True: i += 1 print("hello", i) await asyncio.sleep(1) def plugin_loaded() -> None: def setup() -> None: global echo_task echo_task = asyncio.create_task(echo_forever()) sublime_aio.call_soon_threadsafe(setup) def plugin_unloaded() -> None: def teardown() -> None: global echo_task if echo_task: echo_task.cancel() echo_task = None sublime_aio.call_soon_threadsafe(teardown) class TestCommand(sublime_aio.ViewCommand): async def run(self) -> None: result = sublime.yes_no_cancel_dialog("Do the thing?") print(result)

Not sure what to do about it yet. In future refactorings, this problem should automatically go away. The sublime_aio library has async counterparts for these prompt dialogs: https://github.com/packagecontrol/sublime_aio#window

Actually it doesn’t seem to have async counterparts for the OS-style modal dialogs (yes_no_cancel_dialog, and error_message)…

Based on some more testing the yes/no dialog interacts differently when ran inside the asyncio thread compared to the main thread. On the main thread it seems that the asyncio thread still runs, but no printing to the console occurs.

sublime_aio gained functions for wrapping these blocking dialogs: packagecontrol/sublime_aio#41

For now, I'd like to keep this as-is, and refactor it in a follow-up PR.

rchl · 2026-09-12T22:31:54Z

                    view.run_command('save', {'async': False})
                view.close()  # LSP spec - send didClose for the old file


This could be problematic - triggering save command from asyncio thread and immediately closing the file. Have to be checked.

rwols added 30 commits April 28, 2026 19:53

debugging sublime_aio.ViewEventListener

472705b

Merge branch 'main' into feat/asyncio

ea34714

Fix calls

069ed50

More fix calls

776749a

More work. Some diagnostics show intermittently. Request logic is not…

0227877

… resolving

Tweaks to pull diagnostics handling

2649a6f

Restore method names

54af917

Forgot import

1a4589c

Restore more method names

900f721

ResponseException is not needed, we already have Error

35621cf

Merge branch 'main' into feat/asyncio

dbf4d7d

More rename reverts

f3b2f84

Add more tracing for understanding why requests are not resolving

a9c1692

Start fixing up 'goto' functionality

7004df5

debugging

d27aa93

We have something working

341ff99

Start fixing diagnostic errors because diagnostics work

4ed4e3f

Remove trace() calls

7aba65e

Remove trace() calls

e3ccba2

Remove trace() calls

c2c2a8c

Fixups

af561a8

Remove trace() calls and fixup type hints

7260c90

Reintroduce request_code_actions_async for SessionBuffer

a60ddd2

Consolidate sublime_aio & asyncio functions/classes in plugin/core/ai…

23210e3

…o.py

get_session_buffer_for_uri -> get_session_buffer_for_uri_async

73ed644

Session.session_buffers -> Session.session_buffers_async

6d6e772

_invoke_views_async -> _invoke_views

a84f3dd

Fix notifications being logged twice

f7702c2

Fix LspCheckApplicableCommand... I think

bb51929

rchl reviewed Sep 6, 2026

View reviewed changes

rchl reviewed Sep 7, 2026

View reviewed changes

rwols added 2 commits September 9, 2026 21:28

Address review comments

02af381

Merge branch 'main' into feat/asyncio

13db423