Applying Edits/Undo Stack - GLSP/Langium Integration - VSCode Extension/Node Server

[jmenzies12]

Hey there,

Context:

We've built a VSCode extension that uses Langium documents as the source model for GLSP diagrams. Up until recently our GLSP server was separate from the Language Server in Langium and running in the extension process with access to the VSCode API.

In order to integrate Langium and GLSP better we moved the GLSP server out into it's own process using the socket launcher and this gets triggered after the language server does. Resulting in our GLSP server being able to access SharedServices etc. This also removes access to the VSCode API.

After doing this however we've ran into one main issue in regards to editing the Langium document. Previously, because we had access to the VSCode API, we could simply perform a vscode.workspace.applyEdit from the server side node creation handler and this would apply the edit and sync in fine with the undo/redo stack.

Now what happens is that we have to trigger an applyEdit from the server side that gets sent back up to the language client and that applies it. Unfortunately this has broken the synchronization with the undo stack, essentially resulting in us having to hit undo twice (once for GLSP and then once for Langium).

In an attempt to fix this, we disabled undo for GLSP operations that only affect the Langium document (i.e. node creation/deletion etc.), which sort of worked, however if you move a node and then add a new node, because the applyEdit from the server isn't directly coming from the diagram editor the move is actually first in the undo stack, which results in it undoing the move before undoing the node creation.

I even tried triggering the applyEdit from the GLSPVCodeConnector.handleSetDirtyStateAction, but unfortunately the move still receives priority over the edit that happens after it.

Question/Conclusion:

I'm starting to get the impression that this is due to CustomEditors vs CustomTextEditors, and because the text document and the diagram editor are different that trying to get things to sync up just isn't playing nicely with VSCode. My next rabbit hole essentially, is looking into whether I can convert the GLSP editor over to a CustomTextEditor and then find a way to ensure that operations that do not directly modify the document can still be undone.

I know a few contributors here also contribute to the crossmodel project that uses Langium/GLSP (though I believe it uses the Theia GLSP integration for the frontend instead of VSCode so might not suffer from the exact same problems), so I was wondering if anyone had any insights into this?

[martin-fleck-at]

Hi @jmenzies12,

I don't have too much insights into the difference between CustomEditors vs CustomTextEditors, maybe @tortmayr can shed some light on this. I could imagine, however, that does indeed make a difference.

In CrossModel, we currently send full updates to the Langium language server by having a custom command structure that ¹:

• Serializes the current AST of our document to String as the current state for undo/redo
• Execute the operation to manipulate the AST of the document directly
• Serialize the new AST of our document to String as the new state for undo/redo
• Submit an update of the new AST to a generic Model API (aka Langium)
• (Langium will send an update to all clients with the new update)
• (When loading the source model, we set up a listener for model changes ²)

Doing this should ensure that we are working on the same semantic models from Langium and make sure that GLSP is aware of any changes in Langium as well.

I also had a quick look at what the VS Code API does when you do a vscode.workspace.applyEdit and it seems that it manipulates the text document through their own edit operations ³⁴. However, from what I can see (purely from code, not tested), I'd expect that sending a lsp.Connection.workspace.applyEdit (e.g., ⁵) request would yield the same result as the language client handler transforms this into a vscode.workspace.applyEdit as well ⁶.

Could you explain a bit more how you were syncing the GLSP server with Langium before? I'm not sure I fully grasp your approach. Nevertheless, it seems you experience some issues with the new approach. It would be interesting, where those issues come from. However, you could try to mimic the old behavior by introducing a custom request to communicate between the language server connection and the VS Code extension. A good location for this would be when you launch your language client. For instance, in CrossModel, we register a command to report the port of the GLSP server using the VS Code API ⁷. Using a similar approach you can set up the reverse, e.g., something along the lines of: languageClient.onRequest(CUSTOM_APPLY_EDIT_REQUEST, args => vscode.workspace.applyEdit(...). Essentially, this will be doing the same as the work so if that does not work, we need to figure out in more detail where your approach goes wrong.

I hope this will help you out in some way.

Footnotes

1. https://github.com/CrossBreezeNL/crossmodel/blob/main/extensions/crossmodel-lang/src/glsp-server/common/cross-model-command.ts ↩

2. https://github.com/CrossBreezeNL/crossmodel/blob/main/extensions/crossmodel-lang/src/glsp-server/common/cross-model-storage.ts#L62 ↩

3. https://github.com/microsoft/vscode/blob/main/src/vs/workbench/api/common/extHost.api.impl.ts#L1020 ↩

4. https://github.com/microsoft/vscode/blob/main/src/vs/editor/standalone/browser/standaloneServices.ts#L904 ↩

5. https://github.com/CrossBreezeNL/crossmodel/blob/main/extensions/crossmodel-lang/src/model-server/model-service.ts#L52 ↩

6. lsp.Connection?.workspace.applyEdit ↩

7. https://github.com/CrossBreezeNL/crossmodel/blob/main/extensions/crossmodel-lang/src/extension.ts#L34 ↩

[martin-fleck-at]

Glad you managed to work it out and find a solution to your problem!

Just out of curiosity to see if I get that right:

1. The first operation handler (e.g., create) triggers an async apply edit essentially resulting in the operation handler not having made any modifications by the time it finished. The default operation action handler still triggers a dirty state update ¹ which the GLSPVSCodeConnector will pick up to trigger a document change with GLSP undo/redo behavior attached ². The "empty" operation is put on the command stack.
2. The applyEdit is actually executed which somehow also triggers an operation on the GLSP server so another operation is put on the command stack. This time with proper undo/redo behavior.

What I do not understand is how the second operation is produced and why we even end up with two commands for a single operation. Does that mean that you always have an "empty" undo on the command stack? Would it not be enough to simply await the applyEdit within the operation handler? At least now it is more obvious why there is a difference as the separate process is automatically more async then having the integrated server run in the same process as the extension.

Footnotes

1. https://github.com/eclipse-glsp/glsp-server-node/blob/948f71ac24366102dbc6b9352f5d10b5777b150b/packages/server/src/common/operations/operation-action-handler.ts#L80 ↩

2. https://github.com/eclipse-glsp/glsp-vscode-integration/blob/edec0c24c7521c8d7e2e18f81dcc6138af6a4c99/packages/vscode-integration/src/common/glsp-vscode-connector.ts#L467 ↩

[jmenzies12]

Just out of curiosity to see if I get that right:

1. The first operation handler (e.g., create) triggers an async apply edit essentially resulting in the operation handler not having made any modifications by the time it finished. The default operation action handler still triggers a dirty state update 1 which the GLSPVSCodeConnector will pick up to trigger a document change with GLSP undo/redo behavior attached 2. The "empty" operation is put on the command stack.

2. The applyEdit is actually executed which somehow also triggers an operation on the GLSP server so another operation is put on the command stack. This time with proper undo/redo behavior.

Basically yes, for 2, the applyEdit sent by part 1 from the GLSP server, goes to the language client and applies the edit as an operation on the command stack. With the proper undo/redo behavior.

What I do not understand is how the second operation is produced and why we even end up with two commands for a single operation. Does that mean that you always have an "empty" undo on the command stack? Would it not be enough to simply await the applyEdit within the operation handler? At least now it is more obvious why there is a difference as the separate process is automatically more async then having the integrated server run in the same process as the extension.

Yeah this was puzzling me as well. We do end up with an empty undo on the command stack, if the timing is off the empty undo is latest in the stack, if not then it comes before the applyEdit.

I was hoping that I could await the applyEdit within the operation handler but after refactoring my code to be able to do this the result is still the same. Even if the applyEdit is awaited and finishes within the operation handler we still get end up with two items added to the stack and a double undo with one being empty.

I disabled undo for certain operations on the GLSP side and this removes the empty undo, however the timing still seems to matter hugely, if I move a node (which is still an undoable operation but does not trigger an applyEdit), and then add a new a node, if the applyEdit falls under the operation handler then the applyEdit somehow falls before the move operation instead of after, resulting in even more undo confusion

[martin-fleck-at]

Very interesting indeed! I'm glad you found a way to get this to work properly, even though I also assumed that it should work the other way as well. And thank you for elaborating on this again!

[jmenzies12]

@martin-fleck-at,

Just a quick update, incase you or anyone else is curious.

I spent more time on it and came up with a better solution (this removes the need for delaying the applyEdit and confusion of having conflicting operations at once):

• Created new operation handlers (DocumentOperationHandler) for specific operations that affect the langium document
• Created a custom command for these operations that is non undoable
• Extended the ModelSubmissionHandler with a new function 'submitModelFromDocumentEdit', this is identical to submitModel except the SetDirtyStateAction is always set as true, instead of checking the command stack (due to the lack of undoable command) and then we pass in 'external' as the reason.
• Extended the operation action to use the new ModelSubmissionHandler, this will check if the operation handler is DocumentOperationHandler and if so it will call submitModelFromDocumentEdit with 'external'.
• We then have some custom code to handle the fact that the document edit will be delayed, so essentially the diagram model is changed and then we trigger a second update once we've received a notification from VSCode to say that the document has been updated which we then use to reconcile the diagram model and the source model

[tortmayr]

I don't have too much insights into the difference between CustomEditors vs CustomTextEditors, maybe @tortmayr can shed some light on this. I could imagine, however, that does indeed make a difference.

I don't think it makes much of a difference in practice. When using a CustomTextEditor the underling document is always a TextDocument and it is expected that every change to the source model/text document is done via the vscode API (applyTextEdit).
This means the client/vscode is in full control of managing the source model which contradicts a little bit with the base assumption of GLSP (server is in control and the single source of truth). In the end, you still need to find a way to synchronize the server model state and the client model state via text edits. IMO CustomTextEditors are currently not well supported in GLSP and probably a lot harder to implement. With the Crossmodel project we already have a working solution for langium+GLSP integration witch should work well with normal CustomEditors so I would propose to stick with CustomEditors for now.