Apple's investment into Clangd and refactoring tooling

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Apple's investment into Clangd and refactoring tooling

Sam McCall via cfe-dev
Hi all,

We at Apple have decided to switch focus from supporting the libclang-based tooling infrastructure in order to join forces on the Clangd development efforts. We believe that Clangd is the preferred solution for interactive Clang-based tooling. There has been great work on Clangd already, and we're going to start investing effort as well to make Clangd faster, more capable, and more efficient.

We have some high-level requirements for Clangd in order to make it first-class on our platform and integrate it with our cross-language IDE features:

- We need to support a completely different transport layer protocol in Clangd. We would like to split the implementation of LSP into two layers: the logical LSP layer and the JSON-RPC transport layer. This would allow us to add support for Apple's XPC [1] technology as an alternative to the existing LSP's JSON-RPC transport layer. The new transport layer will be supported on Darwin only. We intend to carry the LSP payloads over XPC.

- We need to support extensions to the protocol beyond LSP's current specification. The extension mechanism should allow us to add new protocol entries and to extend the information provided in the existing requests and responses. These LSP extensions will be supported in both the XPC and the JSON-RPC transport layers.

This proposal presents an outline of our current plan for how these requirements can be satisfied. We welcome the feedback that you might have for us!

## Our Vision for the Transport Layer

The introduction of a new transport layer protocol is a big change to Clangd's codebase. 

In the long term, we envision a solution where Clangd has a clean and an efficient split between the transport layers and LSP itself. This separation would also allow other parties to adopt their own transport layers in a straightforward manner. We would like to come up with a design for transport layer separation in Clangd in the upcoming months. In the meantime, we would love to receive the feedback that you might have for us.

## Initial XPC Bring-up

In the short term, in order to bring up the XPC transport layer more quickly, we plan on utilizing the existing JSON-RPC serialization infrastructure.  We'll overlay JSON on top of XPC, serializing the JSON objects to/from XPC objects.  This is less efficient but will get us to something testable more quickly.

## Initial investment efforts

We would like to invest our time into implementing some of the missing features in Clangd and extending some of the existing features. Our initial use-case will utilize a limited subset of LSP and we plan on focusing on that subset. 

We'll also need to ensure that Clangd works with our cross-language indexer, which drives the IDE integration for global refactoring operations. We'll go back to and continue upstreaming our Clang-based refactoring infrastructure that we introduced in Xcode last year and will ensure that it will be usable from Clangd. We intend to make the refactorings (including global refactorings) work with both Clangd's builtin indexer and our standalone cross-language indexer.

Best regards,
Alex

References:


P.S. Catch me and/or Jan Korous at EuroLLVM today if you'd like to discuss this in person :)

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Apple's investment into Clangd and refactoring tooling

Sam McCall via cfe-dev
Great news! And the things you need definitely make sense to me. Would like to hear more!
For XPC specifically: we also have other transports in use internally at google, which chose to wrap ClangdServer and avoid ClangdLSP server entirely. This may or may not be the right model for you, let's talk.
I have a BoF about build system integration at 4 (maybe also relevant?), would be good to chat before/after, I'll send mail off-list.

On Mon, Apr 16, 2018 at 9:43 AM Alex L via cfe-dev <[hidden email]> wrote:
Hi all,

We at Apple have decided to switch focus from supporting the libclang-based tooling infrastructure in order to join forces on the Clangd development efforts. We believe that Clangd is the preferred solution for interactive Clang-based tooling. There has been great work on Clangd already, and we're going to start investing effort as well to make Clangd faster, more capable, and more efficient.

We have some high-level requirements for Clangd in order to make it first-class on our platform and integrate it with our cross-language IDE features:

- We need to support a completely different transport layer protocol in Clangd. We would like to split the implementation of LSP into two layers: the logical LSP layer and the JSON-RPC transport layer. This would allow us to add support for Apple's XPC [1] technology as an alternative to the existing LSP's JSON-RPC transport layer. The new transport layer will be supported on Darwin only. We intend to carry the LSP payloads over XPC.

- We need to support extensions to the protocol beyond LSP's current specification. The extension mechanism should allow us to add new protocol entries and to extend the information provided in the existing requests and responses. These LSP extensions will be supported in both the XPC and the JSON-RPC transport layers.

This proposal presents an outline of our current plan for how these requirements can be satisfied. We welcome the feedback that you might have for us!

## Our Vision for the Transport Layer

The introduction of a new transport layer protocol is a big change to Clangd's codebase. 

In the long term, we envision a solution where Clangd has a clean and an efficient split between the transport layers and LSP itself. This separation would also allow other parties to adopt their own transport layers in a straightforward manner. We would like to come up with a design for transport layer separation in Clangd in the upcoming months. In the meantime, we would love to receive the feedback that you might have for us.

## Initial XPC Bring-up

In the short term, in order to bring up the XPC transport layer more quickly, we plan on utilizing the existing JSON-RPC serialization infrastructure.  We'll overlay JSON on top of XPC, serializing the JSON objects to/from XPC objects.  This is less efficient but will get us to something testable more quickly.

## Initial investment efforts

We would like to invest our time into implementing some of the missing features in Clangd and extending some of the existing features. Our initial use-case will utilize a limited subset of LSP and we plan on focusing on that subset. 

We'll also need to ensure that Clangd works with our cross-language indexer, which drives the IDE integration for global refactoring operations. We'll go back to and continue upstreaming our Clang-based refactoring infrastructure that we introduced in Xcode last year and will ensure that it will be usable from Clangd. We intend to make the refactorings (including global refactorings) work with both Clangd's builtin indexer and our standalone cross-language indexer.

Best regards,
Alex

References:


P.S. Catch me and/or Jan Korous at EuroLLVM today if you'd like to discuss this in person :)
_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev