Porting / vendoring Chrome C/C++ extension code, add Rust support (sum types etc) and fix https://issues.chromium.org/issues/401522579

[mikaelwaltersson]

Two main issues with the current version this PR addresses

• The build is broken as it is not pinned against a specific version of the https://github.com/ChromeDevTools/devtools-frontend.git repository. There have been some breaking changes since it was last built.
• There is a bug in the Chrome C/C++ Extension that causes all debug symbols to be unavailable if the file path in any of the DWARF compile units is resolved to an invalid URL. The only information available when this happens is a single error message TypeError: Failed to construct 'URL': Invalid URL. See https://issues.chromium.org/issues/401522579 for more information.

Basic Rust types supported added

• The current version can show primitive Rust types like integers, booleans, floats and fixed sized arrays of primitive types but other types from the core library like String, &str, Vec<T> and array slices are not viewable because of how they are internally represented (Unique<u8> pointer to a buffer and 0-sized PhantomData<T> marker struct.)
  Sum types (with the DWARF tag type DW_TAG_variant_part) will ends up as empty structs as there is no such concept in C/C++.
• Support for Rust sum types is added (which also works for common ones in the core library like Option<T> and Result<T, E>.)
• Support for Rust strings and string slices (String and &str) is added.
• Support for Rust vectors and array slices (Vec<T>, VecDeque<T> and &[T]) is added.
• Support for Rust referenced counted pointers (Rc<T> and Weak<T>) is added.
• Support for Rust hash maps and sets (HashMap<K, V> and HashSet<T>) is added.

Other minor fixes

• C and C++ strings with new lines and other special characters are not escaped and so they show up in the variable view only displaying the first line. DevTools in Chrome seems to be handling this on their end but VSCode / vscode-js-debug is not. The PR fixes this by enhancing the display of the strings by calling JSON.stringify on them. See attached screenshots for examples.
• The loading of external .dwo files is broken in the current version. The code that handles this in the original Chrome C/C++ extension is not expecting to be run under Node.
• Enumeration values represented by 8 bit integers shows up as char, this is fixed in the PR.
• Update version number and README.md

Overview of changes made to facilitate the fixes and added features

• Imported / copied the relevant C++ and TypeScript code from the https://github.com/ChromeDevTools/devtools-frontend.git repository. The specific revision used is mention in the commit log.
• Fixes to have typing compatible with current version of @vscode/js-debug
• Ported most of the test suites (including the C++ tests running as WASM modules and the e2e .yaml spec files)
• Ported the build scripts to run both inside a container or locally on Linux or macOS.
• Change to run all TypeScript compilation outside of CMake for better developer experience

Screenshots:

String with tab and newline in Chrome C/C++ extension:

String with tab and newline in current VSCode DWARF debugging extension:

String with tab and newline in patched VSCode DWARF debugging extension:

Rust support:

Other remarks

There is quite a lot of files in this PR so I've tried to structure the commit history in a way that makes it easier to follow.

• f5a1f23562878a7d46cc719247ca4aa64b5efc17 is a verbatim copy of relevant files from the https://github.com/ChromeDevTools/devtools-frontend.git repository.
• 0ee38d9461fd7bc06efdea36fdf4d54f4b5d6721 and 331eba97a6589a1aeab0a2dfc1aacd48a20a91d1 moves / renames the imported files and update some references to the LICENSE file.
• 120a00400106ff3674d12ba8b2d69356aa7278f2 ports / migrates the code and tests and build scripts to be able to run in Node instead of the Chrome DevTools. So this commit is one of the mayor ones the to review along with the last one that adds the Rust type support.
• 7195f4063e4a84fb7700c719424bd0d2da85d859 fixes https://issues.chromium.org/issues/401522579
• 39098556b3d8e8705cca353c0bd7022c8e1c7216 adds Rust types support.

[juancampa]

Wow I'm so excited for this to land. We're building a large-ish wasm application and the lack of debugging has been a real issue. Your effort is highly appreciated @mikaelwaltersson!

[mikaelwaltersson]

Wow I'm so excited for this to land. We're building a large-ish wasm application and the lack of debugging has been a real issue. Your effort is highly appreciated @mikaelwaltersson!

Thanks, unfortunately I suspect this PR might not be merged as it involves transforming the repository from being a thin layer of code and build scripts around the original code in devtools-frontend/extensions/cxx_debugging repository maintained by Google, into a a repository containing all the code for the extension. I can see how that adds a lot of maintenance burden.
In addition this PR is huge (for obvious reasons) so I get why it's a lot of work to review, I've tried my best to structure the commit history so it's easier to see which files where copied across from the devtools-frontend/extensions/cxx_debugging repository and what changes was made to them afterwards.

Ideally I would have been able to just create a new repository and publish my own VSCode extension but way JavaScript debugger extension ms-vscode/js-debug that ships with VSCode is coupled with the ms-vscode/wasm-dwarf-debugging extension makes it impossible to do without forking ms-vscode/js-debug itself as the dependency is hard coded here https://github.com/microsoft/vscode-js-debug/blob/74702375c67744e2f123d89ef950d7b6859c6f16/src/ui/dwarfModuleProviderImpl.ts#L12.

So in lieu of this PR being merged the only way to use this to either locally build this forked version of the extension as ms-vscode/wasm-dwarf-debugging or download the precompiled binaries from this repository and manually install the .vsix file in VSCode.

I've been contemplating raising an issue/proposal in vscode-js-debug for decoupling it from the hard dependency on ms-vscode/wasm-dwarf-debugging and add the ability to use any "WASM debugger" from the VSCode extension market place. That way we could have multiple WASM debugger extensions available for different languages without replacing the JavaScript debugger that comes shipped with VSCode.

[connor4312]

Thanks for the PR! Sorry I didn't see it, I see to have been unsubscribed from notifications in this repo at some point.

Imported / copied the relevant C++ and TypeScript code from the https://github.com/ChromeDevTools/devtools-frontend.git repository. The specific revision used is mention in the commit log.

I wonder if you think it'd be reasonable to avoid pulling these into our sourcetree and instead having a build script that copies them in from the devtools repo -- or were there more changes/additions you had to make to these files once they were ported into this extension?

[mikaelwaltersson]

Thanks for the PR! Sorry I didn't see it, I see to have been unsubscribed from notifications in this repo at some point.

Imported / copied the relevant C++ and TypeScript code from the https://github.com/ChromeDevTools/devtools-frontend.git repository. The specific revision used is mention in the commit log.

I wonder if you think it'd be reasonable to avoid pulling these into our sourcetree and instead having a build script that copies them in from the devtools repo -- or were there more changes/additions you had to make to these files once they were ported into this extension?

No worries, I didn't expect it to be looked at right away either especially as the change set is quite large. I had forgot about it myself until I got a notification that there was a comment on the PR.

Just fixing the issue with the TypeError: Failed to construct 'URL': Invalid URL bug mentioned is probably feasible with a build script that copies the source from https://github.com/ChromeDevTools/devtools-frontend.git and modifies it as build step, you can see the few changes needed in the PR I lodged earlier this year to the Chromium repository (https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/6332180/2/extensions/cxx_debugging/src/ModuleConfiguration.ts).

However to add the Rust types support and fixing the other minor issues mentioned in the PR description requires enough changes to both the C++ code and the TypeScript code for it to be quite hard to create such a build script. Even if you create such a build script that apply patches to the code it would be extremely fragile concerning any changes in the upstream repository.

Here is the diff of the changes made to the imported files, even disregarding changes from auto formatting of the code and refactoring it's still quite a lot (the diff is generated with the command git diff 331eba97a6589a1aeab0a2dfc1aacd48a20a91d1..f5fc47e6599d479b317957a4560dcdf2d5f91d82 -w --patch-with-stat wasm/symbols-backend/src wasm/symbols-backend/lib src).

I looked into importing the original code using git subtree split ... and git subtree add ... to conserve history and make it easier to potentially fetch any new changes from upstream but that would make this repository grow from a 2mb to over one 1gb which I don't think is worth the trouble as there hasn't been much activity in the upstream code the last couple of years.

Disregarding the tests that I ported to run in a node environment instead of in the browser there are really only 21 source code files and 2 CMake build script files imported from the upstream repository. With this in mind I think manually "merging" any future upstream fixes / changes is feasible and that the downside is worth it to not include all the extra unused files that would come with a git subtree ... approach.

extensions/cxx_debugging/lib/ApiContext.cc
extensions/cxx_debugging/lib/ApiContext.h
extensions/cxx_debugging/lib/CMakeLists.txt
extensions/cxx_debugging/lib/Expressions.cc
extensions/cxx_debugging/lib/Expressions.h
extensions/cxx_debugging/lib/Variables.cc
extensions/cxx_debugging/lib/Variables.h
extensions/cxx_debugging/lib/WasmModule.cc
extensions/cxx_debugging/lib/WasmModule.h
extensions/cxx_debugging/lib/WasmVendorPlugins.cc
extensions/cxx_debugging/lib/WasmVendorPlugins.h
extensions/cxx_debugging/lib/api.h
extensions/cxx_debugging/src/CMakeLists.txt
extensions/cxx_debugging/src/CustomFormatters.ts
extensions/cxx_debugging/src/DWARFSymbols.ts
extensions/cxx_debugging/src/DevToolsPluginWorker.ts
extensions/cxx_debugging/src/Formatters.ts
extensions/cxx_debugging/src/MEMFSResourceLoader.ts
extensions/cxx_debugging/src/ModuleConfiguration.ts
extensions/cxx_debugging/src/SymbolsBackend.cc
extensions/cxx_debugging/src/SymbolsBackend.d.ts
extensions/cxx_debugging/src/WasmTypes.ts
extensions/cxx_debugging/src/WorkerRPC.ts

Last but not least, in my opinion the "build script" approach makes it extremely hard to make any contributions here going forward:

• Either the long feedback cycle when making a small change and building everything inside the docker image or needing to install the 3rd party dependencies used by the Chromium team's build scripts (depot_tools, correct version of python etc.)
• The complexity of making changes to the code in such a way that it can easily be applied as a patch at build time.

Let me know if you want me to split it into multiple PR's or if the "staged" commit history is enough to go on for the review.