Tree Sitter problem on Big Endian System

[nealef]

I am attempting to build vscodium for a new platform that happens to be big endian (s390x). The build had been crapping out when using tree-sitter as when it does its magic number and version checking the values it was getting was little endian and it was not catering for this happening on a big endian platform.

I rebuilt tree-sitter from the main branch of the repo using emscripten 4.0.19 (which has endian support) and it now gets past that part of the build. I used npm pack to create a tarball of tree-sitter and replaced node_modules/web-tree-sitter/* with its contents.

However, I am now getting:

> node --max-old-space-size=8192 ./node_modules/gulp/bin/gulp.js compile-build-without-mangling

TypeError: Cannot read properties of undefined (reading 'init')
    at Object.<anonymous> (/build/vscodium/vscode/node_modules/@vscode/l10n-dev/dist/main.js:188:49)
    at Module._compile (node:internal/modules/cjs/loader:1529:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1613:10)
    at Module.load (node:internal/modules/cjs/loader:1275:32)
    at Module._load (node:internal/modules/cjs/loader:1096:12)
    at Module.require (node:internal/modules/cjs/loader:1298:19)
    at require (node:internal/modules/helpers:182:18)
    at Object.<anonymous> (/build/vscodium/vscode/build/lib/i18n.js:28:20)
    at Module._compile (node:internal/modules/cjs/loader:1529:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1613:10)

I did a piece-by-piece replacement and verified the endian issue was solved but I am not sure how to approach the above.

[daiyam]

Have you tried to run ./script/code.sh in the vscode? It will compile and run VSCodium in dev mode not need to do all the packaging thing before making sure it can run.
Also @vscode/l10n-dev might need to be rebuild since it uses tree-sitter (https://github.com/microsoft/vscode-l10n/blob/main/l10n-dev/package.json)

[nealef]

I used dev/build.sh -i to build and just cut and pasted the relevant bit. I think the rebuild of l10n-dev sounds likely so I'll be giving that a try. Thanks. I'll keep the thread updated.

[nealef]

I attempted to build l10n-dev but I crap out with:

> @vscode/l10n-dev@0.0.35 build-wasm
> npm run build-wasm-typescript && npm run build-wasm-tsx


> @vscode/l10n-dev@0.0.35 build-wasm-typescript
> tree-sitter build --wasm ./node_modules/tree-sitter-typescript/typescript --output ./src/ast/tree-sitter-typescript.wasm

Failed to locate a tree-sitter.json file, please ensure you have one, and if you don't then consult the docs

[daiyam]

It should be in l10n-dev/node_modules/tree-sitter-typescript/tree-sitter.json. You need to do npm i in l10n-dev to install the dependencies

[nealef]

Thanks, that worked for the build. I copied the artifacts over but I still get the same failure. What's the best way to narrow down what exactly it is complaining about? I see in tree-sitter.cjs there is an init function defined.

        var LDSO = {
          loadedLibsByName: {},
          loadedLibsByHandle: {},
          init() {
            newDSO("__main__", 0, wasmImports);
          }

and

              var init = moduleExports["__wasm_call_ctors"];
              if (init) {
                if (runtimeInitialized) {
                  init();
                } else {
                  addOnPostCtor(init);
                }
              }

But I have no idea if these are related to the complaint.

If I try building with scripts/code.sh then I don't get very far at all:

# ./scripts/code.sh

> code-oss-dev@1.105.07802 electron
> node build/lib/electron

Downloading electron: [====================================================================================================] 100% ETA: 0.0 seconds
node:internal/process/promises:391
    triggerUncaughtException(err, true /* fromPromise */);
    ^

HTTPError: Response code 404 (Not Found)
    at Request._onResponseBase (file:///build/vscodium/vscode/node_modules/@vscode/gulp-electron/node_modules/got/dist/source/core/index.js:589:31)
    at Request._onResponse (file:///build/vscodium/vscode/node_modules/@vscode/gulp-electron/node_modules/got/dist/source/core/index.js:649:24)
    at ClientRequest.<anonymous> (file:///build/vscodium/vscode/node_modules/@vscode/gulp-electron/node_modules/got/dist/source/core/index.js:667:23)
    at Object.onceWrapper (node:events:639:26)
    at ClientRequest.emit (node:events:536:35)
    at HTTPParser.parserOnIncomingClient (node:_http_client:702:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:118:17)
    at TLSSocket.socketOnData (node:_http_client:544:22)
    at TLSSocket.emit (node:events:524:28)
    at addChunk (node:internal/streams/readable:561:12)
Emitted 'error' event on Stream instance at:
    at Stream.onerror (node:internal/streams/legacy:63:12)
    at Stream.emit (node:events:524:28)
    at get (/build/vscodium/vscode/node_modules/event-stream/index.js:161:14)
    at /build/vscodium/vscode/node_modules/event-stream/index.js:172:15
    at /build/vscodium/vscode/node_modules/@vscode/gulp-electron/src/download.js:159:16 {
  input: undefined,
  code: 'ERR_NON_2XX_3XX_RESPONSE',

[daiyam]

You will to provide a build of Electron for s390x.

Look how PPC is been done:

• https://github.com/VSCodium/vscodium/blob/master/build/linux/package_bin.sh#L26
• https://github.com/VSCodium/vscodium/tree/master/build/linux/ppc64le
• https://github.com/lex-ibm/electron-ppc64le-build-scripts