ESM support for VSCode and extensions

[SimonSiefke]

This PR adds ESM support for VSCode and extensions. Fixes #160416. Fixes #130367

Try Out

git clone git@github.com:SimonSiefke/vscode.git &&
cd vscode &&
git checkout simon/esm &&
yarn &&
yarn compile &&
./scripts/code.sh

A sample ESM extension is located at extensions/hello-esm. Running the command hello esm displays a hello world notification:

Feedback / Next Steps

It is probably too large to review and would be good to split up into smaller PRs! Before continuing it seems it would be good to gather some feedback from the VSCode team!

---

Changes

Type Imports

Some imports are changed from import { IDisposable } from 'vs/base/common/lifecycle'; to import type { IDisposable } from 'vs/base/common/lifecycle'; to work with ESM.

Assert imports

Most of the changes in this PR are due to the assert import. In ESM, when using import * as assert from 'assert' , assert cannot be a function. The change is import assert from 'assert'

Xterm imports

import { Terminal } from '@xterm/headless'

is changed to

import xtermHeadless from '@xterm/headless';

const { Terminal } = xtermHeadless;

This makes the import work in ESM. It seems it could even be changed back when @xterm/headless is published as ESM.

Css Imports

import 'vs/css!./actionbar' is compiled into importCss('./actionbar.css', import.meta.url). The importCss function uses document.createElement('link') to create a stylesheet for actionbar.css.
`

AmdX Loader

The amdx loader is changed from loading scripts with document.createElement('script'); or importScripts to load scripts using import.

ESM extensions

To support ESM extensions, the extension host is started with a custom loader NODE_OPTIONS: --import="vscode/src/extension-loader-register.js"`.

The loader can change how ESM files are imported, so that import vscode from "vscode" works as expected:

// extension-loader.js
export async function resolve(specifier, context, nextResolve) {
	if (specifier === "vscode") {
		return {
			shortCircuit: true,
			format: "module",
			url: new URL("./fake-vscode.js", import.meta.url).toString(),
		};
	}
	return nextResolve(specifier, context);
}

export function load(url, context, nextLoad) {
	return nextLoad(url);
}

When importing vscode, this fake-vscode.js file is imported instead:

// fake-vscode.js
const api = globalThis.vscodeFakeApi
if (!api) {
	throw new Error('vscode api not available')
}

const { window, commands } = api

export { window, commands }

vscodeFakeApi is created by extensionApiFactory() and provides the vscode api properties window, commands and more.

NodeJS Docs for module customization hooks: https://nodejs.org/api/module.html#customization-hooks

Tests

To support ESM for electron integration tests, import maps and preload require are used.

<script type="importmap">
	{
		"imports": {
			"fs": "./import-map/fs.js"
		}
	}
</script>

// preload.js
window.testGlobalRequire = require // expose require function for tests

// import-map/fs.js

const fs = window.testGlobalRequire('testGlobalRequire')

const { createWriteStream, existsSync, readFileSync, readdirSync, rmSync, writeFileSync, unwatchFile, watchFile, watch, readFile, mkdir, chmod, link, unlink, truncate, copyFile, read, access, open, rm, readdir, writeFile, constants, stat, mkdirSync, unlinkSync, fdatasync, statSync, accessSync, symlinkSync, openSync, createReadStream, realpathSync, lstatSync, promises, lstat, fdatasyncSync, closeSync, utimes, appendFile, close, symlink, readlink, rmdir, write, renameSync, rename, realpath } = fs;

export { createWriteStream, existsSync, readFileSync, readdirSync, rmSync, writeFileSync, unwatchFile, watchFile, watch, readFile, mkdir, chmod, link, unlink, truncate, copyFile, read, access, open, rm, readdir, writeFile, constants, stat, mkdirSync, unlinkSync, fdatasync, statSync, accessSync, symlinkSync, openSync, createReadStream, realpathSync, lstatSync, promises, lstat, fdatasyncSync, closeSync, utimes, appendFile, close, symlink, readlink, rmdir, write, renameSync, rename, realpath };

It's a lot of changes for every imported module (assert, child_process, cookie, crypto, electron, events, fs, graceful-fs, istanbul-lib-coverage, istanbul-lib-instrument and more) but it makes the tests work with ESM.

Other

No ESM support for web extensions yet

Making use of NodeJS loader hooks, the ESM support only applies to NodeJS extensions. There would be no ESM support for web extensions yet.

[SimonSiefke]

@lppedd Thanks for the comment! I agree it would be very useful to have ESM support in web extensions also. The issue for web extensions - which I haven't been able to solve - are these imports:

import * as vscode from 'vscode'

The vscode import would need to be mapped to a file, for example /workbench/api/common/extHost.api-esm.js.

It seems import maps for webworkers could solve this problem, once they are supported by browsers.