Updated JavaScript kernel with typescript, top-level async/await, es module import, promise resolution

[mishushakov]

changelog

• replaces ijavascript kernel with a fork that includes a compiler step and promise resolution
• removes typescript compiler step as this is now built-in in the kernel
• tests/docs will follow in a separate PR

kernel changes:

• n-riesco/ijavascript@master...e2b-dev:ijavascript:master
• n-riesco/nel@master...e2b-dev:nel:master

issues:

• fixes Support ES6 modules in JS runtime and top-level await #34
• fixes Async Code Not Executing Correctly in e2b Sandbox Environment E2B#477

Examples

snippet:

import got from "got"

before:

Execution {
  results: [],
  logs: {
    stdout: [],
    stderr: [],
  },
  error: ExecutionError {
    name: "SyntaxError",
    value: "Cannot use import statement outside a module",
    traceback: "evalmachine.<anonymous>:2  import got from \"got\"  ^^^^^^SyntaxError: Cannot use import statement outside a module    at new Script (node:vm:117:7)    at createScript (node:vm:269:10)    at Object.runInThisContext (node:vm:317:10)    at run ([eval]:1020:15)    at onRunRequest ([eval]:864:18)    at onMessage ([eval]:828:13)    at process.emit (node:events:524:28)    at emit (node:internal/child_process:950:14)    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)",
  },
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

after:

Execution {
  results: [],
  logs: {
    stdout: [],
    stderr: [ "(node:322) ExperimentalWarning: vm.USE_MAIN_CONTEXT_DEFAULT_LOADER is an experimental feature and might change at any time\n(Use `node --trace-warnings ...` to show where the warning was created)\n" ],
  },
  error: undefined,
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

(see my note regarding the warning)

snippet:

const s: string = "Hello, world!"
s

before:

Execution {
  results: [],
  logs: {
    stdout: [],
    stderr: [],
  },
  error: ExecutionError {
    name: "SyntaxError",
    value: "Missing initializer in const declaration",
    traceback: "evalmachine.<anonymous>:2  const s: string = \"Hello, world!\"        ^SyntaxError: Missing initializer in const declaration    at new Script (node:vm:117:7)    at createScript (node:vm:269:10)    at Object.runInThisContext (node:vm:317:10)    at run ([eval]:1020:15)    at onRunRequest ([eval]:864:18)    at onMessage ([eval]:828:13)    at process.emit (node:events:524:28)    at emit (node:internal/child_process:950:14)    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)",
  },
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

after:

Execution {
  results: [
    Result {
      isMainResult: true,
      text: "Hello, world!",
      html: undefined,
      markdown: undefined,
      svg: undefined,
      png: undefined,
      jpeg: undefined,
      pdf: undefined,
      latex: undefined,
      json: undefined,
      javascript: undefined,
      raw: [Object ...],
      data: undefined,
      chart: undefined,
      extra: [Object ...],
      formats: [Function: formats],
      toJSON: [Function: toJSON],
    }
  ],
  logs: {
    stdout: [],
    stderr: [],
  },
  error: undefined,
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

snippet:

const res = await fetch("https://example.com")
await res.text()

before:

Execution {
  results: [],
  logs: {
    stdout: [],
    stderr: [],
  },
  error: ExecutionError {
    name: "SyntaxError",
    value: "await is only valid in async functions and the top level bodies of modules",
    traceback: "evalmachine.<anonymous>:2  const res = await fetch(\"https://example.com\")              ^^^^^SyntaxError: await is only valid in async functions and the top level bodies of modules    at new Script (node:vm:117:7)    at createScript (node:vm:269:10)    at Object.runInThisContext (node:vm:317:10)    at run ([eval]:1020:15)    at onRunRequest ([eval]:864:18)    at onMessage ([eval]:828:13)    at process.emit (node:events:524:28)    at emit (node:internal/child_process:950:14)    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)",
  },
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

after:

Execution {
  results: [
    Result {
      isMainResult: true,
      text: "<!doctype html>\\n' +\n  '<html>\\n' +\n  '<head>\\n' +\n  '    <title>Example Domain</title>\\n' +\n  '\\n' +\n  '    <meta charset=\"utf-8\" />\\n' +\n  '    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\\n' +\n  '    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\\n' +\n  '    <style type=\"text/css\">\\n' +\n  '    body {\\n' +\n  '        background-color: #f0f0f2;\\n' +\n  '        margin: 0;\\n' +\n  '        padding: 0;\\n' +\n  '        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\\n' +\n  '        \\n' +\n  '    }\\n' +\n  '    div {\\n' +\n  '        width: 600px;\\n' +\n  '        margin: 5em auto;\\n' +\n  '        padding: 2em;\\n' +\n  '        background-color: #fdfdff;\\n' +\n  '        border-radius: 0.5em;\\n' +\n  '        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\\n' +\n  '    }\\n' +\n  '    a:link, a:visited {\\n' +\n  '        color: #38488f;\\n' +\n  '        text-decoration: none;\\n' +\n  '    }\\n' +\n  '    @media (max-width: 700px) {\\n' +\n  '        div {\\n' +\n  '            margin: 0 auto;\\n' +\n  '            width: auto;\\n' +\n  '        }\\n' +\n  '    }\\n' +\n  '    </style>    \\n' +\n  '</head>\\n' +\n  '\\n' +\n  '<body>\\n' +\n  '<div>\\n' +\n  '    <h1>Example Domain</h1>\\n' +\n  '    <p>This domain is for use in illustrative examples in documents. You may use this\\n' +\n  '    domain in literature without prior coordination or asking for permission.</p>\\n' +\n  '    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\\n' +\n  '</div>\\n' +\n  '</body>\\n' +\n  '</html>\\n",
      html: undefined,
      markdown: undefined,
      svg: undefined,
      png: undefined,
      jpeg: undefined,
      pdf: undefined,
      latex: undefined,
      json: undefined,
      javascript: undefined,
      raw: [Object ...],
      data: undefined,
      chart: undefined,
      extra: [Object ...],
      formats: [Function: formats],
      toJSON: [Function: toJSON],
    }
  ],
  logs: {
    stdout: [],
    stderr: [],
  },
  error: undefined,
  executionCount: 1,
  text: [Getter],
  toJSON: [Function: toJSON],
}

[dobrac]

(node:322) ExperimentalWarning: vm.USE_MAIN_CONTEXT_DEFAULT_LOADER is an experimental feature and might change at any time\n(Use node --trace-warnings ... to show where the warning was created)\n

I would hide this from the user as its more related to our implementation than to the usage.

[mishushakov]

I have changed the way ESM imports are handled, have to verify that it no longer outputs any warnings, otherwise I can add a flag to the jupyter kernel that it should run node with --no-warnings flag

[0div]

Looks promising!

There's one thing i do not understand in the ijavascript fork: how do the dependencies change affect the behaviour in it? I don't see them being imported in the code changes?

Also it would be nice to make a PR for each original repo? nice contrib and that way if it's accepted and merged we can use them again in our deps?

I take it the code-interpeter js tests passed with this?

[mishushakov]

Hey Jonas,
I have replaced the said dependencies with our own versions with some patches. Main difference is we now have a pre-compile step which transforms the source code and the kernel now awaits asynchronous code.

PR to the original repo would be not very productive as the original projects were abandoned, which is why we had to fork them in the first place.

Existing tests pass, I removed the negative scenarios that should no longer be failing. New tests will be in a separate PR. I also confirmed there are indeed no warnings when trying to import the modules. Currently waiting for a deploy of the updated template-manager so we can update the code interpreter template.