e2e tests for examples #516

macintoshhelper · 2020-05-10T00:29:30Z

This PR introduces a new e2e test folder that runs the code from all examples at once.

To avoid instantiating each folder's npm packages, I'm installing every npm package that is needed for specific examples, and aliasing them with Webpack, e.g. chroma-js, etc. This does currently mean that a postinstall script to install the packages in __tests__/examples is required though.

npm run test:e2e appears to be broken since the TypeScript refactor (or later maybe), maybe non commonjs modules are being used somewhere. Importing yoga-layout-prebuilt crashes skpm-test, but that appears to be using commonjs though.

So for now, I'm creating a monolithic test example that is added to the test pipeline with npm run build --prefix __tests__/examples/, and can be manually run in Sketch GUI (or skpm-build --run), or will error out if there is a build breaking issue.

This should make it easy to test all examples with a single command.

This PR could be used later on to generate pngs from all the examples to do a pixel diff test, which would help a lot with the platform bridge refactor, and help out with alignment with React Native, and act as a benchmark for other React renderers to test implementation 🚀. This is something I noticed was mentioned in the React Primitives RFC (using Happo for cross-platform visual regression testing).

TODO

Fix skpm-test and use that instead for the examples test
Finish adding the rest of the examples

fixes #363

src/components/index.ts

macintoshhelper · 2020-05-10T00:50:00Z

package.json

@@ -38,6 +38,7 @@
    "test:unit": "jest --config jest.config.js --no-watchman",
    "test:ci": "npm run test:unit -- --runInBand",
    "test:e2e": "skpm-test",
+    "test:e2e:examples": "npm run build --prefix __tests__/examples/",


Maybe npm run render:once --prefix ... is better for now? Later on this should probably run with test:e2e, together with the other e2e tests.

…act-sketchapp into dev/e2e-example-tests

macintoshhelper · 2020-05-25T01:11:20Z

@mathieudutour I've got e2e tests working again, and have the e2e example tests working now too 🙂.

Took a while to debug it: I found that importing yoga-layout off this branch was crashing Sketch, but the same version of yoga-layout wasn't crashing in a commit snapshot before the TypeScript refactor, so I downgraded @skpm/test-runner to ^0.3 and that fixed it, so should probably open an issue over at https://github.com/skpm/skpm – I'm not really sure how to debug to get enough info, reproduce steps are npm run test:e2e off master pretty much.

There's a bit of clean up to do before a merge, but would be ideal to have a review first. I'm using packages that are installed in __tests__/examples/package.json for now. If a new example is added, then it has to be added manually to the e2e examples test file, together with the dependencies in __tests__/examples/package.json, so maybe this is something that should be mentioned in CONTRIBUTING.md? There should probably be a script to install the skpm/example dependencies too, but not sure on the best way to do this.

I'm still running into #515 (in test runner, and in a normal react-sketchapp project also), which fails the examples test:

 PASS  basic
 PASS  render-context
 PASS  render-in-wrapped-object

 FAIL  examples

  ● examples › should render examples

     React SketchApp cannot render into this layer. You may be trying to render into a layer that does not take children. Try rendering into a LayerGroup, Artboard, or Page.
      at <anonymous> (../../lib/render.js:23:14)
      at <anonymous> (./examples.test.js:72:2)
      at <anonymous> (../../node_modules/promise-polyfill/lib/index.js:79:0)
      at <anonymous> (../../node_modules/@skpm/timers/timeout.js:18:0)

  ✕ should render examples

Patching

src/components/index.ts

import * as Svg from './Svg';

to

import Svg from './Svg';

fixes it and gets the example e2e tests to pass (and const { Path } = Svg; still works).

mathieudutour · 2020-05-25T07:03:49Z

__tests__/skpm/examples.test.js

+  return sketch.getSelectedDocument() || document;
+}
+
+const examplePages = [


a lot fo this is duplicated in main.js. Could easily import it from there

mathieudutour · 2020-05-25T07:04:01Z

__tests__/skpm/.babelrc

@@ -0,0 +1,14 @@
+{


why do you need this file?

I pulled it from before the TypeScript refactor. I believe @babel/plugin-transform-modules-commonjs is needed for @skpm/test-runner? @skpm/test-runner@0.4.1 has this plugin, but without "noInterop": true.

Without the .babelrc file, the tests give this error:

undefined is not an object (evaluating 'module.exports.tests = __skpm_tests__')

mathieudutour · 2020-05-25T07:04:30Z

__tests__/examples/webpack.skpm.config.js

@@ -0,0 +1,24 @@
+const path = require('path');
+
+const aliasedModules = [


could we get this list from the package.json? Just to make sure nothing is missing

mathieudutour · 2020-05-25T07:10:34Z

package.json

@@ -80,7 +81,7 @@
  },
  "devDependencies": {
    "@skpm/babel-preset": "^0.2.1",
-    "@skpm/test-runner": "^0.4.1",
+    "@skpm/test-runner": "^0.3.2",


hum that's unfortunate. I'll try to see what's up

…encies

macintoshhelper · 2020-05-25T16:48:05Z

package.json

@@ -30,13 +30,15 @@
    "docs:watch": "npm run docs:prepare && gitbook serve",
    "docs:publish": "npm run docs:clean && npm run docs:build && cd _book && git init && git commit --allow-empty -m 'update book' && git fetch https://github.com/airbnb/react-sketchapp gh-pages && git checkout -b gh-pages && git add . && git commit -am 'update book' && git push https://github.com/airbnb/react-sketchapp gh-pages --force",
    "lint-staged": "lint-staged",
-    "prepublishOnly": "npm run clean && npm run test:ci && npm run build",
+    "prepublishOnly": "npm run clean && npm run test:ci && npm run install:e2e && npm run build",


npm run install:e2e triggers the __tests__/examples postinstall script. Not sure if this should be left in, it'll fail the prepublish script if the examples skpm-build fails, but not sure if it'll work in CI.

macintoshhelper · 2020-05-25T16:51:36Z

__tests__/examples/webpack.skpm.config.js

+    ...config.resolve,
+    alias: {
+      ...config.resolve.alias,
+      'react-sketchapp': path.resolve(__dirname, '../../lib'),


Seems that @skpm/test-runner needs the commonjs build, and doesn't work with the ES build, strange.

macintoshhelper added 5 commits May 10, 2020 01:14

create initial e2e test examples module

5ff6a22

export app components from examples for pure test imports

d1218ec

migrate emotion example to use @emotion/primitives + @emotion/core

8d95bee

remove flow types from form validation TextBox component

2c11811

fix svg export for es2015 airbnb#515

78c3170

macintoshhelper commented May 10, 2020

View reviewed changes

src/components/index.ts Outdated Show resolved Hide resolved

macintoshhelper added 2 commits May 10, 2020 01:46

add test:e2e:examples script to root package.json

863a2ed

add e2e examples comment [ci skip]

9c0ce94

macintoshhelper commented May 10, 2020

View reviewed changes

macintoshhelper mentioned this pull request May 10, 2020

Async rendering API and "pure" image fetching #472

Open

macintoshhelper changed the title ~~Integration tests for examples~~ e2e tests for examples May 10, 2020

undo import Svg fix attempt

91cd02c

macintoshhelper mentioned this pull request May 10, 2020

Svg import broken #515

Closed

macintoshhelper added 7 commits May 10, 2020 20:11

Merge branch 'master' into dev/e2e-example-tests

5100750

downgrade @skpm/test-runner to fix e2e tests

bd95ed3

use import alias to work with e2e tests

3850931

add babel/webpack setup for e2e example tests

fd830e3

add e2e examples test

1981ae8

Merge branch 'dev/e2e-example-tests' of github.com:macintoshhelper/re…

746b8e7

…act-sketchapp into dev/e2e-example-tests

fix e2e examples webpack module alias

31a6478

mathieudutour requested changes May 25, 2020

View reviewed changes

macintoshhelper added 3 commits May 25, 2020 17:09

migrate skpm webpack.config to __tests__/examples

29779eb

add prepublishOnly script for installing/building e2e examples depend…

38bb39c

…encies

re-use aggregate example App code/imports for e2e example

2bcf86c

macintoshhelper commented May 25, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

e2e tests for examples #516

e2e tests for examples #516

macintoshhelper commented May 10, 2020 •

edited

macintoshhelper May 10, 2020 •

edited

macintoshhelper commented May 25, 2020 •

edited

mathieudutour May 25, 2020

mathieudutour May 25, 2020

macintoshhelper May 25, 2020

mathieudutour May 25, 2020

mathieudutour May 25, 2020

macintoshhelper May 25, 2020

macintoshhelper May 25, 2020

		@@ -0,0 +1,24 @@
		const path = require('path');

		const aliasedModules = [

e2e tests for examples #516

Are you sure you want to change the base?

e2e tests for examples #516

Conversation

macintoshhelper commented May 10, 2020 • edited

macintoshhelper May 10, 2020 • edited

Choose a reason for hiding this comment

macintoshhelper commented May 25, 2020 • edited

mathieudutour May 25, 2020

Choose a reason for hiding this comment

mathieudutour May 25, 2020

Choose a reason for hiding this comment

macintoshhelper May 25, 2020

Choose a reason for hiding this comment

mathieudutour May 25, 2020

Choose a reason for hiding this comment

mathieudutour May 25, 2020

Choose a reason for hiding this comment

macintoshhelper May 25, 2020

Choose a reason for hiding this comment

macintoshhelper May 25, 2020

Choose a reason for hiding this comment

macintoshhelper commented May 10, 2020 •

edited

macintoshhelper May 10, 2020 •

edited

macintoshhelper commented May 25, 2020 •

edited