[Bug]: "Playwright Test for VSCode" v1.1.1 - Unable to "Execute Using Profile" anymore

[mbouchardC3]

Version

Playwright: 1.42.1
Playwright Test for VSCode extension: 1.1.1

Steps to reproduce

1. Go to "Testing" tab (of the Playwright extension)
2. Right-click on a spec file (test case)

Expected behavior

The option "Execute Using Profile" is available and we can debug run the test case (which is the ONLY option that works to debug our project, since the usual "Run | Debug" options don't work for us

Actual behavior

The option "Execute Using Profile" anymore. We instead have an option called "Start Continous Run Using"...
It's pretty bad, considering "Execute Using Profile" was the only way for us to debug our tests locally.
We now need to revert the extension version to continue working with it

Previous working version (1.0.22):

Latest broken version (1.1.1)

Additional context

No response

Environment

System:
    OS: Windows 11 10.0.22631
    CPU: (16) x64 Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
    Memory: 31.36 GB / 63.75 GB
  Binaries:
    Node: 20.11.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.7.0 - C:\Program Files\nodejs\npm.CMD
  IDEs:
    VSCode: 1.89.1 - C:\Users\mbouchard\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
  npmPackages:
    @playwright/test: 1.42.1 => 1.42.1

[mxschmitt]

Have you tried selecting different config files using the new settings menu?

[mbouchardC3]

Have you tried selecting different config files using the new settings menu?

I don't have any playwright.config.ts under my Playwright tab here

[yury-s]

I don't have any playwright.config.ts under my Playwright tab here

Would it be possible for you to share the project where it happens?

[mbouchardC3]

I don't have any playwright.config.ts under my Playwright tab here

Would it be possible for you to share the project where it happens?

I'd rather not share the project, as it has company intellectual property in it.

Any specific information you need that I could try to get for you?

[skonakalla455]

I see even the older version doesn't work. I reverted it to older version 1.0.22 and clicked the TRAINGLE icon to run/debug but it said "The test run did not record any output. "
Can anyone suggest What is the other alternative to debug my playwright type script test?

[trajchelazovski]

@mbouchardC3 I experienced the same initial confusion as you after upgrading to the latest version of the Playwright Test for VSCode extension 1.1.1.

In my case I had multiple projects defined and in order for the tests to show up in the files and test tab and the icons in the files you need to select the projects (previously called Default Profile).

I personally found it better as it was before (in version 1.0.22) since it allowed you to change on the fly the project.

[mbouchardC3]

I see even the older version doesn't work. I reverted it to older version 1.0.22 and clicked the TRAINGLE icon to run/debug but it said "The test run did not record any output. " Can anyone suggest What is the other alternative to debug my playwright type script test?

As mentioned in my initial post, I have the same problem as you. My alternate solution was to right click on the test and click on the "Execute Using Profile" option. Doing it from there works for us, but this option doesn't exist anymore in 1.1.1.

[mbouchardC3]

@mbouchardC3 I experienced the same initial confusion as you after upgrading to the latest version of the Playwright Test for VSCode extension 1.1.1.

In my case I had multiple projects defined and in order for the tests to show up in the files and test tab and the icons in the files you need to select the projects (previously called Default Profile).

I personally found it better as it was before (in version 1.0.22) since it allowed you to change on the fly the project.

I know about this settings change and I welcome it. However, choosing my projects like so only allows me to see them in my test explorer. It doesn't fix my problem that I cannot run/debug my tests locally like I could in the previous version

[skonakalla455]

I just reinstalled the older version and it started working where it allows me to debug the test. At least, this solution will help everyone who needs to debug their tests until the latest version is fixed.

[yury-s]

I'd rather not share the project, as it has company intellectual property in it.

Any specific information you need that I could try to get for you?

We need to understand why in your configuration with multiple playwright configs there is no cogwheel button that allows to select which configs to use. The problem will likely persist if you delete all sensitive stuff from your project and leave just the directory structure and dummy test files.

[mbouchardC3]

Do you need a screenshot from my project route? A sample of my playwright.config.ts?
I'll provide what is needed and filter out company IP if applicable

[yury-s]

Do you need a screenshot from my project route?

No, we cannot debug screenshots.

A sample of my playwright.config.ts? I'll provide what is needed and filter out company IP if applicable

My understanding of #30799 (comment) was that you have multiple playwright.config.ts in your project but you didn't have the cogwheel button like this. If that's the case, we need a minimal subset of your project files where this reproduces (including configs, tests etc).

However, choosing my projects like so only allows me to see them in my test explorer. It doesn't fix my problem that I cannot run/debug my tests locally like I could in the previous version

Can you elaborate on why you cannot debug your tests in this case? If you select the only project that you need in the sidebar and then click debug on a spec file/test case, does it not work or you want something else?

[mbouchardC3]

Do you need a screenshot from my project route?

No, we cannot debug screenshots.

A sample of my playwright.config.ts? I'll provide what is needed and filter out company IP if applicable

My understanding of #30799 (comment) was that you have multiple playwright.config.ts in your project but you didn't have the cogwheel button like this. If that's the case, we need a minimal subset of your project files where this reproduces (including configs, tests etc).

However, choosing my projects like so only allows me to see them in my test explorer. It doesn't fix my problem that I cannot run/debug my tests locally like I could in the previous version

Can you elaborate on why you cannot debug your tests in this case? If you select the only project that you need in the sidebar and then click debug on a spec file/test case, does it not work or you want something else?

Selecting only one project in the sidebar and clicking on the debug button does not work on my end. The test run automatically stops and gives me the error "The test run did not record any output."

I know you can't use screenshots for troubleshooting, but here's my project tree anyway

And here's my playwright.config.ts file

To resume, I have test cases that I store in a folder and in another folder, I have my test suites that uses those exported test cases. Here's an example of one of my test suites:

The suite is able to use the test case, since we export it like so:

And instead of using the test option from Playwright, we use a test extender from our global-before-after-each class

I blurred company info, but hopefully, this will be enough to help you understanding the issue. Thank you for taking time for this!

[skonakalla455]

We have a new version of Playwright Test for VSCODE 1.1.2 and it looks like it broke the older versions as well.
Old versions: Doesn't allow to debug
Latest version: Doesn't allow to debug

[yury-s]

@mbouchardC3 the message "Execute Playwright Suite" (and "Execute Playwright Suite") comes from some other extension, not Playwright.

[mbouchardC3]

@mbouchardC3 the message "Execute Playwright Suite" (and "Execute Playwright Suite") comes from some other extension, not Playwright.

I know. That's not what my issue is about. I don't use this "Execute Playwright Suite" snippet

[yury-s]

We compiled an FAQ explaining the recent changes in the VS Code Extension, it doesn't directly address the issue in this bug, but nevertheless may be helpful for better understanding the new approach to selecting different configurations.

Selecting only one project in the sidebar and clicking on the debug button does not work on my end. The test run automatically stops and gives me the error "The test run did not record any output."

Reading the entire thread again, I believe this is the fundamental issue and we need to focus on understanding why it's not working for you. @mbouchardC3, can you reduce your project to a minimal example that doesn't contain sensitive internal code, and share it with us, so that we can reproduce it locally and see why it's not working as intended?

[mbouchardC3]

@yury-s Sorry for the delay in response, long weekend here.

Anyway, after the release of the 1.1.5 extension AND upgrading playwright/test to latest, it works correctly now, even when I select all my projects!

If some people still struggle with this problem, try all latest versions (and don't forget to npx playwright install afterwards) to see if it works for you.

Thank you guys for the support!

TL;DR : The removal of the option "Execute Using Profile" is normal. Current latest versions (playwright/test 1.44.0 combined with Playwright Test for VS Code 1.1.5) work well together