-
Notifications
You must be signed in to change notification settings - Fork 190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: Allow automatic saving of trace files #1526
Comments
Right now my workaround for |
@yury-s I'd like to start working on this next if that's agreeable with you. If so, can we use this issue to gather requirements? I had some ideas on how we could start tackling this in a phased approach. |
The main open question related to tracing is how we want to manage per test artifacts. E.g in node.js version we have per test output directory where traces, screenshots etc are stored. The traced are copied there from temporary locations after the test has finished, probably we could follow the same pattern in java? |
@yury-s if we allow the user to specify a trace directory, we can have a directory structure like this:
Does that work for you? |
I'd try to follow as closely as possible the API/structure that we have in playwright test in node, because that way we'll be able to avoid some of the mistakes/regrets from the javascript version of playwright. The structure seems fine but I suggest we use more generic
|
Currently we don't provide For
I don't think JUnit exposes any extension hooks into the assertions. I can think of two ways to do this:
This will require some thought for sure. I don't have any ideas as of yet. Would we be able to do this in a phased approach? I think allowing the automatic handling of traces for the contexts provided by the fixtures is important, especially being able to only save on failures as that is currently not possible without some ugly workarounds. |
Good point. Let's let them manage it manually.
Yes, agreed. In Node.js currently Another source of tracing complexity in Node.js is serial mode tests. This is when you can have all the tests in the same file reuse one and the same context. This is one of our regrets and if we were doing it today, serial mode wouldn't be there and we would be more adamant recommending alternatives like One other feature that I believe we'll need to support sooner or later is managing traces for the tests that create more than 1 context. In Node.js version it is implemented via internal hooks on
Yes, we'd need to hook into existing assertions one way or the other. I believe the tracing is useful even without those junit asserts, especially given all the web assertions that playwright provides. So we can think how to added it later.
I think so. Didn't mean to block this work on that. |
馃殌 Feature Request
This is also kind of a bug but I decided to make it a feature request.
Feature Request: I would like the ability to automatically save trace files.
Bug: I cannot save trace file based on test status.
Example
An option in
OptionsFactory
that allows the user to configure when to save trace filesoff
,on
,retain-on-failure
and an option for a root trace directory. The fixtures would then automatically start the trace, and depending on the option and the test result, would save the trace files in the root trace directory.Motivation
As I said, this is a feature request/bug report. I think the benefits are clear but the bug needs to be explained.
Currently, the user can only save a trace file by doing something like this:
There is no way for the user to only save trace files if the test fails (
retain-on-failure
). JUnit recommends implementing the TestWatcher interface. The problem is that the test watcher methods are called after theAfterEachCallback
methods, so when the test watcher method is called, theBrowserContext
is already closed by theBrowserContextExtension
. So even if I use reflection to get at theBrowserContext
inside myTestWatcher
, an exception is thrown.If we allow the fixtures to handle traces this issue can be easily solved
The text was updated successfully, but these errors were encountered: