Releases: artilleryio/artillery
Artillery v2.0.15
AWS Lambda
- AWS Lambda support is now considered stable (no longer experimental) (#2815)
- Fix error happening when multiple lambda tests were created in parallel for the first time (#2802)
- Remove
—container
flag (allrun-lambda
tests now run using AWS Container Images) (#2816) - Remove
worker exited with error
message and instead exit with non-zero exit code when expectations fail in AWS Lambda workers (#2808)
AWS Fargate
- Prevent CLI hanging when workers are terminated early (e.g. due to spot instance termination) #3190
- Remove
worker exited with error
message and instead exit with non-zero exit code when expectations fail in AWS Fargate workers (#2808)
Playwright
- Update Playwright version to 1.44.1 (#3189)
CLI
- Make HTTP tests take
strictCapture
option into account when set in defaults (#2795)
With contributions from: @bernardobridge @hassy
Artillery v2.0.14
This is a hotfix release to fix an issue introduced in Artillery v2.0.13 which would lead to excessive logging when running Playwright tests.
Artillery v2.0.13
AWS Lambda
Lambda Container Images
Artillery will now always use Lambda Container Images to run tests on AWS Lambda. The --container
flag introduced in v2.0.12 is no longer required and will be removed in the next release. Creating Lambda functions from .zip archives is no longer supported.
All existing tests should continue running normally without needing any changes. If you run into any issues please open an issue.
This change brings improved startup times for tests running on AWS Lambda and brings AWS Lambda support to parity with AWS Fargate.
Fixes
- Increase startup timeout time for Lambdas running in a VPC (#2778)
- Exit with non-zero code if a Lambda worker fails (#2766)
- Fix issue that could lead tests to crash when triggered from a Windows machine (#2767)
- Expose current worker ID to Lambda workers as
WORKED_ID
environment variable (#2754)
Playwright
- Accept
0
values fordefaultNavigationTimeout
anddefaultTimeout
settings (#2786) - A warning will be printed if a Playwright
testFunction
cannot be found (#2782)
Monitoring & observability (publish-metrics
plugin)
- Add
url
attribute to page spans when tracing is enabled for Playwright tests (#2779) - Fix issue which could cause Core Web Vitals to not be set as span attributes (#2779)
- A
test_id
attribute is no longer set on all metrics. The built-in$testId
variable may be used to set that attribute when needed (#2779) - Request timing phases such as DNS lookups and SSL handshake timings are now recorded as attributes instead of spans for HTTP tests (#2779)
Slack integration
Fix a number of issues that prevented the Slack plugin from working with tests running on AWS Fargate and AWS Lambda (#2754)
WebSocket
- Errors thrown by hook functions will now be tracked and reported (#2783)
With contributions from: @bbeesley @HugoImaios @InesNi @bernardobridge @hassy
Artillery v2.0.12
AWS Lambda
Artillery can now use Lambda Container Images to run tests on AWS Lambda. Pass the --container
flag to run-lambda
command to use a container instead of a zip file. No further configuration is required.
This change brings two main benefits:
- Much faster startup times for tests
- Ability to use certain features that were previously only available in tests running on AWS Fargate:
- Run test scripts that TypeScript hooks
- Use third-party dependencies
- Use the built-in
publish-metrics
plugin
Container-based Lambda functions will become the default option in a future release of Artillery.
Playwright
- Update Playwright to v1.44.0
- Fix tests failing on Fargate with
ENOSPC
error with Playwright tracing enabled #2717 - Improve reliability of trace collection:
- Reduce trace collection cooldown period to 1-5m
- Increase the number of concurrent trace recordings to 3 up from 1
- Track the number of collected/discarded recordings with
browser.traces_collected
andbrowser.traces_discarded
metrics
CLI & Core
- Make path to the config directory available to test scripts via the special
$dirname
variable #2614 docs - The
--scenario-name
flag will now default to using an exact string match, and fall back to using a regex #2709 - Clarify the meaning of
--tags
flag #2699 - Remove an unnecessary dependency #2703
Slack integration
- Artillery now includes an official Slack integration, which can be used to post test results to a Slack channel automatically #2688 docs
With contributions from: @bernardobridge @InesNi @hassy @43081j @MIJOTHY-V2
Artillery v2.0.11
Playwright
- Display errors from the Playwright engine in Artillery reports and the console (#2677) - docs
- Report errors from failed Playwright assertions (#2677) - docs
Fargate
CLI & Core
- Log an error message when using an unsupported expectation with the
expect
plugin (#2667)
This release brought to you by @InesNi
Artillery v2.0.10
CLI & Core
- Add
parallel
action for making parallel requests in HTTP tests
Fargate
- Add ability to suppress metrics output with
--quiet
flag (#2639) - docs - Add
eu-north-1
to supported regions (#2646)
Lambda
Playwright
- Fix issue preventing stack traces for errors from being displayed (#2643)
- Update Playwright version to
1.43.1
(#2644) - docs
Publish Metrics
- Make reporters using OpenTelemetry compatible with parallel requests (#2628)
This release brought to you by @hassy @InesNi @bernardobridge
Artillery v2.0.9
Publish Metrics
- Sending traces to AWS X-Ray is now available through the CloudWatch reporter (#2555) - Docs
- Allow replacing of all tracing span names with
replaceSpanNameRegex
config option (#2607) - Docs - Fix bug in Datadog reporter causing last batches of traces to occasionally drop (#2610)
- Fix errors happening when
sampleRate
tracing option set (#2601) - Fix errors in tracing happening when using HTTP streaming (#2601)
- Display number of spans exported in the Artillery report (
plugins.publish-metrics.spans.exported
) when tracing is enabled. (#2597)
CLI & Core
- Fix incorrectly declared dependencies in Artillery CLI (#2549)
Artillery v2.0.8
Artillery Cloud
- Fix issue preventing
ARTILLERY_CLOUD_API_KEY
environment variable from being picked up by tests on Fargate when using--dotenv
(#2585)
Metrics by Endpoint
- Report request URLs for requests that resulted in a network error such as
ETIMEOUT
orECONNRESET
(#2574) - docs
CLI and Core
- Fix issue causing some intermediate reports to not be reported (#2565)
- Fix issue with incorrect formatting of time units in “phase started” messages (#2581)
This release brought to you by @hassy @bernardobridge
Artillery v2.0.7
CLI and Core
- Add support for loading custom code from ES modules (#2522) - docs
- Add support for using
async
functions as hooks (#2522) - docs
Playwright
- Update Playwright version to 1.42.1
- Fix bug that prevented traces from Playwright tests from being sent to Datadog and Honeycomb (#2533)
AWS Fargate
- Tests running in Fargate will now use AWS Distro for OpenTelemetry instead of Datadog Agent for sending traces to Datadog (#2481) - docs
- Fix bug that prevented setting CPU and memory allocation with the
--launch-config
flag (#2526)
This release brought to you by @hassy @bernardobridge @InesNi
Artillery v2.0.6
Playwright
- Make
expect
from@playwright/test
work with Typescript support (#2488) - Documentation - Playwright’s baseURL is now set from target (#2493) - Documentation
- Update Playwright version to 1.41.2 (#2492) - Documentation
Artillery Cloud
- Load phases are now shown in Artillery Dashboard for Fargate tests (#2485)
CLI & Core
- Improve Typescript support to work with more packages, by exposing
config.bundling.external
to set packages that can’t be bundled as external (#2488) - Documentation - Make test run id available to test runs via
{{ $testId }}
in scenarios/config andvuContext.vars.$testId
in custom code (#2499) - Routine dependency updates (#2500)
Publish Metrics
- Add
test_id
attribute to metrics and traces sent via the OpenTelemetry reporter (#2505) - Documentation
Fargate
- Improve error message and handling when setting CloudWatch retention policy (#2489)
This release brought to you by @bernardobridge, @InesNi, @hassy