Doesn't Artillery run scenarios in order?

[RemiBardon]

I'm running functional tests with --solo, and because of #2756, I moved my before scenarios inside of the scenarios section. However, when I run artillery, the first "Before" scenario isn't executed as I would expect it to be. I'm pretty sure it's the expected behavior (and that's why the before and after sections exist, but I didn't find an explicit explanation in the Docs.

So my question is simple: Does Artillery run all scenarios in order or not?

If not, I have a subsequent question: For each VU, does Artillery choose one scenario to run or does it run all of them in random order?

[bernardobridge]

Hey @RemiBardon 👋

Does Artillery run all scenarios in order or not?
For each VU, does Artillery choose one scenario to run or does it run all of them in random order?

Artillery chooses one scenario to run randomly based on their weight (to simulate real users arriving at your application and doing different things): https://www.artillery.io/docs/reference/test-script#scenario-weights

If you're interested in running something once in every VU, I'd recommend using beforeScenario, for example.

[RemiBardon]

Oh yes I remember reading about these probabilities! So if I want to cover every feature, I need one file per feature? I was hoping it wasn't the case because I'll have to copy-paste the config section and maintain it everywhere which is a bummer…

If you're interested in running something once in every VU, I'd recommend using beforeScenario, for example.

Well before is what I need, as I want to run something only once (a setup scenario). Also beforeScenario can only point to JS code, right?

[bernardobridge]

So if I want to cover every feature, I need one file per feature? I was hoping it wasn't the case because I'll have to copy-paste the config section and maintain it everywhere which is a bummer…

@RemiBardon you can maintain a config file separate from the scenario, and reuse it for all scenarios by using the --config flag. That's a common pattern with our users.

Well before is what I need, as I want to run something only once (a setup scenario). Also beforeScenario can only point to JS code, right?

Yes, that's right, but you can run any JS code there (including using an HTTP client if necessary).

[RemiBardon]

reuse it for all scenarios by using the --config flag

Oh that's handy, thanks!