[ML] AIOps Log Rate Analysis: Merge fetch queue for keyword and text field candidates.

[walterra]

Summary

Part of #181111.

So far we identified significant text field candidates before keyword field candidates. There were 2 problems with this: 1) The loading behaviour for text fields was not considered in the progress bar. 2) Analysing a text field can be quite slow we weren't taking advantage of the same queue we used to fetch keyword fields. If it was taking quite a while to analysis the text fields, it would appear the progress bar was "stuck". And we had to wait to finish those slow queries before we started fetching p-values for keyword fields.

This PR improves the behaviour by adding text field fetching to the same queue that fetches keyword fields. This means while a maybe slower text field is analysed, we can continue to fetch keyword based fields in parallel given the max concurrency.

It's a bit tricky to test because with the local datasets the analysis might be so fast you won't notice a difference. So for testing you could do the following: In the file significant_items_handler.ts at the top add:

const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

And further down around line 110 add the following delay:

   const pValuesQueue = queue(async function (payload: Candidate) {
      await delay(Math.random() * 10000);

      if (isFieldCandidate(payload)) {
 ...

This will randomly delay the next call for the text/keyword field so you should see the progress bar moving. The difference to before would be that the progress bar would not move while it analyses the text fields.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[peteharverson]

LGTM. Tested using the added delay in the significant items handler and could see the progress bar update while analysing text fields.

[alvarezmelissa87]

Code LGTM ⚡

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: bf08f01

Metrics [docs]

✅ unchanged

History

• 💚 Build #210505 succeeded 27c5b1f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra