Lerna creates identical cache records for different cache hashes

[sashuk]

Current Behavior

When nx cache is enabled, the output of the packages can mismatch the actual changes in the source code when lerna watch is used and code is changed rapidly (around 5 times in 3 seconds on Mac M1 Pro). In particular, different cache entries (obviously with different cache hash) have the same output result.

Please see the reproduction section, but it seems like something described in the scheme below is going on (please note that the problem is reproducible when multiple changes are made to the same file in a matter of second while lerna watch is running)

If nx cache is disabled in repository, problem goes away (but all the benefits of nx caching are gone).

Expected Behavior

When nx cache is enabled, all cached outputs are unique (there are no same outputs for different cache hashes) and they correspond to the actual source code that results in outputs.

Steps to Reproduce

⚠️ The steps below reproduce the issue from scratch in your local copy, but sashuk/picasso-test#2 already has .nx/cache folder added, so feel free to analyze the cache folder itself as it is exactly the same data that is a result of actions in screencasts below (so, there is no need to check it out and run locally)

1. Check out Demo lerna nx problem sashuk/picasso-test#2, install dependencies
2. Run npx nx reset, start nx daemon if it has not started
3. Run yarn start
4. After everything builds, open packages/base/Paper/src/Paper/Paper.tsx
5. Quickly change class name in https://github.com/sashuk/picasso-test/blob/demo-lerna-nx-problem/packages/base/Paper/src/Paper/Paper.tsx#L51 from bg-red-14 to bg-red-15, to bg-red-16, etc, practically like in the video below

Kapture.2024-05-14.at.16.18.47.mp4

6. At some point you will see that there are more than one cache record for the same change bg-red-14 (or any other class name that you have collided), search term ref: ref, className: twMerge('bg-red-14

7. You can identify which of the matching outputs does not belong to its cache hash. In the example below, build:package cached outputs are restored for bg-red-13 and bg-red-14 changes and the cache stored in .nx/cache/1235... folder belong to bg-red-13 cache hash, but it contains the output for bg-red-14 change

Kapture.2024-05-14.at.16.25.03.mp4

Environment

1. Lerna-powered repository (https://github.com/sashuk/picasso-test) has multiple packages that depend on each other
2. nx caching is configured for build:package command https://github.com/sashuk/picasso-test/blob/master/nx.json#L12, the output is the dist-package folder in every package

{
  "...": "",
  "targetDefaults": {
    "build:package": {
      "inputs": ["source", "^source"],
      "outputs": ["{projectRoot}/dist-package/**/*"],
      "cache": true,
      "dependsOn": ["^build:package"]
    }
  }
}

3. Repository has build:package:watch command (https://github.com/sashuk/picasso-test/blob/demo-lerna-nx-problem/package.json#L23) that utilises lerna watch command and rebuilds packages that were updated

{
  "scripts": {
    "build:package:watch": "lerna watch -- lerna run --scope=\\$LERNA_PACKAGE_NAME build:package --include-dependents",
    "start": "yarn -s build:package && yarn build:package:watch"
  }
}

4. npx lerna info

npx lerna info
lerna notice cli v8.1.2
lerna info versioning independent

 Environment info:

  System:
    OS: macOS 14.2.1
    CPU: (8) arm64 Apple M1 Pro
  Binaries:
    Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node
    Yarn: 1.22.17 - /opt/homebrew/bin/yarn
    npm: 10.5.0 - ~/toptal/picasso-test/node_modules/.bin/npm
  Utilities:
    Git: 2.40.1 - /opt/homebrew/bin/git
  npmPackages:
    lerna: ^8.1.2 => 8.1.2 

[JamesHenry]

@sashuk Thanks a lot for this detailed report, but did you try with the latest lerna 8.1.3 please?

[sashuk]

Hello, @JamesHenry! I updated sashuk/picasso-test#2 to lerna@8.1.3, the problem can still be replicated