chore(ci): run workflows based on file changes

[soonum]

This is done to imporve iteration time and feedback for devs.
There is no point to run the full test suite each time. A given development could impact only tfhe-zk-pok for example. In this case only tfhe-zk-pok test would run and thus cutting workflow duration from around 3 hours down to a few minutes.

[IceTDrinker]

Could we avoid the dispatch and have each workflow perform the file checks ?

Basically a first job run on github ubuntu instance generates the changed files that are relevant for the current workflow, if there are changes then continue in the workflow with the instance set up ?

I don't know if the indirection created by the workflow call is a good idea, is that the pattern concrete used ?

[soonum]

Concrete use the dispatch pattern indeed. Here's the pros and cons of this approach:

• Pros:
  • file changes check occurs only in place
• Cons:
  • need to pass around secrets for each target file inducing lot of duplication
  • need to be exhaustive in file checks and define dependencies chain in conditional expression (if ( core_crypto_tests == true && shortint_tests == true [...] )

Regarding your proposal (one file check in each workflow). I think it's a better approach in our case. We define once and for all the file-change check and we carry only one output in the job. Thus we eliminate the need to define long if expression to know if we run the tests or not.

I'll give a shot to your proposal then.

[IceTDrinker]

I think there is just a tiny bit more work to do to manage file dependency chains, that could be detected in the file chnange step

basically we change core, then we want to re run everything that depends on it

csprng likely means core needs to re run etc.

[mayeul-zama]

Thanks!

[IceTDrinker]

looks like we started instances but then ran no tests https://github.com/zama-ai/tfhe-rs/actions/runs/9254705023/job/25457030476?pr=1147 ?

@soonum

[IceTDrinker]

probably a check missing to not start instances at all if no file changed

[soonum]

probably a check missing to not start instances at all if no file changed

You're right. Have to fix this.