feat: let's have compressed keys as an option [WIP]

[bcm-at-zama]

[WIP for now]

let's check how it behaves, is there issues sometimes?

[bcm-at-zama]

eg, python -m pytest -xsvv "tests/sklearn/test_sklearn_models.py::test_separated_inference[True-6-fhe-XGBClassifier0]" --forcing_random_seed 17118411869559400649 --randomly-dont-reset-seed --randomly-seed=4004100345 fails, but it's not deterministic, and it's not the fault to our seeding system:

  assert array_allclose_and_same_shape(a, b, rtol, atol), error_message

E AssertionError: Not equal to tolerance rtol=0, atol=0.001
E a: [[0.06188408 0.93811592]]
E b: [[0.06611255 0.93388745]]
E
E assert False
E + where False = array_allclose_and_same_shape(array([[0.06188408, 0.93811592]]), array([[0.06611255, 0.93388745]]), 0, 0.001)

[BourgerieQuentin]

please retry experiments with coming nightly (fixed in zama-ai/concrete#691)

[bcm-at-zama]

Rebasing + using the new CP (which comes tomorrow, and has a fix for key compression)

[bcm-at-zama]

Can't test for now since concrete-python==2024.3.6 does not exist

[bcm-at-zama]

Trying to run https://github.com/zama-ai/concrete-ml/actions/workflows/update_licenses.yaml workflow even if Mac distribution is not there: I wonder if we need Mac for regular builds, I would say No

[bcm-at-zama]

The action fails because it tries to install CP for Mac

[bcm-at-zama]

At the same time, I have the impression that CP was not built for Linux as well, let's wait

[bcm-at-zama]

Let's launch with the new nightly.

[github-actions]

Coverage passed ✅

Coverage details

---------- coverage: platform linux, python 3.8.18-final-0 -----------
Name    Stmts   Miss  Cover   Missing
-------------------------------------
TOTAL    7125      0   100%

52 files skipped due to complete coverage.

[bcm-at-zama]

Tests are green now!

[andrei-stoian-zama]

• I think we should only set "compress_keys" through the Configuration if that's possible
• We should find out if there is any runtime latency difference with / without compression
• We should set the proper default based on the previous question

[andrei-stoian-zama]

I would rather we set this through the configuration and keep only ML related params here

[bcm-at-zama]

then, would be awesome to have a single place / class for crypto-related configs, and we would change just one place and it would apply to all models we support (linear, tree, qnn, torch). Maybe already the case!

[BourgerieQuentin]

As well there are runtime latency with compression, as you need to decompress on server side.

You should have quicker key generation on client side but longest evaluation time on server side as the keys are decompressed on demand.