test case for adversarial example generation #1036

sonichi · 2023-05-12T17:29:33Z

Why are these changes needed?

Create a test case for generating adversarial instances

Related issue number

Checks

I've used pre-commit to lint the changes in this PR (note the same in integrated in our CI checks).
I've included any doc changes needed for https://microsoft.github.io/FLAML/. See https://microsoft.github.io/FLAML/docs/Contribute#documentation to build and test documentation locally.
I've added tests (if relevant) corresponding to the changes introduced in this PR.
I've made sure all auto checks have passed.

qingyun-wu · 2023-06-08T23:50:22Z

Consider using an agent-based implementation?

sonichi · 2023-06-16T17:49:13Z

test/autogen/test_adv_gen.py

+            test_func=case.test_func,
+            eval_func=case.eval_func,


Add comment about the meaning of these two functions? Do we need a better name for them?

test example draft

efc88be

sonichi assigned srinageshsharma May 12, 2023

sonichi changed the title ~~test example for adversarial example generation~~ test case for adversarial example generation May 12, 2023

srsharm and others added 3 commits May 24, 2023 10:56

Adding adversarial example api backend with LLM example

b9f8dd6

Merge branch 'main' into advgen

ad4d248

regex not working

fa7fa93

sonichi had a problem deploying to openai May 26, 2023 16:11 — with GitHub Actions Failure

sonichi had a problem deploying to openai May 26, 2023 16:11 — with GitHub Actions Error

debug parsing

d49a810

sonichi had a problem deploying to openai May 26, 2023 16:32 — with GitHub Actions Failure

functioning

b765c92

sonichi had a problem deploying to openai May 27, 2023 00:25 — with GitHub Actions Failure

functioning

52b13ec

sonichi had a problem deploying to openai May 27, 2023 00:27 — with GitHub Actions Failure

Adding Hydra to AdvGen, WikiQA test case

1ec0f7e

srinageshsharma had a problem deploying to openai June 2, 2023 12:21 — with GitHub Actions Failure

fix some bug

336be46

sonichi had a problem deploying to openai June 2, 2023 16:21 — with GitHub Actions Failure

sonichi requested a review from aminsaied June 7, 2023 16:19

sonichi requested review from mercerchen and myzwisc June 7, 2023 16:19

Adding programmatic baseline with modified eval

7ff01de

srinageshsharma had a problem deploying to openai June 15, 2023 16:00 — with GitHub Actions Failure

Fixes to manage inconsistent llm outputs

9f7e914

srinageshsharma had a problem deploying to openai June 15, 2023 17:53 — with GitHub Actions Failure

sonichi commented Jun 16, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test case for adversarial example generation #1036

test case for adversarial example generation #1036

sonichi commented May 12, 2023

qingyun-wu commented Jun 8, 2023

sonichi Jun 16, 2023 •

edited

test case for adversarial example generation #1036

Are you sure you want to change the base?

test case for adversarial example generation #1036

Conversation

sonichi commented May 12, 2023

Why are these changes needed?

Related issue number

Checks

qingyun-wu commented Jun 8, 2023

sonichi Jun 16, 2023 • edited

Choose a reason for hiding this comment

sonichi Jun 16, 2023 •

edited