Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration instructor #654

Merged
merged 43 commits into from
May 29, 2024
Merged

Integration instructor #654

merged 43 commits into from
May 29, 2024

Conversation

plaguss
Copy link
Contributor

@plaguss plaguss commented May 21, 2024 •
edited

Description

This PR integrates instructor for structured generation of LLMs.
It includes support for the following LLMs:

  • openai/azure, anthropic, litellm, mistral, cohere and groq.

An example pipeline can be seen in the /examples/structured_generation_with_instructor.py.

The resulting dataset can be seen here.

Also closes #661

@plaguss plaguss added enhancement New feature or request integrations labels May 21, 2024
@plaguss plaguss added this to the 1.2.0 milestone May 21, 2024
@plaguss plaguss linked an issue May 21, 2024 that may be closed by this pull request
[INTEGRATION] Add instructor to generate structured outputs from private APIs #625
Closed
@plaguss plaguss self-assigned this May 21, 2024
@plaguss plaguss linked an issue May 22, 2024 that may be closed by this pull request
[BUG] Cannot import create_distiset directly. #661
Closed
plaguss added 12 commits May 22, 2024 15:57
@plaguss plaguss marked this pull request as ready for review May 23, 2024 14:24
gabrielmbmb
gabrielmbmb requested changes May 27, 2024
View reviewed changes
Copy link
Member

@gabrielmbmb gabrielmbmb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Just some comments related with code style and some missing docstrings.

docs/sections/learn/advanced/structured_generation.md Show resolved Hide resolved
docs/sections/learn/advanced/structured_generation.md Outdated Show resolved Hide resolved
docs/sections/learn/advanced/structured_generation.md Outdated Show resolved Hide resolved
docs/sections/pipeline_samples/examples/index.md Outdated Show resolved Hide resolved
src/distilabel/llms/base.py Show resolved Hide resolved
src/distilabel/steps/tasks/structured_outputs/utils.py Outdated Show resolved Hide resolved
src/distilabel/steps/tasks/structured_outputs/utils.py Outdated Show resolved Hide resolved
src/distilabel/steps/tasks/structured_outputs/utils.py Outdated Show resolved Hide resolved
src/distilabel/steps/tasks/structured_outputs/utils.py Outdated Show resolved Hide resolved
src/distilabel/steps/tasks/structured_outputs/utils.py Outdated Show resolved Hide resolved
plaguss and others added 13 commits May 27, 2024 12:18
@plaguss
Merge branch 'develop' of https://github.com/argilla-io/distilabel in…
f081f05 
…to integration-instructor
@plaguss @gabrielmbmb
Update docs/sections/learn/advanced/structured_generation.md
ddb7b74 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update docs/sections/learn/advanced/structured_generation.md
be9223a 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/instructor.py
05c0470 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/instructor.py
c398a43 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/utils.py
f2f8c27 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/instructor.py
ad8bc91 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/utils.py
fc3d117 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/utils.py
57ca6c8 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss @gabrielmbmb
Update src/distilabel/steps/tasks/structured_outputs/utils.py
a1fb750 
Co-authored-by: Gabriel Martín Blázquez <gmartinbdev@gmail.com>
@plaguss
Merge branch 'integration-instructor' of https://github.com/argilla-i…
2d1028e 
…o/distilabel into integration-instructor
@plaguss
Add changes from code review
fb1f716
@plaguss
Fix type hint per code review
bcd36c0
@plaguss plaguss requested a review from gabrielmbmb May 27, 2024 14:29
@plaguss plaguss merged commit 01b4292 into develop May 29, 2024
4 checks passed
@plaguss plaguss deleted the integration-instructor branch May 29, 2024 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alvarobartt alvarobartt alvarobartt approved these changes

@gabrielmbmb gabrielmbmb Awaiting requested review from gabrielmbmb

Assignees

@plaguss plaguss

Labels
enhancement New feature or request integrations
Projects
None yet
Milestone
1.2.0
3 participants
@plaguss @gabrielmbmb @alvarobartt