Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Include custom steps, requirements in pipeline config file and allow_remote_code flag #573

Open
gabrielmbmb opened this issue Apr 24, 2024 · 0 comments
Labels
enhancement New feature or request
Milestone

Comments

@gabrielmbmb
Copy link
Member

Description

Currently, the pipelines that can be executed directly from the CLI or when loaded with Pipeline.from_yaml are those which include only steps from distilabel package.

The idea is to also serialize custom steps (as safe as possible) and include them in the pipeline config file, so if an user desires to execute the pipeline from the CLI, it can be executed if allow_remote_code=True.

In addition, some pipelines requires some dependencies to be executed (transformers, openai, etc). These requirements should be also added to the pipeline config file and when executing from CLI and if not installed, then prompt the user to consent the install of these requirements.

@gabrielmbmb gabrielmbmb added this to the 1.2.0 milestone Apr 24, 2024
@gabrielmbmb gabrielmbmb added the enhancement New feature or request label Apr 24, 2024
@alvarobartt alvarobartt modified the milestones: 1.2.0, 1.3.0 May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

2 participants