Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft for dependencies cache reuse within same deploy #2769

Draft
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

Andrioden
Copy link
Contributor

@Andrioden Andrioden commented Dec 4, 2023

What issue type does this pull request address?
/kind enhancement

What does this pull request do? Which issues does it resolve?
resolves #2768

Please provide a short message that should be published in the DevSpace release notes
Fixed an issue where DevSpace ...

What else do we need to know?
This is an incomplete PR, my limited understanding of the go language discouraged me to continue as i could not see a pretty way forward to properly preserve the download history. I am also very uncertain how you would want it solved for your codebase.

So either take this an inspiration for what i would like, or maybe give some pointers on how I could move forward.

Copy link

netlify bot commented Dec 4, 2023

Deploy Preview for devspace-docs canceled.

Built without sensitive environment variables

Name Link
🔨 Latest commit ec81def
🔍 Latest deploy log https://app.netlify.com/sites/devspace-docs/deploys/65b7b57c81644a0008b4f25d

@Andrioden Andrioden changed the title Exampel code for dependencies cache reuse within same deploy Draft for dependencies cache reuse within same deploy Dec 5, 2023
Copy link
Collaborator

@lizardruss lizardruss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There may be an existing data structure to use for tracking which dependencies have been resolved.

There's not currently an easy way to e2e test changes to dependency resolution. I'll bring it up for discussion and see if we want to add something to support it.

@@ -31,6 +31,7 @@ type ResolverInterface interface {
// Resolver implements the resolver interface
type resolver struct {
DependencyGraph *graph.Graph
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may be able to use the DependencyGraph to avoid re-downloading / re-pulling dependencies.

@@ -130,7 +131,7 @@ func (r *resolver) resolveRecursive(ctx devspacecontext.Context, basePath, paren
continue
}

dependencyConfigPath, err := util.DownloadDependency(ctx.Context(), basePath, dependencyConfig.Source, ctx.Log())
dependencyConfigPath, err := util.DownloadDependency(ctx.Context(), basePath, dependencyConfig.Source, ctx.Log(), r.DownloadedIds)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be able to skip this by doing a check similar to this line.

Andrioden and others added 21 commits January 29, 2024 14:47
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Fabian Kramm <fab.kramm@googlemail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Luca Di Maio <luca.dimaio1@gmail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Luca Di Maio <luca.dimaio1@gmail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
This is a mitigation for devspace-sh#2448 (comment)

Signed-off-by: Matthias Riegler <matthias.riegler@ankorstore.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: xcming <326658575@qq.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Fabian Kramm <fab.kramm@googlemail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Test is currently failing due to devspace-sh#2415, so this is a temporary measure to allow unrelated PRs to be approved

Signed-off-by: Russell Centanni <russell.centanni@gmail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.2 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.15.2...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.8 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.14.8...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Maarten Deprez <mdp@si-int.eu>
Signed-off-by: André S. Hansen <andre.ok@online.no>
…od, to fix endless wait on container startup with multiple containers and startContainer enabled

Signed-off-by: Maarten Deprez <mdp@si-int.eu>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Maarten Deprez <mdp@si-int.eu>
Signed-off-by: André S. Hansen <andre.ok@online.no>
…mand

Signed-off-by: Russell Centanni <russell.centanni@gmail.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Limiardi Eka Sancerio <limiardi.sancerio@zendesk.com>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Signed-off-by: Maarten Deprez <mdp@si-int.eu>
Signed-off-by: André S. Hansen <andre.ok@online.no>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Avoid uncessary dependency downloading for devspace deploy
6 participants