Pipeline output is not displayed until a newline is reached

[jackwilsdon]

What happened?
Text printed from a pipeline is not output to the terminal until a newline is reached.

What did you expect to happen instead?
Text should be printed without needing a newline.

How can we reproduce the bug? (as minimally and precisely as possible)

1. Create this devspace.yaml:

   version: v2beta1
   pipelines:
     test: |
       echo -n "Text without a newline"
       sleep 1
       echo "... finished!"

2. Run devspace run-pipeline test
3. Note that "Text without a newline" doesn't appear until "... finished!" is printed

Local Environment:

• DevSpace Version: 6.3.3
• Operating System: macOS Sonoma 14.0
• ARCH of the OS: arm64

Anything else we need to know?
This is a bigger issue when using read -p "Some prompt here: ", as the prompt doesn't appear until the user has hit enter (completing the read).

[jackwilsdon]

From what I can tell, this is caused by this code (Scan only returns when it finds a newline):

devspace/pkg/devspace/pipeline/job.go

Lines 101 to 107 in 69e7df3

	parent.Go(func() error {
	s := scanner.NewScanner(stdoutReader)
	for s.Scan() {
	ctx.Log().Info(s.Text())
	}
	return nil
	})

Not sure what the right fix is, as it seems like pkg/util/log doesn't have any support for logging without a newline.

[lizardruss]

@jackwilsdon Thanks for the analysis! We will take a look at a fix. I had previously assumed that this was an issue with the library we were using for the shell emulation, but you're probably right in that it's the scanner using the ScanLines function to tokenize the reader output.

[lizardruss]

@jackwilsdon While not a solution for this particular problem, I've found that read -p "Some prompt here: " works as expected in commands. The read values can then be passed into DevSpace using the --var flag. Hope that's helpful.