zef install fails on a fresh (?) M1 macos brew install of rakudo star

[cognominal]

Context

This is a followup on rakudo/star#199

Expected Behavior

Installation of App::Mi6

Actual Behavior

Installation fails

Steps to Reproduce

zef --debug install App::Mi6
===> Searching for: App::Mi6
===> Found: App::Mi6:ver<3.0.4>:auth<zef:skaji> [via Zef::Repository::Ecosystems<fez>]
===> Dependencies: Pod::To::Markdown, Shell::Command, fez, TAP
===> Searching for missing dependencies: Pod::To::Markdown:ver<0.2.1+>, fez:ver<38+>
===> Found dependencies: fez:ver<55>:auth<zef:tony-o>:api<0> [via Zef::Repository::Ecosystems<fez>]
===> Found dependencies: Pod::To::Markdown:ver<0.2.1>:auth<cpan:SOFTMOTH> [via Zef::Repository::Ecosystems<rea>]
===> Dependencies: Pod::To::HTML
===> Fetching: App::Mi6
===> Fetching: Pod::To::Markdown:ver<0.2.1+>
===> Fetching: fez:ver<38+>
[App::Mi6] Fetching https://360.zef.pm/A/PP/APP_MI6/0869ab01c0d73b78b04600d0b6c5d73359af8914.tar.gz with plugin: Zef::Service::Shell::curl
[Pod::To::Markdown] Fetching https://raw.githubusercontent.com/raku/REA/main/archive/P/Pod%3A%3ATo%3A%3AMarkdown/Pod%3A%3ATo%3A%3AMarkdown%3Aver%3C0.2.1%3E%3Aauth%3Ccpan%3ASOFTMOTH%3E.tar.gz with plugin: Zef::Service::Shell::curl
[fez] Fetching https://360.zef.pm/F/EZ/FEZ/545919bd9786e02834ecf24e3d93febfba196bf3.tar.gz with plugin: Zef::Service::Shell::curl
===> Fetching [OK]: Pod::To::Markdown:ver<0.2.1>:auth<cpan:SOFTMOTH> to /var/folders/n0/_8g462950vgdhwl4s_h_jysw0000gn/T//.zef/1710712745.5407/1710712765.5407.7502.760373204599/Pod%3A%3ATo%3A%3AMarkdown%3Aver%3C0.2.1%3E%3Aauth%3Ccpan%3ASOFTMOTH%3E.tar.gz
===> Fetching [OK]: App::Mi6:ver<3.0.4>:auth<zef:skaji> to /var/folders/n0/_8g462950vgdhwl4s_h_jysw0000gn/T//.zef/1710712745.5407/1710712765.5407.3645.7283472892477/0869ab01c0d73b78b04600d0b6c5d73359af8914.tar.gz
===> Fetching [OK]: fez:ver<55>:auth<zef:tony-o>:api<0> to /var/folders/n0/_8g462950vgdhwl4s_h_jysw0000gn/T//.zef/1710712745.5407/1710712765.5407.4150.6831960378295/545919bd9786e02834ecf24e3d93febfba196bf3.tar.gz
===> Extracting: App::Mi6
Enabled extracting backends [git unzip path] don't understand /var/folders/n0/_8g462950vgdhwl4s_h_jysw0000gn/T//.zef/1710712745.5407/1710712765.5407.3645.7283472892477/0869ab01c0d73b78b04600d0b6c5d73359af8914.tar.gz
You may need to configure one of the following backends, or install its underlying software - [tar]

Your Environment

• raku -v

➜  raku raku -v
Welcome to Rakudo™ v2024.02.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2024.02.
➜  raku

* zef list --installed

➜ raku zef list --installed
===> Found via inst#/opt/homebrew/Cellar/rakudo-star/2024.02/share/perl6/core
rakudo:ver<2024.02>:auth
===> Found via inst#/opt/homebrew/Cellar/rakudo-star/2024.02/share/perl6/site
App::Prove6:ver<0.0.17>:authcpan:LEONT
App::Zef-Deps:ver<0.9.8>:authzef:coke
Config::Parser::json:ver<1.0.0>:authgithub:arjancwidlak
Config::Parser::yaml:ver<1.0.0>:authgithub:scriptkitties
Config::TOML:ver<0.1.2>:auth
Config:ver<3.0.4>:authcpan:TYIL:api<3>
Crane:ver<0.1.1>:auth
DBIish:ver<0.6.6>:authzef:raku-community-modules:api<1>
DateTime::Format:ver<0.1.5>:authzef:raku-community-modules:api<1.0>
DateTime::Parse:ver<0.9.3>:authgithub:sergot
Digest:ver<1.1.0>:authzef:grondilu
Distribution::Builder::MakeFromJSON:ver<0.6>:authgithub:niner
Encode:ver<0.0.4>:authgithub:sergot
File::Directory::Tree:ver<0.1>:authzef:labster
File::Find:ver<0.2.1>:authzef:raku-community-modules
File::Temp:ver<0.0.10>:authzef:rbt
File::Which:ver<1.0.4>
Getopt::Long:ver<0.4.2>
Grammar::Debugger:ver<1.0.1>:authgithub:jnthn
Grammar::Profiler::Simple:ver<0.05>:authzef:raku-community-modules
HTTP::Easy:ver<1.2>:authzef:raku-community-modules
HTTP::Status:ver<0.0.4>:authzef:lizmat
HTTP::UserAgent:ver<1.1.52>:authgithub:sergot
Hash::Merge:ver<2.0.0>:authgithub:scriptkitties:api<2>
IO::Capture::Simple:ver<v.0.0.2>:authzef:jjmerelo
IO::Glob:ver<0.9.0>:authgithub:zostay
IO::Path::XDG:ver<0.2.0>:api<0>
IO::Socket::SSL:ver<0.0.3>:authgithub:sergot
IO::String:ver<0.1.1>:auth
JSON::Class:ver<0.0.21>:authzef:jonathanstowe:api<1.0>
JSON::Fast:ver<0.19>:authcpan:TIMOTIMO
JSON::Marshal:ver<0.0.25>:authzef:jonathanstowe:api<1.0>
JSON::Name:ver<0.0.7>:authzef:jonathanstowe:api<1.0>
JSON::OptIn:ver<0.0.2>:authzef:jonathanstowe
JSON::RPC:ver<1.0.6>:authzef:bbkr
JSON::Tiny:ver<1.0>
JSON::Unmarshal:ver<0.15>:authzef:raku-community-modules
L10N::DE:ver<0.0.1>:authzef:l10n
L10N::EN:ver<0.0.1>:authzef:l10n
L10N::FR:ver<0.0.1>:authzef:l10n
L10N::HU:ver<0.0.1>:authzef:l10n
L10N::IT:ver<0.0.1>:authzef:l10n
L10N::NL:ver<0.0.1>:authzef:l10n
L10N::PT:ver<0.0.1>:authzef:l10n
L10N:ver<0.0.2>:authzef:l10n
LWP::Simple:ver<0.109>:authzef:raku-community-modules
LibraryCheck:ver<0.0.12>:authzef:jonathanstowe:api<1.0>
LibraryMake:ver<1.0.5>:authzef:jjmerelo
License::SPDX:ver<3.22.0>:authzef:jonathanstowe:api<1.0>
License::SPDX:ver<3.23.0>:authzef:jonathanstowe:api<1.0>
Log::Colored:ver<0.1.2>:api<0>
Log:ver<0.3.0>:authcpan:TYIL:api<0>
Log:ver<0.3.1>:authcpan:TYIL:api<0>
META6:ver<0.0.30>:authzef:jonathanstowe:api<1.0>
MIME::Base64:ver<1.2.3>:authzef:raku-community-modules
NativeCall::TypeDiag:auth
NativeHelpers::Blob:ver<0.1.12>:authgithub:salortiz
NativeLibs:ver<0.0.9>:authgithub:salortiz
OO::Monitors:ver<1.1.1>
OpenSSL:ver<0.2.2>:authgithub:sergot
PSGI:ver<1.2.1>:authzef:raku-community-modules
Path::Finder:ver<0.4.7>:authzef:leont
PathTools:ver<0.2.0>:authgithub:ugexe
Pod::Load:ver<0.7.2>:authzef:jjmerelo
Pod::To::BigPage:ver<0.5.2>:auth<Perl 6>
Pod::To::HTML:ver<0.7.1>:authgithub:Raku
Pod::Usage:ver<0.0.1>:authzef:leont
SVG::Plot
SVG:ver<1.0>
Shell::Command:ver<1.1>:authzef:raku-community-modules
System::Query:ver<0.1.6>:authzef:tony-o
TAP:ver<0.3.14>
Temp::Path:ver<1.001008>
Template6:ver<0.14.0>:authzef:raku-community-modules
Template::Mojo:ver<0.2.2>:authzef:raku-community-modules
Template::Mustache:ver<1.2.3>:authgithub:softmoth
Term::termios:ver<0.2.7>:authzef:krunen
Term::termios:ver<0.2>:authgithub:krunen
Terminal::ANSIColor:ver<0.10>:authzef:lizmat
Terminal::ANSIColor:ver<0.9.1>:authzef:lizmat
Terminal::ANSIParser:ver<0.0.3>:authzef:japhb
Terminal::Capabilities:ver<0.0.3>:authzef:japhb
Terminal::LineEditor:ver<0.0.16>:authzef:japhb
Test::META:ver<0.0.20>:authzef:jonathanstowe:api<1.0>
Test::Mock:ver<1.6>:authzef:jnthn
Test::Output:ver<1.001006>:authzef:raku-community-modules
Test::Util::ServerPort:ver<0.0.5>:authzef:jonathanstowe:api<1.0>
Test::When:ver<1.001009>:authzef:raku-community-modules
Testo:ver<1.003009>:authzef:raku-community-modules
Text::MiscUtils:ver<0.0.9>:authzef:japhb
URI:ver<0.3.7>:authzef:raku-community-modules
XML::Writer
XML:ver<0.3.3>:authzef:raku-community-modules
YAMLish:ver<0.1.0>:authzef:leont
sigpipe:ver<0.0.3>:authzef:leont
zef:ver<0.21.4>:authgithub:ugexe:api<0>

[ugexe]

Enabled extracting backends [git unzip path] don't understand /var/folders/n0/_8g462950vgdhwl4s_h_jysw0000gn/T//.zef/1710712745.5407/1710712765.5407.3645.7283472892477/0869ab01c0d73b78b04600d0b6c5d73359af8914.tar.gz

This implies you don't have the tar command available in your PATH. You can look at https://github.com/ugexe/zef/blob/main/lib/Zef/Service/Shell/tar.rakumod for an example of the code that gets run. Specifically the probe method, when it returns true, tells zef that it can find tar and use it.

FWIW I've installed fresh rakudos on various M1/M2/M3 macbooks and used zef successfully as-is, so it is strange that tar seems to not be findable on your system.

[cognominal]

Thx. Tar is accessible thru PATH env var. But something may be indeed wrong with my system. I have seen strange error messages when running docker failing to fork stuff thru bin/sh or whatever. That does not affect docker otherwise. This may be related.

[cognominal]

Problem solved, thx to your help. Zef found a tar executable in PDP11 (!!! see below) format and did not try to find further in the PATH env the path for a correct tar.
So zef could be more robust against that kind of time travel so I don't close the bug report. But really that's on me.

This is really comical. I deserve tar and feathers. And it is indeed doubly a tar problem. A few days ago, I pulled the tar of v7 unix because I wanted to see the sources of the prehistoric shell. I untared it in my home directory, which was a mistake because the tar has no prefix so the /bin files ended up in ~/bin. I was searching for the sources and I did not notice the tar contained the binaries as well so I did not clean it up.

$ cd ~/bin
$ file tar
tar: PDP-11 separate I&D executable
$ ls -l tar
-rwxr-xr-x  1 cog  staff  17724 May 16  1979 tar
$ which tar
/usr/bin/tar

And speaking of shells. Modern shells are robust. zsh does not even whine and execs the good tar.

[ugexe]

I’m not sure what a reasonable action for zef to take is in that scenario. There would have to be a way to determine the type of tar (bsd, gnu, busybox, etc) as well as the version from invoking the command itself, and as far as I know there is no way to do this. I’m also not really a fan of trying every version of a program in PATH to find one that works - that’s not how any shell works when invoking commands so is just as likely to lead to some other confusing behavior for some other edge case. Maybe I’m missing something though… do any other popular tools behave like that?

[cognominal]

Like I said, neither modern shells fall in that trap, nor the which command. So there must be a way to do it. Having my problem (that was clearly my fault) solved, I understand these may be a low priority target for you to work around buggy user settings.

I thank you for your helpful support and for writing zef in the first place.

My real concern would be it is real slow compared to npm but that may be a rakudo problem.

[ugexe]

I mentioned PATH earlier, but really zef just does the equivalent of run 'tar', '--help - it doesn't actually crawl PATH looking for what to run. It sounds like one of your shell files might be setting up PATH for an interactive shell but not for other processes or something. I'm not sure that e.g. which tar does any magic to find the "right" tar or tries multiple tars.

[cognominal]

Once I have suppressed the spurious PDP11 executables in my `~/bin' folder all problems and warning messages have disappeared on my mac. As for calling the right executables when spurious ones appear earlier in the PATH
, I don't know the correct idiom and how it would translate rakuwise.