Can't install REA archives using gnu tar

[patrickbkr]

REA archive filenames contain colons. Those colons trigger gnu tar functionality to try to download a file from a remote PC.
See timo/json_fast#79

I think the best fix would be to prefix the file to extract / introspect with ./ as that triggers gnu tar to not try to do fancy download things. But that needs to be done in a platform agnostic way.

Alternatively one could probe for the edition of tar and if it's supported add the --force-local command line parameter to explicitly disable that functionality.

[lizmat]

Alternately, I could change the filenames in the REA. Not looking forward to it, but it could be done.

[ugexe]

: isn't safe to use on e.g. Windows filesystems, but archive names should be as platform independent as possible. Actually I guess % isn't either (which I thought was ok) so that might be a similar issue. Ultimately you'll want the REA repo to be clone-able by Windows users and for them to access files the same as a non-Windows users, so I'm not really sure the best way to name them.

[ugexe]

That being said: I think that zef tar plugin should avoid the remote feature one way or another.

[lizmat]

ok, lemme see if I can drop the colons and < > from the file names

[ugexe]

S22 suggested naming like JSON-Fast-v1.tar.gz -- of course using a - as a separator doesn't work for a number of other reasons but does suggest the naming was somewhat considered long ago. As a somewhat related aside we should have been naming our distributions without ::

[lizmat]

yeah, using hyphens is not a good idea

[ugexe]

It looks like both bsdtar and gnutar support --force-local, but I don't see it for busybox tar. On Windows C:Projects\ is relative path which is the only reason I'm wary of just prefixing ./. I know I've said I didn't want to have feature detection in the plugins themselves, but if busybox tar isn't treating files with colons as remote files then that is probably the direction I'd lean just to avoid trying to figure out the Windows interactions.

@patrickbkr I know you've brought up busybox support before: can you happen to confirm that busybox will error when passed --force-local, and/or if it parses colons as remote filenames?

[patrickbkr]

@ugexe I have now tried with a busybox tar (booted up an alpine in a container). It does not care about colons in the filename and extracts such files just like files without a colon. When passing the --force-local parameter, it fails with tar: unrecognized option: force-local.

[lizmat]

I wonder whether it would help if the colons were also URL encoded?

Also, if anybody has an idea to use something instead of colons, I'd like to know :-)

[patrickbkr]

@lizmat I guess url encoding would help. This specific problem is caused by colons. URL encoding would get rid of those.
If URL encoding is too ugly, using substitutes for the common offenders is a quick solution. (e.g. substitute - for -- and : for -)

https://superuser.com/a/748264/349521 lists some more chars we should avoid: \/:*?"<>|

[ugexe]

Unfortunately Windows doesn't allow % in file names so percent encoding probably isn't going to help afterall.

[patrickbkr]

% in a filename works fine on my Windows 10. https://en.wikipedia.org/wiki/Filename states that % is reserved on RT-11, an OS discontinued in 1998.

[lizmat]

The colons in the source-urls of the REA are now URL-encoded as well, and this appeared to have fixed Raku/REA#3 . And possibly this one.

[patrickbkr]

@lizmat: Raku/REA#3 is a duplicate of this bug, so yes, installing from REA now fixed. Thanks!

I'd like to keep this bug open though, because making zef resilient to tar trying to access remote systems is still an open todo.

[lizmat]

@patrickbkr Perhaps change the title of this issue then :-)

[patrickbkr]

I prefer to keep the title as is, as the largest part of the discussion actually revolves around the original problem.

[ugexe]

fwiw I have a fix, but appveyor continues to fail in the exact same spot when testing PathTools and I don't know why. It was pointed out the length of the precomp path is 260+ (immediately after the last zef release a rakudo change was added which adds like 15 characters for a temp extension), but the way libuv handles paths that shouldn't be an issue (and indeed the problem doesn't happen on my own windows VM).

[lizmat]

I think this has been fixed by adapting the REA URLs ?