Issue with run amd64 images on mac arm m1 after recreate podman machine

[arzahs]

Issue Description

I got a weird error qemu-x86_64-static: QEMU internal SIGSEGV {code=MAPERR, addr=0x20} when trying to run a container with amd64 arch on the arm Macbook after recreating machine with latest machine init.
It worked well before.

Steps to reproduce the issue

The issue is reproducing for me on all mac machines which has an arm processor.

1. Remove the old podman machine
2. Create a new podman machine using

podman machine init --rootful --now

3. Try to run the image with amd64 architecture. I tried different images with the same result. But for example:

podman run --rm --arch=amd64 node node --help

Describe the results you received

I got error with qemo:

Resolved "node" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/node:18-alpine...
Getting image source signatures
Copying blob sha256:8ae971f79f99381da4a83f2cb63aa502fb847cc81a2f270326753f6289562dfc
Copying blob sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8
Copying blob sha256:45a0166cf96b2a4f328191f78f73e68e0e340450a962ff6fc34013111c014d26
Copying blob sha256:832e0dc1fe41d061d47d41e00abf6a9dab0c399d69bae854ef1bffe1976c2df0
Copying config sha256:1835bef2bac85b1699df958a9ee9d867ec68551e192317c0a36f5e8d8987280b
Writing manifest to image destination
qemu-x86_64-static: QEMU internal SIGSEGV {code=MAPERR, addr=0x20}

Describe the results you expected

podman run --rm --arch=arm64 node node --help
Usage: node [options] [ script.js ] [arguments]
       node inspect [options] [ script.js | host:port ] [arguments]

Options:
  -                           script read from stdin (default if no
                              file name is provided, interactive mode
                              if a tty)
  --                          indicate the end of node options
  --abort-on-uncaught-exception
                              aborting instead of exiting causes a
                              core file to be generated for analysis
  --build-snapshot            Generate a snapshot blob when the
                              process exits. Currently only supported
                              in the node_mksnapshot binary.
  -c, --check                 syntax check script without executing
  --completion-bash           print source-able bash completion
                              script
  -C, --conditions=...        additional user conditions for
                              conditional exports and imports
  --cpu-prof                  Start the V8 CPU profiler on start up,
                              and write the CPU profile to disk
                              before exit. If --cpu-prof-dir is not
                              specified, write the profile to the

podman info output

$ podman info
host:
  arch: arm64
  buildahVersion: 1.35.4
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc40.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 98.08
    systemPercent: 0.78
    userPercent: 1.14
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "40"
  eventLogger: journald
  freeLocks: 2046
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.8.8-300.fc40.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 3804680192
  memTotal: 8300716032
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc40.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-3.fc40.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc40.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240426.gd03c4e2-1.fc40.aarch64
    version: |
      pasta 0^20240426.gd03c4e2-1.fc40.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-2.fc40.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 28m 50.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 1
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 63819460608
  graphRootUsed: 7317950464
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 7
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.0.3
  Built: 1715299200
  BuiltTime: Fri May 10 03:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.2
  Os: linux
  OsArch: linux/arm64
  Version: 5.0.3

### Podman in a container

No

### Privileged Or Rootless

Privileged

### Upstream Latest Release

Yes

### Additional environment details

```bash
podman machine info
host:
    arch: arm64
    currentmachine: podman-machine-default
    defaultmachine: ""
    eventsdir: /var/folders/mk/krgzfwvx6f7cjq2x4jk3572r0000gn/T/storage-run-501/podman
    machineconfigdir: /Users/sergejnelepa/.config/containers/podman/machine/applehv
    machineimagedir: /Users/sergejnelepa/.local/share/containers/podman/machine/applehv
    machinestate: Running
    numberofmachines: 1
    os: darwin
    vmtype: applehv
version:
    apiversion: 5.0.3
    version: 5.0.3
    goversion: go1.22.3
    gitcommit: d08315df35cb6e95f65bf3935f529295c6e54742
    builttime: Fri May 10 18:37:14 2024
    built: 1715355434
    osarch: darwin/arm64
    os: darwin

podman machine ssh
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`
Fedora CoreOS 40.20240504.2.0
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/tag/coreos

Last login: Tue May 14 21:03:08 2024 from 192.168.127.1
root@localhost:~# uname -a
Linux localhost.localdomain 6.8.8-300.fc40.aarch64 #1 SMP PREEMPT_DYNAMIC Sat Apr 27 18:11:03 UTC 2024 aarch64 GNU/Linux
root@localhost:~#

podman machine list
NAME                    VM TYPE     CREATED         LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default  applehv     29 minutes ago  Currently running  4           8GiB        60GiB

Additional information

No response

[tnk4on]

This issue is expected to be resolved with Rosetta support.
#21670

% podman machine inspect --format {{.Rosetta}}
true
% podman run --rm --arch=amd64 node node --help            
Usage: node [options] [ script.js ] [arguments]
       node inspect [options] [ script.js | host:port ] [arguments]

Options:
  -                           script read from stdin (default if no
                              file name is provided, interactive mode
                              if a tty)
...
Documentation can be found at https://nodejs.org/

[arzahs]

@tnk4on But how did it work before? I have another installation with the older machine where it works.

I see the difference only in the machine image.
It works on:

But it doesn't work on new machine:

[arzahs]

So, after additional testing it works with recent fedora 39 build. When you create machine with hardcoded previous
image.

Example:

podman machine init --disk-size 60 --rootful --cpus=4 --memory=8192 --image https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240407.3.0/aarch64/fedora-coreos-39.20240407.3.0-applehv.aarch64.raw.gz

[zoman-abhishek]

This works, really helpful.

[Luap99]

This is most likely a qemu-user-static regression then if it worked before so you should check rpm -q qemu-user-static on the working version and the non working version.

[Luap99]

sounds like https://gitlab.com/qemu-project/qemu/-/issues/2168

I move this to a discussion as this is not really a podman bug and given the Rosetta implementation should make it in the next version

[spolti]

for now, using f39 as suggested by @arzahs have fixed the issue here.

[stevpier]

podman is not working for me on MacOS; I've tried 5.0.3, 5.1.0-RC1, and 4.9.4. I've tried enabling rosetta. I've tried the alternate machine image (I get 'Error: short-name resolution enforced but cannot prompt without a TTY' from that image on all podman versions).

podman info:

host:
  arch: arm64
  buildahVersion: 1.33.7
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc39.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 98.95
    systemPercent: 0.46
    userPercent: 0.6
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "39"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.8.4-200.fc39.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 7829622784
  memTotal: 8301441024
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc39.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.fc39.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc39.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240220.g1e6f92b-1.fc39.aarch64
    version: |
      pasta 0^20240220.g1e6f92b-1.fc39.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 3m 43.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 63819460608
  graphRootUsed: 2254729216
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.9.4
  Built: 1711446045
  BuiltTime: Tue Mar 26 05:40:45 2024
  GitCommit: ""
  GoVersion: go1.21.8
  Os: linux
  OsArch: linux/arm64
  Version: 4.9.4

[agniBit3]

resolved? getting same error