Archive.is blocking archiving from it :(

[gerroon]

I wanted to archive this locally , since the original is a bloomberg article.

https://archive.is/2023.12.19-110503/https://www.bloomberg.com/features/2023-california-spring-water-bluetriton/

I got this :( Is there a way to archive from that site, so I have local copies? I can access the site no problem with the same IP, so this is definitely happening because of the archiving effort.

[pirate]

Is your DNS set to 1.1.1.1? The owner of Archive.is hates Cloudflare and famously shows this "unsolvable captcha" page to troll people using Cloudflare DNS.

Check if there's any chance your container, Docker VM, or host machine are set up to resolve DNS through 1.1.1.1.

The simple way

docker compose run archivebox add 'https://one.one.one.one/help/'

# then view the snapshot output to see if it shows you resolving DNS through cloudflare

The hard way

docker compose up -d
docker compose exec archivebox /bin/bash

apt update -qq
apt install -y dnsutils

nslookup archive.is
dig +trace archive.is

Check for any cloudflare domains or IPs in the DNS trace.

---

The Fix

Change the dns: section in docker-compose.yml to 8.8.8.8 to use Google's DNS servers. And/or change the CURL_USER_AGENT/WGET_USER_AGENT/CHROME_USER_AGENT archivebox config options to match your normal daily browser's user string (i.e. remove the ArchiveBox part from the default).

https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#curl_user_agent

# see the default user agent
archivebox config --get CHROME_USER_AGENT

# change the user agent
archivebox config --set CHROME_USER_AGENT='Mozilla/5.0 ...'

[gerroon]

thanks the user agent seemd to help.