Can the repo map be cached?

[mm-zacharydavison]

Issue

We run a monorepo, and have been able to use aider by dynamically generating an .aiderignore file based on the packages we specifically want to work on.

This gets us a repo map, but it still takes a really long time, as we're giving it around 4000 files.
Given that the repo map doesn't change that often, could it be optionally cached between invocations of aider?

I did notice that the tags for each file are cached, but not the repo map itself.

I don't mind making this change myself, but wanted to check if there was any reason why this wasn't the existing caching strategy.

Version and model info

No response

[paul-gauthier]

Thanks for trying aider and filing this issue.

The repo-map varies dynamically based on the state of the chat. Specifically, right now it depends on which files have been added to the chat. So it would be possible to cache it indexed on the set of added files. In the future the repo map is likely to be even more dynamic though.

Monorepos are definitely a challenge for aider, which hasn't been optimized to work with such a large number of files.

[mm-zacharydavison]

Thanks for the fast response! Makes sense to me.

For us, aider is really useful for automating repetitive refactor tasks, but the repo map building takes a large percentage of the working time.

If there's anything in the backlog we can do to improve monorepo functionality, we have some capacity to invest time.

[mm-zacharydavison]

For anyone who wants a quick-ish fix, we've been using this script to 'focus' aider in your current directory and add context manually outside that.

#!/bin/bash

# This is a wrapper for `https://aider.chat/`.
# Monorepo's can be too big for `aider` to handle the context.
# This script wraps `aider`, ignoring all directories except the directory you run it from.
# You can also add additional context manually using `--context`.
# Any arguments you provide will be passed through to `aider`.

# Recommend installing this in your own `/usr/local/bin` directory so you can call it from PATH.
# `sudo cp faider.sh /usr/local/bin/faider`

# Parse command line arguments
ARGS=("$@")
for index in "${!ARGS[@]}"; do
    arg=${ARGS[index]}
    case $arg in
        --context=*)
        CONTEXT="${arg#*=}"
        unset 'ARGS[index]'
        ;;
    esac
done

# Get the root directory of the git repository
GIT_ROOT=$(git rev-parse --show-toplevel)

# Create .aiderignore file in the root of the git repository
echo "*" > "$GIT_ROOT/.aiderignore"
echo "!$(basename "$PWD")" >> "$GIT_ROOT/.aiderignore"

# If context directories are provided, include them
if [ -n "$CONTEXT" ]; then
    IFS=',' read -ra DIRS <<< "$CONTEXT"
    for dir in "${DIRS[@]}"; do
        echo "!$dir" >> "$GIT_ROOT/.aiderignore"
    done
fi

# Run aider with remaining arguments
aider "${ARGS[@]}"

# Remove .aiderignore file
rm "$GIT_ROOT/.aiderignore"

Some people have anecdotally said they get better results from aider by doing this, even outside big repos, but I don't really have any concrete data on that.

[paul-gauthier]

FYI there's a ~2X performance improvement landing soon for the repo map.

[mm-zacharydavison]

FYI there's a ~2X performance improvement landing soon for the repo map.

Cool! I'm guessing this won't improve the max size it can support though?
Although I guess it could, since GPT-4o context is much bigger.

[paul-gauthier]

The main branch has the new performance improvements for the repo map, including caching components of the repo map.
You can get it by installing the latest version from github:

python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git

If you have a chance to try it, let me know if it works better for you.

[paul-gauthier]

I'm going to close this issue for now, but feel free to add a comment here and I will re-open or file a new issue any time.