Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove cached_ar_key/cached_ar_value in prefill results to save memory. #636

Closed
wants to merge 12 commits into from
Closed

Remove cached_ar_key/cached_ar_value in prefill results to save memory. #636

wants to merge 12 commits into from

Commits on May 1, 2024

  1. Remove cached_ar_key/cached_ar_value in prefill results to save memory.

    @jwyang-google
    jwyang-google committed May 1, 2024
    Configuration menu
    Copy the full SHA
    cac835b View commit details
    Browse the repository at this point in the history

  2. Remove cached_ar_key/cached_ar_value in prefill results to save memory.

    @jwyang-google
    jwyang-google committed May 1, 2024
    Configuration menu
    Copy the full SHA
    abad157 View commit details
    Browse the repository at this point in the history

  3. Remove cached_ar_key/cached_ar_value in prefill results to save memory.

    @jwyang-google
    jwyang-google committed May 1, 2024
    Configuration menu
    Copy the full SHA
    f80b765 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Remove cached_ar_key/cached_ar_value in prefill results to save memory.

    @jwyang-google
    jwyang-google committed May 2, 2024
    Configuration menu
    Copy the full SHA
    193d104 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. fix norm sharding

    @ZhiyuLi-goog @jwyang-google
    ZhiyuLi-goog authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    262f840 View commit details
    Browse the repository at this point in the history

  2. Add Llama2-70b test

    @A9isha @jwyang-google
    A9isha authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    5c54ac2 View commit details
    Browse the repository at this point in the history

  3. Internal change only. 

    PiperOrigin-RevId: 630446330
    @jwyang-google
    maxtext authors authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    e82098e View commit details
    Browse the repository at this point in the history

  4. Add more tests for Mixtral

    @RissyRan @jwyang-google
    RissyRan authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    0d0a45e View commit details
    Browse the repository at this point in the history

  5. Make some AQT dataclasses to use keyword-only fields (1/N) 

    This cl introduces an temporary decorator that will be temporarily used during this migration. The eventual goal is to enforce kw_only=True in all dataclasses unless it's not feasible, aiming to make AQT less error-prune and improve readability.

PiperOrigin-RevId: 631132072
    @jwyang-google
    maxtext authors authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    07b02c5 View commit details
    Browse the repository at this point in the history

  6. Reverts e8b53e5 

    PiperOrigin-RevId: 631465526
    @golechwierowicz @jwyang-google
    golechwierowicz authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    321274b View commit details
    Browse the repository at this point in the history

  7. Replace deprecated np.product with np.prod

    @gobbleturk @jwyang-google
    gobbleturk authored and jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    51375d8 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into prefill-memory-optimize

    @jwyang-google
    jwyang-google committed May 7, 2024
    Configuration menu
    Copy the full SHA
    74b78c4 View commit details
    Browse the repository at this point in the history