Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Arch/mixtral #169

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

Arch/mixtral #169

wants to merge 17 commits into from

Commits on Jan 18, 2024

  1. Added kernel for layerNORMS from triton tutorials[https://triton-lang… 

    ….org/main/getting-started/tutorials/05-layer-norm.html]
    @cm2435
    cm2435 committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    1a21471 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2024

  1. formatting and typo fix

    @cm2435
    cm2435 committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    b60c138 View commit details
    Browse the repository at this point in the history

  2. starting to structure out test suite

    @cm2435
    cm2435 committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    598dda3 View commit details
    Browse the repository at this point in the history

  3. Contributed relu triton kernl, removed a small amount of boilerplate … 

    …from the layernorm, added test suite for kernls.
    @cm2435
    cm2435 committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    3954f16 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. update test to use common torch.allclose function util

    @cm2435
    cm2435 committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    48eb887 View commit details
    Browse the repository at this point in the history

  2. impliment forward and backward kernels for GeLU

    @cm2435
    cm2435 committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    4a7c20c View commit details
    Browse the repository at this point in the history

  3. fixed incorrect bwd pass for GeLU, wrapped functions in pytorch convi… 

    …ence handlers
    @cm2435
    cm2435 committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    b6d224b View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2024

  1. added partial scaling to rope emebeddings

    @cm2435
    cm2435 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    e520e9a View commit details
    Browse the repository at this point in the history

  2. added new seeded dropout kernel

    @cm2435
    cm2435 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    cbd31c3 View commit details
    Browse the repository at this point in the history

  3. updated tests

    @cm2435
    cm2435 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    b62c886 View commit details
    Browse the repository at this point in the history

  4. formatting

    @cm2435
    cm2435 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    e3e41a7 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. uncommented out init.py

    @cm2435
    cm2435 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    440ef5d View commit details
    Browse the repository at this point in the history

  2. doing work to impliment model

    @cm2435
    cm2435 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    a1e2b0d View commit details
    Browse the repository at this point in the history

  3. updated pre_patch for phi2

    @cm2435
    cm2435 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    f2112b1 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. wrote decoder model fwd for phi2

    @cm2435
    cm2435 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    0386c96 View commit details
    Browse the repository at this point in the history

  2. Added torch profiler to look at slowest torch modules

    @cm2435
    cm2435 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    11aa5df View commit details
    Browse the repository at this point in the history

  3. adding tools to be able to profile model fwds to see what to turn int… 

    …o kernels
    @cm2435
    cm2435 committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    6db5b12 View commit details
    Browse the repository at this point in the history