Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[llama] implement rotary_embeddings_llama op #8577

Open
mikevin920 opened this issue May 16, 2024 · 0 comments
Open

[llama] implement rotary_embeddings_llama op #8577

mikevin920 opened this issue May 16, 2024 · 0 comments
Assignees
@mikevin920
Labels
llama2-70b LLM_feature LLMs on Metal

Comments

@mikevin920
Copy link
Contributor

No description provided.

@mikevin920 mikevin920 self-assigned this May 16, 2024
kevinmiTT11 pushed a commit that referenced this issue May 16, 2024
@mikevin920
#8577: first version implement rotary_embeddings_llama, python implem…
fcfc7a3 
…entation verified correctness
kevinmiTT11 pushed a commit that referenced this issue May 17, 2024
@mikevin920
#8577: WIP rotary_embeddings_llama first verision, hang in compute ke…
d1364ed 
…rnel
avoraTT added a commit that referenced this issue May 20, 2024
@avoraTT
#8577: Finished implementation for single core rotary_embeddings_llam…
b99fb07 
…a op. Passing test cases.
kevinmiTT11 pushed a commit that referenced this issue May 21, 2024
@mikevin920
#8577: Cleaned up rotary_embedding_llama unit test
b29805c
kevinmiTT11 pushed a commit that referenced this issue May 21, 2024
@mikevin920
#8577: multicore rotary_embeddings_llama WIP
90adcc2
avoraTT added a commit that referenced this issue May 21, 2024
@avoraTT
#8577: finished multi-core rotary_embedding_llama op implementation. …
3fd821c 
…Passing unit test. Need to test perf.
kevinmiTT11 pushed a commit that referenced this issue May 21, 2024
@mikevin920
#8577: Added cmake, added program cache support and proper testing fo…
37df732 
…r program cache, moved unnesscary runtime args to compile time args
kevinmiTT11 pushed a commit that referenced this issue May 21, 2024
@mikevin920
#8577: Speeds up kernel by reading and computing Wt tiles at a time
15fe140
caixunshiren added a commit that referenced this issue May 22, 2024
@caixunshiren
#8577: increased compute and write granularity to Wt
fc283fa
avoraTT added a commit that referenced this issue May 23, 2024
@avoraTT
#8577: Reduced sin/cos duplication across cores, resulting in perf bo…
81b2416 
…ost. Passing unit test.
avoraTT added a commit that referenced this issue May 23, 2024
@avoraTT
#8577: Passing unit tests with 4k sequence length. Op hangs for 8k se…
5f3e762 
…quence length.
avoraTT added a commit that referenced this issue May 23, 2024
@avoraTT
#8577: Fixed deadlock issue with CBs. Passing unit tests with 8k and …
dac80b6 
…16k sequence length.
avoraTT added a commit that referenced this issue May 24, 2024
@avoraTT
#8577: Added validation checks. Added 16k seq_len test case.
5f558c4
avoraTT added a commit that referenced this issue May 28, 2024
@avoraTT
#8577: Addressed PR comments. Cleaned up validation code, removed deb…
0603b59 
…ug statements. Removed single core implementation because multicore works for single core. Cleaned up cache testing.
avoraTT added a commit that referenced this issue May 28, 2024
@avoraTT
#8577: replacing operation::run_with_autoformat with operation::run.
6adf081
avoraTT added a commit that referenced this issue May 28, 2024
@avoraTT
#8577: Using ttnn instead of ttlib in pytester. Adding runtime/compil…
ab3e8cc 
…e time args. Cleaning up host code.
@caixunshiren caixunshiren mentioned this issue May 28, 2024
Open
5 tasks
avoraTT added a commit that referenced this issue May 30, 2024
@avoraTT
#8577: fixing unit tester for use_program_cache
81d5068
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: first version implement rotary_embeddings_llama, python implem…
3fe3904 
…entation verified correctness
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: WIP rotary_embeddings_llama first verision, hang in compute ke…
fb63c35 
…rnel
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Finished implementation for single core rotary_embeddings_llam…
3c743ce 
…a op. Passing test cases.
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Cleaned up rotary_embedding_llama unit test
2fb8e0f
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: multicore rotary_embeddings_llama WIP
a592099
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: finished multi-core rotary_embedding_llama op implementation. …
423788d 
…Passing unit test. Need to test perf.
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Added cmake, added program cache support and proper testing fo…
a6f6df3 
…r program cache, moved unnesscary runtime args to compile time args
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Speeds up kernel by reading and computing Wt tiles at a time
5850512
avoraTT pushed a commit that referenced this issue May 31, 2024
@caixunshiren @avoraTT
#8577: increased compute and write granularity to Wt
c83580e
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Reduced sin/cos duplication across cores, resulting in perf bo…
f4e62c2 
…ost. Passing unit test.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Passing unit tests with 4k sequence length. Op hangs for 8k se…
094810f 
…quence length.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Fixed deadlock issue with CBs. Passing unit tests with 8k and …
c49e147 
…16k sequence length.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Added validation checks. Added 16k seq_len test case.
b415fb1
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Addressed PR comments. Cleaned up validation code, removed deb…
013b5f9 
…ug statements. Removed single core implementation because multicore works for single core. Cleaned up cache testing.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: replacing operation::run_with_autoformat with operation::run.
bd1070d
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Using ttnn instead of ttlib in pytester. Adding runtime/compil…
6588eb9 
…e time args. Cleaning up host code.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: fixing unit tester for use_program_cache
8d46a44
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: first version implement rotary_embeddings_llama, python implem…
a969da2 
…entation verified correctness
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: WIP rotary_embeddings_llama first verision, hang in compute ke…
b4b0bbc 
…rnel
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Finished implementation for single core rotary_embeddings_llam…
16c08e4 
…a op. Passing test cases.
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Cleaned up rotary_embedding_llama unit test
d08e04b
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: multicore rotary_embeddings_llama WIP
9f59448
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: finished multi-core rotary_embedding_llama op implementation. …
5d3496e 
…Passing unit test. Need to test perf.
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Added cmake, added program cache support and proper testing fo…
23a4988 
…r program cache, moved unnesscary runtime args to compile time args
avoraTT pushed a commit that referenced this issue May 31, 2024
@mikevin920 @avoraTT
#8577: Speeds up kernel by reading and computing Wt tiles at a time
5b4b76e
avoraTT pushed a commit that referenced this issue May 31, 2024
@caixunshiren @avoraTT
#8577: increased compute and write granularity to Wt
abc668c
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Reduced sin/cos duplication across cores, resulting in perf bo…
d0c3aad 
…ost. Passing unit test.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Passing unit tests with 4k sequence length. Op hangs for 8k se…
c7298c8 
…quence length.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Fixed deadlock issue with CBs. Passing unit tests with 8k and …
be403ba 
…16k sequence length.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Added validation checks. Added 16k seq_len test case.
4007a1c
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Addressed PR comments. Cleaned up validation code, removed deb…
88c7f2e 
…ug statements. Removed single core implementation because multicore works for single core. Cleaned up cache testing.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: replacing operation::run_with_autoformat with operation::run.
40a30ee
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: Using ttnn instead of ttlib in pytester. Adding runtime/compil…
c5f662e 
…e time args. Cleaning up host code.
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT
#8577: fixing unit tester for use_program_cache
a2b3fc6
avoraTT added a commit that referenced this issue May 31, 2024
@avoraTT @mikevin920 @caixunshiren
Model team/rotary embeddings llama (#8812)
068bb60 
* #8577: implement fused rotary embeddings op for llama

---------

Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Jack Cai <caixunshiren@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikevin920 mikevin920

Labels
llama2-70b LLM_feature LLMs on Metal
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mikevin920