Skip to content

Issues: pytorch/torchtitan

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

33 Open 47 Closed
33 Open 47 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

numerical difference for SDPA between non-dtensor vs dtensor, when math attention and fp16 are used bug Something isn't working
#317 opened May 8, 2024 by tianyu-l
freqs_cis in llama model should be a non-persistent buffer bug Something isn't working
#316 opened May 8, 2024 by tianyu-l
Question on Model Init question Further information is requested
#312 opened May 6, 2024 by XinDongol
6
add doc for adding custom dataset documentation Improvements or additions to documentation enhancement New feature or request
#311 opened May 5, 2024 by lessw2020
Converting to checkpoint.pd is not working
#307 opened May 4, 2024 by viai957
@wz337
4
freezeing some part of the model enhancement New feature or request
#306 opened May 3, 2024 by tianyu-l
reload existing llama checkpoints enhancement New feature or request
#305 opened May 3, 2024 by tianyu-l
1
add config option to only produce tensorboard logs on rank 0 enhancement New feature or request
#304 opened May 3, 2024 by tianyu-l
@tianyu-l
[Feature] Add gradient accumulation enhancement New feature or request
#292 opened May 1, 2024 by XinDongol
5
Make dataloader stateful? enhancement New feature or request
#291 opened May 1, 2024 by XinDongol Efficient data loading solution for large datasets
1
@gokulavasan @tianyu-l
8
[Feature] Plan to add model_register enhancement New feature or request
#282 opened Apr 28, 2024 by XinDongol
1
numerical issue when running SDPA with DTensor bug Something isn't working help wanted Extra attention is needed
#267 opened Apr 24, 2024 by tianyu-l
Fused RMSNorm incompatible with PP tracing (dynamic stride) bug Something isn't working
#217 opened Apr 10, 2024 by wconstab
@lessw2020
2
add unit test for ongoing numerical verification of fusedRMSNorm better_engineering Repo code quality improvements
#205 opened Apr 5, 2024 by lessw2020
@lessw2020
Make fused RMSNorm a registered op bug Something isn't working enhancement New feature or request
#199 opened Apr 5, 2024 by lessw2020
@tianyu-l
1
Verify that we can do eval / inference enhancement New feature or request
#192 opened Apr 4, 2024 by gnadathur
@kwen2501
1
Add support for MoE model architecture enhancement New feature or request
#184 opened Apr 2, 2024 by gnadathur
1
FSDP2 based HSDP support enhancement New feature or request
#177 opened Mar 29, 2024 by gnadathur torchtitan release 1.0
@awgu
Add HSDP + TP/SP support enhancement New feature or request
#176 opened Mar 29, 2024 by gnadathur torchtitan release 1.0
Checkpoint saves failing for eager mode training bug Something isn't working
#168 opened Mar 27, 2024 by chauhang First OSS release of Torchtrain
@wz337
7
Starting off with different models across ranks and FSDP doesn't synchronise bug Something isn't working
#166 opened Mar 26, 2024 by BadrYoubiIdrissi
4
Loss curve spikes on amalagamated datasets - need full scale shuffler in dataloader enhancement New feature or request
#128 opened Mar 12, 2024 by lessw2020 Efficient data loading solution for large datasets
2
metrics - add L1 gradient norm tracking enhancement New feature or request
#119 opened Mar 8, 2024 by lessw2020
consider - enable streaming attention as default for llama models (1-4M context) enhancement New feature or request
#86 opened Feb 25, 2024 by lessw2020
add 'dry run' flag - one iter, no saving, as quick proof to check basic perf and verify that environ is ready to go enhancement New feature or request
#73 opened Feb 23, 2024 by lessw2020
ProTip! Add no:assignee to see everything that’s not assigned.