Skip to content

Pull requests: mlfoundations/open_lm

New pull request New
29 Open 197 Closed
29 Open 197 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Allow tokenization for gzipped json files in datapreprocess
#285 opened Jun 3, 2024 by taidnguyen Loading…
Update requirements.txt
#284 opened May 29, 2024 by sedrick-keh-tri Loading…
fix for EOS/PAD tokens when not gpt-neox
#283 opened May 24, 2024 by jeffreywpli Loading…
Add option to predownload data from s3 at the start of each checkpoint.
#280 opened May 18, 2024 by GeorgiosSmyrnis Loading…
3
Parameter input rotary-freq
#263 opened Apr 30, 2024 by jmercat Loading…
1
Add loss like Rho-1
#260 opened Apr 27, 2024 by GeorgiosSmyrnis Loading…
Add dMoE
#257 opened Apr 25, 2024 by Muennighoff Loading…
Checkpoint skipping.
#256 opened Apr 21, 2024 by GeorgiosSmyrnis Loading…
Mamba update
#254 opened Apr 18, 2024 by jmercat Loading…
HF Integration
#248 opened Apr 12, 2024 by sedrick-keh-tri Loading…
1
Bug fix to import Llama in OpenLM.
#245 opened Apr 11, 2024 by kushal-tri Loading…
adding cosine rewarmed scheduler
#243 opened Apr 10, 2024 by Tomerporian Loading…
Change GeGLU and add MQA.
#239 opened Mar 31, 2024 by GeorgiosSmyrnis Loading…
Add unit test for source mixing + Fix naming within tars.
#233 opened Mar 12, 2024 by GeorgiosSmyrnis Loading…
3
Allow mixing for pretokenized data.
#230 opened Mar 8, 2024 by GeorgiosSmyrnis Loading…
@GeorgiosSmyrnis
1
Adding depth scale init support
#225 opened Mar 4, 2024 by kalyani7195 Loading…
[WIP] Adding support for FP8 training
#218 opened Feb 21, 2024 by shahromil16 Loading…
@shahromil16
20
[WIP] Attention across documents.
#213 opened Jan 31, 2024 by GeorgiosSmyrnis Loading…
8
added argument description in readme generate text
#211 opened Jan 29, 2024 by jmercat Loading…
remove double counting for jsonl.gzip files
#209 opened Jan 29, 2024 by jeffreywpli Loading…
Fix too many tokens requested edge case.
#201 opened Jan 16, 2024 by GeorgiosSmyrnis Loading…
Update README on tokenization.
#200 opened Jan 15, 2024 by GeorgiosSmyrnis Loading…
7
refactor params
#192 opened Jan 5, 2024 by jmercat Loading…
2
[DRAFT] Instruction tuning
#185 opened Dec 31, 2023 by kernelmachine Loading…
1
Finished the new 160m token/sec benchmark
#182 opened Dec 26, 2023 by sanyalsunny111 Loading…
ProTip! Exclude everything labeled bug with -label:bug.