Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
59 Open 973 Closed
59 Open 973 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fix social_iqa answer choices
#1909 opened May 31, 2024 by haileyschoelkopf Loading…
1
mlx Model (loglikelihood & generate_until)
#1902 opened May 29, 2024 by chimezie Loading…
4
Complete task list from pr 1727
#1901 opened May 29, 2024 by anthony-dipofi Loading…
22
add arc_challenge_mt
#1900 opened May 29, 2024 by jonabur Loading…
Add new Lambada translations
#1897 opened May 28, 2024 by zafstojano Loading…
6
Add LegalBench tasks
#1878 opened May 23, 2024 by zafstojano Loading…
4
Add chat template
#1873 opened May 22, 2024 by KonradSzafer Loading…
17
Test coverage for optimum_lm.py
#1872 opened May 22, 2024 by zafstojano Loading…
Added tests for Anthropic LLMs
#1868 opened May 21, 2024 by zafstojano Loading…
Multiple Choice Questions and Large Languages Models: A Case Study with Fictional Medical Data
#1867 opened May 21, 2024 by maximegmd Loading…
2
Draft - Support ov models via genai
#1862 opened May 20, 2024 by sstrehlk Loading…
1
mmlu-pro for the Italian language
#1860 opened May 19, 2024 by giux78 Loading…
3
[WIP] Fix NeuralMagic tests
#1859 opened May 19, 2024 by haileyschoelkopf Loading…
5
Force BOS token usage in 'gemma' models for VLLM
#1857 opened May 19, 2024 by haileyschoelkopf Loading…
1
Add option in TaskManager to not index library default tasks ; Tests for include_path feature request A feature that isn't implemented yet.
#1856 opened May 19, 2024 by haileyschoelkopf Loading…
1
Fix m_mmlu target
#1853 opened May 18, 2024 by jordane95 Loading…
Implement Exams benchmark
#1852 opened May 17, 2024 by snova-zoltanc Loading…
2
Fix self.max_tokens in anthropic_llms.py
#1848 opened May 16, 2024 by lozhn Loading…
1
Adding LLaVa support
#1832 opened May 13, 2024 by ashvinnihalani Loading…
1
Financial PhraseBank (FPB) Eval Metric
#1815 opened May 9, 2024 by bcicc Loading…
Fix cost_estimate.py
#1810 opened May 8, 2024 by xksteven Loading…
5
Fix --gen_kwargs and VLLM (temperature not respected) bug Something isn't working.
#1800 opened May 7, 2024 by haileyschoelkopf Loading…
Make scripts.write_out error out when no splits match
#1796 opened May 7, 2024 by haileyschoelkopf Loading…
1
Vllm get tokenizer
#1794 opened May 6, 2024 by AguirreNicolas Loading…
1
Create task dharma2 - a small (300 qs) & wide (many topics) dataset
#1753 opened Apr 26, 2024 by UmerHA Loading…
6
ProTip! Adding no:label will show everything without a label.