Pinned to Ideas

• 13 You must be logged in to vote

  💡

  GPU optimization across different cards

  performance Speed related topics hardware Hardware related

  JohannesGaessler started May 13, 2023 in Ideas

  16

Discussions

• 3 You must be logged in to vote

  💡

  Paligemma

  tucnak started May 15, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Asynchronous request parsing & prompt processing in server

  tristandruyen started May 14, 2024 in Ideas

  

  

  0
• 2 You must be logged in to vote

  💡

  Intel Compute Stick Support (Movidius Myriad 2, Myriad X)

  Oichkatzelesfrettschen started May 13, 2024 in Ideas

  

  

  0
• 11 You must be logged in to vote

  💡

  4-bit KV Cache

  research 🔬

  Dampfinchen started Mar 8, 2024 in Ideas

  

  

  

  

  18
• 1 You must be logged in to vote

  💡

  GGUF as directory and quantization parameters GUI

  Nexesenex started May 13, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Feature proposal: Layer shuffling

  foldl started May 9, 2024 in Ideas

  

  

  2
• 14 You must be logged in to vote

  💡

  Add support for Video-LLaVA

  psychodinae started Feb 22, 2024 in Ideas

  

  

  1
• 1 You must be logged in to vote

  💡

  Handing token redefinitions during model conversion

  fairydreaming started May 8, 2024 in Ideas

  

  

  

  

  4
• 3 You must be logged in to vote

  💡

  Self-frankenmerge support?

  marcingomulkiewicz started Apr 30, 2024 in Ideas

  

  

  

  

  5
• 1 You must be logged in to vote

  💡

  Any viable Assembly-level optimizations ideas?

  KaelaSavia started Apr 29, 2024 in Ideas

  

  

  0
• 3 You must be logged in to vote

  💡

  NPU support?

  sebastienbo started Apr 24, 2024 in Ideas

  

  

  0
• 23 You must be logged in to vote

  💡

  Neural Engine Support

  BrianSemiglia started Mar 20, 2023 in Ideas

  

  

  

  

  23
• 7 You must be logged in to vote

  💡

  Optimization to remove PCIe bandwidth limitations for large matrix multiplications on consumer GPU cards

  00prometheus started Mar 5, 2024 in Ideas

  

  

  

  

  6
• 1 You must be logged in to vote

  💡

  Resume interrupted quantization of a model.

  Nexesenex started Apr 6, 2024 in Ideas

  

  

  0
• 7 You must be logged in to vote

  💡

  Flash Attention 2 support possible?

  Dampfinchen started Jul 18, 2023 in Ideas

  

  

  

  

  3
• 3 You must be logged in to vote

  💡

  server should support tools and tool_choice parameter

  segmond started Apr 2, 2024 in Ideas

  

  

  

  

  2
• 1 You must be logged in to vote

  💡

  Model representation.. aren't weights layers actually are "images"?

  drazdra started Mar 27, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Add Support for cognitivecomputations/dolphin-phi-2-kensho and other models that are configured like it

  joseph777111 started Mar 26, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  GGML types IQ1_M and IQ2_M?

  Nexesenex started Mar 22, 2024 in Ideas

  

  

  3
• 2 You must be logged in to vote

  💡

  Possibly possible size optimization, with a better precision

  drazdra started Mar 16, 2024 in Ideas

  

  

  6
• 2 You must be logged in to vote

  💡

  BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences

  sorasoras started Mar 21, 2024 in Ideas

  

  

  0
• 4 You must be logged in to vote

  💡

  Make llama.cpp brew install-able on Mac!

  Vaibhavs10 started Mar 17, 2024 in Ideas

  

  

  1
• 3 You must be logged in to vote

  💡

  AQLM: Extreme Compression of Large Language Models via Additive Quantization

  joseph777111 started Mar 10, 2024 in Ideas

  

  

  0
• 2 You must be logged in to vote

  💡

  IQ3_S Improvements

  PeterReid started Mar 4, 2024 in Ideas

  

  

  5
• 6 You must be logged in to vote

  💡

  When are 1bit quants coming?

  superchargez started Feb 28, 2024 in Ideas

  

  

  

  

  11