Discussions

• 1 You must be logged in to vote

  🙏

  Speculative decoding example in e2e_grpc_speculative_decoding_client.py

  jxchenus asked May 31, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Any Speculative Decoding Performance Numbers?

  jxchenus asked May 30, 2024 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why do I get this error after executing the command python3 scripts/launch_triton_server.py --world_size 1 --model_repo=llama_ifb/?

  SevenEmotion asked Apr 12, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  What's the difference between ensemble and tensorrt_llm_bls?

  Saigut asked Apr 10, 2024 in Q&A · Closed · Answered

  

  

  1