Issues

• Incorrect Results with FlexDecoding

  meta-pytorch/gpt-fast#233 · Filter by author zaptrem opened on Aug 19, 2025
  4

• mixtral-moe example fails to torch.compile

  meta-pytorch/gpt-fast#232 · Filter by author zou3519 opened on Jul 23, 2025
  4

• llama-3.1-8b sometimes giving weird results

  meta-pytorch/gpt-fast#230 · Filter by author zou3519 opened on Jul 23, 2025
  1

• Batteries included: promote some basic version/utils of reasonably fast offline/batched inference into PyTorch core (maybe based on gpt-fast, nano-vllm, torchao)

  meta-pytorch/gpt-fast#229 · Filter by author vadimkantorov opened on Jun 30, 2025
  2

• Error in TP: RuntimeError: get_group_info: no group info associated with the group name

  meta-pytorch/gpt-fast#228 · Filter by author zy-ning opened on Jun 9, 2025

• Rotary Embeddings Implementation

  meta-pytorch/gpt-fast#226 · Filter by author hello-fri-end opened on Apr 9, 2025

• Dynamo error when trying to run eval.py 'Tensor' object has no attribute 'mask_mod'

  meta-pytorch/gpt-fast#223 · Filter by author bobrenjc93 opened on Mar 8, 2025

• 对于源码没有给出的模型（如DeepSeek-R1-Distill-Qwen-1.5B）该如何使用这个方法？

  meta-pytorch/gpt-fast#220 · Filter by author yabuke opened on Feb 11, 2025

• AttributeError: 'Tensor' object has no attribute 'mask_mod'

  meta-pytorch/gpt-fast#219 · Filter by author jiapingW opened on Jan 15, 2025
  3

• Issue: Performance degradation with int8 quantization in multi-batch scenarios

  meta-pytorch/gpt-fast#218 · Filter by author kakarotzzz opened on Jan 14, 2025
  3

• int4 quant broken right now?

  meta-pytorch/gpt-fast#217 · Filter by author jerryzh168 opened on Dec 20, 2024
  3

• Llama 3.1 70B size mismatch for tok_embeddings.weight

  meta-pytorch/gpt-fast#215 · Filter by author albertbn opened on Dec 4, 2024

• Has anyone run this code with bs>1 and speculatively?

  meta-pytorch/gpt-fast#214 · Filter by author deafTim opened on Oct 25, 2024

• Mistake in 191 line if is_speculative=True generate.py ?

  meta-pytorch/gpt-fast#213 · Filter by author deafTim opened on Oct 23, 2024
  1

• Error with meta-llama/Llama-3.2-1B

  meta-pytorch/gpt-fast#211 · Filter by author deafTim opened on Oct 18, 2024
  2

• Request for Smaller Model Options (~1B Parameters)

  meta-pytorch/gpt-fast#210 · Filter by author deafTim opened on Oct 17, 2024

• Error with stories15M and stories110M

  meta-pytorch/gpt-fast#209 · Filter by author deafTim opened on Oct 17, 2024
  8

• Reasons for the poor effect of Speculative Sampling

  meta-pytorch/gpt-fast#198 · Filter by author JoeNan1 opened on Aug 26, 2024
  1

• Activation quantization support

  meta-pytorch/gpt-fast#194 · Filter by author ayyoobimani opened on Aug 12, 2024
  1

• trying to convert huggingface whisper model to pytorch

  meta-pytorch/gpt-fast#189 · Filter by author nullonesix opened on Jul 8, 2024
  1

• tokenizer.model

  meta-pytorch/gpt-fast#186 · Filter by author hasakikiki opened on Jun 27, 2024
  1

• It doesn't accelerate very well at L4

  meta-pytorch/gpt-fast#185 · Filter by author songh11 opened on Jun 25, 2024
  1

• getting different acceptance prob when using torch.compile after making a small change.

  meta-pytorch/gpt-fast#184 · Filter by author kalradivyanshu opened on Jun 22, 2024

• Missing Keys in state_dict

  meta-pytorch/gpt-fast#172 · Filter by author bjohn22 opened on May 6, 2024
  2

• Tensor Parallel Inside notebook

  meta-pytorch/gpt-fast#167 · Filter by author nivibilla opened on Apr 29, 2024
  3