Issues

• Why don't we use the pruned tokens to compute attention in the prefill stage?

  FMInference/H2O#51 · Filter by author xinhaoH opened on Jul 29, 2025

• Inquiry about the H2O version under llama cookbook folder

  FMInference/H2O#50 · Filter by author DeadCardassian opened on Mar 27, 2025

• ValueError: The current device_map had weights offloaded to the disk.

  FMInference/H2O#49 · Filter by author Winddfall opened on Mar 11, 2025

• version is not match

  FMInference/H2O#48 · Filter by author KeBbboy opened on Mar 1, 2025

• How to reproduce the result of infinite-length inputs?

  FMInference/H2O#47 · Filter by author shuhao02 opened on Dec 3, 2024

• For model generation，I fail with ERROR:KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

  FMInference/H2O#45 · Filter by author liyanboSustech opened on Aug 29, 2024

• For HELM testing, is an API, like togetherApiKey, necessary?

  FMInference/H2O#44 · Filter by author kkkvcache opened on Aug 19, 2024

• TypeError: forward() got an unexpected keyword argument 'cache_position'

  FMInference/H2O#42 · Filter by author benja-matic opened on Jul 25, 2024
  5

• ModuleNotFoundError: No module named 'streaming_llm'

  FMInference/H2O#40 · Filter by author benja-matic opened on Jul 25, 2024
  1

• Undefined functions

  FMInference/H2O#39 · Filter by author XdertodX opened on Jul 24, 2024

• Current transformers version v4.40 is not supported

  FMInference/H2O#38 · Filter by author maxjeblick opened on Jul 16, 2024

• Evaluation on discrimative datasets with short prompt.

  FMInference/H2O#37 · Filter by author yinwangsong opened on Jun 24, 2024

• Question about the reproduction of COPA results

  FMInference/H2O#36 · Filter by author hyeseongshin opened on Jun 19, 2024

• Recent Budget Simulation is incorrect

  FMInference/H2O#35 · Filter by author gopikrishnajha opened on Jun 13, 2024

• Eviction doesn't happen at all if recent_budget=0

  FMInference/H2O#33 · Filter by author foreverpiano opened on Jun 12, 2024

• No support of GQA of Llama in real_drop

  FMInference/H2O#32 · Filter by author Tomorrowdawn opened on May 27, 2024
  1

• The setting of experiments

  FMInference/H2O#31 · Filter by author YingYellow opened on May 22, 2024
  1

• Where is the streaming_utils

  FMInference/H2O#30 · Filter by author gawainx opened on Apr 28, 2024
  1

• Supporting new versions of transformers & batch size > 1

  FMInference/H2O#29 · Filter by author huangyuxiang03 opened on Apr 23, 2024

• KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

  FMInference/H2O#25 · Filter by author hasanar1f opened on Apr 2, 2024
  3

• Can not reproduce results by LLAMA-7B on OpenBook QA

  FMInference/H2O#24 · Filter by author AkideLiu opened on Mar 25, 2024
  9

• Replication of results with h2o_flexgen flex_opt.py

  FMInference/H2O#23 · Filter by author g-x-w opened on Mar 13, 2024
  1

• Is the flash attention version ready now?

  FMInference/H2O#21 · Filter by author York-Cheung opened on Feb 28, 2024

• Question about the reproduction of XSUM results

  FMInference/H2O#20 · Filter by author SherrySwift opened on Feb 3, 2024
  14

• Package Error when I reproduce as https://github.com/FMInference/H2O/tree/main/h2o_hf

  FMInference/H2O#19 · Filter by author KylinC opened on Jan 11, 2024
  4