Skip to content

Commit b856371

Browse files
committed
Check jsonl ext in quantize.py so local datasets can also be used
Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com>
1 parent cf17ff3 commit b856371

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎examples/post_training/modelopt/quantize.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,7 @@ def get_calib_dataloader(
236236
237237
Supports either a local path (.jsonl) or a HuggingFace dataset name.
238238
"""
239-
if os.path.isfile(dataset_path_or_name):
239+
if os.path.isfile(dataset_path_or_name) and dataset_path_or_name.endswith(".jsonl"):
240240
# Local file
241241
print_rank_0(f"Loading calibration dataset from local file: {dataset_path_or_name}")
242242
all_texts = []

0 commit comments

Comments
 (0)