Fix DS/QWEN Example

[yiliu30]

User description

Signed-off-by: yiliu30 [email protected]

Type of Change

• Remove the limit 64
• Fix QWen OoM

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Possible Issue Removing the --limit parameter might lead to unexpected behavior if the script is not designed to handle an unlimited number of samples. Ensure that the script can handle the increased load without issues. lm_eval --model vllm \ --model_args "pretrained=${MODEL_PATH},tensor_parallel_size=${TP_SIZE},max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False" \ --tasks $TASK_NAME \ --batch_size $BATCH_SIZE \ --log_samples \ --seed 42 \ --output_path ${OUTPUT_DIR} \ --show_config 2>&1 | tee ${OUTPUT_DIR}/log.txt Possible Issue Removing the --limit parameter might lead to unexpected behavior if the script is not designed to handle an unlimited number of samples. Ensure that the script can handle the increased load without issues. lm_eval --model vllm \ --model_args "pretrained=${MODEL_PATH},tensor_parallel_size=${TP_SIZE},max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False" \ --tasks $TASK_NAME \ --batch_size $BATCH_SIZE \ --log_samples \ --seed 42 \ --output_path ${OUTPUT_DIR} \ --show_config 2>&1 | tee ${OUTPUT_DIR}/log.txt

[PRAgent4INC]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Verify --limit removal impact Ensure that the absence of --limit does not lead to excessive resource usage or unintended behavior. examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseek/run_evaluation.sh [115-117] +--batch_size $BATCH_SIZE \ +--log_samples \ +--seed 42 \ - Suggestion importance[1-10]: 5 __ Why: The suggestion asks to verify the impact of removing --limit, which is a reasonable concern. However, it does not propose any concrete changes or improvements, so the score is moderate.	Low