This repository contains the open-source implementation of TIP (Thought Switching Penalty), a decoding strategy proposed in the paper:
"Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs"
Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, Dong Yu
arXiv:2501.18585 [cs.CL]
Paper Link
TIP addresses the "underthinking" phenomenon in o1-like Large Language Models (LLMs), where models frequently switch between different reasoning thoughts without sufficiently exploring promising paths. This behavior leads to inadequate depth of reasoning and decreased performance, particularly on challenging mathematical problems.
TIP implements a simple yet effective decoding strategy:
- Identify Thought Switching Tokens: Detect tokens that indicate transitions between reasoning approaches (e.g., "Alternatively", "alternatively")
- Apply Penalty: Reduce the probability of thought switching tokens when they appear too frequently within a specified window
- Encourage Depth: Promote deeper exploration of each reasoning path before considering alternatives
# Simplified TIP algorithm
def apply_tip_penalty(token_ids, logits, penalty=3.0, window=600):
last_switch_pos = find_last_thought_switch(token_ids)
tokens_since_switch = len(token_ids) - last_switch_pos
if tokens_since_switch < window:
for switch_token_id in thought_switch_tokens:
logits[switch_token_id] -= penalty
return logits-
Clone the repository:
git clone <repository-url> cd underthinking
-
Install dependencies:
pip install -r requirements.txt
-
Verify installation:
python tip.py --help
# Basic TIP inference with default settings
python tip.py \
--input_file ./question.jsonl \
--output_file ./results \
--model_name_or_path Qwen/QwQ-32B-Preview \
--enable_tip \
--max_tokens 16000
# Advanced usage with custom TIP parameters
python tip.py \
--input_file ./question.jsonl \
--output_file ./results \
--model_name_or_path Qwen/QwQ-32B-Preview \
--enable_tip \
--tip_penalty 5.0 \
--tip_window 800 \
--temperature 0.0 \
--n_gpus 2| Argument | Type | Default | Description |
|---|---|---|---|
--input_file |
str | ./question.jsonl |
Path to input JSONL file |
--output_file |
str | ./output |
Base path for output file |
--model_name_or_path |
str | Qwen/QwQ-32B-Preview |
Model name or path |
| Argument | Type | Default | Description |
|---|---|---|---|
--enable_tip |
flag | False |
Enable TIP penalty |
--tip_penalty |
float | 3.0 |
Penalty value for thought switching tokens |
--tip_window |
int | 600 |
Token window for applying penalty |
| Argument | Type | Default | Description |
|---|---|---|---|
--temperature |
float | 0.0 |
Sampling temperature |
--top_p |
float | 1.0 |
Top-p sampling parameter |
--max_tokens |
int | 16000 |
Maximum tokens to generate |
--n |
int | 1 |
Number of samples per question |
| Argument | Type | Default | Description |
|---|---|---|---|
--n_gpus |
int | 1 |
Number of GPUs for tensor parallelism |
{"question": "Solve the equation: 2x + 3 = 7"}
{"question": "What is the derivative of x^2 + 3x + 1?"}{
"question": "Solve the equation: 2x + 3 = 7",
"response": [
{
"content": "To solve 2x + 3 = 7...",
"stop_reason": "stop"
}
]
}- Underthinking Correlation: Frequent thought switching correlates with incorrect responses
- Token Efficiency: Incorrect answers use 225% more tokens due to 418% more thought switches
- Performance Gains: TIP improves accuracy without requiring model fine-tuning
- Models should exhibit chain-of-thought reasoning
- Support for custom logits processors
- Compatible with vLLM inference engine
You can modify the thought switching detection by editing the identify_thought_switch_tokens() function:
# Add custom patterns
base_phrases = [
'Alternatively,', 'alternatively,',
'On the other hand,', 'However,',
'Let me try a different approach'
]# Multi-GPU setup
python tip.py --n_gpus 4 --model_name_or_path <large-model>
# Memory optimization
python tip.py --max_model_len 16384 --gpu_memory_utilization 0.8-
CUDA Out of Memory:
- Reduce
--max_tokensor--max_model_len - Increase
--n_gpusfor tensor parallelism - Lower
--gpu_memory_utilization
- Reduce
-
Model Loading Errors:
- Verify model path/name
- Check HuggingFace authentication for gated models
- Ensure sufficient disk space
-
Performance Issues:
- Use FP16 or quantization for faster inference
- Adjust batch size based on available memory
Enable detailed logging:
export PYTHONPATH=.
python -c "import logging; logging.basicConfig(level=logging.DEBUG)"
python tip.py --enable_tip --input_file small_test.jsonlIf you use this implementation in your research, please cite:
@article{wang2025thoughts,
title={Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs},
author={Wang, Yue and Liu, Qiuzhi and Xu, Jiahao and Liang, Tian and Chen, Xingyu and He, Zhiwei and Song, Linfeng and Yu, Dian and Li, Juntao and Zhang, Zhuosheng and Wang, Rui and Tu, Zhaopeng and Mi, Haitao and Yu, Dong},
journal={arXiv preprint arXiv:2501.18585},
year={2025}
}Note: This is an open-source reproduction of the TIP method for research purposes. The implementation maintains the core logic described in the paper while providing additional features for practical usage and reproducibility.