Jinyue Chen*, Lingyu Kong*, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, Xiangyu Zhang
- [2024/4/21] 🔥🔥🔥 We have released the web demo in Project Page. Have fun!!
- [2024/4/15] 🔥 We have released the code, weights and the benchmark data.
- Download the ChartSE images and jsons here.
- Modify json path at the beginning of
ChartSE_eval/eval_ChartSE.py. Then run eval script:
python ChartSE_eval/eval_ChartSE.py- Clone this repository and navigate to the code folder
git clone https://github.com/LingyvKong/OneChart.git
cd OneChart/OneChart_code/- Install Package
conda create -n onechart python=3.10 -y
conda activate vary
pip install -e .
pip install -r requirements.txt
pip install ninja
pip install flash-attn --no-build-isolation- Download the OneChart weights here.
python vary/demo/run_opt_v1.py --model-name /onechart_weights_path/Following the instruction, type 1 first, then type image path.
- Prepare the dataset and fill in the data path to
OneChart/OneChart_code/vary/utils/constants.py. Then a example script is:
deepspeed /data/OneChart_code/vary/train/train_opt.py --deepspeed /data/OneChart_code/zero_config/zero2.json --model_name_or_path /data/checkpoints/varytiny/ --vision_tower /data/checkpoints/varytiny/ --freeze_vision_tower False --freeze_lm_model False --vision_select_layer -2 --use_im_start_end True --bf16 True --per_device_eval_batch_size 4 --gradient_accumulation_steps 1 --evaluation_strategy "no" --save_strategy "steps" --save_steps 250 --save_total_limit 1 --weight_decay 0. --warmup_ratio 0.03 --lr_scheduler_type "cosine" --logging_steps 1 --tf32 True --model_max_length 2048 --gradient_checkpointing True --dataloader_num_workers 4 --report_to none --per_device_train_batch_size 16 --num_train_epochs 1 --learning_rate 5e-5 --datasets render_chart_en+render_chart_zh --output_dir /data/checkpoints/onechart-pretrain/- You can pay attention to modifying these parameters according to your needs:
--model_name_or_path,freeze_vision_tower,--datasets,--output_dir
- Vary: the codebase and initial weights we built upon!
Usage and License Notices: The data, code, and checkpoint are intended and licensed for research use only. They are also restricted to use that follow the license agreement of Vary, Opt.
If you find our work useful in your research, please consider citing OneChart:
@misc{chen2024onechart,
title={OneChart: Purify the Chart Structural Extraction via One Auxiliary Token},
author={Jinyue Chen and Lingyu Kong and Haoran Wei and Chenglong Liu and Zheng Ge and Liang Zhao and Jianjian Sun and Chunrui Han and Xiangyu Zhang},
year={2024},
eprint={2404.09987},
archivePrefix={arXiv},
primaryClass={cs.CV}
}