8000 GitHub - gautomdas/llava-gutted: A gutted llava model to quantize llama using 1.58 bits · GitHub
[go: up one dir, main page]

Skip to content

About

A gutted llava model to quantize llama using 1.58 bits

Resources

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

Gutted Llava

DISCLAIMER: Code taken from the official Llava Repo.

This is a gutted llava repo to quantize the llama portion and finetune it.

You need to download the TextVQA images and extract to playground/data/eval/textvqa/train_images.

To run:

python -m llava.serve.cli \
    --model-path liuhaotian/llava-v1.5-7b \
    --image-file "https://llava-vl.github.io/static/images/view.jpg" \
    --load-4bit

To test VQA:

python3 -m llava.eval.model_vqa_loader \
  --model-path liuhaotian/llava-v1.5-7b \
  --question-file ./playground/data/eval/textvqa/TextVQA_0.5.1_val.json \
  --image-folder ./playground/data/eval/textvqa/train_images \
  --answers-file ./playground/data/eval/textvqa/answers/llava-v1.5-7b_subset100.jsonl \
  --temperature 0 \
  --load-4bit

About

A gutted llava model to quantize llama using 1.58 bits

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages

0