2024-04-02 12:37:37 +00:00
|
|
|
We provide diverse examples about fine-tuning LLMs.
|
|
|
|
|
2024-05-06 15:07:55 +00:00
|
|
|
Make sure to execute these commands in the `LLaMA-Factory` directory.
|
|
|
|
|
|
|
|
## Table of Contents
|
|
|
|
|
|
|
|
- [LoRA Fine-Tuning on A Single GPU](#lora-fine-tuning-on-a-single-gpu)
|
|
|
|
- [QLoRA Fine-Tuning on a Single GPU](#qlora-fine-tuning-on-a-single-gpu)
|
|
|
|
- [LoRA Fine-Tuning on Multiple GPUs](#lora-fine-tuning-on-multiple-gpus)
|
|
|
|
- [Full-Parameter Fine-Tuning on Multiple GPUs](#full-parameter-fine-tuning-on-multiple-gpus)
|
|
|
|
- [Merging LoRA Adapters and Quantization](#merging-lora-adapters-and-quantization)
|
|
|
|
- [Inferring LoRA Fine-Tuned Models](#inferring-lora-fine-tuned-models)
|
|
|
|
- [Extras](#extras)
|
|
|
|
|
|
|
|
## Examples
|
|
|
|
|
2024-05-06 14:51:02 +00:00
|
|
|
### LoRA Fine-Tuning on A Single GPU
|
|
|
|
|
|
|
|
#### (Continuous) Pre-Training
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_pretrain.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
2024-05-13 12:39:36 +00:00
|
|
|
#### Multimodal Supervised Fine-Tuning
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llava1_5_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
2024-05-06 14:51:02 +00:00
|
|
|
#### Reward Modeling
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_reward.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### PPO Training
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_ppo.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### DPO Training
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_dpo.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### ORPO Training
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_orpo.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Preprocess Dataset
|
|
|
|
|
|
|
|
It is useful for large dataset, use `tokenized_path` in config to load the preprocessed dataset.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_preprocess.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Evaluating on MMLU/CMMLU/C-Eval Benchmarks
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli eval examples/lora_single_gpu/llama3_lora_eval.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Batch Predicting and Computing BLEU and ROUGE Scores
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_predict.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
### QLoRA Fine-Tuning on a Single GPU
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with 4/8-bit Bitsandbytes Quantization (Recommended)
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_bitsandbytes.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with 4/8-bit GPTQ Quantization
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_gptq.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with 4-bit AWQ Quantization
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_awq.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with 2-bit AQLM Quantization
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_aqlm.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
### LoRA Fine-Tuning on Multiple GPUs
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with Accelerate on Single Node
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/lora_multi_gpu/single_node.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with Accelerate on Multiple Nodes
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/lora_multi_gpu/multi_node.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/lora_multi_gpu/ds_zero3.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
### Full-Parameter Fine-Tuning on Multiple GPUs
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with Accelerate on Single Node
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/full_multi_gpu/single_node.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Supervised Fine-Tuning with Accelerate on Multiple Nodes
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/full_multi_gpu/multi_node.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Batch Predicting and Computing BLEU and ROUGE Scores
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/full_multi_gpu/predict.sh
|
|
|
|
```
|
|
|
|
|
|
|
|
### Merging LoRA Adapters and Quantization
|
|
|
|
|
|
|
|
#### Merge LoRA Adapters
|
|
|
|
|
2024-05-07 09:50:27 +00:00
|
|
|
Note: DO NOT use quantized model or `quantization_bit` when merging LoRA adapters.
|
|
|
|
|
2024-05-06 14:51:02 +00:00
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Quantizing Model using AutoGPTQ
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_gptq.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
### Inferring LoRA Fine-Tuned Models
|
|
|
|
|
|
|
|
#### Use CLI
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat examples/merge_lora/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Use Web UI
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli webchat examples/merge_lora/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Launch OpenAI-style API
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli api examples/merge_lora/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
### Extras
|
|
|
|
|
|
|
|
#### Full-Parameter Fine-Tuning using GaLore
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Full-Parameter Fine-Tuning using BAdam
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/badam/llama3_full_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### LoRA+ Fine-Tuning
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/loraplus/llama3_lora_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### Mixture-of-Depths Fine-Tuning
|
|
|
|
|
|
|
|
```bash
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/mod/llama3_full_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### LLaMA-Pro Fine-Tuning
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bash examples/extras/llama_pro/expand.sh
|
|
|
|
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml
|
|
|
|
```
|
|
|
|
|
|
|
|
#### FSDP+QLoRA Fine-Tuning
|
|
|
|
|
2024-05-06 13:47:00 +00:00
|
|
|
```bash
|
2024-05-06 14:51:02 +00:00
|
|
|
bash examples/extras/fsdp_qlora/single_node.sh
|
2024-04-02 12:37:37 +00:00
|
|
|
```
|