Skip to content

Commit

Permalink
Add compatibility warning
Browse files Browse the repository at this point in the history
  • Loading branch information
saharNooby committed Sep 20, 2023
1 parent 16511b6 commit 6d0f9f8
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions README.md
Expand Up @@ -10,7 +10,9 @@ This project provides [a C library rwkv.h](rwkv.h) and [a convinient Python wrap

Loading LoRA checkpoints in [Blealtan's format](https://github.com/Blealtan/RWKV-LM-LoRA) is supported through [merge_lora_into_ggml.py script](rwkv%2Fmerge_lora_into_ggml.py).

### Quality and performance
⚠️ **Python API was restructured on 2023-09-20**, you may need to change paths/package names in your code when updating `rwkv.cpp`.

## Quality and performance

If you use `rwkv.cpp` for anything serious, please [test all available formats for perplexity and latency](rwkv%2Fmeasure_pexplexity.py) on a representative dataset, and decide which trade-off is best for you.

Expand All @@ -26,7 +28,7 @@ Below table is for reference only. Measurements were made on 4C/8T x86 CPU with
| `FP16` | **15.623** | 117 | 2.82 |
| `FP32` | **15.623** | 198 | 5.64 |

#### With cuBLAS
### With cuBLAS

Measurements were made on Intel i7 13700K & NVIDIA 3060 Ti 8 GB. Latency per token in ms shown.

Expand Down

0 comments on commit 6d0f9f8

Please sign in to comment.