Skip to content

GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface

License

Notifications You must be signed in to change notification settings

KerfuffleV2/rusty-ggml

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

rusty-ggml

GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface.

GG-what?

See:

  1. https://github.com/ggerganov/ggml/
  2. https://github.com/KerfuffleV2/ggml-sys-bleedingedge ( https://crates.io/crates/ggml-sys-bleedingedge )

WIP

Not suitable for general use. Consider this to be pre-alpha code.

v0.0.8 Warning: Keeping this in sync with recent GGML changes has lagged. It compiles and seems to work but there might be weird stuff I haven't caught.

Note: There are special considerations when using GPU features like cublas, hipblas. See the ggml-sys-bleedingedge repo or crate documentation for more information

Example usage: https://github.com/KerfuffleV2/smolrsrwkv/blob/189915ec68b28d057b440f75803d3d056e150733/smolrwkv/src/ggml/graph.rs

Related

For your token sampling needs see https://github.com/KerfuffleV2/llm-samplers ( https://crates.io/crates/llm-samplers )

About

GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages