Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Q4_1_O quantization format that preserves outliers in weights and does dot in FP32 #16

Merged
merged 11 commits into from
Apr 8, 2023
Merged

Add Q4_1_O quantization format that preserves outliers in weights and does dot in FP32 #16

merged 11 commits into from
Apr 8, 2023

Commits on Apr 6, 2023

  1. Add missing labels and symbols for new operators

    @saharNooby
    saharNooby committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    ad3a4eb View commit details
    Browse the repository at this point in the history

  2. Free ggml context when model is garbage collected

    @saharNooby
    saharNooby committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    fa9ad13 View commit details
    Browse the repository at this point in the history

  3. Show file compression ratio

    @saharNooby
    saharNooby committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    058b5cd View commit details
    Browse the repository at this point in the history

  4. Do not quantize head

    @saharNooby
    saharNooby committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    ec99bc1 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2023

  1. Add Q4_1_O format

    @saharNooby
    saharNooby committed Apr 7, 2023
    1 Configuration menu
    Copy the full SHA
    c40941d View commit details
    Browse the repository at this point in the history

  2. Use ggml function for parameter size calculation

    @saharNooby
    saharNooby committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    18bf02f View commit details
    Browse the repository at this point in the history

  3. Add Q4_1_O test

    @saharNooby
    saharNooby committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    e26b408 View commit details
    Browse the repository at this point in the history

  4. Update README.md

    @saharNooby
    saharNooby committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    edd57a1 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2023

  1. Remove reference impl comparison test

    @saharNooby
    saharNooby committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    e04baa0 View commit details
    Browse the repository at this point in the history

  2. Add script that measures perplexity

    @saharNooby
    saharNooby committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    85db23c View commit details
    Browse the repository at this point in the history

  3. Update README.md

    @saharNooby
    saharNooby committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    874826c View commit details
    Browse the repository at this point in the history