Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Var store quantize #3126

Merged
merged 4 commits into from
May 25, 2023
Merged

Var store quantize #3126

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
311bda2
[varStore] Add optional quantization to optimize()
behdad May 24, 2023
d9649a7
[varStore] Add test for quantization
behdad May 25, 2023
ae7394e
[varLib.main] Accept --quantization
behdad May 25, 2023
a08f5eb
[varStore] Update comment
behdad May 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 14 additions & 4 deletions Lib/fontTools/varLib/varStore.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -470,7 +470,7 @@
return chars


def VarStore_optimize(self, use_NO_VARIATION_INDEX=True):
def VarStore_optimize(self, use_NO_VARIATION_INDEX=True, quantization=1):
"""Optimize storage. Returns mapping from old VarIdxes to new ones."""

# Overview:
Expand Down Expand Up @@ -551,8 +551,16 @@

for minor, item in enumerate(data.Item):
row = list(zeroes)
for regionIdx, v in zip(regionIndices, item):
row[regionIdx] += v

if quantization == 1:
for regionIdx, v in zip(regionIndices, item):
row[regionIdx] += v
else:
for regionIdx, v in zip(regionIndices, item):
row[regionIdx] += (
round(v / quantization) * quantization
) # TODO: round towards 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why "round towards 0"?

you can use decimal.Decimal's quantize method with a ROUND_DOWN rounding method:

https://docs.python.org/3/library/decimal.html#decimal.ROUND_DOWN

though it might be quicker to simply do floor(v) if v > 0 else ceil(v)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove the comment. round is fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant was a round that rounds .5 to 0, such that eg. with quantization 2, value 1 will become 0, not 2.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah that's https://docs.python.org/3/library/decimal.html#decimal.ROUND_HALF_DOWN

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i think as a size saving optimization this quantization should make the deltas smaller, would be strange if it made them bigger

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(though the sharing comes from reuse, not the delta magnitude itself)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!


row = tuple(row)

if use_NO_VARIATION_INDEX and not any(row):
Expand Down Expand Up @@ -684,13 +692,15 @@
from fontTools.ttLib.tables.otBase import OTTableWriter

parser = ArgumentParser(prog="varLib.varStore", description=main.__doc__)
parser.add_argument("--quantization", type=int, default=1)

Check warning on line 695 in Lib/fontTools/varLib/varStore.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/varLib/varStore.py#L695 

Added line #L695 was not covered by tests
parser.add_argument("fontfile")
parser.add_argument("outfile", nargs="?")
options = parser.parse_args(args)

# TODO: allow user to configure logging via command-line options
configLogger(level="INFO")

quantization = options.quantization

Check warning on line 703 in Lib/fontTools/varLib/varStore.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/varLib/varStore.py#L703 

Added line #L703 was not covered by tests
fontfile = options.fontfile
outfile = options.outfile

Expand All @@ -703,7 +713,7 @@
size = len(writer.getAllData())
print("Before: %7d bytes" % size)

varidx_map = store.optimize()
varidx_map = store.optimize(quantization=quantization)

Check warning on line 716 in Lib/fontTools/varLib/varStore.py

View check run for this annotation

Codecov / codecov/patch

Lib/fontTools/varLib/varStore.py#L716 

Added line #L716 was not covered by tests

writer = OTTableWriter()
store.compile(writer, font)
Expand Down
54 changes: 54 additions & 0 deletions Tests/varLib/varStore_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,3 +202,57 @@ def test_optimize(numRegions, varData, expectedNumVarData, expectedBytes):
data = writer.getAllData()

assert len(data) == expectedBytes, xml


@pytest.mark.parametrize(
"quantization, expectedBytes",
[
(1, 200),
(2, 180),
(3, 170),
(4, 175),
(8, 170),
(32, 152),
(64, 146),
],
)
def test_quantize(quantization, expectedBytes):
varData = [
[0, 11, 12, 0, 20],
[0, 13, 12, 0, 20],
[0, 14, 12, 0, 20],
[0, 15, 12, 0, 20],
[0, 16, 12, 0, 20],
[10, 300, 0, 0, 20],
[10, 301, 0, 0, 20],
[10, 302, 0, 0, 20],
[10, 303, 0, 0, 20],
[10, 304, 0, 0, 20],
]

numRegions = 5
locations = [{i: i / 16384.0} for i in range(numRegions)]
axisTags = sorted({k for loc in locations for k in loc})

model = VariationModel(locations)

builder = OnlineVarStoreBuilder(axisTags)
builder.setModel(model)

for data in varData:
builder.storeMasters(data)

varStore = builder.finish()
varStore.optimize(quantization=quantization)

dummyFont = TTFont()

writer = XMLWriter(StringIO())
varStore.toXML(writer, dummyFont)
xml = writer.file.getvalue()

writer = OTTableWriter()
varStore.compile(writer, dummyFont)
data = writer.getAllData()

assert len(data) == expectedBytes, xml