Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing decoder sanitation checks #2077

Merged
merged 16 commits into from
Apr 20, 2022
Merged

Conversation

JimBobSquarePants
Copy link
Member

@JimBobSquarePants JimBobSquarePants commented Mar 27, 2022

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 馃懏.
  • I have provided test coverage for my change (where applicable)

Description

A PR to add additional sanitation checks with a view to fix #2075

Any additional sanitation fixes added to other open branches should be added here.

bytesRead = this.currentStream.Read(exifData, 0, (int)exifChunkSize);
if (bytesRead != exifChunkSize)
{
WebpThrowHelper.ThrowInvalidImageContentException("Not enough data to read the exif chunk");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there rules in WebP about where a these chunks should be. I.E are they always before the image data?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ICCP and ANIM chunks should always come before the image data, EXIF and XMP chunks should come after the image data.

https://developers.google.com/speed/webp/docs/riff_container#extended_file_format

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since EXIF and XMP chunks come after, I don't think we should throw for them and skip instead. I would consider those non-critical.

@br3aker
Copy link
Contributor

br3aker commented Mar 29, 2022

I would suggest another sanitation check for jpeg decoder in general. Each marker has specific predefined size which we already have in each marker parser as a parameter - remaining. Checking whether stream has enough bytes via stream.Length - stream.Position >= remaining would eliminate any stream read checks inside marker parsers and would make code cleaner imo.

I can't contribute to this branch directly so I can make this as a separate PR.

@JimBobSquarePants
Copy link
Member Author

Checking whether stream has enough bytes via stream.Length - stream.Position >= remaining would eliminate any stream read checks inside marker parsers and would make code cleaner imo.

I would agree. If you have the time to do that'd very much appreciate it.

@br3aker
Copy link
Contributor

br3aker commented Apr 4, 2022

Sorry for late response, been studying scaled IDCT for other PR. Will try to do a PR for jpeg markers on this week.

@JimBobSquarePants
Copy link
Member Author

I looks increasingly unlikely that we'll get any test images for this but I think we should still push on with the sanitation code.
I'm going to create a release/2.1.1 branch and open a second PR from this branch so we can push on with V3.

@br3aker
Copy link
Contributor

br3aker commented Apr 5, 2022

Actually there's kind of a problem in current jpeg decoder. It shouldn't be a cause of mentioned issue but it's still a bug

Current decoding pipeline:

1. Gather jpeg info (components, tables, frame size, etc.) <- pixel buffer is allocated here
2. For each scan:
       For each mcu row:
           Decode: huffman -> spectral -> rgb -> TPixel
3. Create Image object using allocated pixel buffer

Any exceptions between step 1 and 3 would cause allocated pixel buffer to be gc'd leading to a false positive memory leak reported by memory diagnostics. Any invalid marker between scans can cause this, it shouldn't crash the entire process or even lead to an actual memory leak but it's still a thing.

I'll create a separate issue this week, posting this here just to let you know.

@JimBobSquarePants JimBobSquarePants added this to the 2.1.1 milestone Apr 13, 2022
@br3aker
Copy link
Contributor

br3aker commented Apr 13, 2022

I would suggest another sanitation check for jpeg decoder in general. Each marker has specific predefined size which we already have in each marker parser as a parameter - remaining. Checking whether stream has enough bytes via stream.Length - stream.Position >= remaining would eliminate any stream read checks inside marker parsers and would make code cleaner imo.

Extra jpeg sanity checks implemented at #2084.

@JimBobSquarePants JimBobSquarePants requested a review from a team April 15, 2022 07:10
@JimBobSquarePants
Copy link
Member Author

@SixLabors/core I'm marking this as ready to review. I think with these checks and #2084 we've probably handled any potential issues during decode that trigger #2075

@JimBobSquarePants JimBobSquarePants marked this pull request as ready for review April 15, 2022 07:12
@JimBobSquarePants JimBobSquarePants changed the title WIP. Add missing decoder sanitation checks Add missing decoder sanitation checks Apr 15, 2022
Comment on lines 236 to 239
if (bytesRead != 2)
{
JpegThrowHelper.ThrowInvalidImageContentException("Not enough data to read the SOI marker");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this and similar checks from other marker parsing methods as it's now guaranteed that stream has at least remaining bytes available.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah... That method is actually called from TIFF only.

Copy link
Contributor

@br3aker br3aker Apr 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we still can transform 2 read checks to 1 AvailableBytes check :)
I can't comment on non diff lines but that method creates stream:

using var ms = new MemoryStream(tableBytes);

and then do 2 reads of 2 bytes with sequential if checks.

We can do this check:

if(tableBytes.Length < 4)  throw ...

before even creating a stream and remove 2 checks after reads as those would always be true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. Will have a look.

@br3aker
Copy link
Contributor

br3aker commented Apr 15, 2022

Sorry, I forgot to alter TIFF jpeg loading method here:

while (fileMarker.Marker != JpegConstants.Markers.EOI || (fileMarker.Marker == JpegConstants.Markers.EOI && fileMarker.Invalid))
{
if (!fileMarker.Invalid)
{
// Get the marker length.
int remaining = this.ReadUint16(stream) - 2;
switch (fileMarker.Marker)

It should be this:

int markerContentByteSize = this.ReadUint16(stream) - 2;

// Check whether stream actually has enought bytes to read
// markerContentByteSize is always positive so we cast
// to uint to avoid sign extension
if (stream.RemainingBytes < (uint)markerContentByteSize)
{
    JpegThrowHelper.ThrowNotEnoughBytesForMarker(fileMarker.Marker);
}

swtich(fileMarker.Marker)
{
    // ...
}

@JimBobSquarePants
Copy link
Member Author

Peeps I'm gonna merge this. I want a build that can be tested in #2075 and it's blocking all other progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Access Violation in 2.1, works fine in 1.04
3 participants