Bug 44994 - DeflateStream decompression is incomplete if reading byte-by-byte
Summary: DeflateStream decompression is incomplete if reading byte-by-byte
Status: VERIFIED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: BCL Class Libraries (show other bugs)
Version: unspecified
Hardware: Macintosh Mac OS
: Normal normal
Target Milestone: (C9)
Assignee: João Matos
URL:
Depends on:
Blocks:
 
Reported: 2016-10-03 20:37 UTC by mfulker
Modified: 2016-12-30 13:38 UTC (History)
6 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Working windows console app and Xamarin iOS app with premature end of stream (237.47 KB, application/x-zip-compressed)
2016-10-03 20:37 UTC, mfulker
Details

Description mfulker 2016-10-03 20:37:53 UTC
Created attachment 17850 [details]
Working windows console app and Xamarin iOS app with premature end of stream

When decompressing certain compressed files, reading the DeflateStream byte-by-byte with ReadByte() will omit the last few bytes, seemingly hitting end-of-stream prematurely.
On the same file, using DeflateStream.CopyTo(otherStream) copies the data correctly.

(I've also encountered a problem with EndOfStreamException when trying to use a BinaryReader on the DeflateStream for the same files.)

I'm not sure why only certain files cause it or what the pattern is in the failing files.


In the attached zip are solutions for a Xamarin iOS app and a Windows Console App.
Each has two compressed file resources that they attempt to decompress: "compressed.bin" and "compressed2.bin".
The decompressed size is calculated for each, both via CopyTo into a MemoryStream, and by reading to the end with ReadByte().
The only incorrect case is when reading "compressed.bin" with ReadByte on iOS, which yields 125377 bytes instead of 125387 bytes.
Using ReadByte on Windows or using CopyTo will yield the correct deflated size for the first file.
The other file "compressed2.bin" decompresses correctly on both platforms with both approaches.
Comment 1 mfulker 2016-10-04 21:30:56 UTC
I discovered that appending some zero-value bytes to the end of the file will fix every case of the issue I've seen.

Doing that as a workaround shouldn't cause any other output problems with the Deflate algorithm, should it?
(My knowledge of the Deflate specification is that it works on a block-by-block basis and that each block has an "is last block" header bit, so I don't expect superfluous bytes in input to cause extra bytes on output or anything like that.)

As a temporary workaround (and depending on ETA for a real fix), I might create a read-only Stream wrapper that appends a small number of superfluous bytes to the end of the wrapped FileStream (without actually modifying the file).
Comment 2 mfulker 2016-10-05 17:44:54 UTC
I actually discovered one particular (very small) compressed file also has incorrect results even with CopyTo:

data:application/octet-stream;base64,7cWxCQAgDACwpeBjgqsgXiHU0fd9QzBLErX1EQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADepcxcuU/atm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm3btm37zy8=


The correct decompressed length should be 81942 bytes according to windows.
The decompressed length when using CopyTo on iOS is 81921.
The decompressed length when reading byte-by-byte on iOS is 81697.
Comment 3 Rolf Bjarne Kvinge [MSFT] 2016-10-06 06:16:45 UTC
This might be related to bug #34916.
Comment 4 João Matos 2016-10-27 19:37:20 UTC
https://github.com/mono/mono/pull/3844
Comment 6 Marek Safar 2016-10-28 10:48:01 UTC
@joao could you merge it to Mono 4.8 as well

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.


Create a new report for Bug 44994 on Developer Community or GitHub if you have new information to add and do not yet see a matching report.

  • Export the original title and description: Developer Community HTML or GitHub Markdown
  • Copy the title and description into the new report. Adjust them to be up-to-date if needed.
  • Add your new information.

In special cases on GitHub you might also want the comments: GitHub Markdown with public comments


Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Related Links: