Bug 28777

Summary: GZipStream (DeflateStreamNative) native exception after Flush() with no buffer data: Internal error (no progress possible) Flush
Product: [Mono] Class Libraries Reporter: Gabriel Garcia <gbrlgrct>
Component: SystemAssignee: Bugzilla <bugzilla>
Status: VERIFIED FIXED    
Severity: normal CC: joao.matos, mono-bugs+mono, ramc, sgparry, xamarin
Priority: ---    
Version: master   
Target Milestone: Untriaged   
Hardware: PC   
OS: Linux   
Tags: Is this bug a regression?: ---
Last known good build:
Attachments: Requires System.Xml.Linq.dll
Proposed fix

Description Gabriel Garcia 2015-04-04 18:44:31 UTC
Created attachment 10636 [details]
Requires System.Xml.Linq.dll

**Steps to Reproduce:**

   1. Run the attached sample code.

** Sample code **

using System;
using System.Text;
using System.IO;
using System.IO.Compression;
using System.Xml.Linq;

public class Program
{
    public static void Main(String[] args)
    {
        var doc = new XDocument();
        doc.Add(new XComment("This is a comment"));
        doc.Add(new XElement("Root", "content"));
        
        byte[] output = null;
        
        using (var outStream = new MemoryStream())
        {
            using (var gzipStream = new GZipStream(outStream, CompressionMode.Compress))
            {
                doc.Save(gzipStream);
            }
            
            output = outStream.ToArray();
        }
        
        Console.WriteLine(output?.Length);
    }
}

** Results **

Unhandled Exception:
System.IO.IOException: Internal error (no progress possible) Flush
  at System.IO.Compression.DeflateStreamNative.CheckResult (Int32 result, System.String where) <0x40688b80 + 0x000f7> in <filename unknown>:0 
  at System.IO.Compression.DeflateStreamNative.Flush () <0x40688e40 + 0x00027> in <filename unknown>:0 
  at System.IO.Compression.DeflateStream.Flush () <0x40688dc0 + 0x00037> in <filename unknown>:0 
  at System.IO.Compression.GZipStream.Flush () <0x40688d60 + 0x00023> in <filename unknown>:0 
  at System.Xml.XmlUtf8RawTextWriter.Close () <0x40689300 + 0x0005e> in <filename unknown>:0 
  at System.Xml.XmlRawWriter.Close (WriteState currentState) <0x406892e0 + 0x00013> in <filename unknown>:0 
  at System.Xml.XmlWellFormedWriter.Close () <0x40688280 + 0x0015e> in <filename unknown>:0 
  at System.Xml.XmlWriter.Dispose (Boolean disposing) <0x406881e0 + 0x00035> in <filename unknown>:0 
  at System.Xml.XmlWriter.Dispose () <0x406881c0 + 0x00015> in <filename unknown>:0 
  at System.Xml.Linq.XDocument.Save (System.IO.Stream stream, SaveOptions options) <0x40681460 + 0x00139> in <filename unknown>:0 
  at System.Xml.Linq.XDocument.Save (System.IO.Stream stream) <0x40681350 + 0x00027> in <filename unknown>:0 
  at Program.Main (System.String[] args) <0x4067cd50 + 0x0015f> in <filename unknown>:0 


** Build information **

$ mono --version
Mono JIT compiler version 4.1.0 (master/5486ebf Fri Apr  3 21:08:00 BOT 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          supported, not enabled.
	GC:            sgen
Comment 1 Gabriel Garcia 2015-04-05 10:49:42 UTC
The bug is not present in 3.12.1 release. This bug appears to be a regression.

Running on:

$ /usr/bin/mono --version
Mono JIT compiler version 3.12.1 (tarball Thu Mar 12 06:31:20 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	LLVM:          supported, not enabled.
	GC:            sgen

The program runs successfully and outputs:

104
Comment 2 Gabriel Garcia 2015-04-05 13:48:27 UTC
Reduced test code to:

using System;
using System.IO;
using System.IO.Compression;

public class Program
{
    public static void Main(String[] args)
    {
        var data = new byte [1];

        var backing = new MemoryStream ();
        var compressing = new GZipStream (backing, CompressionMode.Compress);

        compressing.Write (data, 0, 0);
        compressing.Flush (); // throws here

        compressing.Close ();
        backing.Close ();
    }
}

Bug is present in 3.12.1 and master/5486ebf.
Comment 3 Gabriel Garcia 2015-04-05 16:40:39 UTC
Upon further inspection, we are actually dealing with two different errors regarding Flush()'s use of zlib deflate().

First, there's Z_STREAM_ERROR. We get this error when we call Flush() without having written any data in. This is because WriteZStream() has not had the opportunity to initialize z_stream's next_out buffer.

Triggering code:

    var backing = new MemoryStream ();
    var compressing = new GZipStream (backing, CompressionMode.Compress);

    compressing.Flush (); // throws due to Z_STREAM_ERROR

    compressing.Close ();
    backing.Close ();

Second, there's Z_BUF_ERROR. We get this error when we call Flush() and deflate() did not output any data. This is *not* a fatal error.

Triggering code:

    var backing = new MemoryStream ();
    var compressing = new GZipStream (backing, CompressionMode.Compress);

    compressing.Write (new byte[1], 0, 1);
    compressing.Flush (); // uses up next_in buffer
    compressing.Flush (); // throws due to Z_BUF_ERROR

    compressing.Close ();
    backing.Close ();
Comment 4 Gabriel Garcia 2015-04-05 17:02:16 UTC
Created attachment 10639 [details]
Proposed fix
Comment 5 Gabriel Garcia 2015-04-06 19:13:30 UTC
https://github.com/mono/mono/pull/1684
Comment 6 Ram Chandra 2015-08-11 11:14:36 UTC
I have checked this issue with following builds:

Mac OS X 10.10.3
Xamarin Studio: 5.9.5 (build 9)
Mono 4.3.0 (master/6fc86ee)
GTK+ 2.24.23 (Raleigh theme)
Package version: 403000828

Screencast: http://www.screencast.com/t/ZHPvNukDoWIp

Observation: To verify this issue I have checked all the test cases mentioned in bug description, Comment 2 and Comment 3 and I observed that I am not getting any error or exception. Test cases are working fine with fixed build. 

This issue has been fixed. Hence I am closing this issue.
Comment 7 Stephen Parry 2015-08-12 17:31:30 UTC
This appears to still be a bug in stable live 4.0.3.20-oxamarin1. I have seen it reported as fixed in  4.0.1.44-1. 
Reversion? Does it need back porting? Please?
Comment 8 Ram Chandra 2015-08-20 13:52:33 UTC
I have checked this issue with following builds:

Mac OS X 10.10.3
Xamarin Studio : 5.9.6 (build 6)
Mono 4.0.4 ((detached/5ab4c0d)
GTK+ 2.24.23 (Raleigh theme)
Package version: 400030001
Xamarin.Android : 5.1.6.3

Screencast: http://www.screencast.com/t/hswBDkC8

This fixed has been merged with C5SR4 branch.
Comment 9 Stephen Parry 2015-08-26 15:49:24 UTC
Just received 4.0.4.1-0xamarin1 beta package and can confirm that this bug is fixed for my use case. Many thanks to the mono team / Xamarin.