This is Xamarin's bug tracking system. For product support, please use the support links listed in your Xamarin Account.
Bug 9247 - RijndaelManaged/CFB CryptoStream fails on read/write
: RijndaelManaged/CFB CryptoStream fails on read/write
Status: IN_PROGRESS
Product: Class Libraries
Classification: Mono
Component: CORLIB
: 2.10.x
: PC All
: --- normal
: ---
Assigned To: Sebastien Pouliot
:
:
:
:
  Show dependency treegraph
 
Reported: 2013-01-03 17:01 EST by Drew DeVault
Modified: 2013-04-08 12:27 EDT (History)
5 users (show)

See Also:
Tags:
Test Case URL:
External Submit: ---


Attachments
Proof of concept, with included results (4.16 KB, application/zip)
2013-01-03 17:01 EST, Drew DeVault
Details
AES/CFB8 Invalid Decoded Bytes (2.83 KB, application/octet-stream)
2013-01-04 10:38 EST, Ray Kelly
Details
Bad bytes without cryptostream (2.36 KB, application/octet-stream)
2013-01-07 11:52 EST, Ray Kelly
Details
CFB8 TransformBlock invalid block size (2.47 KB, application/octet-stream)
2013-01-08 09:12 EST, Ray Kelly
Details

Description Drew DeVault 2013-01-03 17:01:34 EST
Created attachment 3159 [details]
Proof of concept, with included results

On Microsoft.NET, the attached example works properly. It encrypts correctly,
then decrypts correctly, using an AES/CFB CryptoStream with no padding. On
Mono, however, it fails, writing nothing to the stream.
Comment 1 Sebastien Pouliot 2013-01-03 21:21:48 EST
*** Bug 9201 has been marked as a duplicate of this bug. ***
Comment 3 Ray Kelly 2013-01-04 10:32:53 EST
Thank you Sebastien for looking into these issues.  This bug leaves out my
original issue regarding reading from the cryptostream.   I have also found
another issue in regards to reading.  Could be related.  So let me summarize
all the issues.  All of these are specific to AES/CFB8

1. Cant write to cryptostream.  It accepts the write, but nothing goes over the
wire.

2. Reading the stream stops after a block size not equal to a valid block size.
 Per your comment from my bug marked duplicate:
"CryptoStream has a similar (to bug #7193) and wrong assumption about data
length when used with CFB with a feedback size smaller than the block size.
This mess up the end-of-stream detection."

3. Decrypting chunks sometimes causes the first 16 bytes to be invalid in the
second chunk.  So for instance I decrypt 103 bytes and it all looks good.  I
then decrypt 128 bytes and the fist 16 bytes are garbage and the rest of the
bytes are valid.  I have attached a test that reproduces the issue.


If you need all separate bugs, just let me know.  But hopefully these are all
related.  I know these are tough issues and I certainly would not want to be
the one having to fix these :).   Thanks for what you do!

Ray
Comment 4 Ray Kelly 2013-01-04 10:38:23 EST
Created attachment 3161 [details]
AES/CFB8 Invalid Decoded Bytes

You can see in the console output the original and the decrypted data.
Comment 5 Sebastien Pouliot 2013-01-04 11:43:45 EST
I won't be looking into CryptoStream issues until CFB works correctly - because
they are _likely_ related and, if not, then I'm still unable to test it until
it CFB works (unless you have test cases that does not uses CFB, in this case
please open a separate bug report). 

However you can keep adding new (CryptoStream/CFB) test cases (as attachments)
to this bug report. I will, eventually, get to them and will turn (some of)
them into unit tests to ensure this does not regress in the future.

Also to avoid confusion don't use AES as the name. Why ? 

* AesCryptoServiceProvider/CFB8 works with Mono (and is compatible with MS
implementation). However it is incompatible with RijndaelManaged/CFB8, which is
what you're using (just like MS is).

* Microsoft's AesManaged does not support CFB (and future versions of Mono will
behave identically);
Comment 6 Sebastien Pouliot 2013-01-06 20:13:14 EST
[work in progress, only in master so far]

MS handles CFB differently (and incompatibly) between ciphers.

1. DES, TripleDES and RC2 only supports CFB 8
    * That's all *CryptoServiceProvider (unmanaged for MS)
    * This is what Mono always supported (for every ciphers)
    * Additional test cases were added (to confirm that :-)

2. RijndaelManaged supports CFB[8-256] (in multiple of 8 bits)
    * That was the only *Managed cipher (in the original FX)
    * Mono did not have this different behaviour until recently
    * Partially fixed in #7193, now reverted (since it allowed
      invalid cases wrt #1). A new better fix is in place but
      it only (like before) support CFB8 for None/Zeros
    * Another difference is that the ICryptoTransform returned
      by RijndaelManaged has different values for [Input|
      Output]BlockSize when CFB is used. That's likely what's
      breaking CryptoStream since it depends on the values to
      match the block size (like other ciphers returns)
    * Yet another difference is the handled of PaddingMode.None
      other do not support it (and it behave like Zeros)

3. AesCryptoServiceProvider supports CFB[8|16|24|32|40|48|56|64]
    * Mono only supporteded CFB 8 (like #1) [fixed]
    * Incompatible with RijndaelManaged (even when using 
      the same 128 block size). Some details found in:
     
http://blogs.msdn.com/b/shawnfa/archive/2006/10/09/the-differences-between-rijndael-and-aes.aspx

4. AesManaged does not support CFB mode
    * Not very clear why (I assume the situation was already 
      confusing enough ;-) but mentioned in:
     
http://blogs.msdn.com/b/shawnfa/archive/2007/01/17/new-crypto-algorithms-in-orcas.aspx
    * Mono supported CFB 8 [fixed to match MS behavior]
Comment 7 Ray Kelly 2013-01-06 21:02:11 EST
Thank you for the update.  This is above my head but I think it looks
promising.  In our case with CFB 8 no padding, we still need to wait for a fix
correct?  And if using MonoTouch, we would need to wait for an update from them
correct?

Thank you,
Ray
Comment 8 Sebastien Pouliot 2013-01-07 10:41:33 EST
CFB8 works - but then it worked bug #7193 was fixed (and that's be in MT iirc).
However it did not work with CryptoStream* (as your test case pointed out). The
only immediate fix is to stop using CryptoStream and work directly on byte[]
buffers (i.e. replacing it with calls to TransformBlock).

* not sure why (have not looked into that part yet) but it's even less likely
now since the [Input|Output]BlockSize are now matching MS values (and that's
not valid from CryptoStream point of view).
Comment 9 Ray Kelly 2013-01-07 11:52:09 EST
Sorry, I thought you were aware that does not work either.  I have the same
issue as above.  The first 16 bytes of the 2nd call to TransformFinalBlock are
not valid.  I have attached a new code that demonstrates the issue without
cyrptostream. 

Thanks,
Ray
Comment 10 Ray Kelly 2013-01-07 11:52:56 EST
Created attachment 3168 [details]
Bad bytes without cryptostream
Comment 11 Sebastien Pouliot 2013-01-07 20:38:13 EST
From a quick look your test case does not work on Windows with MS .NET - most
likely because you cannot resume decryption once you called
TransformFinalBlock.
Comment 12 Drew DeVault 2013-01-07 20:41:18 EST
I assure you that it does work on Windows - the results of running it on MS.NET
are included in the archive. I create a new instance of CryptoStream for
decryption, it is not reused.
Comment 13 Sebastien Pouliot 2013-01-07 21:06:41 EST
Drew, I was talking about the attachment from Ray, on comment #10, that does
not use CryptoStream (not your original test case). Sorry about the confusion.
Comment 14 Ray Kelly 2013-01-07 22:24:47 EST
You are correct Sebastien, its the same behavior in MS.NET, sorry about that. 
But you mentioned working on the the buffer itself.   So in my case, while
streaming a chunk comes down 103 bytes, I have to call TransformFinalBlock
because I get an error if I call TransformBlock (invalid block size). How can I
use the buffers themselves when I get uneven block sizes streaming down?
Comment 15 Ray Kelly 2013-01-08 09:12:51 EST
Created attachment 3177 [details]
CFB8 TransformBlock invalid block size
Comment 16 Ray Kelly 2013-01-08 09:16:50 EST
I added a new attachment showing the invalid block size error with
transformblock.  Seems to be same as bug #7193 that you say was fixed.  I am
using:

Runtime:
    Mono 2.10.10 (mono-2-10/4d9ada6)
    GTK 2.24.11
    GTK# (2.12.0.0)
    Package version: 210100000
Apple Developer Tools:
     Xcode 4.3.3 (1178)
     Build 4E3002
Comment 17 Ray Kelly 2013-01-09 16:08:22 EST
Sebastien, I was just wondering if you have an idea when this may be fixed as
my project is stalled.   There doesn't appear to be any workaround to get past
this.  I don't even mind using a patch or alpha version of MT.  Thank you again
for looking into this issue.
Comment 18 Drew DeVault 2013-01-13 02:27:02 EST
If you're looking for a workaround, the one that I've used is Bouncy Castle for
encryption. http://bouncycastle.org/

They provide a hefty amount of crypto and whatever you need is probably in
there. I would really like this bug to be fixed ASAP, though, so I needn't add
another 1.5 MB dependency.
Comment 19 Ray Kelly 2013-01-18 09:26:57 EST
Thanks, I tried Bouncy Castle and it decrypts perfectly.  However, its too
slow.  It cant keep up with the stream.  I compared it to the native Rijndael
(the one your are fixing) and its about 80% faster than Bouncy Castle.  So I
hoping you are able to get this bug all fixed up fairly soon.   Thanks, Ray.
Comment 20 Ray Kelly 2013-01-29 15:46:31 EST
Is there any estimate on when this may be fixed in a release?  I am wondering
if I should go to the app store with bouncy castle or wait for the real fix. 
Thanks.
Comment 21 Ray Kelly 2013-02-28 07:50:22 EST
Any update on this issue?   I see that Mono 2.10.11 is now available, but I
think I would have to wait for Xamarins framework correct?   Thanks.
Comment 22 Sebastien Pouliot 2013-02-28 08:10:25 EST
This bug report  will be updated as soon as a fix is available in GIT. In the
mean time please use Bouncy Castle if you need this "variant" of CFB8 mode.
Comment 23 Ray Kelly 2013-04-08 11:57:34 EDT
I am sorry to check in again, but its been 3 months since the bug was filed and
was really hoping for a quick solution.  Even if its in Mono 3.x that would be
great. Thanks again for your time and efforts.
Comment 24 Drew DeVault 2013-04-08 12:02:30 EDT
By the way, if Mono goes with a pure .NET solution, it won't be much faster.
MS.NET is that fast because it's just a wrapper around native code that uses
the AES instruction set.
Comment 25 Ray Kelly 2013-04-08 12:27:44 EDT
Ok, thank you for the info.  Will be interesting to see the performance
difference between BC and Mono.  Every little bit will help in my case but
getting rid of the dependency will be nice too.

Note You need to log in before you can comment on or make changes to this bug.