|Summary:||RijndaelManaged/CFB CryptoStream fails on read/write|
|Product:||[Mono] Class Libraries||Reporter:||Drew DeVault <sir>|
|Component:||mscorlib||Assignee:||Sebastien Pouliot <sebastien>|
|Severity:||normal||CC:||jdluzen, mono-bugs+mono, sebastien, sir, vbisbest|
|Tags:||Is this bug a regression?:||---|
|Last known good build:|
Proof of concept, with included results
AES/CFB8 Invalid Decoded Bytes
Bad bytes without cryptostream
CFB8 TransformBlock invalid block size
Description Drew DeVault 2013-01-03 17:01:34 UTC
Created attachment 3159 [details] Proof of concept, with included results On Microsoft.NET, the attached example works properly. It encrypts correctly, then decrypts correctly, using an AES/CFB CryptoStream with no padding. On Mono, however, it fails, writing nothing to the stream.
Comment 1 Sebastien Pouliot 2013-01-03 21:21:48 UTC
*** Bug 9201 has been marked as a duplicate of this bug. ***
Comment 2 Sebastien Pouliot 2013-01-03 21:23:40 UTC
Comment 3 Ray Kelly 2013-01-04 10:32:53 UTC
Thank you Sebastien for looking into these issues. This bug leaves out my original issue regarding reading from the cryptostream. I have also found another issue in regards to reading. Could be related. So let me summarize all the issues. All of these are specific to AES/CFB8 1. Cant write to cryptostream. It accepts the write, but nothing goes over the wire. 2. Reading the stream stops after a block size not equal to a valid block size. Per your comment from my bug marked duplicate: "CryptoStream has a similar (to bug #7193) and wrong assumption about data length when used with CFB with a feedback size smaller than the block size. This mess up the end-of-stream detection." 3. Decrypting chunks sometimes causes the first 16 bytes to be invalid in the second chunk. So for instance I decrypt 103 bytes and it all looks good. I then decrypt 128 bytes and the fist 16 bytes are garbage and the rest of the bytes are valid. I have attached a test that reproduces the issue. If you need all separate bugs, just let me know. But hopefully these are all related. I know these are tough issues and I certainly would not want to be the one having to fix these :). Thanks for what you do! Ray
Comment 4 Ray Kelly 2013-01-04 10:38:23 UTC
Created attachment 3161 [details] AES/CFB8 Invalid Decoded Bytes You can see in the console output the original and the decrypted data.
Comment 5 Sebastien Pouliot 2013-01-04 11:43:45 UTC
I won't be looking into CryptoStream issues until CFB works correctly - because they are _likely_ related and, if not, then I'm still unable to test it until it CFB works (unless you have test cases that does not uses CFB, in this case please open a separate bug report). However you can keep adding new (CryptoStream/CFB) test cases (as attachments) to this bug report. I will, eventually, get to them and will turn (some of) them into unit tests to ensure this does not regress in the future. Also to avoid confusion don't use AES as the name. Why ? * AesCryptoServiceProvider/CFB8 works with Mono (and is compatible with MS implementation). However it is incompatible with RijndaelManaged/CFB8, which is what you're using (just like MS is). * Microsoft's AesManaged does not support CFB (and future versions of Mono will behave identically);
Comment 6 Sebastien Pouliot 2013-01-06 20:13:14 UTC
[work in progress, only in master so far] MS handles CFB differently (and incompatibly) between ciphers. 1. DES, TripleDES and RC2 only supports CFB 8 * That's all *CryptoServiceProvider (unmanaged for MS) * This is what Mono always supported (for every ciphers) * Additional test cases were added (to confirm that :-) 2. RijndaelManaged supports CFB[8-256] (in multiple of 8 bits) * That was the only *Managed cipher (in the original FX) * Mono did not have this different behaviour until recently * Partially fixed in #7193, now reverted (since it allowed invalid cases wrt #1). A new better fix is in place but it only (like before) support CFB8 for None/Zeros * Another difference is that the ICryptoTransform returned by RijndaelManaged has different values for [Input| Output]BlockSize when CFB is used. That's likely what's breaking CryptoStream since it depends on the values to match the block size (like other ciphers returns) * Yet another difference is the handled of PaddingMode.None other do not support it (and it behave like Zeros) 3. AesCryptoServiceProvider supports CFB[8|16|24|32|40|48|56|64] * Mono only supporteded CFB 8 (like #1) [fixed] * Incompatible with RijndaelManaged (even when using the same 128 block size). Some details found in: http://blogs.msdn.com/b/shawnfa/archive/2006/10/09/the-differences-between-rijndael-and-aes.aspx 4. AesManaged does not support CFB mode * Not very clear why (I assume the situation was already confusing enough ;-) but mentioned in: http://blogs.msdn.com/b/shawnfa/archive/2007/01/17/new-crypto-algorithms-in-orcas.aspx * Mono supported CFB 8 [fixed to match MS behavior]
Comment 7 Ray Kelly 2013-01-06 21:02:11 UTC
Thank you for the update. This is above my head but I think it looks promising. In our case with CFB 8 no padding, we still need to wait for a fix correct? And if using MonoTouch, we would need to wait for an update from them correct? Thank you, Ray
Comment 8 Sebastien Pouliot 2013-01-07 10:41:33 UTC
CFB8 works - but then it worked bug #7193 was fixed (and that's be in MT iirc). However it did not work with CryptoStream* (as your test case pointed out). The only immediate fix is to stop using CryptoStream and work directly on byte buffers (i.e. replacing it with calls to TransformBlock). * not sure why (have not looked into that part yet) but it's even less likely now since the [Input|Output]BlockSize are now matching MS values (and that's not valid from CryptoStream point of view).
Comment 9 Ray Kelly 2013-01-07 11:52:09 UTC
Sorry, I thought you were aware that does not work either. I have the same issue as above. The first 16 bytes of the 2nd call to TransformFinalBlock are not valid. I have attached a new code that demonstrates the issue without cyrptostream. Thanks, Ray
Comment 10 Ray Kelly 2013-01-07 11:52:56 UTC
Created attachment 3168 [details] Bad bytes without cryptostream
Comment 11 Sebastien Pouliot 2013-01-07 20:38:13 UTC
From a quick look your test case does not work on Windows with MS .NET - most likely because you cannot resume decryption once you called TransformFinalBlock.
Comment 12 Drew DeVault 2013-01-07 20:41:18 UTC
I assure you that it does work on Windows - the results of running it on MS.NET are included in the archive. I create a new instance of CryptoStream for decryption, it is not reused.
Comment 13 Sebastien Pouliot 2013-01-07 21:06:41 UTC
Drew, I was talking about the attachment from Ray, on comment #10, that does not use CryptoStream (not your original test case). Sorry about the confusion.
Comment 14 Ray Kelly 2013-01-07 22:24:47 UTC
You are correct Sebastien, its the same behavior in MS.NET, sorry about that. But you mentioned working on the the buffer itself. So in my case, while streaming a chunk comes down 103 bytes, I have to call TransformFinalBlock because I get an error if I call TransformBlock (invalid block size). How can I use the buffers themselves when I get uneven block sizes streaming down?
Comment 15 Ray Kelly 2013-01-08 09:12:51 UTC
Created attachment 3177 [details] CFB8 TransformBlock invalid block size
Comment 16 Ray Kelly 2013-01-08 09:16:50 UTC
I added a new attachment showing the invalid block size error with transformblock. Seems to be same as bug #7193 that you say was fixed. I am using: Runtime: Mono 2.10.10 (mono-2-10/4d9ada6) GTK 2.24.11 GTK# (220.127.116.11) Package version: 210100000 Apple Developer Tools: Xcode 4.3.3 (1178) Build 4E3002
Comment 17 Ray Kelly 2013-01-09 16:08:22 UTC
Sebastien, I was just wondering if you have an idea when this may be fixed as my project is stalled. There doesn't appear to be any workaround to get past this. I don't even mind using a patch or alpha version of MT. Thank you again for looking into this issue.
Comment 18 Drew DeVault 2013-01-13 02:27:02 UTC
If you're looking for a workaround, the one that I've used is Bouncy Castle for encryption. http://bouncycastle.org/ They provide a hefty amount of crypto and whatever you need is probably in there. I would really like this bug to be fixed ASAP, though, so I needn't add another 1.5 MB dependency.
Comment 19 Ray Kelly 2013-01-18 09:26:57 UTC
Thanks, I tried Bouncy Castle and it decrypts perfectly. However, its too slow. It cant keep up with the stream. I compared it to the native Rijndael (the one your are fixing) and its about 80% faster than Bouncy Castle. So I hoping you are able to get this bug all fixed up fairly soon. Thanks, Ray.
Comment 20 Ray Kelly 2013-01-29 15:46:31 UTC
Is there any estimate on when this may be fixed in a release? I am wondering if I should go to the app store with bouncy castle or wait for the real fix. Thanks.
Comment 21 Ray Kelly 2013-02-28 07:50:22 UTC
Any update on this issue? I see that Mono 2.10.11 is now available, but I think I would have to wait for Xamarins framework correct? Thanks.
Comment 22 Sebastien Pouliot 2013-02-28 08:10:25 UTC
This bug report will be updated as soon as a fix is available in GIT. In the mean time please use Bouncy Castle if you need this "variant" of CFB8 mode.
Comment 23 Ray Kelly 2013-04-08 11:57:34 UTC
I am sorry to check in again, but its been 3 months since the bug was filed and was really hoping for a quick solution. Even if its in Mono 3.x that would be great. Thanks again for your time and efforts.
Comment 24 Drew DeVault 2013-04-08 12:02:30 UTC
By the way, if Mono goes with a pure .NET solution, it won't be much faster. MS.NET is that fast because it's just a wrapper around native code that uses the AES instruction set.
Comment 25 Ray Kelly 2013-04-08 12:27:44 UTC
Ok, thank you for the info. Will be interesting to see the performance difference between BC and Mono. Every little bit will help in my case but getting rid of the dependency will be nice too.