Created attachment 3159 [details]
Proof of concept, with included results
On Microsoft.NET, the attached example works properly. It encrypts correctly, then decrypts correctly, using an AES/CFB CryptoStream with no padding. On Mono, however, it fails, writing nothing to the stream.
*** Bug 9201 has been marked as a duplicate of this bug. ***
Thank you Sebastien for looking into these issues. This bug leaves out my original issue regarding reading from the cryptostream. I have also found another issue in regards to reading. Could be related. So let me summarize all the issues. All of these are specific to AES/CFB8
1. Cant write to cryptostream. It accepts the write, but nothing goes over the wire.
2. Reading the stream stops after a block size not equal to a valid block size. Per your comment from my bug marked duplicate:
"CryptoStream has a similar (to bug #7193) and wrong assumption about data
length when used with CFB with a feedback size smaller than the block size.
This mess up the end-of-stream detection."
3. Decrypting chunks sometimes causes the first 16 bytes to be invalid in the second chunk. So for instance I decrypt 103 bytes and it all looks good. I then decrypt 128 bytes and the fist 16 bytes are garbage and the rest of the bytes are valid. I have attached a test that reproduces the issue.
If you need all separate bugs, just let me know. But hopefully these are all related. I know these are tough issues and I certainly would not want to be the one having to fix these :). Thanks for what you do!
Created attachment 3161 [details]
AES/CFB8 Invalid Decoded Bytes
You can see in the console output the original and the decrypted data.
I won't be looking into CryptoStream issues until CFB works correctly - because they are _likely_ related and, if not, then I'm still unable to test it until it CFB works (unless you have test cases that does not uses CFB, in this case please open a separate bug report).
However you can keep adding new (CryptoStream/CFB) test cases (as attachments) to this bug report. I will, eventually, get to them and will turn (some of) them into unit tests to ensure this does not regress in the future.
Also to avoid confusion don't use AES as the name. Why ?
* AesCryptoServiceProvider/CFB8 works with Mono (and is compatible with MS implementation). However it is incompatible with RijndaelManaged/CFB8, which is what you're using (just like MS is).
* Microsoft's AesManaged does not support CFB (and future versions of Mono will behave identically);
[work in progress, only in master so far]
MS handles CFB differently (and incompatibly) between ciphers.
1. DES, TripleDES and RC2 only supports CFB 8
* That's all *CryptoServiceProvider (unmanaged for MS)
* This is what Mono always supported (for every ciphers)
* Additional test cases were added (to confirm that :-)
2. RijndaelManaged supports CFB[8-256] (in multiple of 8 bits)
* That was the only *Managed cipher (in the original FX)
* Mono did not have this different behaviour until recently
* Partially fixed in #7193, now reverted (since it allowed
invalid cases wrt #1). A new better fix is in place but
it only (like before) support CFB8 for None/Zeros
* Another difference is that the ICryptoTransform returned
by RijndaelManaged has different values for [Input|
Output]BlockSize when CFB is used. That's likely what's
breaking CryptoStream since it depends on the values to
match the block size (like other ciphers returns)
* Yet another difference is the handled of PaddingMode.None
other do not support it (and it behave like Zeros)
3. AesCryptoServiceProvider supports CFB[8|16|24|32|40|48|56|64]
* Mono only supporteded CFB 8 (like #1) [fixed]
* Incompatible with RijndaelManaged (even when using
the same 128 block size). Some details found in:
4. AesManaged does not support CFB mode
* Not very clear why (I assume the situation was already
confusing enough ;-) but mentioned in:
* Mono supported CFB 8 [fixed to match MS behavior]
Thank you for the update. This is above my head but I think it looks promising. In our case with CFB 8 no padding, we still need to wait for a fix correct? And if using MonoTouch, we would need to wait for an update from them correct?
CFB8 works - but then it worked bug #7193 was fixed (and that's be in MT iirc). However it did not work with CryptoStream* (as your test case pointed out). The only immediate fix is to stop using CryptoStream and work directly on byte buffers (i.e. replacing it with calls to TransformBlock).
* not sure why (have not looked into that part yet) but it's even less likely now since the [Input|Output]BlockSize are now matching MS values (and that's not valid from CryptoStream point of view).
Sorry, I thought you were aware that does not work either. I have the same issue as above. The first 16 bytes of the 2nd call to TransformFinalBlock are not valid. I have attached a new code that demonstrates the issue without cyrptostream.
Created attachment 3168 [details]
Bad bytes without cryptostream
From a quick look your test case does not work on Windows with MS .NET - most likely because you cannot resume decryption once you called TransformFinalBlock.
I assure you that it does work on Windows - the results of running it on MS.NET are included in the archive. I create a new instance of CryptoStream for decryption, it is not reused.
Drew, I was talking about the attachment from Ray, on comment #10, that does not use CryptoStream (not your original test case). Sorry about the confusion.
You are correct Sebastien, its the same behavior in MS.NET, sorry about that. But you mentioned working on the the buffer itself. So in my case, while streaming a chunk comes down 103 bytes, I have to call TransformFinalBlock because I get an error if I call TransformBlock (invalid block size). How can I use the buffers themselves when I get uneven block sizes streaming down?
Created attachment 3177 [details]
CFB8 TransformBlock invalid block size
I added a new attachment showing the invalid block size error with transformblock. Seems to be same as bug #7193 that you say was fixed. I am using:
Mono 2.10.10 (mono-2-10/4d9ada6)
Package version: 210100000
Apple Developer Tools:
Xcode 4.3.3 (1178)
Sebastien, I was just wondering if you have an idea when this may be fixed as my project is stalled. There doesn't appear to be any workaround to get past this. I don't even mind using a patch or alpha version of MT. Thank you again for looking into this issue.
If you're looking for a workaround, the one that I've used is Bouncy Castle for encryption. http://bouncycastle.org/
They provide a hefty amount of crypto and whatever you need is probably in there. I would really like this bug to be fixed ASAP, though, so I needn't add another 1.5 MB dependency.
Thanks, I tried Bouncy Castle and it decrypts perfectly. However, its too slow. It cant keep up with the stream. I compared it to the native Rijndael (the one your are fixing) and its about 80% faster than Bouncy Castle. So I hoping you are able to get this bug all fixed up fairly soon. Thanks, Ray.
Is there any estimate on when this may be fixed in a release? I am wondering if I should go to the app store with bouncy castle or wait for the real fix. Thanks.
Any update on this issue? I see that Mono 2.10.11 is now available, but I think I would have to wait for Xamarins framework correct? Thanks.
This bug report will be updated as soon as a fix is available in GIT. In the mean time please use Bouncy Castle if you need this "variant" of CFB8 mode.
I am sorry to check in again, but its been 3 months since the bug was filed and was really hoping for a quick solution. Even if its in Mono 3.x that would be great. Thanks again for your time and efforts.
By the way, if Mono goes with a pure .NET solution, it won't be much faster. MS.NET is that fast because it's just a wrapper around native code that uses the AES instruction set.
Ok, thank you for the info. Will be interesting to see the performance difference between BC and Mono. Every little bit will help in my case but getting rid of the dependency will be nice too.