Bug 25397 - Exception when instantiating an Android.Graphics.Pdf.PdfRenderer object
Summary: Exception when instantiating an Android.Graphics.Pdf.PdfRenderer object
Status: ASSIGNED
Alias: None
Product: Android
Classification: Xamarin
Component: General (show other bugs)
Version: 5.0
Hardware: Macintosh Mac OS
: Normal normal
Target Milestone: master
Assignee: dean.ellis
URL:
Depends on:
Blocks:
 
Reported: 2014-12-15 12:37 UTC by John Pilczak
Modified: 2016-09-20 11:19 UTC (History)
3 users (show)

See Also:
Tags: XATriaged
Is this bug a regression?: ---
Last known good build:


Attachments

Description John Pilczak 2014-12-15 12:37:31 UTC
While attempting to port the PdfRendererBasic java sample to Xamarin.Android, I have run into an exception when instantiating PdfRenderer. When launching the project, at the line where PdfRenderer's constructor is called, it throws an IOException where the exception's message reads "not create document. Error". This is not a regression from the current stable.

Application Output: https://gist.github.com/JohnPilczak/355a79ec73403446714d
Logcat output: https://gist.github.com/JohnPilczak/13f551a8e9dbd04b5082
My Solution: https://www.dropbox.com/s/48mmbz4z5bdjags/PdfRendererBasic.zip?dl=0

Environment Info: https://gist.github.com/JohnPilczak/ba6db77c3e4814ee8e93
Comment 1 Jonathan Pryor 2014-12-15 14:31:48 UTC
This is the same-but-different from Bug #23351: PdfRenderer *requires* that the parameter be "seekable", i.e. that it NOT be compressed:

https://developer.android.com/reference/android/graphics/pdf/PdfRenderer.html#PdfRenderer(android.os.ParcelFileDescriptor)
> Note: The provided file descriptor must be seekable, i.e. its data being randomly accessed,
> e.g. pointing to a file.

Unfortunately, when you add a file as an @(AndroidAsset), the file IS compressed, and thus is NOT seeable, hence the IOException.

The fix is to to either extract the Asset into a "normal" file and use the file, -OR- to use $(AndroidStoreUncompressedFileExtensions) to prevent the PDF file from being compressed within the .apk.
Comment 2 John Pilczak 2014-12-15 16:31:03 UTC
This issue has been occurring even with AndroidStoreUncompressedFileExtensions set to prevent PDF files from being compressed. Doing this had fixed Bug #23351 and allowed the FileDescriptor/ParcelFileDescriptor for the pdf to be opened and passed into the constructor and into the native FPDF were this error appears to be coming from.

Also, according to the source code for PdfRenderer, https://github.com/android/platform_frameworks_base/blob/master/core/jni/android/graphics/pdf/PdfRenderer.cpp#L89 , the error's message appears to be cutting off information at the beginning and end as it should read "cannot create document. Error: <errorcode>".
Comment 3 Jonathan Pryor 2014-12-15 23:11:08 UTC
It took me awhile to realize the problem with the error message, but Android has a bug:

https://github.com/android/platform_frameworks_base/blob/lollipop-release/core/jni/android/graphics/pdf/PdfRenderer.cpp#L85-86
       jniThrowException(env, "java/io/IOException",
                "cannot create document. Error:" + error);

The bug is that C is not Java, and the 3rd parameter is a `const char*`, so `"string constant" + value` doesn't perform a string concat, it instead returns `&"string constant"[value]`, i.e. the substring starting at index `value`.

This was later fixed:

https://github.com/android/platform_frameworks_base/blob/46d8444631b4b1253a76bfcc78a29d26014d022f/core/jni/android/graphics/pdf/PdfRenderer.cpp#L89-90
        jniThrowExceptionFmt(env, "java/io/IOException",
                "cannot create document. Error: %ld", error);

In the meantime, we can deduce that the error code is 3, as sizeof("can")==3 is trimmed from the beginning of the error message.

At this point things get somewhat confusing, as I'm not sure what library they're using, and thus what the error codes mean.

I do know that the FPDF_GetLastError() function used is defined in libpdfium.so. I *think* this is the Foxit PDF reader; if that's true, then error code 3 is "File not found or could not be opened.":

http://www.foxitsoftware.com/german/support/usermanuals/DLL311/group___f_p_d_f_v_i_e_w.html#gae825d36f23e023757bb127fa82b01454
http://www.foxitsoftware.com/german/support/usermanuals/DLL311/group___f_p_d_f_v_i_e_w.html#gad0bf9bd93b30af88f78aafcdf4e75023
Comment 4 Jonathan Pryor 2014-12-16 12:07:06 UTC
@John: Please port the sample to Java (or find the original Java sample). Does it work with Java?

I suspect that it does not.

The throw-site is in the nativeCreate() function, which takes a `size` parameter:
https://github.com/android/platform_frameworks_base/blob/46d8444631b4b1253a76bfcc78a29d26014d022f/core/jni/android/graphics/pdf/PdfRenderer.cpp#L77-96

The `size` parameter in turn comes from fstat(2):
https://github.com/android/platform_frameworks_base/blob/46d84446/graphics/java/android/graphics/pdf/PdfRenderer.java#L135-152

    Libcore.os.lseek(input.getFileDescriptor(), 0, OsConstants.SEEK_SET);
    size = Libcore.os.fstat(input.getFileDescriptor()).st_size;

Note, however, that before it grabs st_size it seeks to the beginning of the file.

Which raises the important question: what *is* the file that `input` refers to?

The file that `input` refers to is for the .apk itself, not a distinct filesystem entry within the .apk [0]. Which in turn strongly implies that using an Asset CANNOT WORK, particularly in combination with the lseek(2) to the beginning of the file.

The file MUST be backed by the filesystem.

This allows the sample to work:

	using (var i = context.Assets.Open ("sample.pdf"))
	using (var o = context.OpenFileOutput ("_sample.pdf",
				FileCreationMode.Private))
		i.CopyTo (o);
	var f = context.GetFileStreamPath ("_sample.pdf");
	fileDescriptor = ParcelFileDescriptor.Open (f, ParcelFileMode.ReadOnly);

[0]: Determined by P/Invoking to fstat(2) and checking struct stat::st_size for context.Assets.OpenFd("sample.pdf").ParcelFileDescriptor.Fd, which returns a size *far* larger than the PDF, but coincidentally matches the size of the .apk...
Comment 5 John Pilczak 2014-12-16 12:40:18 UTC
The Java sample does actually work. I just opened it in Android Studio and tried it on a Nexus 7 with the Lollipop system image along with an emulator and it the sample ran just fine on each meaning that the sample.pdf document was rendered onto the screen and I was able to look at it page by page.

The original Java sample: https://github.com/googlesamples/android-PdfRendererBasic
Comment 6 Jonathan Pryor 2014-12-16 15:46:43 UTC
I've "figured it out"...and I have no words. Much less a fix. (PUNT TIME!)

The problem is due to file ordering within the .apk.

The Java/gradle contents:

> $ unzip -lf Application-release.apk 
> Archive:  Application-release.apk
>   Length     Date   Time    Name
>  --------    ----   ----    ----
>      1896  12-16-14 15:31   AndroidManifest.xml
>    453132  12-16-14 15:31   assets/sample.pdf
>       583  12-16-14 15:31   res/drawable-hdpi-v4/ic_action_info.png
>      3153  12-16-14 15:31   res/drawable-hdpi-v4/ic_launcher.png
>       234  12-16-14 15:31   res/drawable-hdpi-v4/tile.9.png
>       395  12-16-14 15:31   res/drawable-mdpi-v4/ic_action_info.png
>      1929  12-16-14 15:31   res/drawable-mdpi-v4/ic_launcher.png
>       728  12-16-14 15:31   res/drawable-xhdpi-v4/ic_action_info.png
>      4218  12-16-14 15:31   res/drawable-xhdpi-v4/ic_launcher.png
>      1129  12-16-14 15:31   res/drawable-xxhdpi-v4/ic_action_info.png
>      7324  12-16-14 15:31   res/drawable-xxhdpi-v4/ic_launcher.png
>       960  12-16-14 15:31   res/layout/activity_main.xml
>       360  12-16-14 15:31   res/layout/activity_main_real.xml
>      1316  12-16-14 15:31   res/layout/fragment_pdf_renderer_basic.xml
>       452  12-16-14 15:31   res/menu/main.xml
>      5520  12-16-14 15:31   resources.arsc
>    869900  12-16-14 15:31   classes.dex
>     13432  12-16-14 15:30   lib/armeabi/libhelp.so
>     13436  12-16-14 15:30   lib/armeabi-v7a/libhelp.so
>      5212  12-16-14 15:30   lib/x86/libhelp.so
>      1720  12-16-14 15:31   META-INF/MANIFEST.MF
>      1749  12-16-14 15:31   META-INF/CERT.SF
>       665  12-16-14 15:31   META-INF/CERT.RSA
>  --------                   -------
>   1389443                   23 files

(Note: not the "default" upstream contents; above has been annotated to add "libhelp.so" for my debugging.)

If I take the Java/gradle output, REMOVE the META-INF entries, and re-sign in the same way that Xamarin.Android does:

> $ unzip -lv App-aligned.apk 
> Archive:  App-aligned.apk
>  Length   Method    Size  Ratio   Date   Time   CRC-32    Name
> --------  ------  ------- -----   ----   ----   ------    ----
>     1712  Defl:N      777  55%  12-16-14 15:36  c3703f0d  META-INF/MANIFEST.MF
>     1833  Defl:N      839  54%  12-16-14 15:36  38f61afa  META-INF/ANDROIDD.SF
>     1209  Defl:N     1054  13%  12-16-14 15:36  fe5a572b  META-INF/ANDROIDD.RSA
>     1896  Defl:N      700  63%  12-16-14 15:31  3d1d4f25  AndroidManifest.xml
>   453132  Stored   453132   0%  12-16-14 15:31  62461d34  assets/sample.pdf
>      583  Stored      583   0%  12-16-14 15:31  8e9102d0  res/drawable-hdpi-v4/ic_action_info.png
>     3153  Stored     3153   0%  12-16-14 15:31  491df637  res/drawable-hdpi-v4/ic_launcher.png
>      234  Stored      234   0%  12-16-14 15:31  c560f719  res/drawable-hdpi-v4/tile.9.png
>      395  Stored      395   0%  12-16-14 15:31  6d1da940  res/drawable-mdpi-v4/ic_action_info.png
>     1929  Stored     1929   0%  12-16-14 15:31  e2a4cdf0  res/drawable-mdpi-v4/ic_launcher.png
>      728  Stored      728   0%  12-16-14 15:31  9be31f46  res/drawable-xhdpi-v4/ic_action_info.png
>     4218  Stored     4218   0%  12-16-14 15:31  ed860537  res/drawable-xhdpi-v4/ic_launcher.png
>     1129  Stored     1129   0%  12-16-14 15:31  02df764b  res/drawable-xxhdpi-v4/ic_action_info.png
>     7324  Stored     7324   0%  12-16-14 15:31  e121e4e4  res/drawable-xxhdpi-v4/ic_launcher.png
>      960  Defl:N      423  56%  12-16-14 15:31  55e58153  res/layout/activity_main.xml
>      360  Defl:N      204  43%  12-16-14 15:31  bc4101d0  res/layout/activity_main_real.xml
>     1316  Defl:N      525  60%  12-16-14 15:31  a110395c  res/layout/fragment_pdf_renderer_basic.xml
>      452  Defl:N      235  48%  12-16-14 15:31  7dae45ed  res/menu/main.xml
>     5520  Stored     5520   0%  12-16-14 15:31  f9d480b4  resources.arsc
>   869900  Defl:N   358663  59%  12-16-14 15:31  a1547d03  classes.dex
>    13432  Defl:N     5472  59%  12-16-14 15:30  34f1043e  lib/armeabi/libhelp.so
>    13436  Defl:N     5467  59%  12-16-14 15:30  54b68516  lib/armeabi-v7a/libhelp.so
>     5212  Defl:N     1620  69%  12-16-14 15:30  4b99e3d6  lib/x86/libhelp.so
> --------          -------  ---                            -------
>  1390063           854324  39%                            23 files

...and run this app, the Java app FAILS with the same error as Xamarin.Android!

Note that the only "real" difference between the two is where the META-INF are located -- at the "beginning" or "end" of the .apk. If they're at the beginning -- as Xamarin.Android does -- then the app doesn't work.

> $ zip -d Application-release.apk META-INF/MANIFEST.MF
> $ zip -d Application-release.apk META-INF/CERT.SF
> $ zip -d Application-release.apk META-INF/CERT.RSA
> $ /usr/bin/jarsigner -keystore "$HOME/.local/share/Xamarin/Mono for Android/debug.keystore" -storepass android \
	-keypass android -digestalg SHA1 -sigalg md5withRSA \
	-signedjar App-unaligned.apk Application-release.apk \
	androiddebugkey 
> $ /opt/android/sdk-tool/sdk/build-tools/21.0.1/zipalign  4 App-unaligned.apk App-aligned.apk
> $ adb uninstall com.example.android.pdfrendererbasic ; adb install App-aligned.apk
> $ adb shell am start com.example.android.pdfrendererbasic/.MainActivity
Comment 7 Jonathan Pryor 2014-12-16 15:52:36 UTC
@John: Can you please confirm some behavior I'm seeing? I think PdfRenderer is FUBAR, Android-side... ;-)

Add ANOTHER .pdf to the Java assets directory. Change PdfRendererBasicFragment to use the new .pdf; is sample.pdf or the "new" .pdf shown? Change PdfRendererBasicFragment to use sample.pdf again? Which PDF is shown?

I added a "hw.pdf", and when I change PdfRendererBasicFragment to use "hw.pdf" instead of "sample.pdf", sample.pdf is the one that's displayed, NOT hw.pdf!
Comment 8 John Pilczak 2014-12-16 16:07:02 UTC
I can confirm that behavior. I added a new pdf file and changed the PdfRendererBasicFragment class to use the new pdf but the sample.pdf is displayed on the device when I run it for some reason.
Comment 9 Jonathan Pryor 2014-12-16 17:40:36 UTC
The "More than one PDF asset results in showing the wrong PDF" bug is filed upstream as:

https://code.google.com/p/android/issues/detail?id=82838
Comment 10 Jonathan Pryor 2014-12-16 18:12:54 UTC
The "META-INF entries can't be at the beginning of the file" bug is filed upstream as:

https://code.google.com/p/android/issues/detail?id=82841
Comment 11 Jonathan Pryor 2014-12-16 18:14:00 UTC
@Eno: For "fun", perhaps we should figure out why Android Studio/gradle is placing the META-INF entries at the end of the .apk while jarsigner/Xamarin.Android is placing them at the beginning of the .apk?

At least we will then be as bad as Android...

Note You need to log in before you can comment on or make changes to this bug.