Bug 38742 - CVOpenGLESTextureCache not compatible/working with ES 3.0
Summary: CVOpenGLESTextureCache not compatible/working with ES 3.0
Alias: None
Product: iOS
Classification: Xamarin
Component: Xamarin.iOS.dll ()
Version: XI 9.4 (iOS 9.2)
Hardware: PC Mac OS
: --- normal
Target Milestone: Untriaged
Assignee: Bugzilla
Depends on:
Reported: 2016-02-14 01:56 UTC by renan jegouzo
Modified: 2016-02-29 13:22 UTC (History)
2 users (show)

Is this bug a regression?: ---
Last known good build:

Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.

Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:

Description renan jegouzo 2016-02-14 01:56:46 UTC
CVOpenGLESTextureCache not working with opengl ES 3.0 (opentk-1.0)

using OpenTK.Graphics.ES30;

var tCache=CVOpenGLESTextureCache.FromEAGLContext(EAGLContext.CurrentContext);

var texY=tCache.TextureFromImage(buf,true,(OpenTK.Graphics.ES20.All)All.Red,sz.Width,sz.Height,(OpenTK.Graphics.ES20.All)All.Red,(OpenTK.Graphics.ES20.DataType)DataType.UnsignedByte,0,out ret);
var texUV=tCache.TextureFromImage(buf,true,(OpenTK.Graphics.ES20.All)All.Rg,sz.Width/2,sz.Height/2,(OpenTK.Graphics.ES20.All)All.Rg,(OpenTK.Graphics.ES20.DataType)DataType.UnsignedByte,1,out ret);

texY & TexUV returns null
this code was working well using ES 2.0
Comment 1 Rolf Bjarne Kvinge [MSFT] 2016-02-15 10:48:58 UTC
This might very well be related to how 'buf' is created, so could you attach a complete test project we can use to reproduce this?
Comment 2 renan jegouzo 2016-02-16 01:10:49 UTC
just use your test project that work in egl 2.0
Comment 3 renan jegouzo 2016-02-16 01:18:41 UTC
new VideoDataDelegate(delegate(AVCaptureOutput o,CoreMedia.CMSampleBuffer sbuf,AVCaptureConnection con) {
    var buf=sbuf.GetImageBuffer() as CVPixelBuffer;
    // ....
Comment 4 renan jegouzo 2016-02-16 01:22:14 UTC
VideoDataDelegate : IAVCaptureVideoDataOutputSampleBufferDelegate
Comment 5 renan jegouzo 2016-02-16 01:26:22 UTC
but for your test I think the easiest way is to create a CVPixelBuffer by yourself https://developer.apple.com/library/mac/documentation/QuartzCore/Reference/CVPixelBufferRef/#//apple_ref/c/func/CVPixelBufferCreateWithBytes
Comment 6 renan jegouzo 2016-02-16 01:47:57 UTC
in all case you haven't implemented the API for EGL 3.0 and 3.1, so it's normal it's not working..

for your information 
OpenGL ES 3.0 is implemented in the Apple A7, A8, and A9 GPUs
so all device with this CPU/GPU are not compatible with xamarin openGL implementation.

your implementation: (only ES20)

using OpenTK.Graphics.ES20;
using System;

public CVOpenGLESTexture TextureFromImage (CVImageBuffer imageBuffer, bool isTexture2d, All internalFormat, int width, int height, All pixelFormat, DataType pixelType, int planeIndex, out CVReturn errorCode);

also I don't understand why your modified version of OpenTK isn't open source.. and pulled on github..
Comment 7 renan jegouzo 2016-02-16 01:49:39 UTC
ES2.0 means only EGL 1.2, we are at EGL 3.1 now..
Comment 8 renan jegouzo 2016-02-16 01:51:31 UTC
ooops, GLSL 3.1
Comment 9 Rolf Bjarne Kvinge [MSFT] 2016-02-16 09:39:01 UTC
Almost all our opentk code (in OpenTK-1.0.dll) is public here: https://github.com/mono/opentk
Comment 10 Rolf Bjarne Kvinge [MSFT] 2016-02-16 10:07:58 UTC
This works fine for me:

	var tCache=CoreVideo.CVOpenGLESTextureCache.FromEAGLContext(EAGLContext.CurrentContext);
	var sz = new System.Drawing.Rectangle (0, 0, 100, 100);
	CoreVideo.CVReturn ret;

	var buf = new CoreVideo.CVPixelBuffer (sz.Width, sz.Height, CVPixelFormatType.TwoComponent8);
	var texY = tCache.TextureFromImage (buf, true, (OpenTK.Graphics.ES20.All)OpenTK.Graphics.ES30.All.Red, sz.Width, sz.Height, (OpenTK.Graphics.ES20.All)OpenTK.Graphics.ES30.All.Red, (OpenTK.Graphics.ES20.DataType)DataType.UnsignedByte, 0, out ret);
	Console.WriteLine ("texY: {0} ret: {1}", texY, ret);
	var texUV = tCache.TextureFromImage (buf, true, (OpenTK.Graphics.ES20.All)OpenTK.Graphics.ES30.All.Rg, sz.Width / 2, sz.Height / 2, (OpenTK.Graphics.ES20.All)OpenTK.Graphics.ES30.All.Rg, (OpenTK.Graphics.ES20.DataType)DataType.UnsignedByte, 1, out ret);
	Console.WriteLine ("texUV: {0} ret: {1}", texUV, ret);

I've also fixed CVOpenGLESTextureCache.TextureFromImage to take ES30 versions of the All and DataType enums so that you don't have to cast to ES20 enums [1] (although casting enums would not cause anything to fail; but if you still need help, please attach a _complete_ project (not just code snippets) we can use to reproduce any issue).

[1] maccore/master: 7ffe71e6983ece9c008ddbb33c824304ec044e63
Comment 11 renan jegouzo 2016-02-16 14:34:41 UTC
are you sure you have initialized your EAGL context in ES30 ?


new GLKView(CGRect(0,0,100,100), new OpenGLES.EAGLContext(EAGLRenderingAPI.OpenGLES3));

or maybe directly: new OpenGLES.EAGLContext(EAGLRenderingAPI.OpenGLES3)
Comment 12 Rolf Bjarne Kvinge [MSFT] 2016-02-16 14:48:46 UTC
Please provide a _complete_ sample project, that's the only way to ensure we're running the same code.
Comment 13 renan jegouzo 2016-02-16 15:35:15 UTC
but your sample code doesn't work, even in ES20...
Comment 14 renan jegouzo 2016-02-16 15:35:51 UTC
my code is too big to be shared
Comment 15 renan jegouzo 2016-02-16 15:36:31 UTC
but putting here fake code snippet doesn't help rolf..
Comment 16 renan jegouzo 2016-02-16 16:04:02 UTC
I will keep es20 for my project, and after I will port all to swift, and bye bye xamarin..
Comment 17 renan jegouzo 2016-02-18 00:45:31 UTC
seems like there is already a problem with this API
var pixbuf=new CoreVideo.CVPixelBuffer(sz.Width,sz.Height,CoreVideo.CVPixelFormatType.CV420YpCbCr8BiPlanarFullRange,new CVPixelBufferAttributes { OpenGLCompatibility=true, PixelFormatType=CVPixelFormatType.CV32RGBA });

which PixelFormatType will be used in this case ?
Comment 18 Rolf Bjarne Kvinge [MSFT] 2016-02-18 09:20:53 UTC
The ctor is calling the iOS function CVPixelBufferCreate: https://developer.apple.com/library/ios/documentation/QuartzCore/Reference/CVPixelBufferRef/index.html#//apple_ref/c/func/CVPixelBufferCreate, so it'll do whatever that function does.
Comment 20 Rolf Bjarne Kvinge [MSFT] 2016-02-20 00:38:30 UTC
CVPixelBufferCreateResolvedAttributesDictionary is implemented as CVPixelBuffer.GetAttributes.

CVPixelBufferCreateWithPlanarBytes is not implemented yet (I just did), but you can use this in the meantime: https://gist.github.com/rolfbjarne/ca4c60b9c1d4da4a8d50
Comment 21 renan jegouzo 2016-02-20 10:10:31 UTC
maybe you could rename it as CVPixel.ResolveAttributes and set it as static function (like apple) not a member of a CVPixelBuffer instance..
also maybe it returns a new CVPixelBuffer in the output  dictionary; in this case maybe keep the same name CVPixel.CreateWithResolvedAttributes.

cf: "allocator: The allocator to use to create the pixel buffer. Pass NULL to specify the default allocator."
Comment 22 renan jegouzo 2016-02-20 10:24:41 UTC
for CVPixelBufferCreateWithPlanarBytes the most important field of CVPixelBufferCreateWithPlanarBytes is missing "pixelBufferAttributes" in 
Comment 23 Rolf Bjarne Kvinge [MSFT] 2016-02-29 13:22:57 UTC
That's trivial to add (and I've already done it, just reload the gist).