Bug 58778 - Xamarin Profiler 1.5.5-10 errors out trying to open a mlpd when starting new profiling session
Summary: Xamarin Profiler 1.5.5-10 errors out trying to open a mlpd when starting new ...
Status: VERIFIED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: Tools (show other bugs)
Version: XI 10.14 (d15-4)
Hardware: Macintosh Mac OS
: High normal
Target Milestone: 15.3
Assignee: Rolf Bjarne Kvinge [MSFT]
URL:
Depends on:
Blocks:
 
Reported: 2017-08-15 16:32 UTC by Rob Ringham
Modified: 2017-09-06 06:18 UTC (History)
8 users (show)

See Also:
Tags:
Is this bug a regression?: Yes
Last known good build:


Attachments
Screenshots of issues (47.97 KB, application/zip)
2017-08-15 16:32 UTC, Rob Ringham
Details
Xamarin Profiler logs (101.31 KB, application/zip)
2017-08-16 12:40 UTC, Rob Ringham
Details
screen_after_changing_app_bundle_id (119.46 KB, image/png)
2017-08-16 13:01 UTC, Rob Ringham
Details
iOS debug panel screenshots (247.23 KB, application/zip)
2017-08-16 15:45 UTC, Rob Ringham
Details
Build log with -vvvv (932.96 KB, text/plain)
2017-08-16 16:15 UTC, Rob Ringham
Details
Test case (8.17 KB, application/zip)
2017-08-22 09:11 UTC, Rolf Bjarne Kvinge [MSFT]
Details

Description Rob Ringham 2017-08-15 16:32:05 UTC
Created attachment 24196 [details]
Screenshots of issues

just upgraded to the latest version of VS for Mac (7.1 b1297) and Xamarin Profiler (1.5.5-10), and I'm no longer able to get Xamarin Profiler to connect to a deployed Xamarin.iOS app on my iOS device. I had no trouble with a previous version of Xamarin Profiler - this appears to be isolated to the newest version.

When I start profiling the app, the app starts (on my USB-connected iPad), the profiler opens, and gets stuck trying to open a mlpd under /var (see attachment #1 [details] in xam_profiler_attachments.zip: attach_1_open_mlpd.png).

I don't know what this mlpd is (I wasn't trying to load it) - I'm assuming it's a blank mlpd for a new profiling session. It tries this for awhile, then errors out (see attachment #2 [details] in xam_profiler_attachments.zip: attach_2_empty_mlpd.png).

Eventually, I get an error message popping up (see attachment #3 [details] in xam_profiler_attachments.zip: attach_3_error_popup.png).

I've tried the following things, but this error still persists:

- Deleted the /var/folders/19 folder
- Uninstalled and reinstalled Xamarin Profiler
- Deleted the deployed app, rebuilt, deployed again, and tried to profile again
- Looked through the Xamarin Profiler troubleshooting guide for any applicable solution (none found)
- Tried a previous version (1.5.4-19), with the same behavior. I suspect the version I was using yesterday was older than 1.5.4-19.

All with no luck. Relevant content from a log file for a session where this is happening:

XamarinProfiler.Mac.LcdView Information: 0 : [2017-08-15 09:21:45.1188] Sorry, something went wrong trying to run com.temp.app: file /var/folders/19/z02myk096cv53ll0mm_hrvy80000gp/T/n87mh2qq.mlpd exists, but is empty
XamarinProfiler.Mac.LcdView Information: 0 : [2017-08-15 09:21:45.1197] Stopping iOS app com.temp.app
XamarinProfiler.Mac.LcdView Error: 0 : [2017-08-15 09:23:47.5555] error stopping iOS application
System.IO.IOException: USB connect timeout (ENODATA), likely because the app failed to launch. Please review device logs and/or crash reports for more information.
  at Xamarin.MacDev.AggregateAsyncResult.CheckError (System.Boolean cancelled) [0x0001c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/AggregateAsyncResult.cs:68 
  at Xamarin.MacDev.IPhoneCommandConnection.EndExecuteCommand (System.IAsyncResult result) [0x00007] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/IPhoneCommandConnection.cs:146 
  at Xamarin.MacDev.IPhoneCommandConnection.Stop () [0x0004c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/IPhoneCommandConnection.cs:448 
  at XamarinProfiler.Core.Platforms.iOS.MTouchRunner.Stop () [0x0002c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/src/XamarinProfiler.Core/Platforms/iOS/MTouchRunner.cs:205 
XamarinProfiler.Mac.LcdView Error: 0 : [2017-08-15 09:23:47.5555] error stopping iOS application
System.IO.IOException: USB connect timeout (ENODATA), likely because the app failed to launch. Please review device logs and/or crash reports for more information.
  at Xamarin.MacDev.AggregateAsyncResult.CheckError (System.Boolean cancelled) [0x0001c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/AggregateAsyncResult.cs:68 
  at Xamarin.MacDev.IPhoneCommandConnection.EndExecuteCommand (System.IAsyncResult result) [0x00007] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/IPhoneCommandConnection.cs:146 
  at Xamarin.MacDev.IPhoneCommandConnection.Stop () [0x0004c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/external/maciostools/Xamarin.MacDev.Ide/IPhoneCommandConnection.cs:448 
  at XamarinProfiler.Core.Platforms.iOS.MTouchRunner.Stop () [0x0002c] in /Users/builder/data/lanes/4982/292cc19a/source/profiler/src/XamarinProfiler.Core/Platforms/iOS/MTouchRunner.cs:205 
XamarinProfiler.Mac.LcdView Information: 0 : [2017-08-15 09:23:47.5596] Stopping profiler log reader
XamarinProfiler.Mac.LcdView Information: 0 : [2017-08-15 09:23:47.5600] Stopping profiler log reader


Note: this issue is also described here on the Xamarin Profiler forum: https://forums.xamarin.com/discussion/101269/latest-xamarin-profiler-1-5-5-10-appears-to-be-broken-cannot-connect-to-app-on-device#latest
Comment 1 Rob Ringham 2017-08-15 17:06:52 UTC
Update: I've gone back all the way to Xamarin Profiler 1.4.0-1 and I'm still getting the same issue. Maybe something in the 1.5.5-10 update broke or corrupted some local configuration?
Comment 2 Rodrigo Moya 2017-08-16 08:12:59 UTC
Can you please attach the device logs? For that, open Xcode and go to Window->Devices.

also, does it fail also on simulator, or only on device?
Comment 3 Rodrigo Moya 2017-08-16 09:00:57 UTC
hmm, I have been able to reproduce this by trying to profile 2 apps at the same time. Is that your case? Or is the machine where this happens shared with other people that might be profiling at the same time? Or maybe a stale Xamarin Profiler process?
Comment 4 Rob Ringham 2017-08-16 12:39:14 UTC
Hi Rodrigo - that is not my case (profiling 2 apps at the same time). Nor does it appear to be a stale Xamarin Profiler process (have been restarting my Mac regularly throughout this process to see if that has anything to do with it). The problem persists even after restarting, ensuring no stale processes could be running.

An additional test I ran this morning may provide more insight: I created a separate Xamarin project and was able to successfully profile it on my device. However - when I switched back to the main project I'm working on (my client's project), it's right back to the same error. Perhaps an update to Xamarin Profiler corrupted some local DB or cache for this specific app? Is there a way I could clear out that cache to test that hypothesis?
Comment 5 Rob Ringham 2017-08-16 12:40:16 UTC
Created attachment 24233 [details]
Xamarin Profiler logs

Here are relevant logs from yesterday. Please let me know if there are any additional logs I can attach that would be helpful.
Comment 6 Rob Ringham 2017-08-16 12:45:06 UTC
It's also worth pointing out that I can successfully profile my client's app in the iOS Simulator. It's only when I switch back to my actual iPad that this fails (just verified this again) - (but unfortunately, the nature of the testing I'm doing requires me to be testing on an actual iPad).
Comment 7 Rob Ringham 2017-08-16 13:01:22 UTC
Created attachment 24234 [details]
screen_after_changing_app_bundle_id

Another test I just ran - changed the app bundle ID of my client's app, to see if a clean install of the app in a separate sandbox on the iPad would help - unfortunately, I get the same behavior / error (attached screenshot screen_after_changing_app_bundle_id.png).
Comment 8 Rodrigo Moya 2017-08-16 15:32:03 UTC
Oh ok, so if another project can be profiled, I guess if this is a problem in the project options, so can you please go to the iOS project options and take a screenshot of the iOS Debug panel? There is a 'enable profiling' checkbox there, so is it enabled for the configuration you're using for profiling that app on device?

also, you can compare what's in there compared to the other project you created that can be profiled on device.
Comment 9 Rob Ringham 2017-08-16 15:45:41 UTC
Created attachment 24244 [details]
iOS debug panel screenshots

Have included 3 screenshots:

- Client project - iOS Debug panel - Simulator configuration
- Client project - iOS Debug panel - Device configuration
- Separate project - iOS Debug panel - Device configuration (the project I can profile on a device)

This does not seem like a problem with the project's options. Note that when I change the debugger port (10000 for client project) to match the separate project (to 15951), I get the exact same behavior for the client project (mlpd exists, but is empty error in Xamarin Profiler).

Just to reiterate - I was successfully profiling this client project with an earlier version of Xamarin Profiler (not sure which one) on Monday. As soon as the latest update to 1.5.5-10 came through yesterday, this project became un-profilable.
Comment 10 Rodrigo Moya 2017-08-16 15:56:05 UTC
all seems to be good indeed, so let's try another thing:

* in project options/iOS Build, add -vvvv to "additional mtouch arguments entry
* do a full rebuild of the failing iOS project/configuration and attach here the build log from VSfM. hopefully that would tell us something about whether the runtime profiler is included in the app or not.
Comment 12 Rodrigo Moya 2017-08-16 17:34:00 UTC
hmm, that build logs shows what might be the problem:

    /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(748,3): warning MSB6002: The command-line for the "MTouch" task is too long. Command-lines longer than 32000 characters are likely to fail. Try reducing the length of the command-line by breaking down the call to "MTouch" into multiple calls with fewer parameters per call.

@Rolf, could that prevent the runtime profiler to be loaded, even if enable profiling is on?
Comment 13 Jose Medrano 2017-08-16 17:59:52 UTC
Hello!

I could reproduce the issue with an old template example... I was comparing my mtouch msbuild vars in the project configuration... and those were different with the newest template.

**MtouchExtraArgs                = --linkskip=System.Core
**MtouchFastDev                  = False
**MtouchFloat32                  = False
**MtouchNoSymbolStrip            = False

All my properties in <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|iPhone' "> to the current defaults and all works like expected.

Seems there are an issue related to this, maybe your template is too old?

Can you try in your .csproj and PropertyGroup where Config|platform is == "Debug|Iphone" change all to this default properties (remember to backup your current values first)? 

    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\iPhone\Debug</OutputPath>
    <DefineConstants>DEBUG;ENABLE_TEST_CLOUD;</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <CodesignKey>iPhone Developer</CodesignKey>
    <DeviceSpecificBuild>true</DeviceSpecificBuild>
    <MtouchDebug>true</MtouchDebug>
    <MtouchNoSymbolStrip>true</MtouchNoSymbolStrip>
    <MtouchFastDev>true</MtouchFastDev>
    <MtouchProfiling>true</MtouchProfiling>
    <MtouchFloat32>true</MtouchFloat32>
    <CodesignEntitlements>Entitlements.plist</CodesignEntitlements>
    <IOSDebuggerPort>27489</IOSDebuggerPort>
    <MtouchLink>SdkOnly</MtouchLink>
    <MtouchArch>ARMv7, ARM64</MtouchArch>
    <MtouchHttpClientHandler>HttpClientHandler</MtouchHttpClientHandler>

Thanks!
Comment 14 Rob Ringham 2017-08-16 18:38:28 UTC
Hi Jose - that worked! Note this template is relatively new (project started in January 2017) - but this change allows me to profile on the device. Are the changes to the default template documented anywhere that we can keep an eye on?
Comment 15 Jose Medrano 2017-08-16 18:56:55 UTC
Cool! 

Go deeper this issue... MtouchFastDev is not in Debug|iPhone configuration in old templates... and this probably is doing some kind of exception running the usb logging in the device doing profiler waiting forever...
Comment 16 Rolf Bjarne Kvinge [MSFT] 2017-08-17 07:16:40 UTC
(In reply to Rodrigo Moya from comment #12)
> hmm, that build logs shows what might be the problem:
> 
>    
> /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.
> Common.targets(748,3): warning MSB6002: The command-line for the "MTouch"
> task is too long. Command-lines longer than 32000 characters are likely to
> fail. Try reducing the length of the command-line by breaking down the call
> to "MTouch" into multiple calls with fewer parameters per call.
> 
> @Rolf, could that prevent the runtime profiler to be loaded, even if enable
> profiling is on?

No, this would not cause profiler problems, it would just fail to build with strange errors.

Also the command-line limit on macOS is ~256kb, so it's unlikely this is causing problems unless the project is truly big.
Comment 17 Rolf Bjarne Kvinge [MSFT] 2017-08-17 07:22:55 UTC
(In reply to Jose Medrano from comment #15)
> Cool! 
> 
> Go deeper this issue... MtouchFastDev is not in Debug|iPhone configuration
> in old templates... and this probably is doing some kind of exception
> running the usb logging in the device doing profiler waiting forever...

So if I understand correctly, profiling stops working when MtouchFastDev ("Incremental builds" in the project's iOS options) is enabled?

Can you reproduce with VSfM? If so, can you attach the Application Output?
Comment 18 Rolf Bjarne Kvinge [MSFT] 2017-08-17 14:21:16 UTC
This occurs when "Strip native debugging symbols" is checked in the project's iOS Build options (thus the workaround is to uncheck that checkbox).
Comment 19 Rolf Bjarne Kvinge [MSFT] 2017-08-17 14:23:37 UTC
Setting milestone to 15.5 since there's a trivial workaround for 15.4 and earlier.
Comment 20 Rolf Bjarne Kvinge [MSFT] 2017-08-17 14:42:58 UTC
master: https://github.com/xamarin/xamarin-macios/pull/2501
Comment 22 Rolf Bjarne Kvinge [MSFT] 2017-08-22 09:06:40 UTC
We're trying to get this into 15.3, so I'm updating the milestone accordingly.

d15-3: https://github.com/xamarin/xamarin-macios/pull/2520
d15-4: https://github.com/xamarin/xamarin-macios/pull/2521
Comment 23 Rolf Bjarne Kvinge [MSFT] 2017-08-22 09:11:52 UTC
Created attachment 24346 [details]
Test case

QA: use the attached project to reproduce/verify.

Note You need to log in before you can comment on or make changes to this bug.