Bug 36124 - NUnit tests fail when using command line and won't load in test runner
Summary: NUnit tests fail when using command line and won't load in test runner
Status: RESOLVED FIXED
Alias: None
Product: Xamarin Studio
Classification: Desktop
Component: F# Add-in ()
Version: 5.10 (C6SR3)
Hardware: Macintosh Mac OS
: --- normal
Target Milestone: master
Assignee: Dave Thomas
URL:
Depends on:
Blocks:
 
Reported: 2015-11-21 16:45 UTC by Ryan Riley
Modified: 2015-12-15 12:06 UTC (History)
2 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
Freya tests running in XS (195.86 KB, image/gif)
2015-12-15 12:04 UTC, Jason Imison
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Ryan Riley 2015-11-21 16:45:37 UTC
My NUnit tests no longer load in the Xamarin Studio test runner for the Freya project (https://github.com/freya-fs/freya). Also, when attempting to run from the command line using build.sh, I see the following errors:

40) Test Error : Freya.Router.Tests.Router.Router With No Routes Returns Next
   System.MissingMethodException : Method 'FSharpExpr.ValueWithName' not found.
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) <0x4630030 + 0x00093> in <filename unknown>:0 


Running build failed.
Error:
Fake.UnitTestCommon+FailedTestsException: NUnit test failed (40).
  at Fake.NUnitSequential.NUnit (Microsoft.FSharp.Core.FSharpFunc`2 setParams, IEnumerable`1 assemblies) <0x3f92000 + 0x00337> in <filename unknown>:0 
  at FSI_0001.Build+clo@323-11.Invoke (Microsoft.FSharp.Core.Unit _arg4) <0x3f61bc0 + 0x00087> in <filename unknown>:0 
  at Fake.TargetHelper+targetFromTemplate@195[a].Invoke (Microsoft.FSharp.Core.Unit unitVar0) <0x3f54110 + 0x00020> in <filename unknown>:0 
  at Fake.TargetHelper.runSingleTarget (Fake.TargetTemplate`1 target) <0x3f4b698 + 0x000bd> in <filename unknown>:0
Comment 1 Jason Imison 2015-12-15 12:04:50 UTC
Created attachment 14296 [details]
Freya tests running in XS
Comment 2 Jason Imison 2015-12-15 12:06:57 UTC
I tested the Freya tests in XS 6.0 and they are working there (see attachment). There was a fix recently related to backticked test names not working correctly. 

I also tested running the tests from the command line using Mono 4.2.1

08:23 $ ./build.sh
No version specified. Downloading latest stable.
Starting download from https://github.com/fsprojects/Paket/releases/download/2.36.5/paket.exe
Done.
Paket version 2.36.5.0
Downloading FAKE 4.9.1
Downloading Microsoft.Owin.Host.HttpListener 3.0.1
Downloading Microsoft.Owin.Hosting 3.0.1
Downloading Microsoft.Owin 3.0.1
Downloading Microsoft.Owin.SelfHost 3.0.1
Downloading Microsoft.Owin.Diagnostics 3.0.1
Downloading Arachne.Http 2.3.25
Downloading Arachne.Core 2.3.25
Downloading Aether 8.0.0-rc2
Downloading Microsoft.Owin.Testing 3.0.1
Downloading Arachne.Language 2.3.25
Downloading Arachne.Http.Cors 2.3.25
Downloading Arachne.Uri 2.3.25
Downloading Arachne.Uri.Template 2.3.25
Downloading Owin 1.0.0
Downloading Unquote 3.1.0
Downloading Chiron 6.0.0-rc2
Downloading FParsec 1.0.2
Downloading Hekate 2.0.0-rc2
22 seconds - ready.
Building project with version: LocalBuild
Shortened DependencyGraph for Target Default:
<== Default
   <== Publish.Pack
      <== Source
         <== Source.Test
            <== Source.Build
               <== Source.AssemblyInfo
                  <== Source.GitSubmodules
                     <== Source.Clean

The resulting target order is:
 - Source.Clean
 - Source.GitSubmodules
 - Source.AssemblyInfo
 - Source.Build
 - Source.Test
 - Source
 - Publish.Pack
 - Default
Starting Target: Source.Clean
Creating /Users/jason/src/fsharp/freya/bin
Creating /Users/jason/src/fsharp/freya/temp
Finished Target: Source.Clean
Starting Target: Source.GitSubmodules (==> Source.Clean)
git submodule update --init
Submodule 'ui/inspector' (https://github.com/freya-fs/freya.ui.inspector.git) registered for path 'ui/inspector'
Submodule path 'ui/inspector': checked out 'b34943f05c7729242ff922d9547ef86228b6f33e'
Finished Target: Source.GitSubmodules
Starting Target: Source.AssemblyInfo (==> Source.GitSubmodules)
Created AssemblyInfo file "src/Freya.Core/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Lenses.Http/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Lenses.Http.Cors/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Machine/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Machine.Extensions.Http/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Machine.Extensions.Http.Cors/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Machine.Router/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Recorder/AssemblyInfo.fs".
Created AssemblyInfo file "src/Freya.Router/AssemblyInfo.fs".
Finished Target: Source.AssemblyInfo
Starting Target: Source.Build (==> Source.AssemblyInfo)
Building project: Freya.sln
  xbuild  Freya.sln /t:Build  /v:q  /p:RestorePackages="False" /p:Optimize="True" /p:DebugSymbols="True" /p:Configuration="Release" /logger:Fake.MsBuildLogger+ErrorLogger,"/Users/jason/src/fsharp/freya/packages/build/FAKE/tools/FakeLib.dll"
xbuild  Freya.sln /t:Build  /v:q  /p:RestorePackages="False" /p:Optimize="True" /p:DebugSymbols="True" /p:Configuration="Release" /logger:Fake.MsBuildLogger+ErrorLogger,"/Users/jason/src/fsharp/freya/packages/build/FAKE/tools/FakeLib.dll"
XBuild Engine Version 12.0
Mono, Version 4.2.1.0
Copyright (C) 2005-2013 Various Mono authors
Finished Target: Source.Build
Starting Target: Source.Test (==> Source.Build)
/Users/jason/src/fsharp/freya/packages/test/NUnit.Runners/tools/nunit-console.exe "-nologo" "-noshadow" "-labels" "tests/Freya.Core.Tests/bin/Release/Freya.Core.Tests.dll" "tests/Freya.Machine.Tests/bin/Release/Freya.Machine.Tests.dll" "tests/Freya.Machine.Extensions.Http.Tests/bin/Release/Freya.Machine.Extensions.Http.Tests.dll" "tests/Freya.Router.Tests/bin/Release/Freya.Router.Tests.dll" "-xml:bin/TestResults.xml"
mono  /Users/jason/src/fsharp/freya/packages/test/NUnit.Runners/tools/nunit-console.exe "-nologo" "-noshadow" "-labels" "tests/Freya.Core.Tests/bin/Release/Freya.Core.Tests.dll" "tests/Freya.Machine.Tests/bin/Release/Freya.Machine.Tests.dll" "tests/Freya.Machine.Extensions.Http.Tests/bin/Release/Freya.Machine.Extensions.Http.Tests.dll" "tests/Freya.Router.Tests/bin/Release/Freya.Router.Tests.dll" "-xml:bin/TestResults.xml"
WARNING: The runtime version supported by this application is unavailable.
Using default runtime: v4.0.30319
ProcessModel: Default    DomainUsage: Multiple
Execution Runtime: mono-4.0
***** Freya.Core.Tests.Core.apply correctly unwraps and applies Freya computations
***** Freya.Core.Tests.Core.Lens get, set and map behave correctly
***** Freya.Core.Tests.Core.map + apply correctly chain across applicative computations
***** Freya.Core.Tests.Core.map correctly applies a function to a Freya computation
***** Freya.Core.Tests.Core.map2 correctly applies a function to Freya computations
***** Freya.Core.Tests.Integration.freya computation can compose with an OwinAppFunc
***** Freya.Core.Tests.Integration.freya computation can roundtrip to and from OwinAppFunc
***** Freya.Core.Tests.Integration.MidFunc can roundtrip to Freya Pipeline before / after and back
***** Freya.Core.Tests.Integration.MidFunc can work with other Freya Pipelines
***** Freya.Core.Tests.Integration.MidFunc can work with other Freya Pipelines and correctly halt when first halts
***** Freya.Core.Tests.Integration.MidFunc can work with other Freya Pipelines and correctly halt when midFunc returns early
***** Freya.Core.Tests.Integration.MidFunc can work with other Freya Pipelines and correctly halt when second halts
***** Freya.Core.Tests.Integration.MidFunc that terminates early and doesn't call next Halts correctly as a Freya Pipeline
***** Freya.Core.Tests.Integration.MidFunc that throws early and doesn't call next Halts correctly as a Freya Pipeline
***** Freya.Core.Tests.Integration.pipeline executes app and both monads if first returns halt after running the app
***** Freya.Core.Tests.Integration.pipeline executes app and both monads if first returns next after running the app
***** Freya.Core.Tests.Integration.pipeline executes app and both monads if first returns next after running the app with composed pipeline
***** Freya.Core.Tests.Integration.pipeline executes app and only the first monad if first returns halt after running the app with composed pipeline
***** Freya.Core.Tests.Integration.pipeline executes both monads if first returns next
***** Freya.Core.Tests.Integration.pipeline executes both monads if first returns next with composed pipeline
***** Freya.Core.Tests.Integration.pipeline executes both monads if first returns next with wrapped pipeline OwinMidFunc
***** Freya.Core.Tests.Integration.pipeline executes only the first monad if first returns halt
***** Freya.Core.Tests.Integration.pipeline executes only the first monad if first returns terminate with composed pipeline
***** Freya.Core.Tests.Integration.pipeline executes only the first monad if first returns terminate with wrapped pipeline OwinMidFunc
***** Freya.Core.Tests.Integration.test app works
***** Freya.Core.Tests.Pipeline.pipeline executes both monads if first returns next
***** Freya.Core.Tests.Pipeline.pipeline executes only the first monad if first returns terminate
***** Freya.Machine.Tests.todobackend not returns 304 for If-Modified-Since
***** Freya.Machine.Tests.todobackend returns 304 for If-Modified-Since
***** Freya.Machine.Tests.todobackend returns empty array at first
***** Freya.Machine.Extensions.Http.Tests.Domain.Charset Negotiation
***** Freya.Machine.Extensions.Http.Tests.Domain.Encoding Negotiation
***** Freya.Machine.Extensions.Http.Tests.Domain.Language Negotiation
***** Freya.Machine.Extensions.Http.Tests.Domain.MediaType Negotiation
***** Freya.Router.Tests.Router.Router Executes First Full Match
***** Freya.Router.Tests.Router.Router Executes Pipeline Registered First
***** Freya.Router.Tests.Router.Router With Base Route Executes Correctly
***** Freya.Router.Tests.Router.Router With Method Specific Routes Executes Correctly
***** Freya.Router.Tests.Router.Router With Multiple Routes Executes Correctly
***** Freya.Router.Tests.Router.Router With No Routes Returns Next

Tests run: 40, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.3466044 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

Finished Target: Source.Test
Starting Target: Source (==> Source.Test)
Finished Target: Source
Starting Target: Publish.Pack (==> Source)
mono  /Users/jason/src/fsharp/freya/.paket/paket.exe pack output bin  version  3.0.0 releaseNotes  "A lot of stuff - TODO!"
Paket version 2.36.5.0
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
No platform specified; found output path node for the AnyCPU platform after failing to find one for the following:
Packed: /Users/jason/src/fsharp/freya/paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Machine.Router/Freya.Machine.Router.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Lenses.Http.Cors/Freya.Lenses.Http.Cors.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Lenses.Http/Freya.Lenses.Http.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Machine.Extensions.Http.Cors/Freya.Machine.Extensions.Http.Cors.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Recorder/Freya.Recorder.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Machine/Freya.Machine.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Core/Freya.Core.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Router/Freya.Router.fsproj.paket.template
Packed: /Users/jason/src/fsharp/freya/src/Freya.Machine.Extensions.Http/Freya.Machine.Extensions.Http.fsproj.paket.template
0 seconds - ready.
Finished Target: Publish.Pack
Starting Target: Default (==> Publish.Pack)
Finished Target: Default

---------------------------------------------------------------------
Build Time Report
---------------------------------------------------------------------
Target                 Duration
------                 --------
Source.Clean           00:00:00.0014322
Source.GitSubmodules   00:00:01.9896600
Source.AssemblyInfo    00:00:00.0108187
Source.Build           00:00:52.2892103
Source.Test            00:00:03.8312344
Source                 00:00:00.0000303
Publish.Pack           00:00:01.0537109
Default                00:00:00.0010312
Total:                 00:00:59.2137529
Status:                Ok
---------------------------------------------------------------------
✔ ~/src/fsharp/freya [master|✔]
08:24 $ mono --version
Mono JIT compiler version 4.2.1 (explicit/3ddba85 Mon Dec  7 16:23:48 EST 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  x86
	Disabled:      none
	Misc:          softdebug
	LLVM:          yes(3.6.0svn-mono-(detached/a173357)
	GC:            sgen