Tests not run

[keithyipkw]

I have just added bwoken into my project. The tests seemed not to run. The "Complete Duration:" was missing. I also typed some random characters in my test js. bwoken happily accepted them and printed the filenames. Then I restarted my computer, did a fresh installation and run the examples. The results were the same. Enabling verbose gave me empty lines after ** BUILD SUCCEEDED **. I am using Xcode 6.1 (6A1052d)

$ bwoken -v
2.1.0.rc.2

$ruby -v
ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]

$ bwoken test
Building.......
Build Successful!

iphone example.js

ipad example.js

$ bwoken test --verbose

Build settings from command line:
BWOKEN_CONFIGURATION_BUILD_DIR = ~/myproject/build/iphonesimulator
SDKROOT = iphonesimulator8.1
Build settings from configuration file '/Library/Ruby/Gems/2.0.0/gems/bwoken-2.1.0.rc.2/lib/bwoken/configs/bwoken.xcconfig':
CONFIGURATION_BUILD_DIR = $(BWOKEN_CONFIGURATION_BUILD_DIR)
GCC_PREPROCESSOR_DEFINITIONS = $(inherited) TEST_MODE=1
ONLY_ACTIVE_ARCH = NO

.
.
.

** BUILD SUCCEEDED **

[keithyipkw]

instruments rejected my binary because my first device was somehow my Mackbook. Specifying a valid device made instruments to run the test.

This may help other users if bwoken can display some error messages or the output of instruments is not missing. I am not sure if this is the design or a bug when using bwoken with Xcode 6.

[reallyseth]

I am having the same issue. How do you fix if you want to test in the Simulator?

[lawicko]

The same problem here, was able to workaround it by specifying the device. What you need to do is to look up the available devices by doing:
instruments -s
this will print the list of available devices similar to this:

Known Devices:
Jakub’s MacBook Pro [FFBE27F6-3076-59B7-B14F-40E92F1366FD]
iPad (7.1.2) [07056cc29131036c74d215ba52a9be605dc9e64a]
Resizable iPad (8.1 Simulator) [26038DBD-791C-421E-99F4-9153CA726A2F]
Resizable iPhone (8.1 Simulator) [31B2F0AE-3547-4189-A561-CD2088F6C645]
iPad 2 (8.1 Simulator) [683233C4-EC2B-48A3-826B-10EF62A875CD]
iPad Air (8.1 Simulator) [6358A6F5-2FD0-4377-BD32-0A2C3329276D]
iPad Retina (8.1 Simulator) [224533ED-94DA-46CC-B1DB-1781A1C80710]
iPhone 4s (8.1 Simulator) [D114BC7E-A913-4063-A349-C119BAFC06DA]
iPhone 5 (8.1 Simulator) [7AF6D6F4-C6BC-4A47-B83C-3A4B43ABE0DD]
iPhone 5s (8.1 Simulator) [BE82D607-466A-43E5-863D-6A05F217C117]
iPhone 6 (8.1 Simulator) [D183FF05-3023-4FB5-BEA0-290EA881040A]
iPhone 6 Plus (8.1 Simulator) [0AFCC9AB-7C59-48C7-9BFB-FBF4865B7A63]

Note the MacBook Pro being the first entry, which allegedly is the root problem here. Now, you can work around it by calling:
bwoken test --device D183FF05-3023-4FB5-BEA0-290EA881040A

[reallyseth]

Ah yeah, that did it. I'm now able to run tests on a particular device or simulator.

So what's the way to fix this? Would it be best to detect if the system is Yosemite, then ignore the first device? Or would it be better to pass a device ID to ignore in through the command line or through some config? I took a peek at the code, but I'm not a Ruby dev, so I'm not exactly sure how to dig into this. But I'd be up for helping if I can.

[reallyseth]

This works okay with the simulator, but I can't get it to work on a particular device. Is there any tricks to get that working, or is it simulator only for us on Yosemite? Here's what I'm running:

bwoken test --device f961455171b4a5e1b9366544583fxxxxxxxxxxx --verbose --skip-build

[keithyipkw]

Try using instruments directly with verbose to get some error logs. It is not difficult if you follow the instruction here http://blog.manbolo.com/2013/04/09/launching-uiautomation-tests-in-command-line
This is how I figured out the problem.

[listrophy]

Hey all,

I'm on the cusp of figuring this out. I got bwoken to work with Yosemite + Xcode 6 with the simulator, but it just sits there with the app running when on-device. Once I hammer that out, I'll get a release out that works.

If anyone out there can find the correct incantation of instruments to run an app+tests on a device, I'd love to see it!

[ryanjm]

I'm trying to figure this out myself. When I run the test it just sits there and the simulator doesn't launch the app. Am I missing a step?

bwoken test --focus SelectUserTests --simulator --device "AB766BBC-B5B7-428B-B79F-501F0A1F9BE1"
Building...
...
Build Successful!

iphone  SelectUserTests.js

SelectUserTests is just:

UIALogger.logStart("Logging element tree ...");
UIATarget.localTarget().logElementTree();
UIALogger.logPass();

[listrophy]

@ryanjm I mean using instruments itself from the command line, like:

instruments -w "iPhone 5s (8.1 Simulator)" -t Automation build/iphonesimulator/BwokenTestApp.app -D integration/tmp/trace -e UIASCRIPT integration/tmp/javascript/iphone/    example.js -e UIARESULTSPATH integration/tmp/results

The above line works for the simulator, but I can't figure out the correct line to run tests on a device. I can get the app to launch with some different params, but the tests won't run. 😢

[listrophy]

By the way, among other things like transpiling CoffeeScript to JavaScript, bwoken just creates the appropriate string to run instruments from the command line. Once we figure out what string that is, then I can update bwoken to generate that string for you.

[ryanjm]

That makes sense. I'll try to catch up to where you are in the problem and then see if I can help you figure it out. Right now when I run the line directly I'm getting WebKit Threading Violation - initial use of WebKit from a secondary thread. which I saw someone else had an issue with that (#85).

[ryanjm]

@listrophy You do have the "Enable UI Automation" turned on, right? (Settings > Developer). via appium

I do have it turned on with my 4s (my 6 should come in tomorrow) and I'm getting WebKit Threading Violation - initial use of WebKit from a secondary thread. It will launch the app and just stay open which sounds like what you were running into.

I did get the simulator to work though so that it is a start, thanks for the help. I'll see if I can figure out how to run the test on the device.

[ryanjm]

I haven't figured out much yet. The docs say not to use the path for the app name (when targeting a device) but I haven't noticed any difference with that.

targetAppName
The local name of the application. When targeting a device, omit the pathname and .app extension. When targeting a simulator, use the full pathname.
(via developer.apple)

I tried doing a "customized Automation template" but that ended with the same results also.

It looks like it doesn't matter if you use the name or the device id. ... still poking away at this.

[ryanjm]

I was testing with my iPhone 4s yesterday. It is running iOS 8.1.1 so I thought it wouldn't be any different, but it seems to be. The following does run my the test on my iPad Air 2.

instruments -t Automation -D integration/tmp/trace2 -w "Ryan's iPad (8.1.1)" MyAppName -e UIASCRIPT integration/tmp/javascript/iphone/SelectUserTests.js -e UIARESULTSPATH integration/tmp/results

I'm still getting WebKit Threading Violation - initial use of WebKit from a secondary thread. but at least the test runs.

Update: looks like order of arguments matters for it to use the right trace file. I updated the code block above to use the right order.

[listrophy]

@ryanjm Awesome! That gets it working.

Now, for the bad news: we need to find a way to push the app onto the device. This invocation of instruments doesn't allow for a way to upload a new version to the device, since it only uses the app name, not the path to the binary.

[reallyseth]

I used a combination of the xcodebuild command and the ios-deploy command to automate the building and deploying of an updated app to a test device. It looks something like this:

xcodebuild -workspace "/path/to/workspace.xcworkspace" -scheme MySchemeName -sdk "iphoneos8.1" -configuration Debug SYMROOT="/output/path/for/build" DSTROOT="/output/path/for/build"

ios-deploy --justlaunch --bundle "output/path/for/build/myapp.app" --id device-UUID

After that, you can run tests on the testing device with the instruments command.

[ryanjm]

@listrophy Thanks for the clarification, sorry I missed that step.

@reallyseth thanks for sharing those steps. I'll try it out on Monday.

[kdawgwilk]

Has anyone found a fix to use the bwoken command?

[ryanjm]

@kdawgwilk, if I build the app within XCode so it gets loaded onto a simulator or device I can then run the bwoken command with the --device option and that works for me.

[kdawgwilk]

I got it running now on the simulator but after the app is installed. Has a solution been integrated into bwoken to combine the steps found above? I have forked bwoken and started to pick it apart to understand more of what is going on but I'm still far from being able to contribute.