Consume part of SUMT?

[thomthom]

SketchUp/api-issue-tracker#61

[MSP-Greg]

First off, please don't consider SUMT to be equivalent to "I don't like testup-2". I started by revising some of the tests using testup, and, given my background with testing, I just wanted to click in the console, hit the up arrow, and repeat the tests. So, rather than figure out all of testup, I just started from scratch,

It may not be trivial, but I'm willing to work on it (combining the two). As an aside, I'd appreciate if you checked the temp file issue, and whether the UDP is working for you (and on MacOS if you can).

I might also suggest that someone from Trimble could run SUMT.run r:20 (twenty runs of the test suite), and check whether it completes and what the memory/resource status is at the end of 20 runs...

Finally, as to windows, I assume you have some stand-alone Ruby versions. Maybe you should start running trunk? Might as well get familiar with MSYS2, given that's it the future of Windows Ruby.

95% of the work I've done across the whole Ruby eco-system I've done with trunk. The doc site has always been updated using trunk. Etc, etc...

Never asked, are you more a windows type or a MacOS type?

[thomthom]

I run standalone Ruby from one of the Windows Installer builds. The version vary from computer to computer. I often have had matching standalone version to the versions used in Ruby - in order to build gems etc. And also to check if issues exist only in SU Ruby or also in standalone Ruby.

When we get to upgrading Ruby in SU we will have to get familiar with MSYS2.

I'm mainly a Windows guy. I boot into my mac partition only briefly when needed.

[MSP-Greg]

Re Ruby & Gems (as long as you mentioned them), I've got a lot of Ruby versions on my drive(s). From 1.9.3 forward, and hundreds of trunk builds (most are zipped). Anytime I use one, I add a text file to the base directory with the PATH for the build. Ruby-loco trunk has been build over 500 times on Appveyor...

Re Gems, since consecutive SU versions often have the same Ruby version, being able to install gems with the '--user-install' option is helpful, since one doesn't need the same gem installed in two places for use with '17 & '18. I'll get my gem stuff together and start a repo with SU stuff, a good readme or wiki, and explain how to set up SU & RubyGems so --user-install gems can be used. I'll also explain how to update RubyGems, as updates are often security related. FYI, RubyGems includes its own certs (*.pem) for its SSL/TLS connections.

Back to merging. Could we change testup-2 to be more 'load-on-demand'? I set up SUMT that way, and I think it would appreciated by devs, as testing code/plugins are certainly helpful to have 'installed', but they shouldn't load until they're used... Haven't given it that much thought, busy with some 'Ruby world' stuff.

One thing that might work, have one namespace for test related things, one namespace for 'GUI', and one namespace for 'Console'. If that will work, then we/you just need to determine how to merge the test related differences...

[thomthom]

Back to merging. Could we change testup-2 to be more 'load-on-demand'?

Can you elaborate a bit on the load-on-demand nature of SUMT?

Btw, I'll be doing some heavy changes to TestUp the next couple of weeks. You probably want to avoid doing big changes right now - otherwise troublesome conflicts will probably occur.

One thing that might work, have one namespace for test related things, one namespace for 'GUI', and one namespace for 'Console'. If that will work, then we/you just need to determine how to merge the test related differences...

Yea, cleaning up the source of TestUp is also on the plate. Expect to see things moving around. TestUp2 was created in 2013 in somewhat of a rush to create a new test runner compatible with Ruby 2.0. Since then more stuff have been thrown at it and it's time to pay back some of the code depth.
Things will be split into smaller items and more separation of responsibility.

[MSP-Greg]

Can you elaborate a bit on the load-on-demand nature of SUMT?

I've got both TestUp & SUMT set up as extensions. Before performing any operations, below is a simple inspection of their namespaces (edited for width). More items (loaded classes/modules) would also appear in the Minitest namespace.

TestUp.constants.sort
[:AppFiles, :BASE_CONSOLE_CLASS, :Console, :Coverage, :DEBUG, :Debugger, :Editor, :FILENAMESPACE,
:PATH, :PATH_IMAGES, :PATH_JS_SCRIPTS, :PATH_ROOT, :PLUGIN_ID, :PLUGIN_NAME, :PLUGIN_VERSION, 
:Perforce, :PreferencesWindow, :Runs, :SKUI, :ScriptDebugger, :Settings, :SystemFiles, :SystemUI,
:TESTUP_CONSOLE, :TaskbarProgress, :TestDiscoverer, :TestUpWindow, :Win32Helper]

SUMT.constants.sort
[:PATH, :PLUGIN_ID, :PLUGIN_NAME, :PLUGIN_VERSION]

Or, from $LOADED_FEATURES:

E:/GitHub/SUMT/lib/sumt.rb

E:/GitHub/testup-2/src/testup.rb
E:/GitHub/testup-2/src/testup/app_files.rb
E:/GitHub/testup-2/src/testup/console.rb
E:/GitHub/testup-2/src/testup/core.rb
E:/GitHub/testup-2/src/testup/coverage.rb
E:/GitHub/testup-2/src/testup/debug.rb
E:/GitHub/testup-2/src/testup/editor.rb
E:/GitHub/testup-2/src/testup/lib/ruby230/x64/TaskbarProgress.so
E:/GitHub/testup-2/src/testup/p4.rb
E:/GitHub/testup-2/src/testup/preferences_window.rb
E:/GitHub/testup-2/src/testup/runs.rb
E:/GitHub/testup-2/src/testup/settings.rb
E:/GitHub/testup-2/src/testup/system_files.rb
E:/GitHub/testup-2/src/testup/taskbar_progress.rb
E:/GitHub/testup-2/src/testup/test_discoverer.rb
E:/GitHub/testup-2/src/testup/test_window.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/base.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/bridge.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/button.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/checkbox.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/container.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/control.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/control_manager.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/core.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/debug.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/embed_skui.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/enum.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/enum_system_color.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/enum_system_font.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/events.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/font.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/groupbox.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/image.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/json.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/label.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/listbox.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/properties.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/radiobutton.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/rect.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/textbox.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/typecheck.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/version.rb
E:/GitHub/testup-2/src/testup/third-party/SKUI/src/SKUI/window.rb
E:/GitHub/testup-2/src/testup/ui.rb
E:/GitHub/testup-2/src/testup/win32.rb

[thomthom]

Gotcha.

That's actually something I've been thinking of in general for extensions. Some pattern to load extensions on-demand to keep SU startup times low.

[MSP-Greg]

Another thing I'd like to see encouraged (and I'd don't recall what testup-2 does) is not adding to $LOAD_PATH unless done so in a block of code, whether that be a method, block, etc. Or, any changes to $LOAD_PATH should be temporary. Disk I/O is bad.

[thomthom]

That's a check in rubocop-sketchup that I added: https://github.com/SketchUp/rubocop-sketchup
(All though, not so much for I/O issues, but because it can make extension's conflict if they modify the load path in an attempt to shorten their require statements.)

TestUp makes one modification to $LOAD_PATH, in order to have Minitest pick up TestUp's Minitest-plugin:

testup-2/src/testup/minitest_setup.rb

Lines 61 to 71 in a407d39

	# Configure Ruby such that the TestUp reporter can be found without creating
	# gem for it.
	#
	# MiniTest uses Gem.find_files to look for extensions by globbing
	# { }"minitest/*_plugin.rb". To avoid making a gem that needs installing, make
	# use of the fact that by default Gem.find_files will search in $LOAD_PATH.
	#
	# Verify by checking after `MiniTest.run`:
	# Minitest.extensions
	# > ["pride", "testup"]
	$LOAD_PATH << File.join(__dir__, 'minitest_plugins')

I didn't find any other way to make that work.

[MSP-Greg]

The following methods/code handles it in SUMT:

SUMT.load_reporters

and the code at the end of the reporters:

file_reporter.rb

udp_reporter.rb

The code in MT works for all gems that are plugins, but since we know what we want to load... This also allows loading & unloading.