Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing maps to disk #181

Closed
simon-tarr opened this issue Apr 23, 2019 · 18 comments · Fixed by #182 or #187
Closed

Writing maps to disk #181

simon-tarr opened this issue Apr 23, 2019 · 18 comments · Fixed by #182 or #187

Comments

@simon-tarr
Copy link

Hello,

Very new to Julia and Circuitscape.jl, so please forgive a very basic question. I'm wondering how I can write raster images of the outputs? I've opened one of the example .ini files in input/raster/all_to_one/2/allToOneVerify2.ini and there are various flags within the output section which are all set to 'false'. I've tried to set some of these flags to 'true' and I either get no outputs from running the .ini file or I get an error of: ERROR: SystemError: opening file "C:/Users/simon_curmap_1.asc": Permission denied

I'm not sure where the simon_curmap_1.asc bit comes from as nowhere in the .ini file have I specified the path C:/users/ or a file name of simon_curmap_1.asc (In fact, the .ini file is exactly as cloned from Github, except I've toggled a few flags i.e. none of the file paths have been changed).

Is there a full tutorial online using circuitscape.jl that I could perhaps follow? I am aware of the rather comprehensive documentation at https://circuitscape.org/docs/ for the GUI version 4.0 (and some help regarding Python) but there doesn't appear to be any documentation on using Circuitscape in Julia.

Thanks in advance for your help.

@ranjanan
Copy link
Member

Can you paste the contents of the INI file here please?

@simon-tarr
Copy link
Author

simon-tarr commented Apr 24, 2019

Hi @ranjanan, thanks for getting back to me. The contents of the INI file are the same as in input/raster/all_to_one/2/allToOneVerify2.ini except that I've pointed explicitly to the input/output directories and files (rather than use the relative path of input/raster/all_to_one/2/). I have nonetheless copied the INI file below.

I have flagged 'write_volts_maps' to True.

[Options for advanced mode]
ground_file_is_resistances = False
source_file = (Browse for a current source file)
remove_src_or_gnd = keepall
ground_file = (Browse for a ground point file)
use_unit_currents = False
use_direct_grounds = False

[Calculation options]
low_memory_mode = False
solver = cg+amg
print_timings = False

[Options for pairwise and one-to-all and all-to-one modes]
included_pairs_file = None
use_included_pairs = False
point_file = C:/Users/simon.tarr/Documents/Circuitscape.jl/test/input/raster/one_to_all/2/points.asc

[Output options]
write_cum_cur_map_only = False
log_transform_maps = False
output_file = C:/Users/simon.tarr/Documents/Circuitscape.jl/test/output/allToOneVerify2.asc
write_max_cur_maps = False
write_volt_maps = True
set_null_currents_to_nodata = False
set_null_voltages_to_nodata = False
compress_grids = False
write_cur_maps = False

[Short circuit regions (aka polygons)]
use_polygons = True
polygon_file = C:/Users/simon.tarr/Documents/Circuitscape.jl/test/input/raster/one_to_all/2/polygons.asc

[Connection scheme for raster habitat data]
connect_four_neighbors_only = True
connect_using_avg_resistances = True

[Habitat raster or graph]
habitat_file = C:/Users/simon.tarr/Documents/Circuitscape.jl/test/input/raster/one_to_all/2/cellmap.asc
habitat_map_is_resistances = False

[Options for one-to-all and all-to-one modes]
use_variable_source_strengths = False
variable_source_file = None

[Version]
version = unknown

[Mask file]
use_mask = False
mask_file = None

[Circuitscape mode]
data_type = raster
scenario = all-to-one

I've noticed that the file paths input/raster/all_to_one/2/ are /one_to_all/ whereas the last line of the INI file says scenario = all-to-one. This is true of the current commit on the master branch of this repo. Could this be causing any issues?

@ranjanan
Copy link
Member

No the scenario is totally fine. Could you the paste the full stack trace of the error you get when you run compute(file)?

@simon-tarr
Copy link
Author

As requested:

julia> using Circuitscape

julia> compute("C:/users/simon.tarr/Documents/Circuitscape.jl/test/input/raster/all_to_one/2/allToOneVerify2.ini")
[ Info: 2019-04-25 08:04:58 : Precision used: Double
[ Info: 2019-04-25 08:05:01 : Reading maps
[ Info: 2019-04-25 08:05:01 : Resistance/Conductance map has 7 nodes
[ Info: 2019-04-25 08:05:03 : There are 6 points and 1 connected components
[ Info: 2019-04-25 08:05:05 : Solving point 1 of 5
[ Info: 2019-04-25 08:05:07 : Time taken to construct preconditioner = 0.7645297 seconds
[ Info: 2019-04-25 08:05:07 : Time taken to solve linear system = 0.3120092 seconds
ERROR: SystemError: opening file "C:/Users/simon_voltmap_1.asc": Permission denied
Stacktrace:
 [1] (::getfield(Base, Symbol("##696#698")))(::Task) at .\asyncmap.jl:178
 [2] foreach(::getfield(Base, Symbol("##696#698")), ::Array{Any,1}) at .\abstractarray.jl:1866
 [3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::UnitRange{Int64}) at .\asyncmap.jl:178
 [4] #async_usemap#681 at .\asyncmap.jl:154 [inlined]
 [5] #async_usemap at .\none:0 [inlined]
 [6] #asyncmap#680 at .\asyncmap.jl:81 [inlined]
 [7] #asyncmap at .\none:0 [inlined]
 [8] #pmap#215(::Bool, ::Int64, ::Nothing, ::Array{Any,1}, ::Nothing, ::Function, ::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:126
 [9] pmap(::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:101
 [10] #pmap#225(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [11] pmap(::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [12] onetoall_kernel(::Circuitscape.RasData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\raster\onetoall.jl:136
 [13] raster_one_to_all(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\raster\onetoall.jl:10
 [14] _compute(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\run.jl:0
 [15] macro expansion at .\util.jl:213 [inlined]
 [16] compute(::String) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\run.jl:31
 [17] top-level scope at none:0

@simon-tarr
Copy link
Author

Here is some extra information which may help with troubleshooting. Using Circuitscape v4.0 (GUI) and the toy raster files which you can download from this gist, I can generate a sensible looking current flow output between two nodes. Here's a screen grab of the v4.0 settings I'm using:

Capture

I get the following output when plotting the current output in R:

Capture

If I run the same analysis using Circuitscape v5.5.0 GUI (as downloaded here) I get a ton of error messages within the Julia shell. Here's a screen grab of the v5.5.0 GUI with the settings I'm using:

Capture

I do not get any current output files in the output folder that I have specified. I do get a file (which I have named output.out) which seems to show the input settings for me (I guess this is just a general log of the initial settings):

[Circuitscape Mode]
data_type = Raster
scenario = One To All

[Version]
version = 5.0.0

[Habitat raster or graph]
habitat_file = C:\Users\simon.tarr\Desktop\test_data\resistance.asc
habitat_map_is_resistances = false

[Connection Scheme for raster habitat data]
connect_four_neighbors_only = false
connect_using_avg_resistances = false

[Short circuit regions (aka polygons)]
use_polygons = false
polygon_file = False

[Options for advanced mode]
ground_file_is_resistances = true
source_file = 
remove_src_or_gnd = keepall
ground_file = 
use_unit_currents = false
use_direct_grounds = false

[Mask file]
use_mask = false
mask_file = None

[Options for one-to-all and all-to-one modes]
use_variable_source_strengths = false
variable_source_file = None

[Options for pairwise and one-to-all and all-to-one modes]
included_pairs_file = (Browse for a file with pairs to include or exclude)
use_included_pairs = false
point_file = C:\Users\simon.tarr\Desktop\test_data\focal_nodes.asc

[Calculation options]
solver = cg+amg

[Output options]
write_cum_cur_map_only = false
log_transform_maps = false
output_file = c:/users/simon.tarr/desktop/test_data/output.out
write_max_cur_maps = false
write_volt_maps = false
set_null_currents_to_nodata = false
set_null_voltages_to_nodata = false
compress_grids = false
write_cur_maps = true

This is the mess from the Julia shell which runs in the background:

┌ Warning: Error requiring IteratorInterfaceExtensions from Tables:
│ cannot assign variable IteratorInterfaceExtensions.IteratorInterfaceExtensions from module Tables
│ Stacktrace:
│  [1] top-level scope at none:0
│  [2] eval at .\boot.jl:319 [inlined]
│  [3] eval at C:\Users\ranjan\.julia\packages\Tables\zyTIO\src\Tables.jl:1 [inlined]
│  [4] (::getfield(Tables, Symbol("##12#24")))() at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:67
│  [5] err(::getfield(Tables, Symbol("##12#24")), ::Module, ::String) at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:38
│  [6] #11 at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:66 [inlined]
│  [7] withpath(::getfield(Tables, Symbol("##11#23")), ::String) at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:28
│  [8] #10 at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:65 [inlined]
│  [9] listenpkg(::getfield(Tables, Symbol("##10#22")), ::Base.PkgId) at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:13
│  [10] macro expansion at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:64 [inlined]
│  [11] macro expansion at C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:63 [inlined]
│  [12] __init__() at C:\Users\ranjan\.julia\packages\Tables\zyTIO\src\Tables.jl:10
└ @ Requires C:\Users\ranjan\.julia\packages\Requires\9Jse8\src\require.jl:40
pwd() = "D:\\Circuitscape\\Circuitscape\\Circuitscape\\core"
((CircuitscapeUI.Blink).AtomShell)._electron = "..\\lib\\atom\\electron.exe"
((CircuitscapeUI.Blink).AtomShell).mainjs = "..\\res\\main.js"
(CircuitscapeUI.Blink).mainhtml = "..\\res\\main.html"
(CircuitscapeUI.WebIO).bundlepath = "..\\res\\bundle.js"
(CircuitscapeUI.InteractBulma).bulma_tooltip_min_css = "..\\res\\bulma-tooltip.min.css"
(CircuitscapeUI.InteractBulma).bulma_accordion_min_css = "..\\res\\bulma-accordion.min.css"
(CircuitscapeUI.Blink).resources = Dict("blink.js"=>"..\\res\\res\\blink.js","blink.css"=>"..\\res\\res\\blink.css","spinner.css"=>"..\\res\\res\\spinner.css","reset.css"=>"..\\res\\res\\reset.css")
(CircuitscapeUI.Blink).webiosetup = "..\\res\\webio_setup.js"
v = "Raster"
v = "Raster"
v = "Raster"
v = "Raster"
(CircuitscapeUI.AssetRegistry).registry = Dict("/assetserver/e5c73d9d77baa676fffc5eaf5fa68064ceb681ed-bulma-accordion.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma-accordion.min.css","/assetserver/a049b91595583871e8a34da37235d28468c83fd4-tachyons.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\tachyons.min.css","/assetserver/ed9cad2b38a0f1a7231f0c8535a6b163cbb6216d-bulma-checkradio.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma-checkradio.min.css","/assetserver/f3e62c757b7fed89accbca281e7f00bf4b627712-knockout_punches.js"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\knockout_punches.js","/assetserver/e9d48936757fa817eeb83fa85955fa6d5961d3ea-knockout.js"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\knockout.js","/assetserver/89189390bbf07164f8cd1a5d4dd95d18805ec9a3-bundle.js"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bundle.js","/assetserver/5fea10939a1f4cca16581028886279e594443751-cs_logo.ico"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\cs_logo.ico","/assetserver/64ba6458dbc337ea1b749a573c9d4ddbd767de41-bulma-tooltip.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma-tooltip.min.css","/assetserver/ece303ccfa3d9f5fe31d903b06cc6f9b022f0a21-bulma-slider.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma-slider.min.css","/assetserver/e5dd56fe725f77df9c8421802573106e96fed58c-style.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\style.css","/assetserver/7377fd620d0a03fd39ef30437620d6349daa3781-webio_setup.js"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\webio_setup.js","/assetserver/0e4f7f7290084c7666e4da7e64f3c60db18541a4-bulma.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma.min.css","/assetserver/fb0107b715224a01dc39eb577edfec32606d168d-bulma-switch.min.css"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\bulma-switch.min.css","/assetserver/6ca5fbe89562198acdb9c3f2eff5b6c023457343-all.js"=>"D:\\Circuitscape\\Circuitscape\\Circuitscape\\res\\all.js")
filepath changed to C:\Users\simon.tarr\Desktop\test_data\resistance.asc
data_type = "Raster"
scenario = "One To All"
input_graph[] = "C:\\Users\\simon.tarr\\Desktop\\test_data\\resistance.asc"
is_res[] = false
focal[] = "C:\\Users\\simon.tarr\\Desktop\\test_data\\focal_nodes.asc"
source[] = ""
ground[] = ""
out[] = "c:/users/simon.tarr/desktop/test_data/output.out"
write_cur_maps[] = true
write_volt_maps[] = false
num_parallel[] = 1
JSExpr: Unsupported `string` expression, "$(e)"JSExpr: Unsupported `string` expression, "$(e)"

If I run the same analysis using the Julia command line compute("path/to/file.ini") I get the following output (which is very similar-- if not the same) as the stack that I posted in my previous comment:

julia> compute("c:/users/simon.tarr/desktop/test_data/test.ini")
[ Info: 2019-04-25 13:40:11 : Precision used: Double
[ Info: 2019-04-25 13:40:14 : Reading maps
[ Info: 2019-04-25 13:40:15 : Resistance/Conductance map has 83 nodes
[ Info: 2019-04-25 13:40:17 : There are 73 points and 1 connected components
[ Info: 2019-04-25 13:40:18 : Solving point 1 of 2
[ Info: 2019-04-25 13:40:20 : Time taken to construct preconditioner = 0.7303418 seconds
[ Info: 2019-04-25 13:40:21 : Time taken to solve linear system = 0.3168362 seconds
ERROR: SystemError: opening file "C:/Users/simon_voltmap_1.asc": Permission denied
Stacktrace:
 [1] (::getfield(Base, Symbol("##696#698")))(::Task) at .\asyncmap.jl:178
 [2] foreach(::getfield(Base, Symbol("##696#698")), ::Array{Any,1}) at .\abstractarray.jl:1866
 [3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::UnitRange{Int64}) at .\asyncmap.jl:178
 [4] #async_usemap#681 at .\asyncmap.jl:154 [inlined]
 [5] #async_usemap at .\none:0 [inlined]
 [6] #asyncmap#680 at .\asyncmap.jl:81 [inlined]
 [7] #asyncmap at .\none:0 [inlined]
 [8] #pmap#215(::Bool, ::Int64, ::Nothing, ::Array{Any,1}, ::Nothing, ::Function, ::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:126
 [9] pmap(::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:101
 [10] #pmap#225(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [11] pmap(::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [12] onetoall_kernel(::Circuitscape.RasData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\raster\onetoall.jl:136
 [13] raster_one_to_all(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\raster\onetoall.jl:10
 [14] _compute(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\run.jl:0
 [15] macro expansion at .\util.jl:213 [inlined]
 [16] compute(::String) at C:\Users\simon.tarr\.julia\packages\Circuitscape\yuRbe\src\run.jl:31
 [17] top-level scope at none:0

My biggest concern in that output is the program's insistence in reading and writing to C:/Users/ as it generates this error: ERROR: SystemError: opening file "C:/Users/simon_voltmap_1.asc": Permission denied. I am working on a work computer so it's quite possible that the code is trying to dump temporary/output files into C:/Users for which I have no permissions.

@ViralBShah
Copy link
Member

Thank you for the detailed bug report!

@ranjanan
Copy link
Member

Yeah, I found the bug. It's messing up with the . in your username thinking it's an extension. I've put up a fix here.

@ranjanan
Copy link
Member

@simon-tarr I've fixed this via the PR and tagged a new release. I'll ping you here once the release is registered, and then you'll be able to use it.

@simon-tarr
Copy link
Author

@ranjanan that's great thank you very much for the speedy fix.

@ranjanan
Copy link
Member

@simon-tarr the new version just got registered. Could you do using Pkg; Pkg.update() at your Julia prompt and see if your Circuitscape updates to v5.5.2? If it does, restart Julia, do using Circuitscape and compute() as you usually would do.

@simon-tarr
Copy link
Author

Hi @ranjanan I've updated to v.5.5.2 and restarted Julia, as requested. I've hit a new SystemError: opening file message. Previously the error concerned voltmap (ERROR: SystemError: opening file "C:/Users/simon_voltmap_1.asc": Permission denied) but now the error is resistances.out. I've copied the full stack trace below.

compute("C:/users/simon.tarr/desktop/test_data/input.ini")
[ Info: 2019-04-29 08:28:33 : Precision used: Double
[ Info: 2019-04-29 08:28:36 : Reading maps
[ Info: 2019-04-29 08:28:36 : Resistance/Conductance map has 4 nodes
[ Info: 2019-04-29 08:28:39 : Solver used: AMG accelerated by CG
[ Info: 2019-04-29 08:28:39 : Graph has 1 nodes, 4 focal points and 1 connected components
[ Info: 2019-04-29 08:28:39 : Total number of pair solves = 0
[ Info: 2019-04-29 08:28:40 : Time taken to construct preconditioner = 0.7294516 seconds
[ Info: 2019-04-29 08:28:40 : Time taken to construct local nodemap = 0.157624999 seconds
ERROR: SystemError: opening file "c:/users/simon_resistances.out": Permission denied
Stacktrace:
 [1] #systemerror#43(::Nothing, ::Function, ::String, ::Bool) at .\error.jl:134
 [2] systemerror at .\error.jl:134 [inlined]
 [3] #open#309(::Nothing, ::Nothing, ::Nothing, ::Bool, ::Nothing, ::Function, ::String) at .\iostream.jl:283
 [4] #open at .\none:0 [inlined]
 [5] open(::String, ::String) at .\iostream.jl:339
 [6] #open#310(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(Circuitscape, Symbol("##82#84")){Array{Float64,2}}, ::String, ::Vararg{String,N} where N) at .\iostream.jl:367
 [7] open at .\iostream.jl:367 [inlined]
 [8] save_resistances(::Array{Float64,2}, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\out.jl:460
 [9] amg_solver_path(::Circuitscape.GraphData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}, ::Bool) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\core.jl:246
 [10] single_ground_all_pairs(::Circuitscape.GraphData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}, ::Bool) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\core.jl:55
 [11] single_ground_all_pairs at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\core.jl:53 [inlined]
 [12] _pt_file_no_polygons_path(::Circuitscape.RasData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\raster\pairwise.jl:61
 [13] raster_pairwise(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\raster\pairwise.jl:29
 [14] _compute(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\run.jl:42
 [15] macro expansion at .\util.jl:213 [inlined]
 [16] compute(::String) at C:\Users\simon.tarr\.julia\packages\Circuitscape\2LiH5\src\run.jl:31
 [17] top-level scope at none:0

@simon-tarr
Copy link
Author

Hello @ranjanan I was wondering if there was any update to this? I appreciate your assistance thus far and many thanks in advance.

@ranjanan
Copy link
Member

ranjanan commented May 3, 2019

Hi @simon-tarr, sorry about the new error you're facing. I didn't fix all instances of that bug in the code. I have a couple of urgent thiings to do right now, and will get to this in the weekend. Thank you for your patience thus far!

@simon-tarr
Copy link
Author

Hi @ranjanan no worries. As this issue is set as closed, I was just making sure that it hadn't been forgotten about. Looking forward to the fix so that I can explore the model.

@simon-tarr
Copy link
Author

simon-tarr commented May 13, 2019

Hi @ranjanan I'm getting a couple of new errors now, I'm afraid. The first error occurred reasonably quickly after submitting the job:

ERROR: On worker 5:
BoundsError: attempt to access 0-element Array{SharedArrays.SharedArray{Float64,2},1} at index [4]
getindex at .\array.jl:729 [inlined]
maybeview at .\views.jl:125 [inlined]
dotview at .\broadcast.jl:1067 [inlined]
write_cur_maps at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\out.jl:81
macro expansion at .\util.jl:213 [inlined]
postprocess at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\core.jl:596
f at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\core.jl:206
#89 at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\core.jl:221
#112 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:269
run_work_thunk at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:56
macro expansion at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:269 [inlined]
#111 at .\task.jl:259
Stacktrace:
 [1] (::getfield(Base, Symbol("##696#698")))(::Task) at .\asyncmap.jl:178
 [2] foreach(::getfield(Base, Symbol("##696#698")), ::Array{Any,1}) at .\abstractarray.jl:1866
 [3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::UnitRange{Int64}) at .\asyncmap.jl:178
 [4] #async_usemap#681 at .\asyncmap.jl:154 [inlined]
 [5] #async_usemap at .\none:0 [inlined]
 [6] #asyncmap#680 at .\asyncmap.jl:81 [inlined]
 [7] #asyncmap at .\none:0 [inlined]
 [8] #pmap#215(::Bool, ::Int64, ::Nothing, ::Array{Any,1}, ::Nothing, ::Function, ::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:126
 [9] pmap(::Function, ::Distributed.WorkerPool, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:101
 [10] #pmap#225(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [11] pmap(::Function, ::UnitRange{Int64}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Distributed\src\pmap.jl:156
 [12] amg_solver_path(::Circuitscape.GraphData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}, ::Bool) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\core.jl:221
 [13] single_ground_all_pairs(::Circuitscape.GraphData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}, ::Bool) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\core.jl:55
 [14] _pt_file_polygons_path(::Circuitscape.RasData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\raster\pairwise.jl:105
 [15] raster_pairwise(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\raster\pairwise.jl:27
 [16] _compute(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\run.jl:42
 [17] macro expansion at .\util.jl:213 [inlined]
 [18] compute(::String) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\run.jl:31
 [19] top-level scope at none:0

The second error occurred much later in the analysis (after I re-ran it with the same .ini file and rasters) and the stack track is much shorter:

...
[ Info: 2019-05-13 11:40:59 : Solving pair 3160 of 3160
[ Info: 2019-05-13 11:40:59 : Solver used: AMG accelerated by CG
[ Info: 2019-05-13 11:40:59 : Graph has 605 nodes, 2 focal points and 81 connected components
[ Info: 2019-05-13 11:40:59 : Time taken to construct preconditioner = 2.6899e-5 seconds
[ Info: 2019-05-13 11:40:59 : Time taken to construct local nodemap = 0.0001241 seconds
[ Info: 2019-05-13 11:40:59 : Time taken to construct preconditioner = 5.3e-6 seconds
[ Info: 2019-05-13 11:40:59 : Time taken to construct local nodemap = 0.0001246 seconds

ERROR: BoundsError: attempt to access 0-element Array{SharedArrays.SharedArray{Float64,2},1} at index [1]
Stacktrace:
 [1] getindex at .\array.jl:729 [inlined]
 [2] write_cum_maps(::Circuitscape.Cumulative{Float64}, ::Array{Float64,2}, ::Dict{String,String}, ::Circuitscape.RasterMeta, ::Bool, ::Bool) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\out.jl:482
 [3] _pt_file_polygons_path(::Circuitscape.RasData{Float64,Int32}, ::Circuitscape.RasterFlags, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\raster\pairwise.jl:115
 [4] raster_pairwise(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\raster\pairwise.jl:27
 [5] _compute(::Type, ::Type, ::Dict{String,String}) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\run.jl:42
 [6] macro expansion at .\util.jl:213 [inlined]
 [7] compute(::String) at C:\Users\simon.tarr\.julia\packages\Circuitscape\x2r8m\src\run.jl:31
 [8] top-level scope at none:0

This second error appears to be occurring immediately after solving the final pair in my example. Any ideas what may be causing this? I'm running a pairwise raster analysis which worked using Circuitscape v4.0 (with the same resistance and focal point files).

Many thanks
Simon

@ranjanan
Copy link
Member

Hi @simon-tarr I tagged another release v5.5.3 recently. Can you update Circuitscape and check if this particular issue is resolved?

@ranjanan
Copy link
Member

For the new errors, could you please open a new issue? My semester is over at the end of this week and I'll make sure you have a Circuitscape that works properly.

@simon-tarr
Copy link
Author

@ranjanan opened a new issue as requested #188
Thanks for your help thus far in getting the model back up and running again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants