Syntax error, unable to parse arrays?

[ghaines3]

Summary:

I'm brand new to using stan, so this may be something I'm doing wrong, but I keep getting an error when trying to run the individual Cormack-Jolly-Seber model in the stan user manual. I am able to run stan programs without arrays seemingly without a problem, but am unable to get arrays to parse.

Description:

The error code I get looks like this:

SYNTAX ERROR, MESSAGE(S) FROM PARSER:
 error in 'model7d441a1b2ac8_individual_CJS' at line 7, column 2
  -------------------------------------------------
     5:   int<lower=2> cap;
     6:   int<lower=0> I;
     7:   array[121, 20] int<lower=0, upper=1> y;
         ^
     8: }
  -------------------------------------------------

PARSER EXPECTED: <one of the following:
  a variable declaration, beginning with type,
      (int, real, vector, row_vector, matrix, unit_vector,
       simplex, ordered, positive_ordered,
       corr_matrix, cov_matrix,
       cholesky_corr, cholesky_cov
  or '}' to close variable declarations>
Error in stanc(file = file, model_code = model_code, model_name = model_name,  : 
  failed to parse Stan model 'individual_CJS' due to the above error.

I found some reports of a similar error from a little over a year ago, with someone responding that rstan had not quite kept up with the stan syntax, but haven't seen anything more recent, so I assume it must have been fixed.

Reproducible Steps:

I have just updated R, so I am running R v. 4.3.0, and v. 2.21.8 of rstan on macOS Monterey 12.6. The data inputs weren't very well described in the manual, but from what I have worked out, a reproducible example would be something like this (with the code for the stan file copied directly from the link above):

history.array<-list(y=matrix(c(1,0,0,1,0,0,0,0,1,0,0,1,
         0,0,0,0,1,1,0,1,1,0,0,1,
         1,1,0,0,1,1,0,0,0,0,0,0,
         1,1,0,0,1,0,0,1), nrow=11)%>%apply(2,as.integer),
T = as.integer(4),
I = as.integer(11),
phi = rep(.5,3),
p = rep(.5,4))

test_ind_CJS<-stan(
  file = "individual_CJS.stan",
  chains = 4,
  warmup = 1000,
  data = history.array,
  iter = 2000,
  cores = 4,
  refresh = 250,
  control = list(max_treedepth = 10,
                 adapt_delta = 0.95)
)

I also tried using matrix[,] instead of array[,], but that just gave me a different error.

Would really appreciate some insight as to what is going on here, and whether it's an rstan issue or a me issue.

thanks

[bob-carpenter]

RStan isn't up to date with Stan because of restrictive CRAN policies (hard cap on package sizes, no way to coordinate dependencies among packages). It should work if you install version 2.26 from GitHub following the RStan Getting Started guide:

https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started

Check out this doc from the Stan Reference Manual for more information:

https://mc-stan.org/docs/reference-manual/brackets-array-syntax.html

[ghaines3]

Awesome, thanks very much!

[ghaines3]

Hi, sorry to keep bugging you, but I just tried installing from the repository, and it is returning this error, then defaulting to installing 2.21 from cran:

> install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
also installing the dependency ‘StanHeaders’

Warning in install.packages :
  unable to access index for repository https://mc-stan.org/r-packages/bin/macosx/big-sur-x86_64/contrib/4.3:
  cannot open URL 'https://mc-stan.org/r-packages/bin/macosx/big-sur-x86_64/contrib/4.3/PACKAGES'

It looks like this was noticed the other day with cmdstanr, as well

[bob-carpenter]

Maybe a web server was down? I tried it just now and the index seems to be there. @bgoodri may know or you could ask on the Stan Forums.

~/github/bob-carpenter/stan-getting-started (main)$ R

R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
--- Please select a CRAN mirror for use in this session ---
1
also installing the dependency ‘StanHeaders’

--- Please select a CRAN mirror for use in this session ---
--- Please select a CRAN mirror for use in this session ---
trying URL 'https://mc-stan.org/r-packages/bin/macosx/contrib/4.2/StanHeaders_2.26.22.tgz'
Content type 'application/octet-stream' length 3743256 bytes (3.6 MB)
==================================================
downloaded 3.6 MB

trying URL 'https://mc-stan.org/r-packages/bin/macosx/contrib/4.2/rstan_2.26.22.tgz'
Content type 'application/octet-stream' length 5614107 bytes (5.4 MB)
==================================================
downloaded 5.4 MB


The downloaded binary packages are in
	/var/folders/b3/6h4t41094vz281j7nn48ws900001jv/T//RtmpybDdtq/downloaded_packages


> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.1 tools_4.2.1    tcltk_4.2.1   

[ghaines3]

Huh. I just tried again and it's still doing the same thing for me. Could this be because I am running R 4.3? I have put this up on the stan forum (on a thread describing the initial problem I was having), but nothing yet.

> sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Atlantic/Reykjavik
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] compiler_4.3.0   Matrix_1.5-4     generics_0.1.3   cli_3.6.1        tools_4.3.0     
 [6] rstudioapi_0.14  Rcpp_1.0.10      lubridate_1.9.2  grid_4.3.0       timechange_0.2.0
[11] lattice_0.21-8 

[bob-carpenter]

You may also want to try cmdstanr, which is much easier to install and up to date with Stan.

It indeed looks like an issue with R 4.3 as you suspected. I updated to 4.3 and now get this:

> install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
--- Please select a CRAN mirror for use in this session ---
also installing the dependencies ‘colorspace’, ‘utf8’, ‘backports’, ‘ps’, ‘farver’, ‘labeling’, ‘munsell’, ‘RColorBrewer’, ‘viridisLite’, ‘fansi’, ‘magrittr’, ‘pillar’, ‘pkgconfig’, ‘gtable’, ‘checkmate’, ‘matrixStats’, ‘callr’, ‘cli’, ‘crayon’, ‘desc’, ‘prettyunits’, ‘processx’, ‘R6’, ‘rprojroot’, ‘withr’, ‘jsonlite’, ‘curl’, ‘glue’, ‘isoband’, ‘lifecycle’, ‘rlang’, ‘scales’, ‘tibble’, ‘vctrs’, ‘StanHeaders’, ‘inline’, ‘gridExtra’, ‘Rcpp’, ‘RcppParallel’, ‘loo’, ‘pkgbuild’, ‘V8’, ‘ggplot2’, ‘RcppEigen’, ‘BH’

--- Please select a CRAN mirror for use in this session ---
Warning: unable to access index for repository https://mc-stan.org/r-packages/bin/macosx/big-sur-x86_64/contrib/4.3:
  cannot open URL 'https://mc-stan.org/r-packages/bin/macosx/big-sur-x86_64/contrib/4.3/PACKAGES'

Here's the session info:

> sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.3.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.0 tools_4.3.0    tcltk_4.3.0   

[ghaines3]

Thanks for your help, I just did (got the same advice from the stan forum a minute ago), and that seems to have worked.

[WardBrian]

There is an open issue for creating R 4.3 binaries stan-dev/r-packages#10

I'm not sure who is usually managing that repository. Seems like it is often @jgabry or @rok-cesnovar

[jgabry]

Closing now that the new array syntax is finally available in RStan on CRAN!