Dealing with R 4.0

andrjohns · August 11, 2020, 3:57am

Hi Doguhan,

The relevant error in your case is: sh: -c: line 0: syntax error near unexpected token `('

Which I believe is caused by the parentheses in the directory name :C:/Users/daksu/Dropbox/My PC (DESKTOP-O01UCGH)/.... As a temporary fix, can you try moving your R package directory to somewhere without parentheses in the directory name?

daks · August 11, 2020, 4:35am

Thank you so much for your quick response!

I have changed the library path:

.libPaths()
[1] “C:/Users/daksu/Documents/R/win-library/4.0” “C:/Program Files/R/R-4.0.2/library”

When I run

example(stan_model, package = “rstan”, run.dontrun = TRUE)

I get the following resultrstan1.txt (8.9 KB)

Shall I do anything about:

Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
‘-E’ not found

andrjohns · August 11, 2020, 4:39am

That means it’s all working fine now (hooray), you can safely ignore the warning message

esben · August 11, 2020, 8:32am

Sorry for the wait, I was hit by the limit on number of posts on the first day (as a new member)… But here it is: out.txt (19.5 KB). And thanks for the help - Much appreciated!

hpjfromdk · August 11, 2020, 8:33am

Probably not of relevance for the issue but when I load RSTAN, I get:

library(“rstan”)
Loading required package: StanHeaders
Loading required package: ggplot2
Use suppressPackageStartupMessages() to eliminate package startup messages
rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
Do not specify ‘-march=native’ in ‘LOCAL_CPPFLAGS’ or a Makevars file
Warning message:
package ‘rstan’ was built under R version 4.1.0

I have tried many things, including other stanheaders (currently 2.21.0-6 ), building from source etc…
But, my R version is 4.0.2 so it’s perhaps a bit odd that the when loading the package seems to have been built under R version 4.1.0 …

/hpj

andrjohns · August 11, 2020, 9:08am

Not a problem, my fault for asking so many questions! Because those commands ran without issue, it looks like your compiler/toolchain is configured correctly and R can call it without issue. That would imply that there’s an issue with how RStan itself is calling the compiler.

As a last attempt, could you try reinstalling the inline and Rcpp packages, then restarting R and running:

fx <- inline::cxxfunction( signature(x = "integer", y = "numeric" ) , 'return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;' )

esben · August 11, 2020, 9:19am

andrjohns:

As a last attempt, could you try reinstalling the inline and Rcpp packages, then restarting R and running:
fx <- inline::cxxfunction( signature(x = "integer", y = "numeric" ) , 'return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;' )

Sure, first reinstalling the pacages:

Restarting R session…

remove.packages(“inline”)
Removing package from ‘C:/Users/REVSB00E/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
remove.packages(“Rcpp”)
Removing package from ‘C:/Users/REVSB00E/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)

install.packages(“inline”)
Installing package into ‘C:/Users/REVSB00E/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
trying URL ‘https://cran.rstudio.com/bin/windows/contrib/4.0/inline_0.3.15.zip’
Content type ‘application/zip’ length 125049 bytes (122 KB)
downloaded 122 KB

package ‘inline’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
C:\Users\REVSB00E\AppData\Local\Temp\RtmpkxgsJr\downloaded_packages
install.packages(“Rcpp”)
Installing package into ‘C:/Users/REVSB00E/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
trying URL ‘https://cran.rstudio.com/bin/windows/contrib/4.0/Rcpp_1.0.5.zip’
Content type ‘application/zip’ length 3265998 bytes (3.1 MB)
downloaded 3.1 MB

package ‘Rcpp’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
C:\Users\REVSB00E\AppData\Local\Temp\RtmpkxgsJr\downloaded_packages

Inline returns exactly the same error:

fx ← inline::cxxfunction( signature(x = “integer”, y = “numeric” ) , ‘return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;’ )
Error in file(con, “r”) : cannot open the connection
In addition: Warning message:
In file(con, “r”) :
cannot open file ‘file1834672972fc.cpp.err.txt’: No such file or directory

But, just FYI, Rcpp is working (and has been working the whole time for some of my more advanced use cases):

Rcpp::cppFunction(‘int add(int x, int y, int z) {
int sum = x + y + z;
return sum;
}’)

add(1, 2, 3)
[1] 6

andrjohns · August 11, 2020, 9:23am

Great, thanks for checking with Rcpp as well, helps narrow things down. Unfortunately I’m out of ideas and will have to call in for help here.

@bgoodri (sorry to keep pinging you) I think something might have broken in the inline package for some people. This user can compile c++ with no issue via:

system("touch foo.cpp")
system("R CMD SHLIB foo.cpp")

And with the Rcpp package:

Rcpp::cppFunction('int add(int x, int y, int z) {
  int sum = x + y + z;
  return sum;
}')

add(1, 2, 3)
[1] 6

But encounters errors when using the inline package:

fx <- inline::cxxfunction( signature(x = "integer", y = "numeric" ) , 'return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;' )
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
    In file(con, "r") :
    cannot open file 'file1b104434265c.cpp.err.txt': No such file or directory

esben · August 11, 2020, 9:32am

And just to add in; I am working on a completely fresh install from Friday (August 7) as I reinstalled R, R-studio, Rtools, all packages and so on as part of trying to get it to work. Don’t know if this can be part of the explanation why other users apparently aren’t affected as bad…

hpjfromdk · August 11, 2020, 10:47am

First of all many thanks for your all your help efforts… GREATLY APPRECIATED !!
I will try to reinstalll Rccp., But before that…
I tried to fit the 8scools, which terminates “as usual” and left these two files in the Rtemp folder:
callr-env-7985430627f (no extension)
file79872cf2313.cpp
This will also setup the compiler and linker flags…

I then ran a
system(“R CMD SHLIB file79872cf2313.cpp”)
The compilation, and linking seems successful:

ending with >[1] 0
creation of file79872cf2313.cpp.o
creation of file79872cf2313.cpp.dll
…which I take as indicative of the toolchain, compiler and linker working as intended…

Took me a while to figure out how to dump the entire BIG output to a text file, but in case others need it
you can modify the size of the rstudio console buffer :https://community.rstudio.com/t/more-than-1000-lines-output-in-r-studio-console/3288/8

The RStan fit output is in lines 41 to 604 and the R CMD SHLIB output in lines 610 to 4851 in this file 8scoolStan_ex.txt (1.3 MB)

Do you still want me to reinstall Rcpp ?

br/hpj

esben · August 11, 2020, 3:10pm

I have looked a little into the inline package and I believe I have found the problem. Thanks to @hpjfromdk for giving me the clues I needed.

In the inline package, the function compileCode() is sourced in the background and is called from the inline::cxxfunction() that throws the same error as stan(). In this function the system(R CMD SHLIB..) command is called in the following way:

cmd <- paste(R.home(component="bin"), "/R CMD SHLIB ", basename(libCFile), " 2> ", errfile, sep="")
if (verbose) cat("Compilation argument:\n", cmd, "\n")
compiled <- system(cmd, intern=!verbose)

Apparently the " 2 > " command is not understood by my computer and causes the system() command to return character(0) but without writing the error file (.err.txt). The stan and inline errors come from the next line:

> errmsg <- readLines( errfile )

since no error-file is generated, this call returns:

> Error in file(con, "r") : cannot open the connection
> In addition: Warning message:
> In file(con, "r") :
>   cannot open file 'file471012dc4e44.cpp.err.txt': No such file or directory

Now the interesting part:If I edit the system command to:

cmd <- paste(R.home(component="bin"), "/R CMD SHLIB ", basename(libCFile), sep="")

i.e. erase the " 2> ", errfile, and then subsequently comment out the three following lines regarding error handling I can get the function to work.

Hence, with those edits:

> fx <- cxxfunction( signature(x = "integer", y = "numeric" ) , 'return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;' , verbose = FALSE)
> fx(2L,3)
[1] 6

Now, the next step in getting this hack to play with stan, I recon, is to build a new edited version of the inline package and then use that in place of the original (until the problem is resolved in a more sustainable way…).
I’ve not come this far yet, but just thought I would post it in case others can use it and are maybe inspired to post a more elegant solution…

Thanks for the help to all of you!

bgoodri · August 11, 2020, 3:53pm

You can install rstan from CRAN now to get something build with R 4.0.2

bgoodri · August 11, 2020, 3:55pm

Yeah, this is a bit of a mess

But rstan on CRAN is supposed to be working with LOCAL_LIBS rather than PKG_LIBS. Can you tell if / where things are going wrong on your machine?

esben · August 11, 2020, 7:43pm

As far as I can tell, in lines 169-175 of rstan/R/ cxxfunplus.R:

fx <- pkgbuild::with_build_tools(
    cxxfunction(sig = sig, body = body, plugin = plugin, includes = includes, 
                settings = settings, ..., verbose = verbose),
    required = rstan_options("required") &&
    # workaround for packages with src/install.libs.R
      !identical(Sys.getenv("WINDOWS"), "TRUE") &&
      !identical(Sys.getenv("R_PACKAGE_SOURCE"), "") )

And by the way, my edit to inline::cxxfunction() similar to what you pointed out on github solves the problem and leaves me only with the warning regarding g++ that many others are having…

In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
‘C:/rtools40/usr/mingw_/bin/g++’ not found,

I’ll try to look closer into exactly what is passed to the cxxfunction() when the error happens. But that will have to wait till tomorrow…

bgoodri · August 11, 2020, 7:46pm

If you can just try

remotes::install_github("bgoodri/inline")

that would be a big help.

hpjfromdk · August 11, 2020, 9:02pm

used your remotes::install_github(“bgoodri/inline”) and restarted R

After that, in my case, now the:

fit <- stan(file = file.path("D:/R/$_R_Install/Stan","schools.stan"), data = schools_dat,verbose = TRUE)
the :
rfit <- stan(model_code = rats , data = rats_dat,verbose = TRUE)
and the
example(stan_model, package = "rstan", run.dontrun = TRUE)

all complete compilation/linking and sampling from the models without any error !!!

I will compare the sampling results with example sources tomorrow…
But so far the issue symptoms seem to have ceased

Great Job, many thanks to all !!
br/hpj

bgoodri · August 12, 2020, 1:35am

Need @jonah or someone with a Mac to try my inline package both for

example(stan_model, package = "rstan", run.dontrun = TRUE)

some Stan program that parses but does not compile (that is easiest if you have a function that is declared but not defined and you pass allow_undefined to stan_model).

jonah · August 12, 2020, 2:40am

This runs fine.

With this Stan program

functions {
  real foo(real x);
}
parameters {
  real theta;
}
model {
  real phi = foo(theta);
  phi ~ normal(0, 1);
}

and allow_undefined=TRUE I get this error

Error in dyn.load(libLFile) : 
  unable to load shared object '/var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//RtmpA8xhCO/file2d2666c33bcf.so':
  dlopen(/var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//RtmpA8xhCO/file2d2666c33bcf.so, 6): Symbol not found: __ZN59model2d26c85f946_ba618b1f654f36b2234f8583b1052e01_namespace3fooIN4stan4math3varEEEN5boost4math5tools12promote_argsIT_fffffE4typeERKS8_PNSt3__113basic_ostreamIcNSD_11char_traitsIcEEEE
  Referenced from: /var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//RtmpA8xhCO/file2d2666c33bcf.so
  Expected in: flat namespace
 in /var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//RtmpA8xhCO/file2d2666c33bcf.so
Error in sink(type = "output") : invalid connection

esben · August 12, 2020, 5:20am

That also did the trick for me. Now RStan is working for the first time in a week. Thanks for the effort!

bgoodri · August 22, 2020, 5:16pm

OK, that is supposed to happen but is there more to the error message?

Topic		Replies	Views
R 4.0, rstan, and you RStan howto	41	8839	February 5, 2021
Problems getting Stan to run after upgrading to R 4.0.2 RStan	48	9855	March 20, 2023
R 4.0.0 and CRAN MacOS binaries RStan	27	7737	April 6, 2021
Rstan on Windows RStan	177	25996	September 2, 2020
Stan just broke after upgrade to R 4.0.2 under windows 10 required reinstallation - Help needed Interfaces rstan , installation	4	731	July 20, 2020

Dealing with R 4.0

Related Topics