Capturing RStan Error Output

Interfaces RStan
1

I’m trying to capture the error message accompanying a failed model compilation (for other uses). Interactively

> stan_model(file='stan_programs/debug.stan')

returns something like

SYNTAX ERROR, MESSAGE(S) FROM PARSER:
No matches for: 
 
 real[ ] * real[ ]
...

Explicitly capturing the error output prints the error message to screen,

> output <- capture.output(stan_model(file='stan_programs/debug.stan'), type=c("message"))
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
No matches for: 
 
 real[ ] * real[ ]
...

but doesn’t capture it,

> output
NULL

Similarly a tryCatch still lets the same error message be print to screen,

> output <- tryCatch({stan_model(file='stan_programs/debug.stan')},
                   error = function(e) { return(e) })
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
No matches for: 
 
 real[ ] * real[ ]
...

in this case capturing a different error,

> result
<simpleError in stanc(file = file, model_code = model_code, model_name = model_name,     verbose = verbose, obfuscate_model_name = obfuscate_model_name,     allow_undefined = allow_undefined, isystem = isystem): failed to parse Stan model 'debug1a' due to the above error.>

From the RStan repository it looks like the message being printed to screen and evading all means of capture is coming from a C++ exception thrown but I can’t figure out how rcpp is handling that and what kind of message it produces.

Does anyone have any idea how to capture these error messages so that I can reformat them and the like?

Thanks.

  • Operating System - OS X 10.14
  • RStan Version - 2.192.
2 Likes
2

I guess this is similar thing we have in Python, where the C++ side is not always captured.

Maybe try the file based solution mentioned here https://stackoverflow.com/a/49722545

invisible(capture.output(log("A"), type = "message", file = "example.txt"))
3

Would surrounding the Rstan call with a tryCatch work? Something along the lines of:

tryCatch(stan_model(...), error=function(e) print(e))

If you are able to capture the exception through that, then perhaps you will be able to do whatever you want within that function(e) block.

4

That seems to only contain

failed to parse Stan model due to the above error.

I think capture.output is supposed to be the way one would go about this but it gets precluded by the stop() thrown by stanc. So, I think we would have to change things so that stanc always returns something even if the Stan program is invalid.

5

My understanding is that sink, and hence capture.output, is supposed to capture the output message associated with the stop() command. The best I can tell is that the message is printed through the terminal before stop() is called and hence leaks out of anywhere that R could potentially contain it.

This seems to happen with system calls – could https://github.com/stan-dev/rstan/blob/d807b3457de94fa2e9eb78f783d4eb968f1c8a01/rstan/rstan/R/cxxfunplus.R#L197 be the culprit? Would adding an intern option to that system call be possible?

markdown eventually captures this input, I believe by wrapping all calls in an implicit system call with intern set, so I’m hoping that it’s possible. I’m not sure how to wrap an R call to stan_model into a system call, though.

6

I had success using loggr to capture the Stan’s messages when using rstan. There was a bug that truncated multi-line messages that my fork fixes. To use it, I do the following before running rstan::sampling():

library(loggr)
log_file(
    log_file_name
    , .formatter = function(event) event$message
    , subscriptions=c('message', 'warning','stop')
)
7

If you do

foo <- function() {
  message("this is a message")
  stop("this is an error")
}
output <- capture.output(foo(), type = "message")
output

you get

Error: object 'output' not found

So, the stop is precluding the message from getting captured. That is why I was saying we would have to change things around to remove the stop and then check the return code subsequently.

8

I think I can do this if we can ever get rstan 2.21 onto CRAN without segfaulting the other packages that still have pre-CRTP code but are not yet linking to TBB. But if you are writing new code, I would write it to the stanc3 parser. You can get that with

ctx <- V8::v8() # must have V8 installed
ctx$source("https://github.com/stan-dev/stanc3/releases/download/nightly/stanc.js")
stuff <- ctx$call("stanc", "test", "Data {}")
stuff

which will return a R list that has (at least)

$errors
[1] “0”
[2] “\nSyntax error in ‘string’, line 1, column 0 to column 4, parsing error:\n\nExpected "functions {" or "data {" or "transformed data {" or "parameters {" or "transformed parameters {" or "model {" or "generated quantities {".\n\n\n”

The quotation marks are actually ASCII but do not look that way on Discourse.

1 Like
9

What about withCallingHandlers? It sounds like they maybe allow one to hook directly into any message call and hence save the message before the stop is thrown. Or do they not offer much beyond a tryCatch?