Compile errors with stanc3 for higher-order functions

I’ve been getting odd compilation errors with cmdstanr when using higher-order functions. I just downloaded a new version of cmdstan today.

First with integrate_ode_rk45, I get the following message:

> mod <- cmdstan_model(paste0("model/", modelName))
Compiling Stan program...
Uncaught exception:
  
  (Invalid_argument "length mismatch in map2_exn: 5 <> 4 ")

Raised at file "stdlib.ml", line 34, characters 20-45
Called from file "src/list.ml", line 130, characters 13-76
Called from file "src/list.ml" (inlined), line 378, characters 2-36
Called from file "src/middle/UnsizedType.ml", line 68, characters 13-165
Called from file "src/middle/UnsizedType.ml", line 86, characters 12-68
Called from file "list.ml", line 134, characters 35-42
Called from file "src/list0.ml" (inlined), line 32, characters 40-75
Called from file "src/list.ml" (inlined), line 373, characters 27-49
Called from file "src/list.ml" (inlined), line 379, characters 2-18
Called from file "src/middle/UnsizedType.ml", line 84, characters 7-212
Called from file "src/list.ml", line 204, characters 19-22
Called from file "src/list.ml" (inlined), line 209, characters 22-39
Called from file "src/middle/Stan_math_signatures.ml", line 249, characters 4-128
Called from file "src/frontend/Semantic_check.ml", line 322, characters 4-121
Called from file "src/frontend/Semantic_check.ml", line 334, characters 8-38
Called from file "src/frontend/Semantic_check.ml", line 695, characters 4-57
Called from file "src/frontend/Semantic_check.ml", line 931, characters 12-51
Called from file "src/frontend/Semantic_check.ml", line 1406, characters 9-80
Called from file "src/frontend/Semantic_check.ml", line 1434, characters 4-58
Called from file "src/frontend/Semantic_check.ml", line 1667, characters 18-51
Called from file "list.ml", line 117, characters 24-34
Called from file "src/list0.ml" (inlined), line 22, characters 40-81
Called from file "src/frontend/Semantic_check.ml", line 1666, characters 6-128
Called from file "src/frontend/Semantic_check.ml", line 1728, characters 13-63
Called from file "src/frontend/Frontend_utils.ml", line 24, characters 10-51
Called from file "src/stanc/stanc.ml", line 187, characters 18-53
Called from file "src/stanc/stanc.ml", line 244, characters 9-16
make: *** No rule to make target `/var/folders/ty/36ws994x4l33ksszcdwjtrv40000gp/T/RtmpEluPgF/model-47401e5c31e2.d', needed by `/var/folders/ty/36ws994x4l33ksszcdwjtrv40000gp/T/RtmpEluPgF/model-47401e5c31e2'.  Stop.
Error in processx::run(command = make_cmd(), args = c(tmp_exe, include_paths,  : 
  System command 'make' failed, exit status: 2, stderr (last 10 lines):
E> Called from file "src/frontend/Semantic_check.ml", line 1434, characters 4-58
E> Called from file "src/frontend/Semantic_check.ml", line 1667, characters 18-51
E> Called from file "list.ml", line 117, characters 24-34
E> Called from file "src/list0.ml" (inlined), line 22, characters 40-81
E> Called from file "src/frontend/Semantic_check.ml", line 1666, characters 6-128
E> Called from file "src/frontend/Semantic_check.ml", line 1728, characters 13-63
E> Called from file "src/frontend/Frontend_utils.ml", line 24, characters 10-51
E> Called from file "src/stanc/stanc.ml", line 187, characters 18-53
E> Called from file "src/stanc/stanc.ml", line 244, characters 9-16
E> make: *** No rule to make target `/var/folders/ty/36ws994x4l33ksszcdwjtrv40000gp/T/RtmpEluPgF/model-47401e5c31e2.d', needed by `/var/folders/ty/36ws994x4l33ksszcdwjtrv40000gp
[...]
Type .Last.error.trace to see where the error occured

Really not quite sure what causes this error. Also, I don’t think the unit tests would catch this.

Next, when I try using a prototype function, laplace_marginal_poisson_log(), from the branch try_laplace-approximation2 (note the 2) I get the following:

../model/disease_map_ela.hpp:422:19: error: no matching function for call to
      'laplace_marginal_poisson_log'
        density = laplace_marginal_poisson_log<false>(y, n_samples, ye,
                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../model/disease_map_ela.hpp:661:14: note: in instantiation of function template
      specialization
      'disease_map_ela_model_namespace::disease_map_ela_model::log_prob<false,
      false, double>' requested here
      return log_prob<propto__,jacobian__,T_>(vec_params_r, vec_params_i...
             ^
/Users/charlesm/Desktop/Code/laplace_approximation/writeup/StanCon2020/script/cmdstan/stan/src/stan/model/model_base_crtp.hpp:90:50: note: 
      in instantiation of function template specialization
      'disease_map_ela_model_namespace::disease_map_ela_model::log_prob<false,
      false, double>' requested here
    return static_cast<const M*>(this)->template log_prob<false, false, double>(
                                                 ^
../model/disease_map_ela.hpp:162:3: note: in instantiation of member function
      'stan::model::model_base_crtp<disease_map_ela_model_namespace::disease_map_ela_model>::log_prob'
      requested here
  ~disease_map_ela_model() final { }
  ^
/Users/charlesm/Desktop/Code/laplace_approximation/writeup/StanCon2020/script/cmdstan/stan/lib/stan_math/stan/math/laplace/laplace_marginal_poisson_log.hpp:39:6: note: 
      candidate template ignored: invalid explicitly-specified argument for
      template parameter 'T0'
  T1 laplace_marginal_poisson_log
     ^
/Users/charlesm/Desktop/Code/laplace_approximation/writeup/StanCon2020/script/cmdstan/stan/lib/stan_math/stan/math/laplace/laplace_marginal_poisson_log.hpp:58:6: note: 
      candidate template ignored: invalid explicitly-specified argument for
      template parameter 'T0'
  T1 laplace_marginal_poisson_log

A look at the created C++ file shows the call to laplace_marginal_poisson_log is the following

density = laplace_marginal_poisson_log<false>(y, n_samples, ye,
                    K_functor__(), phi, x, delta, delta_int, theta_0,
                    pstream__);

Everything here is fine except the <false> part. The C++ code I wrote is not designed to handle such a boolean. The code exposing the function to the parser is at line 1600 of stan_math_signature.ml. The unit tests I wrote pass, but obviously they wouldn’t catch such a mistranslation.

Huh, with the integrate one you have probably uncovered a bug that was maybe added with the variadic ODE PR that was merged today.

I will look into this. it would really help if could you post a minimal example where this fails.
Thanks!

EDIT: I tried all the test models we have for the old integrate interface and they all compile fine, is this maybe on your branch? If not, I would need a minimal example.

EDIT2: I took the liberty of changing the thread title to reflect the issue more (its not something cmdstanr specific.Hope you dont mind.

For the laplace one, the issue is that for laplace the propto template parameter is not used and should not get generated. The logic to handle that is here:

Since it has the _log suffix that was used for distributions, the parser treats it like any distibution and prepends the template parameter. If the _log suffix will stay, this should not be handled as a distribution I guess: https://github.com/stan-dev/stanc3/blob/master/src/middle/Utils.ml#L16

Something along the lines of:

let is_distribution_name s =
  (not
     ( String.is_suffix s ~suffix:"_cdf_log"
     || String.is_suffix s ~suffix:"_ccdf_log" 
     || String.equal s "laplace_marginal_poisson_log"))
  && List.exists
       ~f:(fun suffix -> String.is_suffix s ~suffix)
       distribution_suffices

Best solution would be if the functon would not end on _log if possible.

This was an oversight in the laplace PR. (Ignore me, this is on a branch)
This would get caught if we had tests I am proposing here: https://github.com/stan-dev/stanc3/issues/636
(Sorry for taking this opportunity for pushing my agenda here, but I think its important we add this).
That or if the model was also added to the stan repo. But I would prefer not to duplicate stuff.

1 Like

Nope, that is an older bug. The error points here


It was fixed on the closures branch.
1 Like

Oh, thanks! When does it come up? Nevermind, I can see where it might come up.

The bug is that when checking if a function signature matches what a higher-order-function expects it only checks that each argument has the expected type but crashes if the number of arguments does not match. integrate_ode_rk45 takes an ODE system that has five input arguments (time, state, params, real data, int data). Presumably @charlesm93 defined an ODE system function that only takes four arguments.

2 Likes

Thanks everyone for the help.

laplace_marginal_poisson_log is really the name I’d like to use to be consistent with the density named poisson_log. Can I create an exception for certain functions? If needed, I’ll change the name for now (my goal being to have something usable for StanCon, i.e. August 13th).

@nhuurre Your analysis is correct. The function I passed had the wrong number of arguments and the cmdstan version I used was outdated.

Will this be used as laplace_marginal_poisson_log_lpmf similar to poisson_log_lpmf?

No. It isn’t an lpmf in the traditional sense, because the data isn’t passed as a single object (scalar or vector). Rather, the users passes sufficient statistics.

So no laplace_marginal_poisson_log_lpmf.

Ok. Then if its not intended for use with the tilde statement, we can special case it in is_distribution in Utils.ml I think that should cover everything.

Sounds good. I’ll take a look at the file.

Sorry, I was responding on a phone yesterday, so didnt go into detail.

Basically this: https://github.com/stan-dev/stanc3/blob/master/src/middle/Utils.ml#L16

Should be:

let is_distribution_name s =
  (not
     ( String.is_suffix s ~suffix:"_cdf_log"
     || String.is_suffix s ~suffix:"_ccdf_log" 
     || String.equal s "laplace_marginal_poisson_log" ))
  && List.exists
       ~f:(fun suffix -> String.is_suffix s ~suffix)
       distribution_suffices

or something close to that.