Assigning N values to a M elements array (N>M) creates N elements array, with no error returned

I have a bug in my code, but Stan does not complain

I am assigning 279 values to the vector lambda_sigma_MPI which has been defined just 36 elements long. Weirdly enough this produces a 279 long lambda_sigma_MPI vector.

I would expect C++ requirements to be more stringent.

Can you help me figuring out what is going on?

Code.

[...]
	vector[2*M] lambda_sigma_MPI[n_shards];
	print(lambda_sigma_MPI[1]);
	for( i in 1:(n_shards) ) {

	  vector[ (M*2) - (G_per_shard[i]*2) ] buffer = rep_vector(0.0,(M*2) - (G_per_shard[i]*2));

		lambda_sigma_MPI[i] =
  		append_row(
  		  append_row(
  		    append_row(
      		  lambda_log[(G_per_shard_idx[i]+1):(G_per_shard_idx[i+1])],
      		  sigma[(G_per_shard_idx[i]+1):(G_per_shard_idx[i+1])]
      		),
      		buffer 
      	),
      	exposure_rate // This is in excess, Stan does not complain!
      );

	}

print(lambda_sigma_MPI[1]);
[...]

First defined array

[nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan]

The array becomes much longer

[-0.0496774,0.595961,1.03753,0.0649089,-0.499356,-0.616169,0.148922,-1.89846,-1.89519,1.84231,-1.40401,-0.138188,1.27948,0.231069,1.78851,-1.51458,0.031641,-1.70146,0.457391,0.31896,0.155028,3.01326,0.291948,2.25668,5.72644,0.203418,0.373941,0.809195,4.23869,0.491712,1.15213,0.91687,0.374412,1.41859,0.393286,5.01292,0.210023,-0.100856,-0.449241,1.31634,0.00470277,-1.03057,1.45933,1.9353,-0.700987,0.676599,-0.49359,-1.71404,1.3759,1.45259,0.503186,-1.53609,1.06142,0.728205,1.03482,-1.0087,-1.84758,0.552733,0.335901,-1.71078,-1.53266,-0.478691,-0.93572,-1.27517,-1.70485,-0.451256,-1.88265,0.454993,1.35763,1.04594,0.828966,-0.174494,1.30642,1.41468,-0.575567,-0.213569,1.73675,-1.72428,1.45046,-0.107095,-0.407834,-1.26263,-1.003,0.494981,0.826307,1.91744,0.634951,-0.493431,0.112875,1.78925,0.28481,1.08939,-1.96713,1.23898,-0.267274,-0.00949125,-0.376032,-0.55174,1.84119,0.897458,-1.54812,1.41001,1.76401,1.19041,1.01101,1.31242,1.00478,1.62729,0.85891,-0.307529,1.76401,-0.48152,-0.666308,-0.754774,0.674606,-0.48722,-0.575738,-0.717865,-1.82183,1.11639,-0.7591,1.86681,1.78815,-0.898625,1.53084,-1.04286,0.675433,0.0741882,-0.870926,1.24151,-0.0201724,-1.24325,1.26374,-0.680789,1.15283,0.52379,-1.19901,-0.458115,0.860948,-1.21598,0.242596,-0.339379,-0.744218,-0.643777,1.91259,-0.621658,-1.14437,-1.20626,-0.562274,-1.24617,-0.0940032,0.780497,0.523713,0.407986,-0.973179,0.600587,0.813714,-0.380672,-1.36992,-0.995027,-0.313834,1.23108,1.20302,-0.331825,0.0743259,1.28139,1.94422,-0.572329,0.101252,-0.619873,1.74964,0.924074,-0.465266,1.11537,1.34973,1.32021,1.40056,1.17056,-1.05908,1.4312,-1.94098,-0.985709,1.77898,-0.659867,-1.37223,-0.0605706,-0.491428,-1.0131,0.751121,1.18368,1.53465,1.82427,0.424789,-1.70698,-1.91073,-0.738409,-0.181382,-0.400701,0.815832,0.213034,1.01518,-0.704623,1.39922,1.82995,0.911035,0.819625,-1.05318,1.66108,0.0939394,-0.0993963,-1.18811,0.796197,-0.0867104,1.94642,-0.171875,-1.90609,-0.447039,1.01101,1.81107,-0.448851,-0.963961,0.979362,0.214951,1.3968,-1.56933,-0.260907,1.11971,-1.08058,-1.4232,1.01431,-1.1483,0.157093,1.78057,0.154565,-1.52198,-1.36719,-1.57376,-0.190888,1.29453,-1.13779,-1.55347,-0.542038,0.857132,1.9849,-1.51809,-0.933988,1.7382,1.67067,-0.114745,0.267339,1.07603,-1.23462,1.90386,0.206362,0.641333,-0.825287,1.17978,1.34138,0.708278,1.23175,1.35365,0.81702,-1.91903,-0.79913,-1.01539,-0.326542,-0.892931,1.11934,1.94478,-1.75328,-1.49716,-1.65166,-1.93525,-0.976041,0.340353,-0.00588534,-0.980069,0.419339,0.765796]

in the generated code, variable vector[2*M] lambda_sigma_MPI[n_shards]; becomes:

std::vector<Eigen::Matrix<double, Eigen::Dynamic, 1> > lambda_sigma_MPI(n_shards, Eigen::Matrix<double, Eigen::Dynamic, 1>((2 * M)));

and the assignment statement becomes:

stan::model::assign(lambda_sigma_MPI, 
                            stan::model::cons_list(stan::model::index_uni(i), stan::model::nil_index_list()), 
                            append_row(append_row(append_row(stan::model::rvalue(lambda_log, stan::model::cons_list(stan::model::index_min_max((get_base1(G_per_shard_idx, i, "G_per_shard_idx", 1) + 1), get_base1(G_per_shard_idx, (i + 1), "G_per_shard_idx", 1)), stan::model::nil_index_list()), "lambda_log"), stan::model::rvalue(sigma, stan::model::cons_list(stan::model::index_min_max((get_base1(G_per_shard_idx, i, "G_per_shard_idx", 1) + 1), get_base1(G_per_shard_idx, (i + 1), "G_per_shard_idx", 1)), stan::model::nil_index_list()), "sigma")), buffer), exposure_rate), 
                            "assigning variable lambda_sigma_MPI");
                }

and there’s no problem appending to a std::vector.

the Stan compiler checks variable shape - i.e., number of dimensions.
the generated c++ code checks that variable dimensions are non-negative, e.g., assuming your code has variable:

int G_per_shard_idx[M*2];

the corresponding c++ code checks for valid sizes before declaring the variable:

validate_non_negative_index("G_per_shard_idx", "(M * 2)", (M * 2));
G_per_shard_idx = std::vector<int>((M * 2), int(0));

so yes, you can mess up. maybe the new compiler will be smarter.

1 Like

Thanks Mitzi,

a naive question may be, if I can do this:

vector[3] a = vector[7] b;

resulting in vector[7] a

why do we specify vector a dimension at all?

you can’t do that -
this program:

transformed data {
  vector[7] a;
  vector[3] b = a;
}

produces the following runtime error:

Exception: assign: Rows of left-hand-side (3) and rows of right-hand-side (7) must match in size  (in 'f2.stan' at line 3)

however in your program, there are intermediate results that aren’t being checked that should be.
we plan to fix this…

Thanks!

(yes this behaviors could be bug prone.)