Problem background is similar to :
https://discourse.mc-stan.org/t/problems-converging-using-custom-gamma2-distribution/14684
The data is different, and there is a g_swath parameter instead of g_shape.
2 g_size levels
2 g_noise levels
2 g_swath levels
6 g_interps levels
20 g_reps in each g_sizeg_noises_swath combination with g_interps repeated measures.
When running the model below for 20 g_reps, everything looks great. However, the bulk and tail ess is low for the intercept only.
brms_20red6_gamma_mdl4_1
Family: gamma
Links: mu = log; shape = log
Formula: y ~ g_size * g_noise * g_swath * g_interps + (1 | g_rep)
shape ~ g_size * g_noise * g_swath * g_interps
Data: t_longSubset20red6_unord (Number of observations: 960)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Group-Level Effects:
~g_rep (Number of levels: 20)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.04 0.01 0.03 0.06 1.00 1222 1732
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept 5.31 0.01 5.28 5.33 1.01 842 1677
shape_Intercept 7.30 0.22 6.85 7.72 1.00 2638 3213
g_size25 0.19 0.05 0.09 0.29 1.00 1602 2034
g_noise19 2.94 0.01 2.92 2.96 1.00 1919 2370
g_swathLM -0.03 0.03 -0.10 0.04 1.00 1803 2163
g_interpsEM006 -0.57 0.01 -0.58 -0.56 1.00 2239 2205
g_interpsEM018 -0.60 0.01 -0.61 -0.59 1.00 2171 2256
g_interpsEM033 -0.61 0.01 -0.62 -0.60 1.00 2168 2314
g_interpsEM035 -0.61 0.01 -0.62 -0.60 1.00 2145 2238
g_interpsEM037 -1.14 0.01 -1.17 -1.12 1.00 4353 3262
g_size25:g_noise19 -0.27 0.09 -0.44 -0.10 1.00 1441 2342
g_size25:g_swathLM 0.39 0.09 0.21 0.57 1.00 1699 2355
g_noise19:g_swathLM -0.03 0.04 -0.11 0.06 1.00 1827 2135
g_size25:g_interpsEM006 0.57 0.07 0.44 0.70 1.00 2212 2479
g_size25:g_interpsEM018 0.59 0.06 0.46 0.72 1.00 1823 2666
g_size25:g_interpsEM033 0.60 0.07 0.47 0.73 1.00 2247 2545
g_size25:g_interpsEM035 0.59 0.06 0.47 0.72 1.00 2153 2632
g_size25:g_interpsEM037 0.70 0.08 0.54 0.85 1.00 2887 2782
g_noise19:g_interpsEM006 -0.00 0.01 -0.02 0.02 1.00 2008 2385
g_noise19:g_interpsEM018 -0.01 0.01 -0.03 0.01 1.00 1985 2299
g_noise19:g_interpsEM033 -0.01 0.01 -0.02 0.01 1.00 1962 2385
g_noise19:g_interpsEM035 -0.01 0.01 -0.03 0.01 1.00 1923 2447
g_noise19:g_interpsEM037 -1.39 0.05 -1.50 -1.29 1.00 4944 3052
g_swathLM:g_interpsEM006 0.34 0.05 0.24 0.45 1.00 2367 3128
g_swathLM:g_interpsEM018 0.35 0.05 0.25 0.46 1.00 2430 2436
g_swathLM:g_interpsEM033 0.36 0.05 0.26 0.47 1.00 2474 2568
g_swathLM:g_interpsEM035 0.36 0.05 0.26 0.46 1.00 2370 2578
g_swathLM:g_interpsEM037 0.46 0.08 0.30 0.63 1.00 3394 3370
g_size25:g_noise19:g_swathLM -0.12 0.15 -0.42 0.19 1.00 1409 2089
g_size25:g_noise19:g_interpsEM006 0.00 0.12 -0.23 0.24 1.00 2274 2963
g_size25:g_noise19:g_interpsEM018 0.01 0.12 -0.22 0.25 1.00 2041 2533
g_size25:g_noise19:g_interpsEM033 0.00 0.12 -0.22 0.23 1.00 2149 2942
g_size25:g_noise19:g_interpsEM035 -0.02 0.12 -0.25 0.20 1.00 2043 2238
g_size25:g_noise19:g_interpsEM037 1.08 0.17 0.74 1.44 1.00 3333 3215
g_size25:g_swathLM:g_interpsEM006 -0.34 0.13 -0.58 -0.09 1.00 2358 2972
g_size25:g_swathLM:g_interpsEM018 -0.34 0.13 -0.60 -0.08 1.00 2032 3038
g_size25:g_swathLM:g_interpsEM033 -0.36 0.13 -0.61 -0.10 1.00 2335 2976
g_size25:g_swathLM:g_interpsEM035 -0.34 0.13 -0.59 -0.09 1.00 2490 2848
g_size25:g_swathLM:g_interpsEM037 -0.25 0.15 -0.56 0.06 1.00 2726 3142
g_noise19:g_swathLM:g_interpsEM006 -0.01 0.07 -0.14 0.12 1.00 2696 3207
g_noise19:g_swathLM:g_interpsEM018 -0.01 0.07 -0.15 0.13 1.00 2538 2829
g_noise19:g_swathLM:g_interpsEM033 -0.01 0.07 -0.15 0.12 1.00 2464 2962
g_noise19:g_swathLM:g_interpsEM035 -0.01 0.07 -0.15 0.12 1.00 2519 2997
g_noise19:g_swathLM:g_interpsEM037 0.95 0.16 0.63 1.28 1.00 3580 2884
g_size25:g_noise19:g_swathLM:g_interpsEM006 0.01 0.22 -0.42 0.44 1.00 2172 2934
g_size25:g_noise19:g_swathLM:g_interpsEM018 0.01 0.22 -0.40 0.45 1.00 1846 3116
g_size25:g_noise19:g_swathLM:g_interpsEM033 0.01 0.22 -0.43 0.46 1.00 2417 2605
g_size25:g_noise19:g_swathLM:g_interpsEM035 0.03 0.22 -0.40 0.47 1.00 2478 2924
g_size25:g_noise19:g_swathLM:g_interpsEM037 -0.85 0.31 -1.47 -0.26 1.00 2855 3236
shape_g_size25 -4.29 0.35 -5.00 -3.64 1.00 2141 2405
shape_g_noise19 -0.44 0.33 -1.10 0.19 1.00 2579 2881
shape_g_swathLM -3.45 0.36 -4.14 -2.75 1.00 2256 2717
shape_g_interpsEM006 1.88 0.40 1.09 2.64 1.00 2859 2587
shape_g_interpsEM018 2.57 0.43 1.71 3.40 1.00 2852 2964
shape_g_interpsEM033 3.42 0.47 2.52 4.34 1.00 2621 3104
shape_g_interpsEM035 3.43 0.48 2.49 4.37 1.00 2580 3078
shape_g_interpsEM037 -1.51 0.37 -2.23 -0.82 1.00 2713 3159
shape_g_size25:g_noise19 -0.27 0.46 -1.18 0.62 1.00 2060 2526
shape_g_size25:g_swathLM 2.89 0.47 1.95 3.80 1.00 2351 2754
shape_g_noise19:g_swathLM 0.88 0.47 -0.07 1.83 1.00 2506 2812
shape_g_size25:g_interpsEM006 -1.57 0.54 -2.61 -0.53 1.00 2538 2070
shape_g_size25:g_interpsEM018 -2.21 0.55 -3.29 -1.12 1.00 2792 2328
shape_g_size25:g_interpsEM033 -3.05 0.58 -4.19 -1.96 1.00 2711 2489
shape_g_size25:g_interpsEM035 -3.05 0.58 -4.21 -1.90 1.00 2631 2906
shape_g_size25:g_interpsEM037 1.09 0.52 0.08 2.11 1.00 2511 2826
shape_g_noise19:g_interpsEM006 0.30 0.53 -0.73 1.33 1.00 2885 2569
shape_g_noise19:g_interpsEM018 -0.00 0.53 -1.01 1.05 1.00 2689 3010
shape_g_noise19:g_interpsEM033 0.42 0.66 -0.85 1.72 1.00 2538 3193
shape_g_noise19:g_interpsEM035 0.24 0.63 -0.99 1.52 1.00 2157 2343
shape_g_noise19:g_interpsEM037 -2.30 0.51 -3.26 -1.28 1.00 2448 2741
shape_g_swathLM:g_interpsEM006 -2.21 0.55 -3.30 -1.14 1.00 2662 2712
shape_g_swathLM:g_interpsEM018 -2.90 0.55 -3.99 -1.85 1.00 2680 3280
shape_g_swathLM:g_interpsEM033 -3.73 0.57 -4.86 -2.62 1.00 2533 3272
shape_g_swathLM:g_interpsEM035 -3.75 0.59 -4.90 -2.61 1.00 2621 2677
shape_g_swathLM:g_interpsEM037 -0.15 0.52 -1.12 0.89 1.00 2654 2950
shape_g_size25:g_noise19:g_swathLM -0.93 0.59 -2.09 0.24 1.00 2476 2676
shape_g_size25:g_noise19:g_interpsEM006 -0.48 0.69 -1.86 0.86 1.00 2926 3394
shape_g_size25:g_noise19:g_interpsEM018 -0.24 0.70 -1.63 1.14 1.00 2465 2977
shape_g_size25:g_noise19:g_interpsEM033 -0.66 0.77 -2.13 0.83 1.00 2698 3206
shape_g_size25:g_noise19:g_interpsEM035 -0.53 0.74 -1.99 0.91 1.00 2278 3046
shape_g_size25:g_noise19:g_interpsEM037 1.50 0.69 0.14 2.86 1.00 2575 2808
shape_g_size25:g_swathLM:g_interpsEM006 1.76 0.71 0.33 3.17 1.00 2749 2930
shape_g_size25:g_swathLM:g_interpsEM018 2.39 0.70 1.04 3.77 1.00 2925 2857
shape_g_size25:g_swathLM:g_interpsEM033 3.18 0.72 1.80 4.61 1.00 2953 2817
shape_g_size25:g_swathLM:g_interpsEM035 3.17 0.72 1.76 4.54 1.00 2912 2837
shape_g_size25:g_swathLM:g_interpsEM037 0.20 0.69 -1.09 1.55 1.00 2775 3037
shape_g_noise19:g_swathLM:g_interpsEM006 -0.56 0.70 -1.90 0.84 1.00 2951 2574
shape_g_noise19:g_swathLM:g_interpsEM018 -0.30 0.69 -1.64 1.06 1.00 2771 3013
shape_g_noise19:g_swathLM:g_interpsEM033 -0.74 0.77 -2.27 0.79 1.00 2637 3319
shape_g_noise19:g_swathLM:g_interpsEM035 -0.56 0.76 -2.06 0.88 1.00 2576 2588
shape_g_noise19:g_swathLM:g_interpsEM037 0.86 0.69 -0.47 2.16 1.00 2588 2979
shape_g_size25:g_noise19:g_swathLM:g_interpsEM006 0.78 0.91 -0.99 2.57 1.00 3241 2942
shape_g_size25:g_noise19:g_swathLM:g_interpsEM018 0.60 0.88 -1.13 2.28 1.00 2869 3194
shape_g_size25:g_noise19:g_swathLM:g_interpsEM033 1.04 0.95 -0.86 2.86 1.00 3242 3245
shape_g_size25:g_noise19:g_swathLM:g_interpsEM035 0.90 0.91 -0.86 2.70 1.00 2797 3074
shape_g_size25:g_noise19:g_swathLM:g_interpsEM037 -0.58 0.88 -2.29 1.13 1.00 2985 2939
Samples were drawn using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
However, when increasing the reps from 20 to 200. I run into problems with bulk and tail ess overall.
brms_200red6_gamma_mdl4_1
Family: gamma
Links: mu = log; shape = log
Formula: y ~ g_size * g_noise * g_swath * g_interps + (1 | g_rep)
shape ~ g_size * g_noise * g_swath * g_interps
Data: t_longSubset200red6_unord (Number of observations: 9600)
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup samples = 4000
Group-Level Effects:
~g_rep (Number of levels: 200)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.04 0.00 0.04 0.05 1.08 49 95
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept 5.30 0.00 5.29 5.30 1.09 33 181
shape_Intercept 7.27 0.09 7.07 7.44 1.02 219 637
g_size25 0.18 0.02 0.14 0.21 1.01 349 694
g_noise19 2.94 0.00 2.93 2.95 1.01 645 1143
g_swathLM -0.03 0.01 -0.04 -0.01 1.01 619 1377
g_interpsEM006 -0.57 0.00 -0.58 -0.57 1.00 728 1463
g_interpsEM018 -0.60 0.00 -0.61 -0.60 1.00 687 1537
g_interpsEM033 -0.61 0.00 -0.61 -0.60 1.00 688 1392
g_interpsEM035 -0.61 0.00 -0.62 -0.61 1.00 694 1651
g_interpsEM037 -1.12 0.00 -1.13 -1.11 1.00 1447 2164
g_size25:g_noise19 -0.17 0.02 -0.22 -0.12 1.01 343 746
g_size25:g_swathLM 0.37 0.03 0.32 0.43 1.00 355 730
g_noise19:g_swathLM -0.04 0.01 -0.06 -0.02 1.01 534 1323
g_size25:g_interpsEM006 0.57 0.02 0.53 0.62 1.01 463 1028
g_size25:g_interpsEM018 0.59 0.02 0.55 0.64 1.00 468 953
g_size25:g_interpsEM033 0.60 0.02 0.56 0.65 1.00 493 1182
g_size25:g_interpsEM035 0.60 0.02 0.55 0.64 1.01 461 767
g_size25:g_interpsEM037 0.66 0.03 0.60 0.71 1.00 576 1188
g_noise19:g_interpsEM006 -0.00 0.00 -0.01 0.00 1.01 622 1233
g_noise19:g_interpsEM018 -0.01 0.00 -0.02 -0.01 1.01 713 1260
g_noise19:g_interpsEM033 -0.01 0.00 -0.01 -0.00 1.01 673 1200
g_noise19:g_interpsEM035 -0.01 0.00 -0.01 -0.00 1.01 682 1251
g_noise19:g_interpsEM037 -1.41 0.02 -1.45 -1.37 1.00 1187 1675
g_swathLM:g_interpsEM006 0.35 0.01 0.32 0.38 1.01 795 1463
g_swathLM:g_interpsEM018 0.36 0.01 0.33 0.38 1.00 921 1914
g_swathLM:g_interpsEM033 0.37 0.01 0.34 0.39 1.01 919 1920
g_swathLM:g_interpsEM035 0.37 0.01 0.34 0.39 1.01 945 1930
g_swathLM:g_interpsEM037 0.43 0.02 0.39 0.47 1.01 869 1605
g_size25:g_noise19:g_swathLM -0.26 0.04 -0.35 -0.18 1.01 376 612
g_size25:g_noise19:g_interpsEM006 0.00 0.03 -0.06 0.06 1.01 445 1260
g_size25:g_noise19:g_interpsEM018 0.01 0.03 -0.05 0.07 1.00 445 862
g_size25:g_noise19:g_interpsEM033 0.00 0.03 -0.06 0.07 1.00 470 1177
g_size25:g_noise19:g_interpsEM035 -0.01 0.03 -0.08 0.05 1.00 493 1428
g_size25:g_noise19:g_interpsEM037 1.18 0.05 1.08 1.27 1.00 713 1310
g_size25:g_swathLM:g_interpsEM006 -0.35 0.04 -0.43 -0.27 1.00 498 1153
g_size25:g_swathLM:g_interpsEM018 -0.35 0.04 -0.43 -0.27 1.00 474 1204
g_size25:g_swathLM:g_interpsEM033 -0.36 0.04 -0.45 -0.28 1.00 522 1099
g_size25:g_swathLM:g_interpsEM035 -0.35 0.04 -0.43 -0.27 1.00 527 1156
g_size25:g_swathLM:g_interpsEM037 -0.22 0.05 -0.31 -0.12 1.00 569 1239
g_noise19:g_swathLM:g_interpsEM006 -0.02 0.02 -0.06 0.01 1.00 790 1599
g_noise19:g_swathLM:g_interpsEM018 -0.02 0.02 -0.06 0.01 1.00 781 1589
g_noise19:g_swathLM:g_interpsEM033 -0.03 0.02 -0.06 0.01 1.01 816 1612
g_noise19:g_swathLM:g_interpsEM035 -0.03 0.02 -0.06 0.01 1.01 848 1457
g_noise19:g_swathLM:g_interpsEM037 0.94 0.05 0.85 1.04 1.00 977 1747
g_size25:g_noise19:g_swathLM:g_interpsEM006 0.02 0.06 -0.09 0.14 1.00 519 805
g_size25:g_noise19:g_swathLM:g_interpsEM018 0.02 0.06 -0.10 0.14 1.00 471 1076
g_size25:g_noise19:g_swathLM:g_interpsEM033 0.03 0.06 -0.09 0.15 1.00 542 1073
g_size25:g_noise19:g_swathLM:g_interpsEM035 0.04 0.06 -0.08 0.16 1.00 604 1026
g_size25:g_noise19:g_swathLM:g_interpsEM037 -0.96 0.09 -1.13 -0.79 1.00 631 1262
shape_g_size25 -4.36 0.13 -4.62 -4.10 1.01 251 543
shape_g_noise19 -0.14 0.13 -0.40 0.12 1.01 254 618
shape_g_swathLM -2.76 0.14 -3.02 -2.48 1.02 206 582
shape_g_interpsEM006 2.01 0.15 1.72 2.32 1.02 225 634
shape_g_interpsEM018 2.11 0.14 1.82 2.37 1.01 416 1024
shape_g_interpsEM033 4.02 0.20 3.65 4.43 1.01 260 585
shape_g_interpsEM035 4.02 0.19 3.65 4.38 1.02 232 590
shape_g_interpsEM037 -1.62 0.14 -1.89 -1.34 1.02 198 448
shape_g_size25:g_noise19 0.17 0.18 -0.19 0.53 1.01 259 853
shape_g_size25:g_swathLM 2.19 0.19 1.82 2.53 1.01 261 572
shape_g_noise19:g_swathLM 0.31 0.19 -0.05 0.67 1.02 245 1062
shape_g_size25:g_interpsEM006 -1.95 0.20 -2.35 -1.53 1.01 244 632
shape_g_size25:g_interpsEM018 -2.05 0.19 -2.43 -1.66 1.01 382 751
shape_g_size25:g_interpsEM033 -3.95 0.23 -4.41 -3.51 1.01 239 551
shape_g_size25:g_interpsEM035 -3.96 0.23 -4.39 -3.51 1.02 224 923
shape_g_size25:g_interpsEM037 1.22 0.19 0.84 1.60 1.01 263 471
shape_g_noise19:g_interpsEM006 -0.06 0.20 -0.44 0.32 1.01 266 846
shape_g_noise19:g_interpsEM018 -0.26 0.19 -0.62 0.12 1.01 413 863
shape_g_noise19:g_interpsEM033 -1.11 0.26 -1.64 -0.61 1.01 303 556
shape_g_noise19:g_interpsEM035 -1.36 0.25 -1.83 -0.88 1.01 226 637
shape_g_noise19:g_interpsEM037 -3.03 0.19 -3.39 -2.66 1.02 241 709
shape_g_swathLM:g_interpsEM006 -2.72 0.21 -3.13 -2.32 1.02 211 722
shape_g_swathLM:g_interpsEM018 -2.86 0.20 -3.25 -2.47 1.01 450 1081
shape_g_swathLM:g_interpsEM033 -4.76 0.24 -5.23 -4.31 1.01 227 690
shape_g_swathLM:g_interpsEM035 -4.76 0.23 -5.20 -4.31 1.01 286 890
shape_g_swathLM:g_interpsEM037 -0.28 0.20 -0.66 0.11 1.02 205 793
shape_g_size25:g_noise19:g_swathLM -0.58 0.25 -1.05 -0.08 1.01 271 667
shape_g_size25:g_noise19:g_interpsEM006 -0.01 0.27 -0.57 0.50 1.01 281 661
shape_g_size25:g_noise19:g_interpsEM018 0.18 0.26 -0.34 0.70 1.01 363 793
shape_g_size25:g_noise19:g_interpsEM033 1.02 0.31 0.40 1.65 1.01 255 537
shape_g_size25:g_noise19:g_interpsEM035 1.22 0.30 0.63 1.77 1.02 229 894
shape_g_size25:g_noise19:g_interpsEM037 2.10 0.26 1.57 2.60 1.01 289 698
shape_g_size25:g_swathLM:g_interpsEM006 2.61 0.28 2.07 3.17 1.01 226 544
shape_g_size25:g_swathLM:g_interpsEM018 2.76 0.28 2.21 3.30 1.01 450 1198
shape_g_size25:g_swathLM:g_interpsEM033 4.63 0.29 4.07 5.24 1.01 241 789
shape_g_size25:g_swathLM:g_interpsEM035 4.64 0.29 4.08 5.20 1.01 294 765
shape_g_size25:g_swathLM:g_interpsEM037 0.31 0.26 -0.20 0.82 1.01 296 680
shape_g_noise19:g_swathLM:g_interpsEM006 -0.11 0.27 -0.65 0.42 1.01 263 804
shape_g_noise19:g_swathLM:g_interpsEM018 0.08 0.26 -0.43 0.61 1.01 451 1097
shape_g_noise19:g_swathLM:g_interpsEM033 0.92 0.31 0.32 1.56 1.01 303 754
shape_g_noise19:g_swathLM:g_interpsEM035 1.16 0.30 0.58 1.76 1.01 288 890
shape_g_noise19:g_swathLM:g_interpsEM037 1.63 0.27 1.10 2.13 1.03 224 797
shape_g_size25:g_noise19:g_swathLM:g_interpsEM006 0.23 0.37 -0.48 0.96 1.01 278 440
shape_g_size25:g_noise19:g_swathLM:g_interpsEM018 0.05 0.37 -0.68 0.77 1.00 411 1204
shape_g_size25:g_noise19:g_swathLM:g_interpsEM033 -0.77 0.39 -1.52 0.00 1.01 263 894
shape_g_size25:g_noise19:g_swathLM:g_interpsEM035 -0.99 0.38 -1.72 -0.22 1.01 307 894
shape_g_size25:g_noise19:g_swathLM:g_interpsEM037 -1.31 0.36 -2.01 -0.60 1.02 308 1161
Samples were drawn using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
Warning message:
Parts of the model have not converged (some Rhats are > 1.05). Be careful when analysing the results! We recommend running more iterations and/or setting stronger priors.