# Request for Mathematica code review: generating test cases

**URL:** <https://discourse.mc-stan.org/t/request-for-mathematica-code-review-generating-test-cases/12696>\
**Category:** Developers\
**Tags:** math\
**Created:** [January 18, 2020, 9:16pm UTC](https://discourse.mc-stan.org/t/request-for-mathematica-code-review-generating-test-cases/12696 "2020-01-18T21:16:24Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [January 23, 2020, 8:10pm UTC](https://discourse.mc-stan.org/t/request-for-mathematica-code-review-generating-test-cases/12696/8 "2020-01-23T20:10:02Z")

</div>

> [@bgoodri](#):
>
> I would just reiterate a couple of things that I have said before that few people agree with:
> 
> 1. I think we should be testing that known identities hold. For example, since lnΓ(x)=lnΓ(x+1)−lnx\ln \Gamma\left(x\right) = \ln \Gamma\left(x + 1\right) - \ln x for all positive xx , plug in an xx to that equation and verify that the equation holds and that all derivatives of interest are the same on both sides.

I think you may have misinterpreted my earlier comments. I think this is a good idea, just not always necessary if we can do thorough tests versus well-defined base cases like the lgamma from the standard library.

To really get traction on this, it should be automated to allow us to plug in two functions and a range of values and make sure all derivatives have the same value. I can help write that. It’d look something like this:

```no-highlight
template <class F, class G>
expect_same(const ad_tolerances& tol, const F& f, const G& g,
         Eigen::VectorXd x);

```

where the vector packs all the arguments to `f` or `g`. We cold get fancier and put in a parameter pack for `x` and try to use the serialization framework to reduce it to a vector to supply to our functionals.

As @martinmodrak points out,

> [@martinmodrak](#):
>
> Another issue with formulas is that I always have to worry about the numerics of the formula itself.

That’s why I included tolerances.

> [@bgoodri](#):
>
> If we want arbitrary precision tests to compare to, then the gold standard is [Arb](http://arblib.org/index.html)

Wouldn’t we need to write all our functions to deal with their data types? Or recode them?

> [@martinmodrak](#):
>
> IMHO you are making perfect the enemy of the good here.

I don’t get this comment. From my perspective, both @martinmodrak and @bgoodri are suggesting new, complementary tests that go beyond what we have now. If people want to put in effort making tests better, more power to them!

> [@martinmodrak](#):
>
> Using Arb might well be a better way forward in the long run, but I personally lack the skill and patience to work with C/C++ more than I have to.

I don’t have any opinion about Arb vs. Mathematica in terms of coverage or accuracy. I’d never heard of Arb before @bgoodri brought it up. The problem I have is that I can’t read that Mathematica code. So the discussion we should have is whether everyone’s OK introducing one more language into the project after OCaml and C++.

> [@martinmodrak](#):
>
> I am not sure this thread should be turned into a discussion about best practices for tests in Stan math.

But @martinmodrak, you’re the one who opened this can of worms by saying that our current tests were not best practice because they were low precision. I wasn’t offended by that, by the way—I’m totally open to having people improve things I did. That’s how we make the project better.

---

_[View the full topic](https://discourse.mc-stan.org/t/request-for-mathematica-code-review-generating-test-cases/12696)._