Unit Tests: Finite-Diff

Developers
1

I’m writing unit tests in Torsten for the autodiff. A while ago, I did a series of preliminary tests with run_diagnose in cmdStan.

I want to test autodiff against finite diff. I’ve been following the example of what was done for integrate_ode_rk45. Straightforward, but coding extensive. Is this the best approach? Is there maybe a way to harness the run_diagnose function for these tests?

2

@charlesm93, I just have questions:

  • what’s Torsten?
  • is testing at the CmdStan level the right level? if you’re testing finite diffs, shouldn’t it be at the math level?
  • what’s run_diagnose?
  • what was done for integrate_ode_rk45? Are you talking about the tests in the math library? Which file in particular?
3
  • Torsten is the library of specialized pharmacometrics functions.
  • Testing at the CmdStan level is not right. As said, these were preliminary tests, and I am currently implementing tests at the math level.
  • Sorry, not run_diagnose but diagnose
  • I am looking at test/unit/math/rev/arr/functor/util.hpp. In it, a set of functions to do finite-diff on integrate_ode_rk45 specifically is defined. This leads me to think that for each function I want to test with finite-diff, I will need to create a similar set of functions. On the other hand, I’m wondering if there is a method to automatically do finite-diff in Stan, since we have a diagnose tool.
4

Hi @charlesm93, I’m still confused:

What does Torsten depend on? Is it dependent on Math and Stan? Just on Math? I’m still trying to figure out how broad Torsten is. And it’s probably not “the” library, but “a” library unless I missed something.

This implies that it’s either something that’s part of the Math library or built on top of it? Can it be used standalone as a library that sits outside of the Math library? Or does it wrap the Math library?

Or… is it built on Stan?

[For anyone else reading, @charlesm93 is referring to the Math library]
See the functors in the math library in stan/math/prim/mat/functor/*.hpp. I think that might be what you’re looking for.

5

Stan provides a general finite diff test for whole programs.

Other tests for functions, etc., should be unit tests down
in stan-math.

At the level of Torsten, you should just assume that Stan
works. Each level can’t try to retest all the lower levels.
The tests should just go in the lower levels directly to save
redundancy.

  • Bob
6

@syclik: Yes, Torsten is “a” library of built-in Stan functions for pharmacometrics applications, and its development is part of the deliverables for the ONR grant. The functions are coded in C++ inside the Math library and exposed to Stan’s language. See http://metrumrg.com/publications/2016/10/margossian.ACOP.2016.10.html

Thank you for pointing me to the functor files, this looks promising.

7

Yes, that makes sense. Luckily, this seems to be the path I’ve been on.

8

Daniel:

See:

It’s why we were talking about making Stan extensible at
the C++ level.

Everyone:

The question is how much of Torsten or other domain-specific
functions into Stan and how to support extensions. Ideally,
Torsten would be distributable as a plug-in extension to Stan.

Charles:

To use the autodiff functors in math/prim/mat/functor, you need
to code up a functor for the function you want to differentiate.
There shouldn’t be autodiff code elsewhere, just conversions to
functors. Is there some left somewhere?

So the tests are at the C++ level of the math functions, not
through the higher-level interfaces. I think we’re all on the
same page here.

  • Bob

syclik Developer
December 14
Hi @charlesm93, I’m still confused:

charlesm93:
Torsten is the library of specialized pharmacometrics functions.

What does Torsten depend on? Is it dependent on Math and Stan? Just on Math?

RStan?