# Switch from \`std::vector\` to \`Eigen::Tensor\` in compiled models?

**URL:** <https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914>\
**Category:** Developers\
**Tags:** features\
**Created:** [September 16, 2017, 6:18pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914 "2017-09-16T18:18:02Z")\
**Posts on this page:** 17\
**Page:** 2

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [October 16, 2017, 4:31am UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/21 "2017-10-16T04:31:07Z")

</div>

> [@bgoodri](#):
>
> Just to be clear: I don’t think the language has to change to do this. Someone can still declare real x[M,N,L] in their Stan programs, but stanc could generate code for an `Eigen::Tensor` named x of rank 3 and a double or var as the scalar type.

Yup. That part is perfectly clear.

The part that isn’t clear at all is what’s the gain? (I really don’t know what we gain from this. Not saying that there aren’t any; I just don’t know what they are.)

To summarize the potential problems:

- maintenance: shifting API means that single use of `Eigen::Tensor` may not be compatible across different Eigen versions, so the math version would then have to be linked to an Eigen version. (Right now, we don’t have that sort of requirement, but maybe we should?)
- RStan installation: not sure if RcppEigen already ships `Eigen::Tesnor`. If it’s an incompatible version with Stan Math, then the installation process for RStan becomes much more complicated.
- dependency management: we’re now in a wag the dog situation where the Stan Math library wouldn’t be able to shift off of Eigen versions because of RStan’s dependency on RcppEigen’s version.
- compiler issues: given how much effort we put in to work around multiple versions of Eigen and different OSes and compilers, I’m going to assume getting `Eigen::Tensor` to work is going to be difficult. I’m guessing we haven’t figured out how to compile against g++7 yet? Given how much trouble PyStan already has compiling models with Eigen, I’m going to guess this is going to make it worse.

Anyway, I’m not trying to disparage. I just want to know that the pros (or even the potential pros) outweigh the list of things I know we’re going to face.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [October 16, 2017, 7:57pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/22 "2017-10-16T19:57:59Z")

</div>

> [@syclik](#):
>
> I really don’t know what we gain from this.

Pro: Mostly memory locality and reduced malloc overhead (and hence reduced fragmentation in theory) and a bit of reduced memory usage. This will make any vectorized operations on the collection faster.

Con: You won’t get values by reference any more, so it may require copying or a whole lot of work writing expression template views. It’ll take arithmetic to look things up (though that’ll almost certainly be faster than the pointer chasing we use now).

> [@syclik](#):
>
> I just want to know that the pros (or even the potential pros) outweigh the list of things I know we’re going to face.

I don’t think they do here.

As you said earlier, I think if we need efficient 3D tensor operations, we should write 3D tensors. Then arrays will remain `std::vector` with all the pros and cons that entails.

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [October 17, 2017, 11:09am UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/23 "2017-10-17T11:09:05Z")

</div>

Since the elements of a std::vector are stored contiguously there’s no penalty to converting things into Eigen at a later point and keeping the public function signatures simple (i.e., std::vector). ([https://stackoverflow.com/questions/247738/is-it-safe-to-assume-that-stl-vector-storage-is-always-contiguous](https://stackoverflow.com/questions/247738/is-it-safe-to-assume-that-stl-vector-storage-is-always-contiguous))

Not requiring interfaces to know anything about Eigen seems to me like an overwhelming advantage for maintainability.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [October 22, 2017, 2:51am UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/24 "2017-10-22T02:51:22Z")

</div>

While `vector<double>` is memory-contiguous, `vector<vector<double>>` is not. Each of the inner vectors will do its own `malloc` and the outer vector will store the elements of the inner vector type, which only includes a pointer to the inner values.

Sorry that Eigen’s proving to be such a thorn in PyStan’s side. I wish I could be more help there.

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [October 22, 2017, 2:59pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/25 "2017-10-22T14:59:19Z")

</div>

No vector\<vector\> s are used in any signatures in the generated  
stan model code as far as I can tell. Do you see them somewhere?

For example, log\_prob’s signature is:

```
template <bool propto __, bool jacobian__ , typename T__>
T__log_prob(vector<T__>& params_r__,
              vector<int>& params_i__,
              std::ostream* pstream__ = 0) const;

```

Seems like it’s just vector we need to worry about.

[edit: escaped code; added const]

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [October 29, 2017, 3:48pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/26 "2017-10-29T15:48:21Z")

</div>

The only reason we still have `vector<T>` is that Allen says it’s hard to use Eigen types in Python. We’re also generating this signature, which I believe we use everywhere else other than PyStan:

```
    template <bool propto, bool jacobian, typename T_>
    T_ log_prob(Eigen::Matrix<T_,Eigen::Dynamic,1>& params_r,
                std::ostream* pstream = 0) const;

```

It also removes the integer parameters.

P.S. Allen, you can use three back ticks to open and close code blocks.

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [October 29, 2017, 8:56pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/27 "2017-10-29T20:56:11Z")

</div>

It’s not just Python specifically that will prefer std over Eigen. It’s  
any programming language that is not R. So Go, Rust, Julia, etc… To the  
extent that they do support interfacing with C++ code they’re far more  
likely to support calling functions which have std::vector arguments  
than calling functions which have Eigen arguments.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 5:37pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/28 "2017-11-01T17:37:39Z")

</div>

Aren’t you going to have to bind the function you want to call anyway with things like the standard streams? All that’s required for Eigen is this:

```
std::vector<double> x;
...
Eigen::VectorXd x2 = Eigen::Map<Eigen::VectorXd>(x.data(), x.size());

```

Internally, everything is done with `VectorXd` types, so we need to do this copy anyway if things are called with `std::vector<double>`.

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [November 1, 2017, 6:01pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/29 "2017-11-01T18:01:39Z")

</div>

If we’re not on the same page we’re at least not far apart.

What you just wrote does not involve a copy. data() returns a pointer  
and Map doesn’t do any copying. I would propose that code such as the  
code block you included be kept inside Stan C++ code. Interfaces should  
only have to know about flat, memory-contiguous std::vector`s.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 6:33pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/30 "2017-11-01T18:33:05Z")

</div>

> [@ariddell](#):
>
> What you just wrote does not involve a copy.

It involves an allocation and copy in the assignment to `VectorXd` — that’s just the way expression templates work. The copies are carried out in the `operator()=` function (which may just autodelegate to a copy constructor).

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 6:37pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/31 "2017-11-01T18:37:34Z")

</div>

How about a compromise where there’s a standalone function that deals with it:

```
namespace stan {
namespace model {

template <bool propto, bool jacobian, typename T, typename M, typename RNG>
T log_prob(const M& m, const std::vector<T>& y, RNG& rng, std::ostream& o) {
  return m.log_prob<propto, jacobian, T>(
     Eigen::Map<Eigen::VectorXd>(y.data(), y.size),
     rng, o);
}
} // namespace model
} // namespace stan

```

Then we can keep the model code simple.

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [November 1, 2017, 7:57pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/32 "2017-11-01T19:57:05Z")

</div>

Perfect. That’s exactly what I had in mind.

I’m still surprised about the copy. Surely there’s a way to avoid that.  
From what I’ve read it seems like people are at least behaving like you  
aren’t making a copy. See

> <https://stackoverflow.com/questions/26094379/typecasting-eigenvectorxd-to-stdvector>

  
and  

> <https://stackoverflow.com/questions/26094379/typecasting-eigenvectorxd-to-stdvector>

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 8:31pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/33 "2017-11-01T20:31:42Z")

</div>

Not sure why you’re surprised as it’s standard operating procedure for expression templates. The `VectorXd` instance manages its own memory using RAII. Construction copies as does assignment—it has to, because the result winds up managing its own memory. You can also look at the code.

The `Map` wraps other memory—that doesn’t need a copy.

At least that first link is also copying to `std::vector` for the same reason—it manages its own memory (unless you override the allocator for `std::vector` to not have the default behavior).

This is also made clear in the doc for `operator=` in `Matrix` (for which VectorXd is a typedef for `Matrix<double, -1, 1>`), which explicitly states that a copy is involved.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 8:34pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/34 "2017-11-01T20:34:29Z")

</div>

Oops, forgot the link to the doc for `Eigen::Matrix`:

[https://eigen.tuxfamily.org/dox/classEigen\_1\_1Matrix.html](https://eigen.tuxfamily.org/dox/classEigen_1_1Matrix.html)

which states in the doc for `operator=`, i.e.,

```
Matrix& Eigen::Matrix< Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols >
::operator=(const EigenBase<OtherDerived>& other);

```

that it

```
Copies the generic expression other into *this.

```

---

<div class="post-metadata">

**Author:** ![ariddell](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@ariddell](https://discourse.mc-stan.org/u/ariddell)\
**Post date:** [November 1, 2017, 9:09pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/35 "2017-11-01T21:09:36Z")

</div>

What about the code block which you provided which doesn’t contain an assignment:

```
namespace stan {
namespace model {

    template <bool propto, bool jacobian, typename T, typename M, typename RNG>
    T log_prob(const M& m, const std::vector<T>& y, RNG& rng, std::ostream& o) {
  return m.log_prob<propto, jacobian, T>(
     Eigen::Map<Eigen::VectorXd>(y.data(), y.size),
     rng, o);
}
} // namespace model
} // namespace stan

```

Is there still a copy here? (It’s this block I was responding to)

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 10:29pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/36 "2017-11-01T22:29:08Z")

</div>

It contains an implicit constructor because the function it is calling takes a `VectorXd&` argument. Whenever you call an argument, you’re essentially assigning to the function argument variables.

The assignment operator `operator=(const T&)` and the copy construtor `T(const T&)` almost always do the same thing. And it’s almost always a copy, hence the name.

The way to reason about it is to keep track which container classes manage memory for their elements. `Map` does not manage memory, but `std::vector` and `Eigen::Matrix` do. A pointer in C or C++ does not manage memory.

Whenever you construct a container or assign to a container that manages its own memory, there’s a copy involved.

C++11 has move semantics which avoid some of this copying by transferring control of the memory from the right-hand-side element to the left hand side. This invalidates the right-hand side object.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [November 1, 2017, 10:30pm UTC](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914/37 "2017-11-01T22:30:43Z")

</div>

I should also add that the compiler is clever in some function return statements, allowing reuse of the memory for an object instead of a default copying assignment. That won’t work here becuase it’s not like types and the lifespan of the object being copied isn’t controlled by the function.

[Previous page](https://discourse.mc-stan.org/t/switch-from-std-vector-to-eigen-tensor-in-compiled-models/1914.md?page=1)