Segfault when linking MatlabEngine library

I made some progress in getting Stan to read input data directly from MATLAB’s workspace. I am however struggling with linking the libraries into main code. I put together a separate, simple test code to read the variables and store them in var_i_ and var_r_ like vectors and it works OK with the options below:

g++ -g -std=c++1y -pthread -ID:/Matlab/R2022b_x64/extern/include -LD:/Matlab/R2022b_x64/extern/bin/win64 read_struct.cpp -lMatlabEngine -lMatlabDataArray -o out.exe

The problems start when I transfer the code into Stan’s source. I boiled it down to a simple mod to main.cpp which still causes Segmentation fault. I suspect the problem is related to the way/order I link dynamic libraries but I am getting to the point when overwhelmed with the complexity.

I understand this is not the mainstream activity for Stan developers but I’d really appreciate if anybody could give me some hints where to look in order to identify the problem. Some details are below:

I modified cmdstan/make/local

# Adding other arbitrary C++ compiler flags
CXXFLAGS += -funroll-loops
CXXFLAGS += -Wno-nonnull
CXXFLAGS += -Wno-ignored-attributes
CXXFLAGS += -g -pthread
CXXFLAGS += -ID:/Matlab/R2022b_x64/extern/include



LDFLAGS += -LD:/Matlab/R2022b_x64/extern/bin/win64 
LDLIBS += -lMatlabEngine -lMatlabDataArray -lpthread

and cmdstan/src/cmdstan/main.cpp

#include <cmdstan/command.hpp>
#include <stan/services/error_codes.hpp>
#include "MatlabEngine.hpp"

void findMAT() {
  using namespace matlab::engine;
  std::vector<String> names = findMATLAB();

int main(int argc, const char *argv[]) {

gdb fault and stack trace:

(gdb) s
0x00007ff66f84b1e3 in (anonymous namespace)::initSession ()
    at D:/Matlab/R2022b_x64/extern/include/MatlabEngine/detail/engine_factory_impl.hpp:20
20              cpp_engine_create_session();
(gdb) bt 10
#0  0x00007ff66f84b1e3 in (anonymous namespace)::initSession ()
    at D:/Matlab/R2022b_x64/extern/include/MatlabEngine/detail/engine_factory_impl.hpp:20
#1  matlab::engine::findMATLAB[abi:cxx11]() ()
    at D:/Matlab/R2022b_x64/extern/include/MatlabEngine/detail/engine_factory_impl.hpp:54
#2  0x00007ff66f971118 in findMAT () at src/cmdstan/main.cpp:8
#3  main (argc=<optimized out>, argv=<optimized out>) at src/cmdstan/main.cpp:12
(gdb) s
The specified procedure could not be found.

Thread 1 received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) info sharedlibrary
From                To                  Syms Read   Shared Object Library
0x00007ffdbf131000  0x00007ffdbf327548  Yes (*)     C:\WINDOWS\SYSTEM32\ntdll.dll
0x00007ffdbe841000  0x00007ffdbe8fe310  Yes (*)     C:\WINDOWS\System32\kernel32.dll
0x00007ffdbc891000  0x00007ffdbcb69990  Yes (*)     C:\WINDOWS\System32\KernelBase.dll
0x00007ffdbcf11000  0x00007ffdbd00faf8  Yes (*)     C:\WINDOWS\System32\ucrtbase.dll
0x00007ffdb9591000  0x00007ffdb95a50c8  Yes (*)     D:\Matlab\R2022b_x64\extern\bin\win64\libMatlabEngine.dll
0x00007ffdb88e1000  0x00007ffdb893b6dc  Yes         D:\Stan\cmdstan\stan\lib\stan_math\lib\tbb\tbb.dll
0x00007ffda4c71000  0x00007ffda4cfc830  Yes (*)     C:\WINDOWS\SYSTEM32\msvcp140.dll
0x00007ffda4c41000  0x00007ffda4c4b13c  Yes (*)     C:\WINDOWS\SYSTEM32\vcruntime140_1.dll
0x00007ffda4c51000  0x00007ffda4c6a1a0  Yes (*)     C:\WINDOWS\SYSTEM32\vcruntime140.dll
0x00007ffdb8d81000  0x00007ffdb8dba084  Yes         D:\Stan\cmdstan\stan\lib\stan_math\lib\tbb\tbbmalloc.dll
(*): Shared library is missing debugging information.

Both my stand alone C++ code and unmodified Stan compile and run with no problems. I am working on Win10 with g++ v10.3.0 in UCRT64 environment.

In the example where you’re compiling a standalone executable with no stan dependencies you produce a out.exe. Can you look at the output of ldd out.exe? I’m mainly wondering if matlab links any of the same libraries we do and that causes an issue.

Hi Brian, thanks for responding so quickly.

ldd bernoulli.exe (with modified cmdstan/src/cmdstan/main.cpp as per note above)

ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffdbf130000)
KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffdbe840000)
KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffdbc890000)
ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffdbcf10000)
libMatlabEngine.dll => /d/Matlab/R2022b_x64/extern/bin/win64/libMatlabEngine.dll (0x7ffdb9590000)
tbb.dll => /d/Stan/cmdstan/stan/lib/stan_math/lib/tbb/tbb.dll (0x7ffdb88e0000)
VCRUNTIME140.dll => /c/WINDOWS/SYSTEM32/VCRUNTIME140.dll (0x7ffda4c50000)
MSVCP140.dll => /c/WINDOWS/SYSTEM32/MSVCP140.dll (0x7ffda4c70000)
VCRUNTIME140_1.dll => /c/WINDOWS/SYSTEM32/VCRUNTIME140_1.dll (0x7ffda4c40000)
vcruntime140_1.dll => /c/Windows/System32/vcruntime140_1.dll (0x1a857f90000)

It looks like VCRUNTIME140_1.dll is loaded twice…

Output from ldd out.exe:

ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffdbf130000)
KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffdbe840000)
KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffdbc890000)
ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffdbcf10000)
libMatlabEngine.dll => /d/Matlab/R2022b_x64/extern/bin/win64/libMatlabEngine.dll (0x7ffdb9590000)
MSVCP140.dll => /c/WINDOWS/SYSTEM32/MSVCP140.dll (0x7ffda4c70000)
VCRUNTIME140_1.dll => /c/WINDOWS/SYSTEM32/VCRUNTIME140_1.dll (0x7ffda4c40000)
VCRUNTIME140.dll => /c/WINDOWS/SYSTEM32/VCRUNTIME140.dll (0x7ffda4c50000)

I looked into it a bit more last night and my bet is on tbb.dll compatibility. MATLAB has got its own version and I am looking into the details of it. Do think it could be an explanation?

It definitely could. Especially if MATLAB on Windows is compiled using the Microsoft compilers (which I suspect it is), then their version of TBB will not mix well with the one we build

I’ve just checked again and I think we’ve found the culprit. I copied Stan’s compiled tbb.dll into MATLAB lib folder and my test code also breaks now with segfault.

Can you think of work around? I asked MATLAB help desk how to compile tbb from let’s say intel website. I am limited to what I can do but I just may able to to try and compile Stan in Visual Studio. Is it worth to have a go?

I have had success compiling Stan against the Visual Studio ABI by using clang. If you install the MSVC build tools and check off the optional Clang/LLVM extension you can probably proceed by making sure you link against the TBB from MATLAB (or any other TBB which is built by MSVC, I used one from conda).

Unfortunately, this doesn’t play very nicely with our makefile setup, so you need to invoke the compiler fairly manually. Here is a powershell script I used before to do this for a CmdStan model: cmdstan-build.ps1 · GitHub