What does n_jobs actually do?

Interfaces PyStan
1

Operating System: Windows 10
Python Version: 3.6.3
Interface Version: IPython 6.2.1 (installed through conda) and Command prompt
Compiler/Toolkit: m2w64-toolchain=5.3.0

After compiling my Stan code and storing it in a variable called multivariate_model, I run the following statement:

multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,n_jobs=4)

What is it that n_jobs parameter is actually doing? Is it “repeating” the computation, performing all iterations on both chains 4 times (i.e. 8 chains in total, 4 on each threads)? Or is it finding some interesting way to split the sampling parameters into different threads? Or further, is it splitting up the multiple iterations of a single chain into multiple threads, and then in this case 2 threads work on a chain simultaneously?

I ask this because when I ran my code in the IPython console and on a Jupyter Notebook, things behaved pretty well, but when I tried running my script through the windows command line with $ python myscript.py, things just went went haywire: python compiled my code multiple times and generated several errors. Here are just the first few hundred lines of the errors:

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.

Done compiling in 40 seconds! Started sampling!

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.

Done compiling in 43 seconds! Started sampling!

Traceback (most recent call last):
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 80, in _map_parallel
    pool = multiprocessing.Pool(processes=n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 119, in Pool
    context=self.get_context())
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 174, in __init__
    self._repopulate_pool()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
    w.start()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\ff3883\Dropbox\UT_Austin\Research\Bayesian_Tests\multivariate_regression_stan_simplified.py", line 93, in <module>
    multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,refresh=1,n_jobs=4)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 726, in sampling
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 83, in _map_parallel
    pool.close()
UnboundLocalError: local variable 'pool' referenced before assignment

Done compiling in 43 seconds! Started sampling!

Done compiling in 43 seconds! Started sampling!


Traceback (most recent call last):
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 80, in _map_parallel
    pool = multiprocessing.Pool(processes=n_jobs)
Traceback (most recent call last):
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 119, in Pool
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 80, in _map_parallel
    context=self.get_context())
    pool = multiprocessing.Pool(processes=n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 174, in __init__
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 119, in Pool
    self._repopulate_pool()
    context=self.get_context())
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 174, in __init__
    w.start()
    self._repopulate_pool()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\process.py", line 105, in start
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
    self._popen = self._Popen(self)
    w.start()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 322, in _Popen
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\process.py", line 105, in start
    return Popen(process_obj)
    self._popen = self._Popen(self)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 322, in _Popen
    prep_data = spawn.get_preparation_data(process_obj._name)
    return Popen(process_obj)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
    _check_not_importing_main()
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    is not going to be frozen to produce an executable.''')
    _check_not_importing_main()
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main

    is not going to be frozen to produce an executable.''')

During handling of the above exception, another exception occurred:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.Traceback (most recent call last):

  File "<string>", line 1, in <module>

During handling of the above exception, another exception occurred:

  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 105, in spawn_main
Traceback (most recent call last):
    exitcode = _main(fd)
  File "<string>", line 1, in <module>
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 114, in _main
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 105, in spawn_main
    prepare(preparation_data)
    exitcode = _main(fd)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 225, in prepare
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 114, in _main
    _fixup_main_from_path(data['init_main_from_path'])
    prepare(preparation_data)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 225, in prepare
    run_name="__mp_main__")
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 263, in run_path
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    pkg_name=pkg_name, script_name=fname)
    run_name="__mp_main__")
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 96, in _run_module_code
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 263, in run_path
    mod_name, mod_spec, pkg_name, script_name)
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 85, in _run_code
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 96, in _run_module_code
    exec(code, run_globals)
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\ff3883\Dropbox\UT_Austin\Research\Bayesian_Tests\multivariate_regression_stan_simplified.py", line 93, in <module>
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 85, in _run_code
    multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,refresh=1,n_jobs=4)
    exec(code, run_globals)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 726, in sampling
  File "C:\Users\ff3883\Dropbox\UT_Austin\Research\Bayesian_Tests\multivariate_regression_stan_simplified.py", line 93, in <module>
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
    multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,refresh=1,n_jobs=4)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 83, in _map_parallel
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 726, in sampling
    pool.close()
UnboundLocalError: local variable 'pool' referenced before assignment
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 83, in _map_parallel
    pool.close()
UnboundLocalError: local variable 'pool' referenced before assignment

Done compiling in 43 seconds! Started sampling!

Traceback (most recent call last):
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 80, in _map_parallel
    pool = multiprocessing.Pool(processes=n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 119, in Pool
    context=self.get_context())
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 174, in __init__
    self._repopulate_pool()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
    w.start()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\ff3883\Dropbox\UT_Austin\Research\Bayesian_Tests\multivariate_regression_stan_simplified.py", line 93, in <module>
    multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,refresh=1,n_jobs=4)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 726, in sampling
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 83, in _map_parallel
    pool.close()
UnboundLocalError: local variable 'pool' referenced before assignment
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_9609296e3b2a76587c28ec8e223a4579 NOW.

Done compiling in 43 seconds! Started sampling!

Traceback (most recent call last):
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 80, in _map_parallel
    pool = multiprocessing.Pool(processes=n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 119, in Pool
    context=self.get_context())
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 174, in __init__
    self._repopulate_pool()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
    w.start()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\ff3883\Dropbox\UT_Austin\Research\Bayesian_Tests\multivariate_regression_stan_simplified.py", line 93, in <module>
    multivariate_fit = multivariate_model.sampling(data=multivariate_data_dict,iter=1000,chains=2,refresh=1,n_jobs=4)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 726, in sampling
    ret_and_samples = _map_parallel(call_sampler_star, call_sampler_args, n_jobs)
  File "C:\Users\ff3883\AppData\Local\Continuum\anaconda2\envs\bayes_env\lib\site-packages\pystan\model.py", line 83, in _map_parallel
    pool.close()
UnboundLocalError: local variable 'pool' referenced before assignment

It seems like the “multiprocessing” library is really misbehaving here and I just wanted to get a sense of how the parallel processing actually works.

Thank you!!!

2

In this case it will run each chain on separate core. So it’s should be the same as calling sampling with n_jobs=2.

The error comes how multiprocessing works on windows. If you run your code from command line, put your code under a function (e.g. main) and then call the function under if __name__ == '__main__' block. Then the n_jobs should work.

def main():
    ...

if __name__ == '__main__':
    main()
1 Like
3

Yep, this worked, thank you for the tip!

I really need to cut my poor coding habits and avoid using globals all the time… Hahahahaha…

4

multiprocessing is what PyStan uses to sample in parallel. Sampling in
parallel means running independent samplers (with different random
initializations).

multiprocessing does not work on Windows in an interactive environment
like IPython/Jupyter. You can read more about this limitation here
https://docs.python.org/3.3/library/multiprocessing.html#windows .

The short-term fix to this is to not use PyStan on Windows in an
interactive environment. Use it in a stand-alone script.

The durable fix is to stop using multiprocessing and use threads
instead, as this is better supported on Windows. The reason we can’t do
this right now is that part of Stan is not threadsafe. The relevant
issue is tracked here https://github.com/stan-dev/math/issues/551

p.s. It seems like there have been many changes in multiprocessing in
recent versions of Python. I haven’t looked closely at what changed but
it’s possible there’s something that could make this situation better.

5

Python has lexical scoping by default, which means that although an enclosed scope can access values in its enclosing scope, it cannot modify them (unless they’re declared global with the global keyword). A closure binds values in the enclosing environment to names in the local environment. The local environment can then use the bound value, and even reassign that name to something else, but it can’t modify the binding in the enclosing environment. UnboundLocalError happend because when python sees an assignment inside a function then it considers that variable as local variable and will not fetch its value from enclosing or global scope when we execute the function. To modify a global variable inside a function, you must use the global keyword.