Rstan fails now when library(rstan) is called simultaneously from a computer cluster

General
1

Hello @jonah @bgoodri ,

Rstan fails now when library(rstan) is called simultaneously from a computer cluster.

I run 30 independent jobs that need library(rstan), and a minimal reproducible example

library(rstan)

Will cause failure. Can this depend on some change made in the web link that rstan does? (I also reinstalled rstan but problem persists). Does rstan does some web query at the time of library(rstan)?

Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ COAD.tcga.harmonized.counts.allgenes.rds failed with exit code 1
rule 5 failed, moving any outputs to makeflow.failed.5
2021/03/05 10:19:05.60 makeflow[21278] notice: will retry failed job Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ COAD.tcga.harmonized.counts.allgenes.rds
submitting job: Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ COAD.tcga.harmonized.counts.allgenes.rds
submitted job 1276679
job 1276674 completed
Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CHOL.tcga.harmonized.counts.allgenes.rds failed with exit code 1
rule 4 failed, moving any outputs to makeflow.failed.4
2021/03/05 10:19:05.63 makeflow[21278] notice: will retry failed job Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CHOL.tcga.harmonized.counts.allgenes.rds
submitting job: Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CHOL.tcga.harmonized.counts.allgenes.rds
submitted job 1276680
job 1276675 completed
Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CESC.tcga.harmonized.counts.allgenes.rds failed with exit code 1
rule 3 failed, moving any outputs to makeflow.failed.3
2021/03/05 10:19:05.65 makeflow[21278] notice: will retry failed job Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CESC.tcga.harmonized.counts.allgenes.rds
submitting job: Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ CESC.tcga.harmonized.counts.allgenes.rds
submitted job 1276681
job 1276676 completed
Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BRCA.tcga.harmonized.counts.allgenes.rds failed with exit code 1
rule 2 failed, moving any outputs to makeflow.failed.2
2021/03/05 10:19:05.67 makeflow[21278] notice: will retry failed job Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BRCA.tcga.harmonized.counts.allgenes.rds
submitting job: Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BRCA.tcga.harmonized.counts.allgenes.rds
submitted job 1276682
job 1276677 completed
Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BLCA.tcga.harmonized.counts.allgenes.rds failed with exit code 1
rule 1 failed, moving any outputs to makeflow.failed.1
2021/03/05 10:19:05.69 makeflow[21278] notice: will retry failed job Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BLCA.tcga.harmonized.counts.allgenes.rds
submitting job: Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ BLCA.tcga.harmonized.counts.allgenes.rds
submitted job 1276683
job 1276678 completed
Rscript dev/TCGA_makeflow_pipeline/create_input.R ~/PhD/deconvolution/ ACC.tcga.harmonized.counts.allgenes.rds failed with exit code 1
2

In the current rstan (2.21), the stan command downloads a javascript compiler in the background to make sure your model compiles with what will be the next version of rstan (it doesn’t upload your model or anything).

rstan itself uses an old compiler to actually build the model (not the one that was downloaded), and you can turn off the download check with:

library(rstan)
rstan_options(javascript = FALSE)
1 Like
3

Thanks! It turned out it was a problem with callr. Mystery… I am reinstalling everything R in that machine as I could not find a solution.

1 Like