Hey Bob,
afaik we’ve had credits to spend on spot instances those may have expired at the end of 2019.
Can you verify that we aren’t using any accounts where we have to pay for cycles for continuous integration?
I’m getting mail to devops@mc-stan.org from AWS threatening to shut our account down 1/30/2020 for non-payment. I don’t have the login for that AWS account, so can’t investigate further. That’s why I’ve been trying to get more info from @seantalts or @jonah.
Can you contact me via email (you can use devops@mc-stan.org!) so that we can coordinate to make sure this isn’t the same account you’re using. Thanks.
I would love to help you but my account doesn’t have access to billing, I think only @seantalts can help.
OK. I’ll try to follow up with @seantalts—he says he sent the info to @jonah, but @jonah hasn’t responded to my email from several days ago about it.
1 Like
Hi Bob,
I’ll send you an email and I can follow this up.
Sue.
1 Like
Is there a problem with the gpu machine? I think this has been stuck for about half a day waiting for a gpu machine to free up:
https://jenkins.mc-stan.org/blue/organizations/jenkins/Math%20Pipeline/detail/PR-1612/9/pipeline
Hey @mcol there were some custom performance tests running. I’ve let them run since yday since there wasn’t any load on the queue. I’ve now cancelled and the queue will execute all builds and PRs normally. Sorry for trouble.
I’d be happy to input new payment information into the account, and I know how to get that information from NumFOCUS, but I’ve been blocked for several months waiting on the new SGB to approve it. I didn’t realize this before but the old SGB only approved TWG Director funds for contractors, not for equipment or services (thanks @syclik).
I think these were broken, stuck infinitely compiling. I won’t have time to look into it.
@seantalts No worries, thanks! I’ve already opened a discussion with Steve and we’ll look into it.
1 Like
Not sure why I was tagged? The TWG Director was authorized to spend it without restriction on equipment or services.
Edit: see Stan - Meeting Minutes
The first $50k was allocated generally. The next $75k was allocated specifically for contractors. This was directly as a result of the request made to the SGB.
Yep, exactly - you pointed that out in an exchange a few months ago. I agree with your assessment, so I was thanking you for helping me follow protocol.
1 Like
The following build is stuck and I can’t get it to rerun (clicking the circular arrow button after logging in does nothing): https://jenkins.mc-stan.org/blue/organizations/jenkins/Math%20Pipeline/detail/PR-1620/3/pipeline
@mcol
It’s fine now: https://jenkins.mc-stan.org/blue/organizations/jenkins/Math%20Pipeline/detail/PR-1620/4/pipeline
2 Likes
Thank you!
1 Like
It looks like authorization for the first $50K has lapsed and, as Sean points out, the $75K is restricted. We should resolve at the next @SGB meeting this coming Monday and arrange payment through NumFOCUS ASAP.
4 Likes
@serban-nicusor The issue with building sundials has appeared again (https://jenkins.mc-stan.org/blue/organizations/jenkins/Stan/detail/downstream_tests/1162/pipeline):
ar: creating lib/stan_math/lib/sundials_4.1.0/lib/libsundials_kinsol.a
ar: unable to rename 'lib/stan_math/lib/sundials_4.1.0/lib/libsundials_kinsol.a'; reason: Permission denied
ar: creating lib/stan_math/lib/sundials_4.1.0/lib/libsundials_idas.a
Ah, thanks for posting @mcol.
@rok_cesnovar looks like the PR didn’t fix the issue :(
It did, we just didnt do the same fix on the Stan repository. I am on it.
1 Like
My bad, thanks for the quick reply and help!
1 Like
Docs can now be generated through a Jenkins job found here https://jenkins.mc-stan.org/job/BuildDocs/
It will build the docs, add redirects and create a PR in stan-dev/docs.
Also generates cmdstan manual which can be found in the job artifacts, example: https://jenkins.mc-stan.org/job/BuildDocs/14/
When done with all the docs the job will be there in more detail both usage and technical.