Cognitive Science and Neuroscience StanConnect 2021 Call for abstracts

Publicity Events
,
1

Dear all,

As part of StanConnect 2021, we intend to propose a session on cognitive science and neuroscience on November 26, 2021, time to be confirmed (depending on the location of the majority of the speakers) 2021-11-19T16:00:00Z.

We’re looking for contributed talks for the proposal to StanConnect. If you’re are interested in giving a talk, please reply in this thread with a title, a 250-words abstract, and your location/time zone. We’ll receive abstracts during 2021-04-09T19:00:00Z → 2021-04-21T21:59:00Z. The selection will be based on an open poll in this thread for 24 hours during 2021-04-21T22:00:00Z.

Contributed talks should feature Stan implementations that are useful for interesting problems in cognitive science and neuroscience, including computational modeling, psychometrics, classification and prediction, and domain-specific workflows.

Submitting an abstract means the commitment to a 10-20 minutes talk (tbd) and the submission or link to a self-contained notebook, such as knitr/rmarkdown or Jupyter (by 2021-11-11T23:00:00Z). The notebook should include separate files containing the Stan program, data, initializations if used, and a permissive license for everything such as CC BY 4.0.

The organizers of this StanConnect session welcome and encourage contributions from all scientists. We strongly value diversity and are committed to creating an equitable environment where human diversity is welcomed and respected. While no list can hope to be comprehensive, we explicitly honor diversity in: age, culture, ethnicity, gender identity or expression, language, national origin, political beliefs, profession, race, religion, sexual orientation, and socioeconomic status.

This thread is only for abstracts For questions about StanConnect 2021 in general see here, for the organization of this session, see here.

Best,
Bruno

3 Likes
2

Workflow techniques for the robust use of Bayes factors

Inferences about hypotheses are ubiquitous in the cognitive sciences. Bayes factors provide one general way to compare different hypotheses by their compatibility with the observed data. Those quantifications can then also be used to choose between hypotheses. While Bayes factors provide an immediate approach to hypothesis testing, they are highly sensitive to details of the data/model assumptions. Moreover it’s not clear how straightforwardly this approach can be implemented in practice, and in particular how sensitive it is to the details of the computational implementation. Here, we investigate these questions for Bayes factor analyses in the cognitive sciences. We explain the statistics underlying Bayes factors as a tool for Bayesian inferences and discuss that utility functions are needed for principled decisions on hypotheses. We study how Bayes factors misbehave under different conditions. This includes a study of errors in the estimation of Bayes factors. Importantly, it is unknown whether Bayes factor estimates based on bridge sampling are unbiased for complex analyses. We are the first to use simulation-based calibration as a tool to test the accuracy of Bayes factor estimates. Moreover, we study how stable Bayes factors are against different MCMC draws. We moreover study how Bayes factors depend on variation in the data. We also look at variability of decisions based on Bayes factors and how to optimize decisions using a utility function. We outline a Bayes factor workflow that researchers can use to study whether Bayes factors are robust for their individual analysis. Reproducible code is available from OSF | Bayes factors.

Location: Berlin/Potsdam, Germany

9 Likes
3

Curvish: An R package for asking questions about development

John C. Flournoy, Graham Baum, Patrick Mair & Leah H. Somerville

Understanding systematic variation of neuroimaging data over time is a central project in developmental cognitive neuroscience, e.g., when describing normative development (Tamnes et al, 2017), or functional dynamics underlying risky behavior during adolescence (Casey, et al., 2016). This research requires that we accurately model developmental trends and derive meaningful quantities from them with appropriate uncertainty. For example, to ask at what age there is a peak in functional BOLD response, we have to find the maximum value of our trend function as well its uncertainty. One approach that has been taken is to bootstrap simple polynomials, but this risks oversimplifying the functional form and biasing estimates. More complex questions strain this approach. Modeling trends using Bayesian splines provides flexible descriptions of development, and posterior probability distributions which can be manipulated to ask complex questions with ease. The curvish package extends brms (Bürkner, 2017) to help users estimate and manipulate Bayesian spline models in cross-sectional and longitudinal data. It provides functions to compute derivatives; the X value where the trend or its derivative is maximized, minimized, or equal to some value; and regions of the trend that are different from another point on the trend or some value. Finally, it replaces false-positive error correction for multiple comparisons with control of the total Type-S (sign) error probability.

San Diego, USA (PST aka GMT-7)

3 Likes
4

Using computational modeling parameters to measure working memory processes

The memory measurement model (M3; Oberauer & Lewandowsky, 2018) is a cognitive measurement model designed to isolate parameters associated with different processes in working memory. It assumes that different categories of representations in working memory get activated through distinct processes. Transforming the activation of the different item categories into their respective recall probabilities then allows to estimate the contributions of different memory processes to working memory performance.
So far, parameter recovery was assessed only for group level parameters of the M3. In contrast to experimental research, individual differences research relies on variation in subject parameters. The quality of parameter recovery of subject parameters has, however, not yet been investigated. To analyze parameter recovery of subject parameters of the M3, we ran a parameter recovery simulation to assess the model performance in recovering subject-level parameters dependent on different experimental conditions. In this talk, we will present the results of this parameter recovery study that used a multivariate parametrization of the model implemented in STAN using the no-u-turn sampler (Hoffman & Gelman, 2011).
The results of the simulation indicate that our implementation of the M3 recovers subject parameters acceptably. Based on differences between experimental conditions, we will provide recommendations for using the M3 in individual differences research. Altogether, our parameter recovery study showed that the M3 is easily scalable to different experimental paradigms with sufficient recovery performance.

Location Heidelberg, Germany (CET)

6 Likes
5

Using Stan with R for inference in psychology and cognitive neuroscience

Monica Thieu and Paul Alexander Bloom

Bayesian regression modeling can be a valuable tool for robust inference in psychology and cognitive neuroscience research. However, Bayesian modeling may not be accessible for many researchers in these fields without experience writing probabilistic programs from scratch. R tools like brms (Bürkner, 2017) and rstanarm (Gabry et al., 2019) built on a Stan back-end leverage the power and flexibility of such models through an approachable interface for those familiar with other R regression packages. Here, we provide a survey of Bayesian approaches to common problems within inference in psychology and cognitive neuroscience, including bounded outcome measures, hierarchical data structures, longitudinal modeling of development, and discriminating within-participant from between-participant relationships. We also demonstrate how these Bayesian approaches can be used within ‘multiverse’ analyses to examine robustness of results to a variety of analytical decision points (Steegen et al., 2016). In efforts to expand the accessibility of Bayesian modeling in Stan to the psychology and cognitive neuroscience communities more broadly, we include self-contained tutorials for each example.

Location: New York, USA (EST)

6 Likes
6

Estimating the Drift Diffusion Model in Stan

The Drift Diffusion Model (DDM) is arguably one of the most used process models in cognitive psychology and neuroscience. It describes decision processes as a first passage Wiener process where relative evidence between two choice option is accumulated over time until one of two decision boundaries (thresholds) is reached. In its original form the DDM maps participant choices and response times to four latent variables: Boundary separation, starting point bias, non-decision time and drift-rate. Since the DDM has a tractable likelihood function, its parameters can be estimated in Stan. However, there are a few things to consider when implementing the model. First, the Wiener first-passage time distribution in Stan is set up for accumulation to one boundary instead of two boundaries as the DDM assumes. Second, the model makes certain assumptions about the parameters which can cause practical problems with chain initialization and divergencies. In this talk, I will show how the model code can be adapted to mimic a Wiener process between two boundaries. I will further suggest that the issues with the DDM assumptions can be accounted for by using advantageous parameter transformations and by fixing the initial samples. Overall, the talk is supposed to provide a good starting point for the creation of more complex diffusion models to capture specific aspects of cognitive processes while benefitting from Stan’s computational efficiency.

Location: Basel, Switzerland (CET)

6 Likes
7

It’s complicated: Some observations on the nuanced constraints of the multivariate normal in high dimensions

The multivariate normal is a structure that enjoys widespread use in many more complex models, including serving as the backbone of the increasingly popular hierarchical (a.k.a. “multi-level” ) class of models. Despite it’s general flexibility in permitting possibly-correlated variates to mutually-inform and thereby improve the accuracy and precision of inference, as the dimensionality of the multivariate normal increases, an otherwise-subtle structural constraint grows in strength to a degree that the multivariate normal becomes inflexible and actively suppresses the mutual-informativity that drives its usefulness for lower-dimensional models. To elucidate this behaviour, this talk will recount the author’s experience observing it in the context of modelling data from 2-alternative speeded-choice tasks, where an inferential workflow that added increasing-but-principled complexity (including: location-scale inference for the continuous response time outcome; simultaneous inference for response times and error rates; contrast reparameterization to enable direct inference on test-retest reliability) led to final inferences that increasingly strongly contradicted both domain expertise and results from simpler models. The talk will also discuss ongoing explorations of the performance of an alternative to the multivariate normal developed during this process that seeks to maintain mutual-informativity even in high dimensions.

Location: Halifax, Canada
Time zone: AST (UTC-4)

3 Likes
8

Comparing encoding- and retrieval-based models of agreement attraction in Stan
Dario Paape, Serine Avetisyan, Sol Lago, and Shravan Vasishth

Agreement attraction occurs when the finite verb of a clause agrees with an incorrect noun phrase, resulting in ungrammatical sentences being perceived as acceptable (“The artists who the sculptor hate”). Two broad classes of accounts aim to capture this illusory acceptability. Retrieval-based accounts assume that the attractor noun “artists” is sometimes incorrectly retrieved from memory as the subject of the verb “hate”, licensing plural agreement. By contrast, encoding-based accounts assume that the plural feature carried by “artists” can erroneously migrate onto the head of the subject noun phrase, causing comprehenders to misconstruct its number (“sculptors"). We implemented the retrieval-based account as a lognormal race model (Nicenboim & Vasishth, 2018), and the encoding-based account as a multinomial processing tree (e.g., Riefer & Batchelder, 1988). Using Stan, we fitted both models to experimental data from Eastern Armenian. Graphical posterior predictive checks and 10-fold cross-validation show that the encoding-based model provides a better predictive fit to the data, and that the fit can be further improved by assuming that the verb itself can be a source of migrating features during encoding.

Location: Berlin/Potsdam, Germany

3 Likes
9

Resolving the multiple testing issue in neuroimaging through Bayesian multilevel modeling
Gang Chen

Two intertwining aspects of information loss are involved in conventional neuroimaging data analysis. First, through massively univariate analysis in which the same model is simultaneously applied to all spatial units (e.g., voxels, regions, matrix elements, DTI tracks), multiple testing adjustment is typically adopted to compensate for multiplicity, but the associated thresholding reduces the continuum of statistical evidences to binary classification. The severe penalty is embodied by a pursuit of rigorous controllability of false positives under the conventional framework through leveraging spatial contiguity (e.g., clusters of neighboring spatial units). The second aspect of information loss is due to an implicit and mostly unrecognized assumption that all potential effects have the same likelihood of being observed, equating to a prior of uniform distribution from -\infty to +\infty. When a bell-shaped distribution (e.g., Gaussian) more accurately characterizes data variability across space, adopting the stance of complete ignorance leads to excessively heavy penalties, inefficient modeling, poor generalizability, overfitting and compromised predictability.

A Bayesian multilevel framework can effectively resolve multiplicity, reduce information loss, and avoid artificial dichotomization. Specifically, we construct an integrative model that incorporates all spatial units into one model. Through partial pooling, information is leveraged across all spatial units, and model performance can be verified and compared through posterior predictive checks and information criteria. Unlike the conventional massively univariate approach that focuses on individual “trees” without any consideration for the forest, spatial information is regularized in Bayesian multilevel modeling through “seeing the forest for the trees”. Furthermore, multiplicity is resolved because one high-dimensional joint posterior distribution is obtained to infer various effects of interest. In addition, we emphasize full result reporting through a “highlight but not hide” approach: gradating the statistical evidence without dichotomization.

Location: Washington DC, USA (EDT / GMT-0400)

5 Likes
10

Implementation of the Diffusion Decision Model with Across-Trial Variability in the Drift Rate
Kendal Foster and Henrik Singmann

The Ratcliff diffusion decision model (DDM) is the most prominent model for jointly modelling binary responses and associated response times. The implementation currently available in Stan only provides the four-parameter variant, the Wiener model, with non-decision time, boundary separation, drift rate, and starting point. We present a Stan implementation of the five-parameter DDM variant that additionally allows for across-trial variabilities in the drift rate. Importantly, the drift rate variability is expressed analytically and not numerically. The five parameter version is implemented in a numerically stable manner combining both “small-time” and “large-time” approximations using the methods recently introduced by Foster and Singmann (2021, http://arxiv.org/abs/2104.01902).

Location: Coventry (UK) and London (UK)

5 Likes
11

A Discrete-State Model of Multiple-Response SAT

Pavel Logacev

The response-signal paradigm, also known as the speed-accuracy tradeoff (SAT) technique, has been widely used to study the time course of cognitive processes in fields as diverse as memory, attention, and sentence processing. In this experimental paradigm, participants are required to respond to a stimulus at one of several specific time lags after the stimulus onset. The differences between experimental conditions with regard to the shape of the resulting speed-accuracy tradeoff function are assumed to reflect differences in the time course of the cognitive processes involved in the experimental task.

Because the SAT technique requires a relatively large number of trials, which is not feasible in sentence processing experiments, SAT experiments in this field routinely use the so-called multiple-response variant of the response-signal paradigm (MR-SAT), in which participants respond to the stimulus multiple times per trial.

So far, the analysis of such data did not take into account that the series of responses on any given experimental trial are unlikely to be independent. I present an analysis method for MR-SAT data which accounts this serial dependence between responses by modelling the process as a two-high-threshold model in which the thresholds, as well as the guessing parameters vary as a function of time between stimulus and response. Under these assumptions, the model, which is implemented in Stan, can be used to estimate the completion time distribution of the decision-making process.

Location: Istanbul/Turkey, GMT+3

2 Likes
12

Dear all,

Thanks for the great abstracts. Please use the poll below to vote for 3 talks. The poll will be open for 24 hours.

Cognitive Science and Neuroscience StanConnect 2021 potential talks
  • Workflow techniques for the robust use of Bayes factors
  • Curvish: An R package for asking questions about development
  • Using computational modeling parameters to measure working memory processes
  • Using Stan with R for inference in psychology and cognitive neuroscience
  • Estimating the Drift Diffusion Model in Stan
  • It’s complicated: Some observations on the nuanced constraints of the multivariate normal in high dimensions
  • Comparing encoding- and retrieval-based models of agreement attraction in Stan
  • Resolving the multiple testing issue in neuroimaging through Bayesian multilevel modeling
  • Implementation of the Diffusion Decision Model with Across-Trial Variability in the Drift Rate
  • A Discrete-State Model of Multiple-Response SAT

0 voters

1 Like