LIVIVO - Search results -

Search results

Result 1 - 10 of total 149

Search options

Article: Prioritizing replay when future goals are unknown.

Sagiv, Yotam / Akam, Thomas / Witten, Ilana B / Daw, Nathaniel D

bioRxiv : the preprint server for biology

2024

Abstract: Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. ... ...

Abstract	Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, and it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Our work extends the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses.
Language	English
Publishing date	2024-03-04
Publishing country	United States
Document type	Preprint
DOI	10.1101/2024.02.29.582822
Database	MEDical Literature Analysis and Retrieval System OnLINE

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Details ▾

Article: Proactive and reactive construction of memory-based preferences.

Nicholas, Jonathan / Daw, Nathaniel D / Shohamy, Daphna

bioRxiv : the preprint server for biology

2023

Abstract: We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related ... ...

Abstract	We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories are encoded (proactively). Here we offer a resolution to this inconsistency. We demonstrate behavioral and neural evidence for both strategies and for how they tradeoff rationally depending on the associative structure of memory. Using fMRI to decode patterns of brain responses unique to categories of images in memory, we found that proactive memory access is more common and allows more efficient inference. However, participants also use reactive access when choice options are linked to more numerous memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time in conditions when this strategy is most favorable.
Language	English
Publishing date	2023-12-11
Publishing country	United States
Document type	Preprint
DOI	10.1101/2023.12.10.570977
Database	MEDical Literature Analysis and Retrieval System OnLINE

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Details ▾

Article ; Online: Are we of two minds?

Daw, Nathaniel D

Nature neuroscience

2018 Volume 21, Issue 11, Page(s) 1497–1499

MeSH term(s)	Corpus Striatum ; Uncertainty
Language	English
Publishing date	2018-10-22
Publishing country	United States
Document type	Journal Article ; Comment
ZDB-ID	1420596-8
ISSN	1546-1726 ; 1097-6256
ISSN (online)	1546-1726
ISSN	1097-6256
DOI	10.1038/s41593-018-0258-2
Database	MEDical Literature Analysis and Retrieval System OnLINE

In stock of ZB MED Cologne/Königswinter

Zs.A 4891: Show issues			Location: Je nach Verfügbarkeit (siehe Angabe bei Bestand) bis Jg. 1994: Bestellungen von Artikeln über das Online-Bestellformular Jg. 1995 - 2021: Lesesall (2.OG) ab Jg. 2022: Lesesaal (EG)
Zs.MG 67: Show issues

Order via subito

This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

Details ▾
- See ZB MED holdings
- Order with fees

Article ; Online: Linear reinforcement learning in planning, grid fields, and cognitive control.

Piray, Payam / Daw, Nathaniel D

Nature communications

2021 Volume 12, Issue 1, Page(s) 4942

Abstract: It is thought that the brain's judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. ...

Abstract	It is thought that the brain's judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. Building on control engineering, here we introduce a model for decision making in the brain that reuses a temporally abstracted map of future events to enable biologically-realistic, flexible choice at the expense of specific, quantifiable biases. It replaces the classic nonlinear, model-based optimization with a linear approximation that softly maximizes around (and is weakly biased toward) a default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably flexible replanning with biases and cognitive control. It also provides insight into how the brain can represent maps of long-distance contingencies stably and componentially, as in entorhinal response fields, and exploit them to guide choice even under changing goals.
MeSH term(s)	Brain/physiology ; Cognition ; Decision Making/physiology ; Humans ; Learning/physiology ; Models, Neurological ; Reinforcement, Psychology
Language	English
Publishing date	2021-08-16
Publishing country	England
Document type	Journal Article ; Research Support, Non-U.S. Gov't ; Research Support, U.S. Gov't, Non-P.H.S.
ZDB-ID	2553671-0
ISSN	2041-1723 ; 2041-1723
ISSN (online)	2041-1723
ISSN	2041-1723
DOI	10.1038/s41467-021-25123-3
Database	MEDical Literature Analysis and Retrieval System OnLINE

Order via subito

Article: Context-sensitive valuation and learning.

Hunter, Lindsay E / Daw, Nathaniel D

Current opinion in behavioral sciences

2021 Volume 41, Page(s) 122–127

Abstract: A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible ... ...

Abstract	A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible choices; in other situations, like choosing between options learned in different contexts, irrational choices can result. We argue that what unites and demystifies these various phenomena is that the brain's goal is not assessing utility as an end in itself, but rather comparing different options to choose the better one. In the presence of uncertainty, noise, or costly computation, adjusting options to the context can produce more accurate choices.
Language	English
Publishing date	2021-06-09
Publishing country	Netherlands
Document type	Journal Article
ISSN	2352-1546
ISSN	2352-1546
DOI	10.1016/j.cobeha.2021.05.001
Database	MEDical Literature Analysis and Retrieval System OnLINE

Order via subito

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Article ; Online: A model for learning based on the joint estimation of stochasticity and volatility.

Piray, Payam / Daw, Nathaniel D

Nature communications

2021 Volume 12, Issue 1, Page(s) 6587

Abstract: Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. However, ... ...

Abstract	Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. However, learning rates are jointly determined by the comparison between volatility and a second factor, moment-to-moment stochasticity. Yet much previous research has focused on simplified cases corresponding to estimation of either factor alone. Here, we introduce a learning model, in which both factors are learned simultaneously from experience, and use the model to simulate human and animal data across many seemingly disparate neuroscientific and behavioral phenomena. By considering the full problem of joint estimation, we highlight a set of previously unappreciated issues, arising from the mutual interdependence of inference about volatility and stochasticity. This interdependence complicates and enriches the interpretation of previous results, such as pathological learning in individuals with anxiety and following amygdala damage.
MeSH term(s)	Algorithms ; Animals ; Anxiety ; Anxiety Disorders ; Decision Making ; Haplorhini ; Humans ; Learning/physiology ; Uncertainty ; Volatilization
Language	English
Publishing date	2021-11-15
Publishing country	England
Document type	Journal Article ; Research Support, Non-U.S. Gov't ; Research Support, U.S. Gov't, Non-P.H.S.
ZDB-ID	2553671-0
ISSN	2041-1723 ; 2041-1723
ISSN (online)	2041-1723
ISSN	2041-1723
DOI	10.1038/s41467-021-26731-9
Database	MEDical Literature Analysis and Retrieval System OnLINE

Order via subito

Article: Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning.

Venditto, Sarah Jo C / Miller, Kevin J / Brody, Carlos D / Daw, Nathaniel D

bioRxiv : the preprint server for biology

2024

Abstract: Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) ...

Abstract	Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step,reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.
Language	English
Publishing date	2024-03-05
Publishing country	United States
Document type	Preprint
DOI	10.1101/2024.02.28.582617
Database	MEDical Literature Analysis and Retrieval System OnLINE

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Details ▾

Article ; Online: An item response theory analysis of the matrix reasoning item bank (MaRs-IB).

Zorowitz, Samuel / Chierchia, Gabriele / Blakemore, Sarah-Jayne / Daw, Nathaniel D

Behavior research methods

2023 Volume 56, Issue 3, Page(s) 1104–1122

Abstract: Matrix reasoning tasks are among the most widely used measures of cognitive ability in the behavioral sciences, but the lack of matrix reasoning tests in the public domain complicates their use. Here, we present an extensive investigation and ... ...

Abstract	Matrix reasoning tasks are among the most widely used measures of cognitive ability in the behavioral sciences, but the lack of matrix reasoning tests in the public domain complicates their use. Here, we present an extensive investigation and psychometric validation of the matrix reasoning item bank (MaRs-IB), an open-access set of matrix reasoning items. In a first study, we calibrate the psychometric functioning of the items in the MaRs-IB in a large sample of adult participants (N = 1501). Using additive multilevel item structure models, we establish that the MaRs-IB has many desirable psychometric properties: its items span a wide range of difficulty, possess medium-to-large levels of discrimination, and exhibit robust associations between item complexity and difficulty. However, we also find that item clones are not always psychometrically equivalent and cannot be assumed to be exchangeable. In a second study, we demonstrate how experimenters can use the estimated item parameters to design new matrix reasoning tests using optimal item assembly. Specifically, we design and validate two new sets of test forms in an independent sample of adults (N = 600). We find these new tests possess good reliability and convergent validity with an established measure of matrix reasoning. We hope that the materials and results made available here will encourage experimenters to use the MaRs-IB in their research.
MeSH term(s)	Adult ; Humans ; Reproducibility of Results ; Problem Solving ; Cognition ; Psychometrics ; Surveys and Questionnaires
Language	English
Publishing date	2023-04-05
Publishing country	United States
Document type	Journal Article
ZDB-ID	231560-9
ISSN	1554-3528 ; 0743-3808 ; 1554-351X
ISSN (online)	1554-3528
ISSN	0743-3808 ; 1554-351X
DOI	10.3758/s13428-023-02067-8
Database	MEDical Literature Analysis and Retrieval System OnLINE

Full text online

Accessible to users with ZB MED library card

In stock of ZB MED Bonn / Germany

Z 76/512: Show issues

Order via subito

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Details ▾

Article ; Online: Uncertainty alters the balance between incremental learning and episodic memory.

Nicholas, Jonathan / Daw, Nathaniel D / Shohamy, Daphna

eLife

2022 Volume 11

Abstract: A key question in decision making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus ... ...

Abstract	A key question in decision making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.
Language	English
Publishing date	2022-12-02
Publishing country	England
Document type	Journal Article
ZDB-ID	2687154-3
ISSN	2050-084X ; 2050-084X
ISSN (online)	2050-084X
ISSN	2050-084X
DOI	10.7554/eLife.81679
Database	MEDical Literature Analysis and Retrieval System OnLINE

Order via subito

Article ; Online: A simple model for learning in volatile environments.

Piray, Payam / Daw, Nathaniel D

PLoS computational biology

2020 Volume 16, Issue 7, Page(s) e1007963

Abstract: Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over ... ...

Abstract	Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.
MeSH term(s)	Algorithms ; Computer Simulation ; Humans ; Learning/physiology ; Models, Psychological ; Signal Processing, Computer-Assisted ; Task Performance and Analysis
Language	English
Publishing date	2020-07-01
Publishing country	United States
Document type	Journal Article ; Research Support, Non-U.S. Gov't ; Research Support, U.S. Gov't, Non-P.H.S.
ZDB-ID	2193340-6
ISSN	1553-7358 ; 1553-734X
ISSN (online)	1553-7358
ISSN	1553-734X
DOI	10.1371/journal.pcbi.1007963
Database	MEDical Literature Analysis and Retrieval System OnLINE

Order via subito

To top

More links

Kategorien

Inter-library loan at ZB MED

More links

Kategorien

Inter-library loan at ZB MED

More links

Kategorien

In stock of ZB MED Cologne/Königswinter

Order via subito

More links

Kategorien

Order via subito

More links

Kategorien

Order via subito

Inter-library loan at ZB MED

More links

Kategorien

Order via subito

More links

Kategorien

Inter-library loan at ZB MED

Full text online

More links

Kategorien

In stock of ZB MED Bonn / Germany

Order via subito

Inter-library loan at ZB MED

More links

Kategorien

Order via subito

More links

Kategorien

Order via subito