LIVIVO - Search results -

Search results

Result 1 - 10 of total 10

Search options

Book ; Online: Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning

Waelchli, Daniel / Weber, Pascal / Koumoutsakos, Petros

2023

Abstract: The discovery of individual objectives in collective behavior of complex dynamical systems such as fish schools and bacteria colonies is a long-standing challenge. Inverse reinforcement learning is a potent approach for addressing this challenge but its ... ...

Abstract	The discovery of individual objectives in collective behavior of complex dynamical systems such as fish schools and bacteria colonies is a long-standing challenge. Inverse reinforcement learning is a potent approach for addressing this challenge but its applicability to dynamical systems, involving continuous state-action spaces and multiple interacting agents, has been limited. In this study, we tackle this challenge by introducing an off-policy inverse multi-agent reinforcement learning algorithm (IMARL). Our approach combines the ReF-ER techniques with guided cost learning. By leveraging demonstrations, our algorithm automatically uncovers the reward function and learns an effective policy for the agents. Through extensive experimentation, we demonstrate that the proposed policy captures the behavior observed in the provided data, and achieves promising results across problem domains including single agent models in the OpenAI gym and multi-agent models of schooling behavior. The present study shows that the proposed IMARL algorithm is a significant step towards understanding collective dynamics from the perspective of its constituents, and showcases its value as a tool for studying complex physical systems exhibiting collective behaviour.
Keywords	Computer Science - Machine Learning ; Computer Science - Artificial Intelligence ; Computer Science - Multiagent Systems
Subject code	006
Publishing date	2023-05-17
Publishing country	us
Document type	Book ; Online
Database	BASE - Bielefeld Academic Search Engine (life sciences selection)

Full text online

Full text

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Book ; Online: Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence

Mojgani, Rambod / Waelchli, Daniel / Guan, Yifei / Koumoutsakos, Petros / Hassanzadeh, Pedram

2023

Abstract: Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as ... ...

Abstract	Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as small-scale eddies in atmospheric and oceanic turbulence. Thus, such small-scale processes have to be represented as a function of the resolved scales via closures (parametrization). The accuracy of these closures is particularly important for capturing climate extremes. Traditionally, such closures are based on heuristics and simplifying assumptions about the unresolved physics. Recently, supervised-learned closures, trained offline on high-fidelity data, have been shown to outperform the classical physics-based closures. However, this approach requires a significant amount of high-fidelity training data and can also lead to instabilities. Reinforcement learning is emerging as a potent alternative for developing such closures as it requires only low-order statistics and leads to stable closures. In Scientific Multi-Agent Reinforcement Learning (SMARL) computational elements serve a dual role of discretization points and learning agents. We leverage SMARL and fundamentals of turbulence physics to learn closures for prototypes of atmospheric and oceanic turbulence. The policy is trained using only the enstrophy spectrum, which is nearly invariant and can be estimated from a few high-fidelity samples (these few samples are far from enough for supervised/offline learning). We show that these closures lead to stable low-resolution simulations that, at a fraction of the cost, can reproduce the high-fidelity simulations' statistics, including the tails of the probability density functions. The results demonstrate the high potential of SMARL for closure modeling for GCMs, especially in the regime of scarce data and indirect observations.
Keywords	Computer Science - Machine Learning ; Computer Science - Computational Engineering ; Finance ; and Science ; Physics - Atmospheric and Oceanic Physics ; Physics - Computational Physics ; Physics - Fluid Dynamics
Subject code	006
Publishing date	2023-12-01
Publishing country	us
Document type	Book ; Online
Database	BASE - Bielefeld Academic Search Engine (life sciences selection)

Full text online

Full text

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Book ; Online: Remember and Forget Experience Replay for Multi-Agent Reinforcement Learning

Weber, Pascal / Wälchli, Daniel / Zeqiri, Mustafa / Koumoutsakos, Petros

2022

Abstract: We present the extension of the Remember and Forget for Experience Replay (ReF-ER) algorithm to Multi-Agent Reinforcement Learning (MARL). {ReF-ER} was shown to outperform state of the art algorithms for continuous control in problems ranging from the ... ...

Abstract	We present the extension of the Remember and Forget for Experience Replay (ReF-ER) algorithm to Multi-Agent Reinforcement Learning (MARL). {ReF-ER} was shown to outperform state of the art algorithms for continuous control in problems ranging from the OpenAI Gym to complex fluid flows. In MARL, the dependencies between the agents are included in the state-value estimator and the environment dynamics are modeled via the importance weights used by ReF-ER. In collaborative environments, we find the best performance when the value is estimated using individual rewards and we ignore the effects of other actions on the transition map. We benchmark the performance of ReF-ER MARL on the Stanford Intelligent Systems Laboratory (SISL) environments. We find that employing a single feed-forward neural network for the policy and the value function in ReF-ER MARL, outperforms state of the art algorithms that rely on complex neural network architectures.
Keywords	Computer Science - Machine Learning ; Computer Science - Artificial Intelligence ; Mathematics - Optimization and Control ; Statistics - Machine Learning
Subject code	006 ; 629
Publishing date	2022-03-24
Publishing country	us
Document type	Book ; Online
Database	BASE - Bielefeld Academic Search Engine (life sciences selection)

Full text online

Full text

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Article ; Online: Data-driven inference of the reproduction number for COVID-19 before and after interventions for 51 European countries.

Karnakov, Petr / Arampatzis, Georgios / Kičić, Ivica / Wermelinger, Fabian / Wälchli, Daniel / Papadimitriou, Costas / Koumoutsakos, Petros

Swiss medical weekly

2020 Volume 150, Page(s) w20313

Abstract: The reproduction number is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. Its estimated value is a measure of the necessity and, eventually, effectiveness of interventions imposed in various countries. Here we present ... ...

Abstract	The reproduction number is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. Its estimated value is a measure of the necessity and, eventually, effectiveness of interventions imposed in various countries. Here we present an online tool for the data-driven inference and quantification of uncertainties for the reproduction number, as well as the time points of interventions for 51 European countries. The study relied on the Bayesian calibration of the SIR model with data from reported daily infections from these countries. The model fitted the data, for most countries, without individual tuning of parameters. We also compared the results of SIR and SEIR models, which give different estimates of the reproduction number, and provided an analytical relationship between the respective numbers. We deployed a Bayesian inference framework with efficient sampling algorithms, to present a publicly available graphical user interface (https://cse-lab.ethz.ch/coronavirus) that allows the user to assess and compare predictions for pairs of European countries. The results quantified the rate of the disease’s spread before and after interventions, and provided a metric for the effectiveness of non-pharmaceutical interventions in different countries. They also indicated how geographic proximity and the times of interventions affected the progression of the epidemic.
MeSH term(s)	Basic Reproduction Number/statistics & numerical data ; Bayes Theorem ; Betacoronavirus/isolation & purification ; COVID-19 ; Communicable Disease Control/methods ; Communicable Disease Control/statistics & numerical data ; Coronavirus Infections/epidemiology ; Coronavirus Infections/prevention & control ; Coronavirus Infections/transmission ; Disease Transmission, Infectious/prevention & control ; Disease Transmission, Infectious/statistics & numerical data ; Epidemiologic Measurements ; Epidemiological Monitoring ; Europe/epidemiology ; Humans ; Pandemics/prevention & control ; Pandemics/statistics & numerical data ; Pneumonia, Viral/epidemiology ; Pneumonia, Viral/prevention & control ; Pneumonia, Viral/transmission ; SARS-CoV-2 ; Uncertainty
Keywords	covid19
Language	English
Publishing date	2020-07-10
Publishing country	Switzerland
Document type	Journal Article
ZDB-ID	2036179-8
ISSN	1424-3997 ; 1424-7860
ISSN (online)	1424-3997
ISSN	1424-7860
DOI	10.4414/smw.2020.20313
Database	MEDical Literature Analysis and Retrieval System OnLINE

In stock of ZB MED Cologne/Königswinter

Ua VI Zs.152: Show issues

Location:
Je nach Verfügbarkeit (siehe Angabe bei Bestand)
bis Jg. 2021: Bestellungen von Artikeln über das Online-Bestellformular
ab Jg. 2022: Lesesaal (EG)

Order via subito

This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

Details ▾
- See ZB MED holdings
- Order with fees

Article ; Online: Optimal allocation of limited test resources for the quantification of COVID-19 infections.

Chatzimanolakis, Michail / Weber, Pascal / Arampatzis, Georgios / Wälchli, Daniel / Kičić, Ivica / Karnakov, Petr / Papadimitriou, Costas / Koumoutsakos, Petros

Swiss medical weekly

2020 Volume 150, Page(s) w20445

Abstract: The systematic identification of infected individuals is critical for the containment of the COVID-19 pandemic. Currently, the spread of the disease is mostly quantified by the reported numbers of infections, hospitalisations, recoveries and deaths; ... ...

Abstract	The systematic identification of infected individuals is critical for the containment of the COVID-19 pandemic. Currently, the spread of the disease is mostly quantified by the reported numbers of infections, hospitalisations, recoveries and deaths; these quantities inform epidemiology models that provide forecasts for the spread of the epidemic and guide policy making. The veracity of these forecasts depends on the discrepancy between the numbers of reported, and unreported yet infectious, individuals. We combine Bayesian experimental design with an epidemiology model and propose a methodology for the optimal allocation of limited testing resources in space and time, which maximises the information gain for such unreported infections. The proposed approach is applicable at the onset and spread of the epidemic and can forewarn of a possible recurrence of the disease after relaxation of interventions. We examine its application in Switzerland; the open source software is, however, readily adaptable to countries around the world. We find that following the proposed methodology can lead to vastly less uncertain predictions for the spread of the disease, thus improving estimates of the effective reproduction number and the future number of unreported infections. This information can provide timely and systematic guidance for the effective identification of infectious individuals and for decision-making regarding lockdown measures and the distribution of vaccines.
MeSH term(s)	Bayes Theorem ; COVID-19/diagnosis ; COVID-19/epidemiology ; COVID-19/prevention & control ; COVID-19/transmission ; COVID-19 Testing/methods ; Communicable Disease Control/methods ; Diagnostic Services/supply & distribution ; Epidemiological Monitoring ; Forecasting ; Health Policy ; Humans ; Random Allocation ; Resource Allocation/methods ; SARS-CoV-2 ; Switzerland/epidemiology
Language	English
Publishing date	2020-12-16
Publishing country	Switzerland
Document type	Journal Article
ZDB-ID	2036179-8
ISSN	1424-3997 ; 1424-7860
ISSN (online)	1424-3997
ISSN	1424-7860
DOI	10.4414/smw.2020.20445
Database	MEDical Literature Analysis and Retrieval System OnLINE

In stock of ZB MED Cologne/Königswinter

Ua VI Zs.152: Show issues

Location:
Je nach Verfügbarkeit (siehe Angabe bei Bestand)
bis Jg. 2021: Bestellungen von Artikeln über das Online-Bestellformular
ab Jg. 2022: Lesesaal (EG)

Order via subito

Details ▾
- See ZB MED holdings
- Order with fees

Article: Data-driven inference of the reproduction number for COVID-19 before and after interventions for 51 European countries

Karnakov, Petr / Arampatzis, Georgios / Kicic, Ivica / Wermelinger, Fabian / Wälchli, Daniel / Papadimitriou, Costas / Koumoutsakos, Petros

Swiss Med Wkly

Abstract	The reproduction number is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. Its estimated value is a measure of the necessity and, eventually, effectiveness of interventions imposed in various countries. Here we present an online tool for the data-driven inference and quantification of uncertainties for the reproduction number, as well as the time points of interventions for 51 European countries. The study relied on the Bayesian calibration of the SIR model with data from reported daily infections from these countries. The model fitted the data, for most countries, without individual tuning of parameters. We also compared the results of SIR and SEIR models, which give different estimates of the reproduction number, and provided an analytical relationship between the respective numbers. We deployed a Bayesian inference framework with efficient sampling algorithms, to present a publicly available graphical user interface (https://cse-lab.ethz.ch/coronavirus) that allows the user to assess and compare predictions for pairs of European countries. The results quantified the rate of the disease’s spread before and after interventions, and provided a metric for the effectiveness of non-pharmaceutical interventions in different countries. They also indicated how geographic proximity and the times of interventions affected the progression of the epidemic.
Keywords	covid19
Publisher	WHO
Document type	Article
Note	WHO #Covidence: #651678
Database	COVID19

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Details ▾

Article ; Online: Data driven inference of the reproduction number (R0) for COVID-19 before and after interventions for 51 European countries

Karnakov, Petr / Arampatzis, Georgios / Kičić, Ivica / Wermelinger, Fabian / Wälchli, Daniel / Papadimitriou, Costas / Koumoutsakos, Petros

medRxiv

Abstract: The reproduction number (R0) is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. The estimation of its value with respect to the key threshold of 1.0 is a measure of the need, and eventually effectiveness, of ... ...

Abstract	The reproduction number (R0) is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. The estimation of its value with respect to the key threshold of 1.0 is a measure of the need, and eventually effectiveness, of interventions imposed in various countries. Here we present an online tool for the data driven inference and quantification of uncertainties for R0 as well as the time points of interventions for 51 European countries. The study relies on the Bayesian calibration of the simple and well established SIR model with data from reported daily infections. The model is able to fit the data for most countries without individual tuning of parameters. We deploy an open source Bayesian inference framework and efficient sampling algorithms to present a publicly available GUI (https://www.cse-lab.ethz.ch/coronavirus/) that allows the user to assess custom data and compare predictions for pairs of European countries. The results provide a ranking based on the rate of the disease9s spread suggesting a metric for the effectiveness of social distancing measures. They also serve to demonstrate how geographic proximity and related times of interventions can lead to similarities in the progression of the epidemic.
Keywords	covid19
Language	English
Publishing date	2020-05-23
Publisher	Cold Spring Harbor Laboratory Press
Document type	Article ; Online
DOI	10.1101/2020.05.21.20109314
Database	COVID19

Full text online

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Article ; Online: Data-driven inference of the reproduction number for COVID-19 before and after interventions for 51 European countries

Karnakov, Petr / Arampatzis, Georgios / Kičić, Ivica / Wermelinger, Fabian / Wälchli, Daniel / Papadimitriou, Costas / Koumoutsakos, Petros

Swiss Medical Weekly, 150

2020

Abstract	The reproduction number is broadly considered as a key indicator for the spreading of the COVID-19 pandemic. Its estimated value is a measure of the necessity and, eventually, effectiveness of interventions imposed in various countries. Here we present an online tool for the data-driven inference and quantification of uncertainties for the reproduction number, as well as the time points of interventions for 51 European countries. The study relied on the Bayesian calibration of the SIR model with data from reported daily infections from these countries. The model fitted the data, for most countries, without individual tuning of parameters. We also compared the results of SIR and SEIR models, which give different estimates of the reproduction number, and provided an analytical relationship between the respective numbers. We deployed a Bayesian inference framework with efficient sampling algorithms, to present a publicly available graphical user interface (https://cse-lab.ethz.ch/coronavirus) that allows the user to assess and compare predictions for pairs of European countries. The results quantified the rate of the disease’s spread before and after interventions, and provided a metric for the effectiveness of non-pharmaceutical interventions in different countries. They also indicated how geographic proximity and the times of interventions affected the progression of the epidemic. ISSN:1424-7860 ISSN:1424-3997
Keywords	COVID-19 ; Bayesian inference ; SIR model ; Interventions ; covid19
Subject code	339
Language	English
Publishing date	2020-07-13
Publisher	EMH Swiss Medical Publishers
Publishing country	ch
Document type	Article ; Online
Database	BASE - Bielefeld Academic Search Engine (life sciences selection)

Full text online

Full text

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Book ; Online: Korali

Martin, Sergio M. / Wälchli, Daniel / Arampatzis, Georgios / Economides, Athena E. / Karnakov, Petr / Koumoutsakos, Petros

Efficient and Scalable Software Framework for Bayesian Uncertainty Quantification and Stochastic Optimization

2020

Abstract: We present Korali, an open-source framework for large-scale Bayesian uncertainty quantification and stochastic optimization. The framework relies on non-intrusive sampling of complex multiphysics models and enables their exploitation for optimization and ...

Abstract	We present Korali, an open-source framework for large-scale Bayesian uncertainty quantification and stochastic optimization. The framework relies on non-intrusive sampling of complex multiphysics models and enables their exploitation for optimization and decision-making. In addition, its distributed sampling engine makes efficient use of massively-parallel architectures while introducing novel fault tolerance and load balancing mechanisms. We demonstrate these features by interfacing Korali with existing high-performance software such as Aphros, Lammps (CPU-based), and Mirheo (GPU-based) and show efficient scaling for up to 512 nodes of the CSCS Piz Daint supercomputer. Finally, we present benchmarks demonstrating that Korali outperforms related state-of-the-art software frameworks. Comment: 12 pages, 12 figures
Keywords	Computer Science - Distributed ; Parallel ; and Cluster Computing ; 60-08 ; G.3
Subject code	006
Publishing date	2020-05-27
Publishing country	us
Document type	Book ; Online
Database	BASE - Bielefeld Academic Search Engine (life sciences selection)

Full text online

Full text

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

Article ; Online: Optimal Testing Strategy for the Identification of COVID-19 Infections

Chatzimanolakis, Michail / Weber, Pascal / Arampatzis, Georgios / Wälchli, Daniel / Karnakov, Petr / Kičić, Ivica / Papadimitriou, Costas / Koumoutsakos, Petros

medRxiv

Abstract: The systematic identification of infectious, yet unreported, individuals is critical for the containment of the COVID-19 pandemic. We present a strategy for identifying the location, timing and extent of testing that maximizes information gain for such ... ...

Abstract	The systematic identification of infectious, yet unreported, individuals is critical for the containment of the COVID-19 pandemic. We present a strategy for identifying the location, timing and extent of testing that maximizes information gain for such infections. The optimal testing strategy relies on Bayesian experimental design and forecasting epidemic models that account for time dependent interventions. It is applicable at the onset and spreading of the epidemic and can forewarn for a possible recurrence of the disease after relaxation of interventions. We examine its application in Switzerland and show that it can provide timely and systematic guidance for the effective identification of infectious individuals with finite testing resources. The methodology and the open source code are readily adaptable to countries around the world.
Keywords	covid19
Language	English
Publishing date	2020-07-26
Publisher	Cold Spring Harbor Laboratory Press
Document type	Article ; Online
DOI	10.1101/2020.07.20.20157818
Database	COVID19

Full text online

Inter-library loan at ZB MED

Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

To top

Full text online

More links

Kategorien

Inter-library loan at ZB MED

Full text online

More links

Kategorien

Inter-library loan at ZB MED

Full text online

More links

Kategorien

Inter-library loan at ZB MED

More links

Kategorien

In stock of ZB MED Cologne/Königswinter

Order via subito

More links

Kategorien

In stock of ZB MED Cologne/Königswinter

Order via subito

Kategorien

Inter-library loan at ZB MED

Full text online

Kategorien

Inter-library loan at ZB MED

Full text online

More links

Kategorien

Inter-library loan at ZB MED

Full text online

More links

Kategorien

Inter-library loan at ZB MED

Full text online

Kategorien

Inter-library loan at ZB MED