LIVIVO - The Search Portal for Life Sciences

zur deutschen Oberfläche wechseln
Back to previous search Search history
Advanced search

Your last searches

  1. AU="Gilmer, Justin"
  2. AU="Routh, Andrew L"

Search results

Result 1 - 10 of total 27

Search options

  1. Book ; Online: Replacing softmax with ReLU in Vision Transformers

    Wortsman, Mitchell / Lee, Jaehoon / Gilmer, Justin / Kornblith, Simon

    2023  

    Abstract: Previous research observed accuracy degradation when replacing the attention softmax with a point-wise activation such as ReLU. In the context of vision transformers, we find that this degradation is mitigated when dividing by sequence length. Our ... ...

    Abstract Previous research observed accuracy degradation when replacing the attention softmax with a point-wise activation such as ReLU. In the context of vision transformers, we find that this degradation is mitigated when dividing by sequence length. Our experiments training small to large vision transformers on ImageNet-21k indicate that ReLU-attention can approach or match the performance of softmax-attention in terms of scaling behavior as a function of compute.
    Keywords Computer Science - Computer Vision and Pattern Recognition ; Computer Science - Machine Learning
    Publishing date 2023-09-15
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  2. Article ; Online: MoSDeF-GOMC: Python Software for the Creation of Scientific Workflows for the Monte Carlo Simulation Engine GOMC.

    Crawford, Brad / Timalsina, Umesh / Quach, Co D / Craven, Nicholas C / Gilmer, Justin B / McCabe, Clare / Cummings, Peter T / Potoff, Jeffrey J

    Journal of chemical information and modeling

    2023  Volume 63, Issue 4, Page(s) 1218–1228

    Abstract: MoSDeF-GOMC is a python interface for the Monte Carlo software GOMC to the Molecular Simulation Design Framework (MoSDeF) ecosystem. MoSDeF-GOMC automates the process of generating initial coordinates, assigning force field parameters, and writing ... ...

    Abstract MoSDeF-GOMC is a python interface for the Monte Carlo software GOMC to the Molecular Simulation Design Framework (MoSDeF) ecosystem. MoSDeF-GOMC automates the process of generating initial coordinates, assigning force field parameters, and writing coordinate (PDB), connectivity (PSF), force field parameter, and simulation control files. The software lowers entry barriers for novice users while allowing advanced users to create complex workflows that encapsulate simulation setup, execution, and data analysis in a single script. All relevant simulation parameters are encoded within the workflow, ensuring reproducible simulations. MoSDeF-GOMC's capabilities are illustrated through a number of examples, including prediction of the adsorption isotherm for CO
    MeSH term(s) Workflow ; Monte Carlo Method ; Ecosystem ; Computer Simulation ; Software
    Language English
    Publishing date 2023-02-15
    Publishing country United States
    Document type Journal Article ; Research Support, U.S. Gov't, Non-P.H.S.
    ZDB-ID 190019-5
    ISSN 1549-960X ; 0095-2338
    ISSN (online) 1549-960X
    ISSN 0095-2338
    DOI 10.1021/acs.jcim.2c01498
    Database MEDical Literature Analysis and Retrieval System OnLINE

    More links

    Kategorien

    In stock of ZB MED Cologne/Königswinter

    Zs.A 1230: Show issues Location:
    Je nach Verfügbarkeit (siehe Angabe bei Bestand)
    bis Jg. 1994: Bestellungen von Artikeln über das Online-Bestellformular
    Jg. 1995 - 2021: Lesesall (1.OG)
    ab Jg. 2022: Lesesaal (EG)

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

  3. Book ; Online: MNIST-C

    Mu, Norman / Gilmer, Justin

    A Robustness Benchmark for Computer Vision

    2019  

    Abstract: We introduce the MNIST-C dataset, a comprehensive suite of 15 corruptions applied to the MNIST test set, for benchmarking out-of-distribution robustness in computer vision. Through several experiments and visualizations we demonstrate that our ... ...

    Abstract We introduce the MNIST-C dataset, a comprehensive suite of 15 corruptions applied to the MNIST test set, for benchmarking out-of-distribution robustness in computer vision. Through several experiments and visualizations we demonstrate that our corruptions significantly degrade performance of state-of-the-art computer vision models while preserving the semantic content of the test images. In contrast to the popular notion of adversarial robustness, our model-agnostic corruptions do not seek worst-case performance but are instead designed to be broad and diverse, capturing multiple failure modes of modern models. In fact, we find that several previously published adversarial defenses significantly degrade robustness as measured by MNIST-C. We hope that our benchmark serves as a useful tool for future work in designing systems that are able to learn robust feature representations that capture the underlying semantics of the input.
    Keywords Computer Science - Computer Vision and Pattern Recognition ; Computer Science - Machine Learning
    Subject code 006
    Publishing date 2019-06-05
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  4. Article ; Online: High-throughput screening of tribological properties of monolayer films using molecular dynamics and machine learning.

    Quach, Co D / Gilmer, Justin B / Pert, Daniel / Mason-Hogans, Akanke / Iacovella, Christopher R / Cummings, Peter T / McCabe, Clare

    The Journal of chemical physics

    2022  Volume 156, Issue 15, Page(s) 154902

    Abstract: Monolayer films have shown promise as a lubricating layer to reduce friction and wear of mechanical devices with separations on the nanoscale. These films have a vast design space with many tunable properties that can affect their tribological ... ...

    Abstract Monolayer films have shown promise as a lubricating layer to reduce friction and wear of mechanical devices with separations on the nanoscale. These films have a vast design space with many tunable properties that can affect their tribological effectiveness. For example, terminal group chemistry, film composition, and backbone chemistry can all lead to films with significantly different tribological properties. This design space, however, is very difficult to explore without a combinatorial approach and an automatable, reproducible, and extensible workflow to screen for promising candidate films. Using the Molecular Simulation Design Framework (MoSDeF), a combinatorial screening study was performed to explore 9747 unique monolayer films (116 964 total simulations) and a machine learning (ML) model using a random forest regressor, an ensemble learning technique, to explore the role of terminal group chemistry and its effect on tribological effectiveness. The most promising films were found to contain small terminal groups such as cyano and ethylene. The ML model was subsequently applied to screen terminal group candidates identified from the ChEMBL small molecule library. Approximately 193 131 unique film candidates were screened with approximately a five order of magnitude speed-up in analysis compared to simulation alone. The ML model was thus able to be used as a predictive tool to greatly speed up the initial screening of promising candidate films for future simulation studies, suggesting that computational screening in combination with ML can greatly increase the throughput in combinatorial approaches to generate in silico data and then train ML models in a controlled, self-consistent fashion.
    MeSH term(s) Friction ; High-Throughput Screening Assays ; Machine Learning ; Molecular Dynamics Simulation
    Language English
    Publishing date 2022-04-22
    Publishing country United States
    Document type Journal Article
    ZDB-ID 3113-6
    ISSN 1089-7690 ; 0021-9606
    ISSN (online) 1089-7690
    ISSN 0021-9606
    DOI 10.1063/5.0080838
    Database MEDical Literature Analysis and Retrieval System OnLINE

    More links

    Kategorien

    In stock of ZB MED Bonn / Germany

    Z 4' 49/16: Show issues

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  5. Book ; Online: Improving Training Stability for Multitask Ranking Models in Recommender Systems

    Tang, Jiaxi / Drori, Yoel / Chang, Daryl / Sathiamoorthy, Maheswaran / Gilmer, Justin / Wei, Li / Yi, Xinyang / Hong, Lichan / Chi, Ed H.

    2023  

    Abstract: Recommender systems play an important role in many content platforms. While most recommendation research is dedicated to designing better models to improve user experience, we found that research on stabilizing the training for such models is severely ... ...

    Abstract Recommender systems play an important role in many content platforms. While most recommendation research is dedicated to designing better models to improve user experience, we found that research on stabilizing the training for such models is severely under-explored. As recommendation models become larger and more sophisticated, they are more susceptible to training instability issues, i.e., loss divergence, which can make the model unusable, waste significant resources and block model developments. In this paper, we share our findings and best practices we learned for improving the training stability of a real-world multitask ranking model for YouTube recommendations. We show some properties of the model that lead to unstable training and conjecture on the causes. Furthermore, based on our observations of training dynamics near the point of training instability, we hypothesize why existing solutions would fail, and propose a new algorithm to mitigate the limitations of existing solutions. Our experiments on YouTube production dataset show the proposed algorithm can significantly improve training stability while not compromising convergence, comparing with several commonly used baseline methods.

    Comment: Accepted at KDD 2023; 12 pages
    Keywords Computer Science - Machine Learning
    Subject code 006
    Publishing date 2023-02-17
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  6. Book ; Online: Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

    Choi, Dami / Xin, Derrick / Dadkhahi, Hamid / Gilmer, Justin / Garg, Ankush / Firat, Orhan / Yeh, Chih-Kuan / Dai, Andrew M. / Ghorbani, Behrooz

    2023  

    Abstract: In this paper, we empirically study the optimization dynamics of multi-task learning, particularly focusing on those that govern a collection of tasks with significant data imbalance. We present a simple yet effective method of pre-training on high- ... ...

    Abstract In this paper, we empirically study the optimization dynamics of multi-task learning, particularly focusing on those that govern a collection of tasks with significant data imbalance. We present a simple yet effective method of pre-training on high-resource tasks, followed by fine-tuning on a mixture of high/low-resource tasks. We provide a thorough empirical study and analysis of this method's benefits showing that it achieves consistent improvements relative to the performance trade-off profile of standard static weighting. We analyze under what data regimes this method is applicable and show its improvements empirically in neural machine translation (NMT) and multi-lingual language modeling.
    Keywords Computer Science - Computation and Language ; Computer Science - Machine Learning
    Publishing date 2023-12-11
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  7. Article ; Online: MoSDeF, a Python Framework Enabling Large-Scale Computational Screening of Soft Matter: Application to Chemistry-Property Relationships in Lubricating Monolayer Films.

    Summers, Andrew Z / Gilmer, Justin B / Iacovella, Christopher R / Cummings, Peter T / McCabe, Clare

    Journal of chemical theory and computation

    2020  Volume 16, Issue 3, Page(s) 1779–1793

    Abstract: We demonstrate how the recently developed Python-based Molecular Simulation and Design Framework (MoSDeF) can be used to perform molecular dynamics screening of functionalized monolayer films, focusing on tribological effectiveness. MoSDeF is an open- ... ...

    Abstract We demonstrate how the recently developed Python-based Molecular Simulation and Design Framework (MoSDeF) can be used to perform molecular dynamics screening of functionalized monolayer films, focusing on tribological effectiveness. MoSDeF is an open-source package that allows for the programmatic construction and parametrization of soft matter systems and enables TRUE (transferable, reproducible, usable by others, and extensible) simulations. The MoSDeF-enabled screening identifies several film chemistries that simultaneously show low coefficients of friction and adhesion. We additionally develop a Python library that utilizes the RDKit cheminformatics library and the scikit-learn machine learning library that allows for the development of predictive models for the tribology of functionalized monolayer films and use this model to extract information on terminal group characteristics that most influence tribology, based on the screening data.
    Language English
    Publishing date 2020-03-02
    Publishing country United States
    Document type Journal Article
    ISSN 1549-9626
    ISSN (online) 1549-9626
    DOI 10.1021/acs.jctc.9b01183
    Database MEDical Literature Analysis and Retrieval System OnLINE

    More links

    Kategorien

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  8. Book ; Online: Adversarial Examples Are a Natural Consequence of Test Error in Noise

    Ford, Nic / Gilmer, Justin / Carlini, Nicolas / Cubuk, Dogus

    2019  

    Abstract: Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to small ... ...

    Abstract Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to small modifications of a correctly handled input. Less surprisingly, image classifiers also lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise. In this paper we provide both empirical and theoretical evidence that these are two manifestations of the same underlying phenomenon, establishing close connections between the adversarial robustness and corruption robustness research programs. This suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions. Based on our results we recommend that future adversarial defenses consider evaluating the robustness of their methods to distributional shift with benchmarks such as Imagenet-C.
    Keywords Computer Science - Machine Learning ; Computer Science - Computer Vision and Pattern Recognition ; Statistics - Machine Learning
    Subject code 006
    Publishing date 2019-01-29
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  9. Article ; Online: Examining the self-assembly of patchy alkane-grafted silica nanoparticles using molecular simulation.

    Craven, Nicholas C / Gilmer, Justin B / Spindel, Caroline J / Summers, Andrew Z / Iacovella, Christopher R / McCabe, Clare

    The Journal of chemical physics

    2021  Volume 154, Issue 3, Page(s) 34903

    Abstract: In this work, molecular dynamics simulations are used to examine the self-assembly of anisotropically coated "patchy" nanoparticles. Specifically, we use a coarse-grained model to examine silica nanoparticles coated with alkane chains, where the poles of ...

    Abstract In this work, molecular dynamics simulations are used to examine the self-assembly of anisotropically coated "patchy" nanoparticles. Specifically, we use a coarse-grained model to examine silica nanoparticles coated with alkane chains, where the poles of the grafted nanoparticle are bare, resulting in strongly attractive patches. Through a systematic screening process, the patchy nanoparticles are found to form dispersed, string-like, and aggregated phases, dependent on the combination of alkane chain length, coating chain density, and the fractional coated surface area. Correlation analysis is used to identify the ability of various particle descriptors to predict bulk phase behavior from more computationally efficient single grafted nanoparticle simulations and demonstrates that the solvent-accessible surface area of the nanoparticle core is a key predictor of bulk phase behavior. The results of this work enhance our knowledge of the phase space of patchy nanoparticles and provide a powerful approach for future screening of these materials.
    Language English
    Publishing date 2021-01-26
    Publishing country United States
    Document type Journal Article
    ZDB-ID 3113-6
    ISSN 1089-7690 ; 0021-9606
    ISSN (online) 1089-7690
    ISSN 0021-9606
    DOI 10.1063/5.0032658
    Database MEDical Literature Analysis and Retrieval System OnLINE

    More links

    Kategorien

    In stock of ZB MED Bonn / Germany

    Z 4' 49/16: Show issues

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  10. Book ; Online: Pre-training helps Bayesian optimization too

    Wang, Zi / Dahl, George E. / Swersky, Kevin / Lee, Chansoo / Mariet, Zelda / Nado, Zachary / Gilmer, Justin / Snoek, Jasper / Ghahramani, Zoubin

    2022  

    Abstract: Bayesian optimization (BO) has become a popular strategy for global optimization of many expensive real-world functions. Contrary to a common belief that BO is suited to optimizing black-box functions, it actually requires domain knowledge on ... ...

    Abstract Bayesian optimization (BO) has become a popular strategy for global optimization of many expensive real-world functions. Contrary to a common belief that BO is suited to optimizing black-box functions, it actually requires domain knowledge on characteristics of those functions to deploy BO successfully. Such domain knowledge often manifests in Gaussian process priors that specify initial beliefs on functions. However, even with expert knowledge, it is not an easy task to select a prior. This is especially true for hyperparameter tuning problems on complex machine learning models, where landscapes of tuning objectives are often difficult to comprehend. We seek an alternative practice for setting these functional priors. In particular, we consider the scenario where we have data from similar functions that allow us to pre-train a tighter distribution a priori. To verify our approach in realistic model training setups, we collected a large multi-task hyperparameter tuning dataset by training tens of thousands of configurations of near-state-of-the-art models on popular image and text datasets, as well as a protein sequence dataset. Our results show that on average, our method is able to locate good hyperparameters at least 3 times more efficiently than the best competing methods.

    Comment: ICML2022 Workshop on Adaptive Experimental Design and Active Learning in the Real World. arXiv admin note: substantial text overlap with arXiv:2109.08215
    Keywords Computer Science - Machine Learning ; Computer Science - Artificial Intelligence ; Statistics - Machine Learning
    Subject code 006
    Publishing date 2022-07-07
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

To top