LIVIVO - The Search Portal for Life Sciences

zur deutschen Oberfläche wechseln
Advanced search

Search results

Result 1 - 10 of total 454

Search options

  1. Book ; Online: Semantic Specialization for Knowledge-based Word Sense Disambiguation

    Mizuki, Sakae / Okazaki, Naoaki

    2023  

    Abstract: A promising approach for knowledge-based Word Sense Disambiguation (WSD) is to select the sense whose contextualized embeddings computed for its definition sentence are closest to those computed for a target word in a given sentence. This approach relies ...

    Abstract A promising approach for knowledge-based Word Sense Disambiguation (WSD) is to select the sense whose contextualized embeddings computed for its definition sentence are closest to those computed for a target word in a given sentence. This approach relies on the similarity of the \textit{sense} and \textit{context} embeddings computed by a pre-trained language model. We propose a semantic specialization for WSD where contextualized embeddings are adapted to the WSD task using solely lexical knowledge. The key idea is, for a given sense, to bring semantically related senses and contexts closer and send different/unrelated senses farther away. We realize this idea as the joint optimization of the Attract-Repel objective for sense pairs and the self-training objective for context-sense pairs while controlling deviations from the original embeddings. The proposed method outperformed previous studies that adapt contextualized embeddings. It achieved state-of-the-art performance on knowledge-based WSD when combined with the reranking heuristic that uses the sense inventory. We found that the similarity characteristics of specialized embeddings conform to the key idea. We also found that the (dis)similarity of embeddings between the related/different/unrelated senses correlates well with the performance of WSD.

    Comment: Accepted by EACL 2023. 14 pages
    Keywords Computer Science - Computation and Language
    Subject code 006 ; 401
    Publishing date 2023-04-22
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  2. Book ; Online: Reducing Sequence Length by Predicting Edit Operations with Large Language Models

    Kaneko, Masahiro / Okazaki, Naoaki

    2023  

    Abstract: Large Language Models (LLMs) have demonstrated remarkable performance in various tasks and gained significant attention. LLMs are also used for local sequence transduction tasks, including grammatical error correction (GEC) and formality style transfer, ... ...

    Abstract Large Language Models (LLMs) have demonstrated remarkable performance in various tasks and gained significant attention. LLMs are also used for local sequence transduction tasks, including grammatical error correction (GEC) and formality style transfer, where most tokens in a source text are kept unchanged. However, it is inefficient to generate all target tokens because a prediction error of a target token may cause a catastrophe in predicting subsequent tokens and because the computational cost grows quadratically with the target sequence length. This paper proposes to predict a set of edit operations for the source text for local sequence transduction tasks. Representing an edit operation with a span of the source text and changed tokens, we can reduce the length of the target sequence and thus the computational cost for inference. We apply instruction tuning for LLMs on the supervision data of edit operations. Experiments show that the proposed method achieves comparable performance to the baseline in four tasks, paraphrasing, formality style transfer, GEC, and text simplification, despite reducing the length of the target text by as small as 21\%. Furthermore, we report that the instruction tuning with the proposed method achieved the state-of-the-art performance in the four tasks.

    Comment: Work in progress
    Keywords Computer Science - Computation and Language
    Subject code 006
    Publishing date 2023-05-19
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  3. Book ; Online: Causal Reasoning through Two Layers of Cognition for Improving Generalization in Visual Question Answering

    Nguyen, Trang / Okazaki, Naoaki

    2023  

    Abstract: Generalization in Visual Question Answering (VQA) requires models to answer questions about images with contexts beyond the training distribution. Existing attempts primarily refine unimodal aspects, overlooking enhancements in multimodal aspects. ... ...

    Abstract Generalization in Visual Question Answering (VQA) requires models to answer questions about images with contexts beyond the training distribution. Existing attempts primarily refine unimodal aspects, overlooking enhancements in multimodal aspects. Besides, diverse interpretations of the input lead to various modes of answer generation, highlighting the role of causal reasoning between interpreting and answering steps in VQA. Through this lens, we propose Cognitive pathways VQA (CopVQA) improving the multimodal predictions by emphasizing causal reasoning factors. CopVQA first operates a pool of pathways that capture diverse causal reasoning flows through interpreting and answering stages. Mirroring human cognition, we decompose the responsibility of each stage into distinct experts and a cognition-enabled component (CC). The two CCs strategically execute one expert for each stage at a time. Finally, we prioritize answer predictions governed by pathways involving both CCs while disregarding answers produced by either CC, thereby emphasizing causal reasoning and supporting generalization. Our experiments on real-life and medical data consistently verify that CopVQA improves VQA performance and generalization across baselines and domains. Notably, CopVQA achieves a new state-of-the-art (SOTA) on PathVQA dataset and comparable accuracy to the current SOTA on VQA-CPv2, VQAv2, and VQA RAD, with one-fourth of the model size.
    Keywords Computer Science - Artificial Intelligence
    Subject code 120
    Publishing date 2023-10-09
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  4. Article ; Online: Reducing anastomotic leakage in TaTME by mucosal coverage of staple lines: a pilot study with preliminary results.

    Deguchi, Katsuya / Hirano, Yasumitsu / Okazaki, Naoto

    BMC surgery

    2023  Volume 23, Issue 1, Page(s) 155

    Abstract: Purpose: We have performed a single stapled anastomosis with double purse-string sutures as a Trans anal Total Mesorectal Excision (TaTME) reconstruction for low rectal cancer. We report an attempt to control local infection and reduce anastomotic ... ...

    Abstract Purpose: We have performed a single stapled anastomosis with double purse-string sutures as a Trans anal Total Mesorectal Excision (TaTME) reconstruction for low rectal cancer. We report an attempt to control local infection and reduce anastomotic leakage (AL) at this anastomotic site.
    Patients and methods: Fifty-one patients who underwent TaTME for low rectal cancer from April 2021 to October 2022 were included. TaTME was performed by two teams, and reconstruction was performed by anastomosis with a single stapling technique (SST). After the anastomosis was thoroughly cleaned, Z sutures were placed parallel to the staple line to suture the mucosa on the oral and anal side of the staple line and to cover the staple line circumferentially. Data on operative time, Distal Margin (DM), recurrence and postoperative complications including AL were prospectively collected.
    Results: The mean age of patients was 67 years. There were 36 males and 15 females. The overall mean operative time was 283.1 min, and the mean Distal Margin was 2.2 cm. Postoperative complications were observed in 5.9% of the patients, but no AL was observed, nor any serious complications with Clavien-Dindo ≥ 3 grade. Of the 49 cases excluding Stage 4, postoperative recurrence was observed in 2 cases (4.9%).
    Conclusion: In patients with lower rectal cancer who underwent TaTME, additional mucosal coverage of the anastomotic staple line by transanal manipulation after reconstruction may be associated with a reduction in the incidence of postoperative AL. Further studies including late anastomotic complications are needed.
    MeSH term(s) Male ; Female ; Humans ; Aged ; Anastomotic Leak/etiology ; Rectum/surgery ; Pilot Projects ; Rectal Neoplasms/surgery ; Rectal Neoplasms/complications ; Postoperative Complications/epidemiology ; Laparoscopy/methods ; Treatment Outcome
    Language English
    Publishing date 2023-06-10
    Publishing country England
    Document type Journal Article
    ZDB-ID 2050442-1
    ISSN 1471-2482 ; 1471-2482
    ISSN (online) 1471-2482
    ISSN 1471-2482
    DOI 10.1186/s12893-023-02071-x
    Database MEDical Literature Analysis and Retrieval System OnLINE

    More links

    Kategorien

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

  5. Book ; Online: Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting

    Kaneko, Masahiro / Bollegala, Danushka / Okazaki, Naoaki / Baldwin, Timothy

    2024  

    Abstract: There exist both scalable tasks, like reading comprehension and fact-checking, where model performance improves with model size, and unscalable tasks, like arithmetic reasoning and symbolic reasoning, where model performance does not necessarily improve ... ...

    Abstract There exist both scalable tasks, like reading comprehension and fact-checking, where model performance improves with model size, and unscalable tasks, like arithmetic reasoning and symbolic reasoning, where model performance does not necessarily improve with model size. Large language models (LLMs) equipped with Chain-of-Thought (CoT) prompting are able to make accurate incremental predictions even on unscalable tasks. Unfortunately, despite their exceptional reasoning abilities, LLMs tend to internalize and reproduce discriminatory societal biases. Whether CoT can provide discriminatory or egalitarian rationalizations for the implicit information in unscalable tasks remains an open question. In this study, we examine the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks. For this purpose, we construct a benchmark for an unscalable task where the LLM is given a list of words comprising feminine, masculine, and gendered occupational words, and is required to count the number of feminine and masculine words. In our CoT prompts, we require the LLM to explicitly indicate whether each word in the word list is a feminine or masculine before making the final predictions. With counting and handling the meaning of words, this benchmark has characteristics of both arithmetic reasoning and symbolic reasoning. Experimental results in English show that without step-by-step prediction, most LLMs make socially biased predictions, despite the task being as simple as counting words. Interestingly, CoT prompting reduces this unconscious social bias in LLMs and encourages fair predictions.
    Keywords Computer Science - Computation and Language
    Subject code 410
    Publishing date 2024-01-28
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  6. Article ; Online: Short-term and Long-term Outcomes After Laparoscopic Surgery for Pathological Stage T4a and T4b Colon Cancer.

    Ishiyama, Yasuhiro / Hirano, Yasumitsu / Tanaka, Hiroto / Fujii, Takatsugu / Okazaki, Naoto / Hiranuma, Chikashi / Deguchi, Katsuya

    Journal of gastrointestinal cancer

    2024  

    Language English
    Publishing date 2024-01-30
    Publishing country United States
    Document type Journal Article
    ZDB-ID 2452514-5
    ISSN 1941-6636 ; 1559-0739 ; 1941-6628 ; 1537-3649
    ISSN (online) 1941-6636 ; 1559-0739
    ISSN 1941-6628 ; 1537-3649
    DOI 10.1007/s12029-024-01017-7
    Database MEDical Literature Analysis and Retrieval System OnLINE

    Full text online

    More links

    Kategorien

    In stock of ZB MED Cologne/Königswinter

    Zs.A 2184: Show issues Location:
    Je nach Verfügbarkeit (siehe Angabe bei Bestand)
    bis Jg. 1994: Bestellungen von Artikeln über das Online-Bestellformular
    Jg. 1995 - 2021: Lesesall (1.OG)
    ab Jg. 2022: Lesesaal (EG)

    Order via subito

    This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.

  7. Book ; Online: DREEAM

    Ma, Youmi / Wang, An / Okazaki, Naoaki

    Guiding Attention with Evidence for Improving Document-Level Relation Extraction

    2023  

    Abstract: Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the relationship between an entity pair, has been shown to help DocRE systems ... ...

    Abstract Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the relationship between an entity pair, has been shown to help DocRE systems focus on relevant texts, thus improving relation extraction. However, evidence retrieval (ER) in DocRE faces two major issues: high memory consumption and limited availability of annotations. This work aims at addressing these issues to improve the usage of ER in DocRE. First, we propose DREEAM, a memory-efficient approach that adopts evidence information as the supervisory signal, thereby guiding the attention modules of the DocRE system to assign high weights to evidence. Second, we propose a self-training strategy for DREEAM to learn ER from automatically-generated evidence on massive data without evidence annotations. Experimental results reveal that our approach exhibits state-of-the-art performance on the DocRED benchmark for both DocRE and ER. To the best of our knowledge, DREEAM is the first approach to employ ER self-training.

    Comment: Accepted by EACL 2023
    Keywords Computer Science - Computation and Language
    Subject code 004
    Publishing date 2023-02-16
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  8. Book ; Online: Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples

    Kaneko, Masahiro / Bollegala, Danushka / Okazaki, Naoaki

    2023  

    Abstract: Numerous types of social biases have been identified in pre-trained language models (PLMs), and various intrinsic bias evaluation measures have been proposed for quantifying those social biases. Prior works have relied on human annotated examples to ... ...

    Abstract Numerous types of social biases have been identified in pre-trained language models (PLMs), and various intrinsic bias evaluation measures have been proposed for quantifying those social biases. Prior works have relied on human annotated examples to compare existing intrinsic bias evaluation measures. However, this approach is not easily adaptable to different languages nor amenable to large scale evaluations due to the costs and difficulties when recruiting human annotators. To overcome this limitation, we propose a method to compare intrinsic gender bias evaluation measures without relying on human-annotated examples. Specifically, we create multiple bias-controlled versions of PLMs using varying amounts of male vs. female gendered sentences, mined automatically from an unannotated corpus using gender-related word lists. Next, each bias-controlled PLM is evaluated using an intrinsic bias evaluation measure, and the rank correlation between the computed bias scores and the gender proportions used to fine-tune the PLMs is computed. Experiments on multiple corpora and PLMs repeatedly show that the correlations reported by our proposed method that does not require human annotated examples are comparable to those computed using human annotated examples in prior work.

    Comment: EACL 2023
    Keywords Computer Science - Computation and Language
    Publishing date 2023-01-27
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  9. Book ; Online: SAIE Framework

    Loem, Mengsay / Kaneko, Masahiro / Okazaki, Naoaki

    Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks

    2023  

    Abstract: Large Language Models (LLMs) can justify or criticize their predictions through discussion with other models or humans, thereby enhancing their intrinsic understanding of instances. While proactive discussions enhance performance, this approach is ... ...

    Abstract Large Language Models (LLMs) can justify or criticize their predictions through discussion with other models or humans, thereby enhancing their intrinsic understanding of instances. While proactive discussions enhance performance, this approach is currently limited to the inference phase. In this context, we posit a hypothesis: learning interactive discussions during training can improve understanding for the instances in the training step and proficiency in logical/critical thinking ability and verbalized expression of the model in the inference step. Our proposed SAIE training method involves both supportive and adversarial discussions between the learner and partner models. The learner model receives a remark from the partner through the discussion, and the parameters of the learner model are then updated based on this remark. That is, the teacher signal dynamically adjusts in response to the evolving model output throughout the training step. By bolstering the capacity for discussion and comprehension of instances, our experiments across datasets, including GSM8K, CommonsenseQA, and MMLU, reveal that models fine-tuned with our method consistently surpass those trained with standard fine-tuning techniques. Moreover, our approach demonstrates superior performance in multi-agent inference scenarios, boosting the models' reasoning abilities at the inference step.

    Comment: Work in progress
    Keywords Computer Science - Computation and Language
    Subject code 006
    Publishing date 2023-11-14
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

  10. Book ; Online: How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection

    Koike, Ryuto / Kaneko, Masahiro / Okazaki, Naoaki

    2023  

    Abstract: Against the misuse (e.g., plagiarism or spreading misinformation) of Large Language Models (LLMs), many recent works have presented LLM-generated-text detectors with promising detection performance. Spotlighting a situation where users instruct LLMs to ... ...

    Abstract Against the misuse (e.g., plagiarism or spreading misinformation) of Large Language Models (LLMs), many recent works have presented LLM-generated-text detectors with promising detection performance. Spotlighting a situation where users instruct LLMs to generate texts (e.g., essay writing), there are various ways to write the instruction (e.g., what task-oriented constraint to include). In this paper, we discover that even a task-oriented constraint in instruction can cause the inconsistent performance of current detectors to the generated texts. Specifically, we focus on student essay writing as a realistic domain and manually create the task-oriented constraint for each factor on essay quality by Ke and Ng (2019). Our experiment shows that the detection performance variance of the current detector on texts generated by instruction with each task-oriented constraint is up to 20 times larger than the variance caused by generating texts multiple times and paraphrasing the instruction. Our finding calls for further research on developing robust detectors that can detect such distributional shifts caused by a task-oriented constraint in the instruction.
    Keywords Computer Science - Computation and Language
    Subject code 420
    Publishing date 2023-11-14
    Publishing country us
    Document type Book ; Online
    Database BASE - Bielefeld Academic Search Engine (life sciences selection)

    Full text online

    More links

    Kategorien

    Inter-library loan at ZB MED

    Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.

To top