Go to AAAI 2025 Workshop AI4Research homepageDYNA-SKILL: Dynamic Self-Prompting Knowledge Graphs for Improving Logical Reasoning in Language Models
Keywords: nlp, graph dataset, language model
TL;DR: We introduce a Custom Graph Dataset with dynamically generated relations to enhance logical reasoning in language models, surpassing the limitations of predefined relations in commonsense knowledge graphs like ATOMIC and COMET
Abstract: Despite recent advances in large language models (LLMs), logical reasoning remains a challenging area, particularly for complex, multi-step reasoning in open-domain contexts. To address this, we introduce the Custom Graph Dataset, a novel graph-based knowledge resource designed to enhance LLMs’ reasoning capabilities. Using a Self-Prompting mechanism, our approach automatically generates both predefined and dynamic relations, creating a dual-triple structure (Head-Relation-Tail and Tail-Dynamic Relation-Additional Tail) that supports richer multi-step reasoning. This Self-Prompting-driven process captures a broad and adaptable range of logical connections, combining predefined relational knowledge with dynamically generated, context-specific relations. Experimental results demonstrate that models finetuned on this dataset significantly outperform both baseline and control models, particularly on reasoning-intensive
benchmarks like Commonsense QA, Riddle Sense, and ARC Challenge. Notably, the dataset includes 133 unique dynamic relations, such as Analogous, Contextual, and Complementary, which contribute to nuanced, context-sensitive reasoning. While general-purpose data offers benefits for some tasks, our findings validate that a targeted, logic-specific dataset can substantially improve LLMs’ reasoning
skills. This work underscores the potential of flexible, Self-Prompting-generated knowledge structures to advance LLM reasoning capabilities, suggesting future directions in combining structured and unstructured data to optimize inference.
Archival Option: The authors of this submission do *not* want it to appear in the archival proceedings.
Submission Number: 20
Loading