Graph-Theoretic Characterization of Knowledge Representation in Large Language Models
Keywords:
Large Language Models, Graph Theory, Knowledge Representation, Semantic Networks, Knowledge Graphs, Explainable Artificial Intelligence, Network Science, Natural Language ProcessingAbstract
Large Language Models (LLMs) have achieved remarkable success in natural language processing by learning rich semantic representations from large-scale textual corpora. However, understanding how knowledge is structurally organized within these models remains a significant challenge. This study proposes a novel graph-theoretic framework for characterizing knowledge representation in LLMs by transforming model-generated semantic information into weighted knowledge graphs and analyzing their topological properties. The proposed framework integrates entity extraction, semantic relation identification, graph construction, and network analysis to evaluate semantic organization across multiple state-of-the-art LLMs. Experiments were conducted using four benchmark datasets WikiText, SQuAD, Natural Questions, and PubMedQA and five representative LLMs, namely GPT, LLaMA, Gemini, Qwen, and DeepSeek. The generated semantic graphs were evaluated using graph-theoretic metrics, including degree centrality, graph density, clustering coefficient, modularity, PageRank, and graph entropy, together with conventional NLP metrics such as precision, recall, F1-score, semantic similarity, and hallucination rate. Experimental results show that Gemini achieved the best overall performance with a Graph Score of 97.4 and an NLP Score of 96.9, followed by DeepSeek and GPT, while statistical analysis confirmed significant differences in graph topology and semantic organization (p < 0.05) across the evaluated models. These findings demonstrate that graph-theoretic characterization provides deeper insights into knowledge connectivity, semantic coherence, and structural complexity than conventional embedding-based evaluation methods. The proposed framework offers an interpretable, scalable, and effective approach for evaluating knowledge representation in LLMs, contributing to explainable artificial intelligence and graph-based assessment of trustworthy language models.
References
Bhatt, A., Vaghela, N., & Dudhia, K. (2024). Generating knowledge graphs from large language models: A comparative study of GPT-4, LLaMA 2, and BERT. arXiv. https://doi.org/10.48550/arXiv.2412.07412
DeepSeek-AI. (2024). DeepSeek LLM: Scaling open-source language models with long-termism. arXiv. https://doi.org/10.48550/arXiv.2401.02954
DeLong, L. N., Mir, R. F., & Fleuriot, J. D. (2025). Neurosymbolic AI for reasoning over knowledge graphs: A survey. IEEE Transactions on Neural Networks and Learning Systems, 36(5), 7822-7842. https://doi.org/10.1109/TNNLS.2024.3420218
Fan, W., Wang, S., Huang, J., Chen, Z., Song, Y., Tang, W., Mao, H., Liu, H., Liu, X., Yin, D., & Li, Q. (2024). Graph machine learning in the era of large language models. arXiv. https://doi.org/10.48550/arXiv.2404.14928
OpenAI. (2023). GPT-4 technical report. arXiv. https://doi.org/10.48550/arXiv.2303.08774
Ren, X., Tang, J., Yin, D., Chawla, N., & Huang, C. (2024). A survey of large language models for graphs. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 6616-6626. https://doi.org/10.1145/3637528.3671460
Touvron, H., Lavril, T., Izacard, G., et al. (2023). LLaMA: Open and efficient foundation language models. arXiv. https://doi.org/10.48550/arXiv.2302.13971
Team Gemini. (2023). Gemini: A family of highly capable multimodal models. arXiv. https://doi.org/10.48550/arXiv.2312.11805
Team Qwen. (2023). Qwen technical report. arXiv. https://doi.org/10.48550/arXiv.2309.16609
Wu, X., Li, Y., Zhang, Z., & Chen, H. (2024). A review of graph neural networks and pretrained language models for knowledge graph reasoning. Neurocomputing, 609, 128490. https://doi.org/10.1016/j.neucom.2024.128490
Published
Issue
Section
Copyright (c) 2026 Journal of Advanced Research in Business Law and Technology Management
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
We, the undersigned, give an undertaking to the following effect with regard to our article entitled
“_______________________________________________________________________________________________________________________________________________________________________________
________________________________________________________________________________” submitted for publication in (Journal title)________________________________________________ _______________________________________________________Vol.________, Year _________:-
1. The article mentioned above has not been published or submitted to or accepted for publication in any form, in any other journal.
2. We also vouchsafe that the authorship of this article will not be contested by anyone whose name(s) is/are not listed by us here.
3. I/We declare that I/We contributed significantly towards the research study i.e., (a) conception, design and/or analysis and interpretation of data and to (b) drafting the article or revising it critically for important intellectual content and on (c) final approval of the version to be published.
4. I/We hereby acknowledge ADRs conflict of interest policy requirement to scrupulously avoid direct and indirect conflicts of interest and, accordingly, hereby agree to promptly inform the editor or editor's designee of any business, commercial, or other proprietary support, relationships, or interests that I/We may have which relate directly or indirectly to the subject of the work.
5. I/We also agree to the authorship of the article in the following sequence:-
Authors' Names (in sequence) Signature of Authors
1. _____________________________________ _____________________________________
2. _____________________________________ _____________________________________
3. _____________________________________ _____________________________________
4. _____________________________________ _____________________________________
5. _____________________________________ _____________________________________
6. _____________________________________ _____________________________________
7. _____________________________________ _____________________________________
8. _____________________________________ _____________________________________
Important
(I). All the authors are required to sign independently in this form in the sequence given above. In case an author has left the institution/ country and whose whereabouts are not known, the senior author may sign on his/ her behalf taking the responsibility.
(ii). No addition/ deletion/ or any change in the sequence of the authorship will be permissible at a later stage, without valid reasons and permission of the Editor.
(iii). If the authorship is contested at any stage, the article will be either returned or will not be
processed for publication till the issue is solved.
