Note Non arxiv papers: Empowering language models with knowledge graph reasoning for open-domain question answering(https://aclanthology.org/2022.emnlp-main.650.pdf) An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks(https://doi.org/10.48550/arXiv.2210.16773) B. Abu-Salih, “Domain-specific knowledge graphs: A survey,” Journal of Network and Computer Applications, vol. 185, p. 103076, 2021.(https://arxiv.org/abs/2011.00235)

Note attempt to encode the graph structure into implicit feature sequences as part of the input sequence [42]. Unlike the previous verbalizing approaches, this usually involves training a graph encoder to encode the graph structure into a sequence of features and fine-tuning the LLMs to adapt to the new input format. [42] demonstrates drastic performance improvement on problems including substructure counting, maximum triplet sum, shortest path, and bipartite matching, indicating that fine-tuning LLM

Note published at AAAI 2024.

Note "GreaseLM [67] proposes to have a language encoding component and a graph encoding component in each layer. These two components interact through a modality-fusion layer (MInt layer), where a special structure token is added to the text Transformer input, and a special node is added to the graph encoding layer."