Deciphering microbial gene function using natural language processing

The rapid accumulation of genomic data, particularly from metagenomics, presents a significant challenge: deciphering the function of numerous uncharacterized microbial genes. These genes hold immense potential for biotechnology and medicine, with applications ranging from genome manipulation tools to antimicrobial development. Previous research has shown that the genomic context – the genes surrounding a gene of interest – provides valuable clues about its function. This is particularly true for prokaryotes where co-functioning genes are often clustered. The CRISPR-Cas system serves as a prime example, where the co-occurrence of *cas* genes signifies a specific system type. Natural Language Processing (NLP) leverages contextual information to understand meaning in text, drawing parallels to gene function prediction. NLP models trained on vast text corpora learn semantic relationships between words, generating 'word embeddings' that capture meaning. Recently, NLP approaches have been applied to protein sequences and biosynthetic gene clusters, demonstrating their potential in biological contexts. This paper explores applying NLP at a higher level of representation, modeling 'gene semantics' based on gene family co-occurrence within genomic context.

The field has seen various attempts to infer gene function from genomic context, including approaches like analyzing conserved bidirectionally transcribed gene pairs and using gene neighborhood analysis to predict functional associations. Several studies have highlighted the importance of genomic context in prokaryotes, particularly in identifying functional gene clusters. The CRISPR-Cas system's various types have been analyzed extensively, illustrating how gene co-occurrence reveals functional relationships. In the realm of NLP, techniques using word embeddings generated by models like word2vec have revolutionized text analysis. Previous applications of NLP to biological problems included predicting protein properties based on amino acid context, classifying biosynthetic gene clusters using Pfam domains, and applying NLP to DNA k-mers for taxonomic classification. However, a universal model of "gene semantics" utilizing complete genes as the unit of analysis was lacking. This study aims to address this gap by applying NLP to an unprecedented scale of microbial genomic data.

The researchers compiled a comprehensive genomic corpus from publicly available assembled metagenomes and genomes (excluding plants, fungi, and animals), encompassing over 2.5 terabases of data. After removing redundancies and short contigs, the dataset contained ~360 million genes. Genes were clustered into families based on KEGG ortholog groups and sequence similarity. Gene families with sufficient representation (≥24 genes) were considered 'words' in the genomic corpus, creating a vocabulary of 563,589 'words'. The word2vec algorithm was employed to generate gene embeddings, creating a 'gene annotation space' where genes with similar contexts are positioned closely. The researchers then trained four classifiers – support vector machine (SVM), random forest, XGBoost, and deep neural network (DNN) – using the gene embeddings to predict functional categories. A taxonomy-based cross-validation was performed to evaluate the model's performance on unseen genomes, effectively testing the model's generalizability across evolutionary distances. A leave-one-taxonomic-group-out cross-validation strategy was implemented to address potential taxonomic biases. The DNN model was selected due to its superior performance and efficiency. Following training, the model was used to predict the function of 56,617 unannotated gene families. The reliability of predictions was assessed based on both prediction score and classifier accuracy for the respective functional categories. A rarefaction analysis was conducted to determine the discovery potential of different functional categories, estimating how many additional gene families could be discovered within each category. Finally, co-occurring gene families with similar predicted functions were examined to identify potentially novel systems.

The study found that genes with similar functions tend to cluster together in the gene embedding space. However, the analysis also revealed instances where genes with identical annotations were located distantly, suggesting functional specialization in different genomic contexts. The DNN classifier demonstrated high performance in predicting gene function, particularly for categories with strong contextual constraints (e.g., secretion systems, prokaryotic defense systems). The accuracy for predicting recently discovered prokaryotic defense systems was exceptionally high (98.6%). The rarefaction analysis highlighted the significant discovery potential for prokaryotic defense systems, secretion systems, and two-component systems, indicating a large number of uncharacterized genes in these categories. The analysis revealed two putative secretion-related systems: one in three *Clostridium* genera and another associated with the type IV pilus system in *Veillonella*. Further, the study identified a novel, widespread putative prokaryotic defense system containing DNA binding and cleaving domains. This defense system, present in various bacterial kingdoms, displayed two types with different gene contents. The core components included a Z1 domain protein, a PD-(D/E)XK motif protein and a cytosine methyltransferase. Comparison with established homology-based methods (PSI-BLAST, HMMER, and HHblits) demonstrated that the NLP approach was often superior or comparable, particularly in detecting genes lacking significant sequence homology.

The results demonstrate the effectiveness of using genomic context to infer gene function, even across large evolutionary distances. The high accuracy in predicting defense systems, particularly those recently discovered, validates the approach's ability to identify novel genes without relying on sequence homology. The findings highlight the significant 'dark matter' of uncharacterized genes in categories like defense and secretion systems, indicating vast potential for future discovery. The identification of new putative bacterial membrane-bound machineries and a widespread prokaryotic defense system showcases the practical implications of this method. The study's success in analyzing an extremely large dataset using a relatively simple NLP architecture (word2vec) suggests the potential for even greater success with more advanced models. The differences in prediction accuracy across different functional categories might be attributed to differences in contextual signal strength and the degree to which genes are shared between different pathways. The rarefaction analysis provides a powerful tool for prioritizing future research efforts toward under-explored gene categories.

This study introduces a powerful new method for deciphering microbial gene function using NLP techniques. The method's high accuracy, particularly for recently discovered defense systems, and its ability to identify functional categories with high discovery potential highlight its value. The discovery of novel secretion systems and a prokaryotic defense system demonstrates its practical utility. Future research could explore incorporating more sophisticated NLP architectures, integrating additional genomic information (e.g., promoter and terminator sequences), and combining gene embeddings with other features to enhance predictive power and further unveil the intricacies of the microbial world.

The study's filtering of infrequent genes and short contigs might have excluded some rare genes and genes located on small mobile elements. The use of standard gene predictions might have missed small open reading frames (ORFs). The choice of functional categories and the level of detail in the KEGG hierarchy might influence the results. Finally, while the DNN model demonstrated high performance, more advanced architectures could potentially improve accuracy.