Author(s)

Junhong Chen ^1,2, Kaihui Peng ³

Affiliation(s)

¹ School of Software Engineering, South China University of Technology, Guangzhou, China
² LeiHuo Studio, NetEase, Hangzhou, China
³ Faculty of Business and Economics, University of Malaya, Kuala Lumpur, Malaysia

Corresponding Author

Junhong Chen

ABSTRACT

This paper proposes a text similarity calculation model based on multi-scale convolutional neural networks and attention mechanisms. The model is capable of extracting information at different granularities within the text, enabling it to learn from multiple layers of information and thereby improving the accuracy of the text similarity calculation task. After training, the model can generate sentence vectors suitable for cosine similarity computation, which allows the model to pre-generate vectors for the text in the repository. During actual retrieval, only the sentence vector of the text to be searched is needed to calculate similarity with the pre-generated vectors in the repository.

KEYWORDS

Text similarity calculation; Attention mechanism; Pre-trained model

CITE THIS PAPER

Junhong Chen, Kaihui Peng, Siamese Network-Based Text Similarity Algorithm Research. Journal of Artificial Intelligence Practice (2024) Vol. 7: 123-131. DOI: http://dx.doi.org/10.23977/jaip.2024.070315.

REFERENCES

[1] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space [J]. arXiv preprint arXiv:1301.3781, 2013.
[2] Ramos J. Using tf-idf to determine word relevance in document queries[C]//Proceedings of the first instructional conference on machine learning. 2003, 242(1): 29-48. 
[3] Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for chinese bert[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.
[4] Koch G, Zemel R, Salakhutdinov R. Siamese neural networks for one-shot image recognition[C]//ICML deep learning workshop. 2015, 2(1): 1-30.
[5] Cai Z, Fan Q, Feris R S, et al. A unified multi-scale deep convolutional neural network for fast object detection[C]//Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer International Publishing, 2016: 354-370.
[6] Vaswani A. Attention is all you need [J]. Advances in Neural Information Processing Systems, 2017.
[7] Huang A. Similarity measures for text document clustering[C]//Proceedings of the sixth new zealand computer science research student conference (NZCSRSC2008), Christchurch, New Zealand. 2008, 4: 9-56.
[8] Mairal J, Koniusz P, Harchaoui Z, et al. Convolutional kernel networks[J]. Advances in neural information processing systems, 2014, 27.
[9] Niu Z, Zhong G, Yu H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62.
[10] Yu D, Wang H, Chen P, et al. Mixed pooling for convolutional neural networks[C]//Rough Sets and Knowledge Technology: 9th International Conference, RSKT 2014, Shanghai, China, October 24-26, 2014, Proceedings 9. Springer International Publishing, 2014: 364-375.
[11] Liu X, Chen Q, Deng C, et al. Lcqmc: A large-scale chinese question matching corpus[C]//Proceedings of the 27th International Conference on Computational Linguistics. 2018: 1952-1962.
[12] Chen J, Chen Q, Liu X, et al. The bq corpus: A large-scale domain-specific chinese corpus for sentence semantic equivalence identification[C]//Proceedings of the 2018 conference on empirical methods in natural language processing. 2018: 4946-4951.
[13] Joulin A, Grave E, Bojanowski P, et al. Bag of tricks for efficient text classification[J]. arXiv preprint arXiv:1607.01759, 2016.
[14] Reimers N. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[J]. arXiv preprint arXiv:1908.10084, 2019.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Computing, Performance and Communication Systems
  
  
• Advances in Computer, Signals and Systems
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Automation and Machine Learning
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks