Author(s)

Shubin Li, Rongfeng Zheng, Anmin Zhou and Liang Liu

Corresponding Author

Liang Liu

ABSTRACT

Buffer overflow attack is one of the typical attacks over the internet, it aims to make the overflow overwrite the legitimate data. How to detect Buffer overflow attack becomes a hot topic in research. Protocol uncertainties and varying attack modes will seriously affect the efficiency and the accuracy of attack detection. Recently, machine learning is widely applied in network traffic detection and data processing, meanwhile tradition buffer overflow detection method based on feature matching is difficult to detect the attack load hidden in network traffic. In order to address these challenges, we apply machine learning to detect remote buffer overflow attack, to enhance the classification ability of machine learning models, we propose a unique set of feature extraction rules after analyzing a large number of attack codes. This method is not only capable of identifying various attack forms and unknown attack types but also has the ability of supporting protocol independent detection and identifying attacks using various protocols based on TCP or UDP. In order to evaluate the performance of our method, three machine learning algorithms are selected to establish detection models and the model base on random forest algorithm perform best., moreover, comparison experiments with other detection methods are also carried out based on the same dataset. The experimental results show our method can detect remote buffer overflow attack with adaptability, efficiency and accuracy.

KEYWORDS

Buffer overflow; machine learning; network traffic detection; feature extraction