Date of Graduation

5-2021

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Engineering (PhD)

Degree Level

Graduate

Department

Computer Science & Computer Engineering

Advisor/Mentor

Thompson, Dale R.

Committee Member

Panda, Brajendra N.

Second Committee Member

Cronan, Timothy P.

Third Committee Member

Li, Qinghua

Keywords

Cybersecurity; DNS Tunneling; DNS-based attacks; Intrusion detection system; Network Security

Abstract

Individuals and organizations rely on the Internet as an essential environment for personal or business transactions. However, individuals and organizations have been primary targets for attacks that steal sensitive data. Adversaries can use different approaches to hide their activities inside the compromised network and communicate covertly between the malicious servers and the victims. The domain name system (DNS) protocol is one of these approaches that adversaries use to transfer stolen data outside the organization's network using various forms of DNS tunneling attacks. The main reason for targeting the DNS protocol is because DNS is available in almost every network, ignored, and rarely monitored. In this work, the primary aim is to design a reliable and robust network-based solution as a detection system against DNS-based attacks using various techniques, including visualization, machine learning techniques, and statistical analysis. The network-based solution acts as a DNS proxy server that provides DNS services as well as detection and prevention against DNS-based attacks, which are either embedded in malware or used as stand-alone attacking tools. The detection system works in two modes: real-time and offline modes. The real-time mode relies on the developed Payload Analysis (PA) module. In contrast, the offline mode operates based on two of the contributed modules in this dissertation, including the visualization and Traffic Analysis (TA) modules. We conducted various experiments in order to test and evaluate the detection system against simulated real-world attacks. Overall, the detection system achieved high accuracy of 99.8% with no false-negative rate. To validate the method, we compared the developed detection system against the open-source detection system, Snort intrusion detection system (IDS). We evaluated the two detection systems using a confusion matrix, including the recall, false-negatives rate, accuracy, and others. The detection system detects all case scenarios of the attacks while Snort missed 50% of the performed attacks. Based on the results, we can conclude that the detection system is significant and original improvement of the present methods used for detecting and preventing DNS-based attacks.

Download

Share

COinS