Date of Graduation

5-2024

Document Type

Thesis

Degree Name

Bachelor of Science in Computer Science

Degree Level

Undergraduate

Department

Computer Science and Computer Engineering

Advisor/Mentor

Li, Qinghua

Committee Member/Reader

Gauch, Susan

Committee Member/Second Reader

Panda, Brajendra

Abstract

Artificial intelligence has progressed rapidly in recent years, greatly revolutionizing the world and helping to automate increasingly complex tasks. However, there are still some disciplines where the problem of automation has not yet been thoroughly tackled, such as in software vulnerability management. Vulnerability management is a critical component of cybersecurity for an organization. Until recently, learning about a vulnerability required a security operator to manually search and read through online information and security advisories to find needed information. Doing so is a significantly time-consuming task, as these advisories tend to be quite lengthy and packed with information about various technologies. To simplify this process, we first present a new approach that can extract crucial information about a vulnerability from provided documents using large language models (LLMs) such as ChatGPT, generating a concise summary of the vulnerability from the documents. This approach can save much of the operator’s time that would need to be spent reading the lengthy documents. While this approach works well for those that already have documents about vulnerabilities to summarize, operators still need to find relevant documents for the vulnerability of interest first, e.g., through web search. To further automate this process, we then introduce a modified implementation of the basic summarization approach as a web browser extension, which can automatically generate a summary about a vulnerability of interest from web search results. That way, an operator only needs to manually type a search about a vulnerability, e.g., “CVE-2023-31147”, in a search engine, and then our solution can automatically generate a concise summary of valuable information about the vulnerability. This enhancement integrates a robust and automated source gathering method and introduces several new filtering parameters that allow users to specify requirements pertaining to the documents and summaries generated by the approach. The results of these approaches are then evaluated on their ability to include various important pieces of information in the generated summaries. We find that the basic approach of summarizing over provided documents performs exceptionally well. The web browser extension has a slight decrease in accuracy as the cost of automation but is still very accurate.

Keywords

Large Language Models; Prompt Engineering; Automatic Summarization; Vulnerability Management

Download

Share

COinS