Yuna Hur  Heuiseok Lim

Vol. 1,  No. 1, pp. 1-18, Aug.  2024
10.23246/AAIRJ.2024.01.01.03

Deep Learning document similarity Machine Learning named entity recognition Security Threat Information Keyword Extraction summarization topic modeling

PDF

As intrusion attack techniques become more intelligent and sophisticated, security incidents are increasing. In order to predict and respond to cyber attacks, a number of security companies quickly identify the methods, types and characteristics of attack techniques and are publishing Security Intelligence Reports(SIRs) on them. However, the SIRs are not formatted for each security company, and a large number of unstructured SIRs are publishing ever-increasing. In this paper, we propose a framework that uses five analytic techniques to formulate a report and extract key information in order to reduce the time required to extract information on large unstructured reports (SIRs) efficiently. Since there is no set of correct answers for SIRs data, we introduce 4 analytical techniques through unsupervised learning method. 4 analytical technologies include Keyword Extraction model, which extracts keywords, Topic Modeling, which classifies the contents of the SIRs, Summarization Model, which summarizes the SIRs contents, and Document Similarity, which searches the documents most similar to the SIRs. Finally, data is constructed to extract threat information from SIRs. We apply it to Named Entity Recognition (NER) to recognize a word belonging to IP, Domain / URL, Hash, and Malware and propose a model to judge which type of word belongs.