Developing and evaluating an IT specification extraction system

Abstract

Purpose - This paper seeks to establish an extraction system for an information technology (IT) product specification named ITSIES which combines the natural language process (NLP) with the ontology concept and also to evaluate the system's effectiveness in advance. Design/methodology/approach - The development of the system is based on a prototype design and performance validation. This study adopts four classes of IT specification (PC, Unix server, Monitor, and Printer) that follow IBM's and HP's product lines as the baseline information in order to construct the extraction system in GATE (General Architecture for Text Engineering) tools and to examine the IT product specification with other brands and patterns. Additionally indices are adopted such as precision, recall, and F-measure as the matrices for evaluating system performance. Findings - The performance shows that the average recall, precision, and F-measure are all over 90 per cent, revealing that the JAPE (Java Annotation Patterns Engine) grammar rules in the IT domain are reasonably good and generally in line with expectations. Originality/value - The paper proposes an integrative framework to examine IT product specification information and demonstrates that the system is effective for IT application.