Methodology for evaluating CI software packages

Abstract

Competitive intelligence (CI) is important for businesses to remain competitive. Software technologies have been developed to make the CI process simpler. These software technologies need to be able to carry out CI effectively by performing all the stages in the CI Cycle, conforming to the British Standard software quality characteristics, extract information from large data sets by having certain Additional Features and be cost effective. Three evaluation frameworks were developed based on the CI Cycle, British Standard and Additional Features. This methodology and the marketing literature of the software were used to evaluate four CI software packages. Information on cost and availability of a free trial version was also taken from the marketing literature. From the evaluation results, the software were able to support at least one CI Cycle stage and at least one British Standard characteristics but none of them fully had any of the Additional Features. One of them has a free trial version, while two of them had information about the cost of their commercial version. It is recommended that before choosing CI software, CI practitioners determine and prioritize their intelligence needs and then test which CI software can meet them. Also, CI software vendors need to provide more information in their marketing literature on the cost and availability of a free trial version of their software and features pertaining to the CI Cycle, British Standard and Additional Features. The British Standard and Additional Features have not been used in the previous CI software evaluation studies.

Keywords

Additional features Big Data British standards competitive advantage competitive intelligence (CI)information analysis information gathering software quality

Introduction

The purpose of this article is to develop a methodology for evaluating commercial competitive intelligence (CI) software packages. Intelligence is both a process and a product which has been found useful in various fields such as the military and in businesses dating from as far back as 1000BC (Gainor and Bouthillier, 2014; Juhari and Stephens, 2004). In businesses, it is known as CI. According to Gainor and Bouthillier (2014), ‘CI involves surveying and monitoring the competitive environment and developing intelligence to inform decision making’ while according to Fuld & Company (n.d.), CI is ‘the collection and analysis of information to anticipate competitive activity, see past market disruptions and dispassionately interpret events’. From the various definitions given above, it can be seen that the underlying aim of CI is gathering and analysing information so as to facilitate decision-making (Bercik, 2014; Bouthillier and Shearer, 2003; Fuld & Company, n.d.). The benefits of CI to organizations include gaining and maintaining a competitive advantage; discovering potential threats and opportunities which facilitates decision-making; innovation in terms of products, services and technological choices; improving the possibility of patent filing and approval; and enhancing activities such as finance, operations and marketing within the organization (Ahituv et al., 1998; Bercik, 2014; Fuld, 2012; Fuld & Company, n.d.; Rouach and Santi, 2001; Teo and Choo, 2001). In addition to the various definitions of CI, authors and practitioners such as Bose (2008), Fuld & Company (2003, 2006/2007, 2008/2009), Bouthillier and Shearer (2003) and Pellissier and Nenzhelele (2013) have suggested that CI is in stages, with one stage leading to the next, to form what is known as the CI Cycle. This is shown in the various models in Figure 1 below.

Figure 1.

CI Cycle models. (a) SCIP model of the CI Cycle (Source: CI Cycle, Bose, 2008). (b) Fuld & Co., Intelligence Cycle (The Intelligence Cycle, Fuld & Co., 2003; 2006; 2008). (c) Bouthillier and Shearer information processing framework (Source: Information processing model of competitive intelligence cycle, Bouthillier and Shearer, 2013). (d) Universal CI process model (Source: Competitive intelligence process model: Pellissier and Nenzhelele, 2013). CI: competitive intelligence.

Before going ahead to the literature review in the next section, the objectives of this study are considered and then discussed under their relevant headings.

Objectives of the study are:

to identify a range of available CI software;

to determine if these software technologies are able to support the CI Cycle;

to ascertain whether these software packages are of high quality as determined by the British Standards Institute ISO/IEC 25010:2011 requirements;

to determine whether these software packages have additional features and therefore can handle large or unstructured data sets;

to identify if there are commercial and free versions of the software packages;

to identify the differences between the commercial and free versions of the software packages; and

to investigate the cost of the commercial versions of these software packages.

Literature review

Software technologies and CI

Technological advancements have made most businesses to utilize software technologies in carrying out CI (Underwood, 2002); however, not all the software packages which vendors claim can perform CI can really do so. The first objective of the study will, therefore, be addressed here by identifying a range of available CI software.

Previous CI software evaluation studies

The results of past CI evaluation studies have also corroborated the idea that not all CI software packages can actually perform CI functions. For instance, in the studies carried out by Fuld & Company (2003, 2006/2007, 2008/2009) using the Fuld & Co CI model (Figure 1(b) above), with each year, the various CI software packages improved in their capabilities to carry out the various stages in a CI model, especially the planning and direction stage, while information gathering is in two stages – through published information and primary source collection. However, the model lacks in evaluating the information storage capacity of the software packages which other CI models such as Bouthillier and Shearer’s (2003) information processing framework and the Universal process model (Pellissier and Nenzhelele, 2013) shown in Figure 1(c) and (d) above, respectively, are able to do. Similarly, Bouthillier and Shearer (2003) and Juhari (2009) carried out their studies using the information processing framework (shown in Figure 1(c) above). The result of the Bouthillier and Shearer (2003) study revealed that CI software packages did not fully support the stage of analysis of information, even though this stage is regarded as the most important stage of the CI Cycle (Bouthillier and Shearer, 2003; Fuld & Company, 2003). The study by Juhari (2009) showed that the software packages were able to support all of the stages of this model with the exception of two software packages. The shortcoming of Bouthillier and Shearer and Juhari studies are that unlike the Fuld & Company model and the Universal process model, the information processing framework did not take planning and direction into consideration.

Research design

Evaluating CI software based on the CI Cycle

For the second objective (i.e. to determine if these software technologies are able to support the CI Cycle) to be achieved without the shortcomings of the models identified in previous studies, a model which integrates all the models (in Figure 1 above) has been adapted. For the sake of this research, it will be called the integrated CI model and is shown in Figure 2 below.

Figure 2.

Integrated CI model. Source: Adapted from Bose (2008), Fuld & Company (2003, 2006/2007, 2008/2009), Bouthillier and Shearer (2003) and Pellissier and Nenzhelele (2013). CI: competitive intelligence.

Identification of CI needs

CI software should use key intelligence topics (KITs) or key intelligence questions (KIQs) to identify CI needs of the organization and then structure these KITs as part of a workflow process (AWARE, n.d.; Fuld & Company, 2006/2007; Limacher, 2014). This means that the CI software should ask the user questions about the intelligence needs (KITs or KIQs) of the organization. These questions could be on topics such as the name and location of the competitor company as well as competitor products and services. When these questions have been answered by the user, the topics should then be organized on the user’s home or dashboard page.

Information collection

With regard to information collection, CI software should be able to collect information from sources internal to the organization such as discussion forums within the organization, internal databases, marketing plans and technical documents. The software should also be able to collect information from sources external to the organization such as advertisements, websites, products and published information of competitor companies available on online databases such as LexisNexis and Factiva. Additionally, the CI software should filter the information it collects based on the identified KITs and KIQs, eliminate repetitive or redundant information from the sources of information, monitor sources of information based on the user’s requirements and alert users when there are new information sources (AWARE, n.d.; Bouthillier and Shearer, 2003).

Information sorting, capturing and storing

CI software should be capable of effectively capturing qualitative information from various sources. The captured information should then be categorized using an index. The software should also be able to link related information sources together and present it to the user. Lastly, the software should have a database for storing information (Bouthillier and Shearer, 2003; Fuld & Company, 2008/2009; Pellissier and Nenzhelele, 2013).

Information analysis

This stage of the CI Cycle is important to the entire CI process as information is converted to intelligence at this stage. For software to be able to analyse information therefore, it should be able to provide various analytical techniques such as Strengths, Weaknesses, Opportunities and Threats (SWOT), benchmarking, trend analysis and personality profiling tools. The software should be able to summarize the content of information sources, suggest ideas that will inform decision-making and provide visualization tools such as bar charts and tag cloud for mapping data (AWARE, n.d.; Bercik, 2014; Bouthillier and Shearer, 2003; Fuld, 2012; Gainor and Bouthillier, 2014; Jones, 2014; Pellissier and Nenzhelele, 2013).

Intelligence communication

In carrying out this function, CI software should be able to send and receive information from the users of the software. This communication could be through emails, Short Message Service (SMS) alerts, report creation and linking to other programs such as Microsoft Office Word and PDF (AWARE, n.d.; Bercik, 2014; Bouthillier and Shearer, 2003; Fuld & Company, 2006/2007; Pellissier and Nenzhelele, 2013).

Evaluating CI software based on the British Standards

In addition to CI software packages being able to carry out the stages in the CI Cycle effectively, it is important that these software packages are safe, satisfy the user and are able to perform the functions for which they were developed. These characteristics of a software package have been described by the British Standards Institute of being of a high quality (British Standards Institute, 2011) and will be investigated as the third objective of this study (to ascertain whether these software packages are of high quality as determined by the British Standards Institute ISO/IEC 25010:2011 requirements). The various characteristics and how the software packages are expected to conform to them are given below.

Functional suitability

This is defined as ‘the degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions’ (British Standards Institute, 2011). This implies that CI software should be capable of performing the function for which it was purchased within the organization. Since this function is to perform CI, the software packages must be able to carry out the various stages in the CI Cycle as described in the section ‘Evaluating CI software based on the CI Cycle’ above.

Performance efficiency

It is used to measure the performance of the software relative to the amount of resources used under certain conditions (British Standards Institute, 2011). This means that CI software should be able to produce results within the shortest amount of time on whatever system it is being used.

Usability

It is defined as ‘degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’ (British Standards Institute, 2011). This means that CI software should be easy to use without the user having to spend a long amount of time learning how to use it; information should be presented in a visually appealing manner; and it should be possible for encountered errors to be reported to either the user or software vendor where possible (Jadhav and Sonar, 2011). The ability of a CI software to use visualization to present data is also an evaluation criteria in the information analysis stage of the CI Cycle; therefore, for this research, visualization is evaluated under information analysis.

Reliability

‘This is the capability of the software package to perform specified functions for a specified period of time under specified conditions’ (British Standards Institute, 2011). A reliable CI software should provide a means for backing up data, recovering lost information and should be able to run consistently without crashing (Jadhav and Sonar, 2011).

Security

CI software should be secure by providing the right level of authorization to users in order to access information thereby ensuring confidentiality, integrity and authenticity of data and information (British Standards Institute, 2011). This implies that the use of strong passwords is necessary to access a CI software and the software should keep a log of user activities (Jadhav and Sonar, 2011).

Maintainability

Software packages are required to remain effective regardless of the number of modules they have or the number of users who can access them simultaneously (British Standards Institute, 2011; Jadhav and Sonar, 2011). In order for CI software to be regarded as maintainable therefore, it should have modules within it and it should be possible for multiple users to access it simultaneously.

Portability

With different types of hardware specifications and operating system in the market today, software packages are required to be able to work on several different types of operating system or hardware specifications. This means that CI software should be able to work on different types of operating systems such as Microsoft Windows and Mac OS. CI software should also be able to work on different hardware such as 64-bit personal computers (PCs) and mobile devices (British Standards Institute, 2011; Jadhav and Sonar, 2011).

Compatibility

A compatible software is one that can perform its functions and exchange information with other components or systems at the same time while sharing the same hardware or software environment without causing detrimental effect to the user’s PC, a device or the program itself (British Standards Institute, 2011). A compatible CI software should, therefore, be able to function effectively on the user’s PC and work well without causing any detrimental effect to the user’s PC.

Evaluating CI software based on Additional Features

Since the emergence of the Internet has brought about an ‘almost inexhaustible supply of data’ and more recently, Big Data (Anon, 2013; Stonebraker, 2013; Underwood, 2002), it is important that these software packages are able to handle these Big Data (Ohlhorst, 2013). According to the McKinsey Global Institute, Big Data are ‘datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyse’ (McKinsey Global Institute, 2011). A similar definition by Ohlhorst is that ‘Big Data defines a situation where datasets have grown to enormous sizes and therefore cannot be handled effectively by conventional information technologies’ (Ohlhorst, 2013). These data sets are usually large in size (e.g. in terabytes or petabytes); unstructured (e.g. emails, geographical maps, images and videos); can easily be prone to misinterpretation and statistical errors if inaccurate; and are time sensitive, that is, need to be captured and analysed as soon as they flow into the organization, yet still be available for future use from its archive. Despite these features, the data sets still need to be able to meet the information needs of the users (Anon, 2013; Keiser, 2013; Ohlhorst, 2013). Consequently, these software packages need to have the capability to handle these large or unstructured data sets so as to harness useful information from them. This capability has been called Additional Features for the sake of this research and will be investigated as the fourth objective, that is, to determine whether these software packages have additional features and therefore can handle large or unstructured data sets. These Additional Features and how CI software packages are expected to have them are described below.

Content management

So as to aid easy information extraction from these large data sets (Stonebraker, 2013), CI software should allow Metadata and Taxonomy classifications to be created and edited. It should also be able to present information to users using visualization tools and automatically translate any information which is not in the user’s default language setting. In addition, CI software should be able to store information and perform searching and indexing of documents (Intelligence Plaza, 2014). The ability to provide features that will be useful for storing content in database and visualization tools are the criteria evaluated under the information sorting, capturing and storage and information analysis stages of the CI Cycle, respectively. Therefore, they will not be considered again under this feature for the purpose of this research.

Data sourcing and inputting features

In order to obtain up-to-date information on competitors from the Web (Anon, 2013; Ohlhorst, 2013), CI software needs to be able to crawl the Web, monitor various information sources, use Rich Site Summary (RSS) feeds to collect information and allow microblogging to be carried out on the various information sources. Additionally, these software need to be able to accept user input through mobile devices and various types of Web browsers, so that users can search for and obtain information as soon as it is needed (Intelligence Plaza, 2014).

Security

CI software should allow its information to be accessed only by authorized users. Since this is also a characteristic of the British Standard, this feature is covered under the British Standard.

Dissemination

It is important that information is made available to users when needed (Anon, 2013; Ohlhorst, 2013). Therefore, CI software should be able to ‘push’ services from the intelligence team to the CI users. This implies that CI software should have the feature for generating newsletters, creating automated, personalized email alerts, sending email to a user group and integrating with external user interfaces such as RSS, XML and SharePoint when needed.

Self-service access

Software technologies used in carrying out CI need to have the capacity to ‘pull’ information and provide visualization and analytics tools that allow for a better representation of information extracted from these data sets (Bayandin and Kretov, 2012; Intelligence Plaza, 2014; Stonebraker, 2013). This implies that users should be able to customize the dashboard of CI software; the software should have analysis tools for text-based and quantitative data (e.g. charts and text mining) and benchmark a competitor’s products and other competitor information.

Collaboration

According to Jones (2014), CI practitioners should be given a range of methods to present their data. For CI software to be able to offer users with a wide range of presentation methods therefore, it should have facilities for commenting on content items and discussion forums and/or threads. Also, with the increase in use of social media and mobile devices within businesses, it is important that these software technologies are able to incorporate facilities that allow collaboration via social networking.

Evaluating CI software based on cost and availability

In order to give users an idea of how these software packages will benefit them if the commercial version is purchased, software vendors often offer free trial versions of their software packages to their customers. Free CI software is beneficial to both the users and the software vendor. It helps the user to determine if the software is able to meet the intelligence needs of the organization without purchasing a full commercial software, it gives users the opportunity to try the software before the commercial version is purchased (Scher, 2009) and saves the cost usually associated with purchasing commercial software (Bouthillier and Shearer, 2003). On the part of the vendor, it acts as an economic incentive by leading users to upgrade to the full-featured commercial versions of these software packages or related packages (LINFO, 2006; Scher, 2009). Therefore, in order to determine if CI software is able to meet the intelligence needs of the organization without purchasing a full commercial software, information about the cost and version availability is obtained from the software vendors. This will be addressed through the last three objectives of this study, that is, to identify if there are commercial and free versions of the software package, to identify the differences between the commercial and free versions of the software packages and to investigate the cost of the commercial versions of these software packages.

Research methodology

In order to identify CI software packages for evaluation, various means such as checking the CI resource index website (http://www.bidigital.com/ci/), setting up Google alerts and Web search using the terms ‘CI software’ were employed. Next, the method of convenience sampling was used to choose 10 software packages, and then a pilot review (Bryman and Bell, 2011) was carried out on 3 of 10 software packages.¹

Of these three software reviewed for the pilot, two of them were accessed by downloading their 30-day free trial versions and installing them on a PC, while a video demonstration of the third software was viewed on the site of the software vendor. Two methods, namely, literature survey and technical review, were employed. For the literature survey, information was gathered on these software packages from the marketing literature provided by their vendor; while for the technical review, two sets of evaluation frameworks were adapted or developed and used. The first evaluation framework was adapted from the Bouthillier and Shearer CI framework (2003), while the second framework was developed using the British Standards Institute requirements. These frameworks were used to test if the software packages could carry out the stages of the CI Cycle and conform to the British Standards requirements, respectively. As for the Additional Features, information was gathered about this by taking note of the features of the software packages and matching them with the software requirements for Additional Features identified above. Based on the results of the pilot carried out, the evaluation frameworks for the CI Cycle and British Standards were modified to contain those values that are possible to evaluate given a situation where there was access to only video demonstrations of the software packages, while a third evaluation framework was developed to evaluate the software packages based on the Additional Features. This resulted in the three evaluation frameworks being used in the main study.

In order to proceed with the main study, the websites of those software vendors whose software were not used in the pilot review were visited in order to contact them for video demonstrations or free trial versions of their CI software packages. Of those contacted, one vendor provided a 30-day free trial version of their software and two vendors provided links (and where needed, login credentials) to the video demonstrations (and recorded webinars) of their software. The fourth software was reviewed using the video demonstration available publicly on the website of the vendor. In total, four software packages were evaluated and they were accessed using different means. In reporting the findings of the research, the software packages are anonymized and labelled as S1, S2, S3 and S4. S1 and S2 were developed by the same software vendor. S1 was accessed through video demonstrations and recorded webinars available on the website of the vendors; S2 was accessed through a 30-day free trial version provided by the software vendor, while S3 and S4 were accessed through video demonstrations available on the website of their vendors. The login credentials needed to access these materials were provided by their respective vendors. The evaluation of these software packages took place over a 2-month period.

Quantitative, qualitative and mixed methods approaches were considered for data collection. In using the quantitative method, the software packages were accessed for their ability to carry out various functions (based on the evaluation frameworks for the CI Cycle, British Standards and Additional Features), and a score (from 0 to 5) was assigned to each software based on its performance in carrying out these functions or the availability/non-availability of a function. For the qualitative method, information on the functions of these software packages (based on the CI Cycle, British Standards and Additional Features) was collected from their various published marketing literature. Lastly, for the mixed methods, the information based on the functions of the software packages collected using the quantitative approach was compared with the information collected using the qualitative approach.

Results

CI cycle

Table 1 below summarizes how the software packages were able to perform with regard to the stages of the CI Cycle in comparison with the mean possible score for each of the stages.

Table 1.

Software performance based on the CI Cycle stages.

Stages of CI	S1	S2	S3	S4	Mean possible score ( $\bar{s p}$ )
Identification of CI needs	5	–	4.5	5	5
Information collection	4.2	4.2	4.6	4	5
Information sorting, capturing and storage	4.5	5	5	4.5	5
Information analysis	2.83	3	3.33	2.67	4.33
Intelligence communication	2.4			0.8	3.4

Note: CI: competitive intelligence.

From Table 1 above and the information contained from the marketing literature of the software packages, it was revealed that the various software packages have their strengths and weaknesses. S1 and S4 are better at identification of CI needs than the other software packages; S3 is better at information collection than the other software packages, while S2 and S3 are better than the other software packages in information sorting, capturing and storage. Choosing a software package for information analysis depends on the needs of the CI practitioner as the various software packages perform differently in this stage. S3 offers a wider range of analytical tools and a deeper level of analysis than the other software packages, while S4 is the only software that recommends suitable action. Therefore, a practitioner interested in a software package that can recommend suitable action is more likely to go for S4 than the other software packages, even though S4 does not offer a wide range of analytical tools like S3. Lastly, S1 is a better option than the other software packages if the CI practitioner is seeking a CI software package that can link and export reports in addition to creating reports and distributing information via email.

British Standards

Table 2 below summarizes how the software packages were able to perform with regard to the British Standard in comparison with the mean possible score for each of the characteristics.

Table 2.

Software performance based on the British Standard.

British Standards software quality characteristics	S1	S2	S3	S4	Mean possible score ( $\bar{s p}$ )
Performance efficiency	4	5	3	5	5
Usability	3.5	1.25	1	0.8	4
Reliability	4.5	–	–	–	5
Security	1	0.5	0.5	0.5	3.5
Maintainability	3	2	2	2	4
Portability	2	–	2	–	4
Compatibility	5	–	–	–	5

Additionally, from the qualitative data collected, the marketing literature of the software packages did not mention features related to reliability and compatibility, and it was not possible to evaluate the software packages based on these two characteristics except S1 due to the use of video demonstrations (and recorded webinars) in evaluating the software packages. For the other characteristics, S2 and S4 are better than the other software packages in performance efficiency; S1 is able to satisfy the user in terms of usability more than the other software packages, while both S1 and S3 are portable CI software packages. For security and maintainability, choosing a software package should be dependent on the security needs and the number of CI users within the organisation, respectively.

Additional Features

Table 3 below gives a summary of how the software packages were able to perform with regard to the Additional Features in comparison with the mean possible score for each of the features.

Table 3.

Software performance based on the Additional Features.

Additional Features	S1	S2	S3	S4	Mean possible score ( $\bar{s p}$ )
Content management	3.75	4.25	1	1.25	5
Data sourcing and inputting	3	1.4	2.4	1.4	3.8
Dissemination	3.75	3.5	3.75	2.5	5
Self-service access	4.25	4.25	4.5	3	5
Collaboration	3	2.5	4.5	2	5

From the table as well as the marketing literature, all the software packages excluding S3 can effectively perform in at least one function required for the Additional Features. S2 is better than the other software packages at content management and S1 is better at data sourcing and inputting, while S3 is better at self-service access than the other software packages. For dissemination, all the software packages are good in this feature except S4, while all the software packages and especially S3 are good at collaboration.

Trial availability and cost

S1 has a free trial version, while S2 has a version which users can pay for to test in light of their intelligence needs, called a ‘proof of concept’ version. S1 also provides information on the cost of a full commercial version, while the full cost of S2 varies depending on the needs of the purchasing organization. No information was provided on the cost of a commercial version and the availability of a free trial version for software packages S3 and S4 does not provide information.

For the differences between the commercial and free versions of the software packages, S1 does not work differently from the trial version as the aim of the company is to help users see how the full version of the software would benefit them. For S2, the proof of concept version is limited in functionality compared with the full version, while no information was available for the remaining two with regard to the differences between their commercial and free versions.

Lastly, with regard to cost, the full version S1 costs €60/$80 per user per month. The proof of concept version for S2 costs €5000, while the cost of the full version varies depending on factors such as size of implementation, need for maintenance and hardware/network cost. No information was available for S3 and S4 with regard to cost.

According to Taylor (1986), when cost is saved by the client on a system, it raises the value of such a system. Bouthillier and Shearer (2003) and Scher (2009) also suggest that the availability of free CI software gives CI practitioners in businesses the opportunity to try out the software before purchasing a full version of it, therefore, reducing the cost attached to it and enhancing its value to the potential buyer.

This implies that CI practitioners within businesses seeking to choose cost-effective CI software for their intelligence needs are more likely to choose software packages S1 and S2 over S3 or S4, since both S1 and S2 provide adequate information on the availability of a free trial versions cost and the cost of a commercial version.

Discussion and conclusion

This study has advanced the methodology used in evaluating CI software packages. Specifically, three evaluation frameworks were adapted and used in evaluating identified CI software packages.

The first framework was adapted from the existing models of the CI Cycle which was then labelled an integrated CI model. Just like other CI models such as that of Bose (2008) and Fuld & Company (2003, 2006/2007, 2008/2009), this integrated model consisted of five stages. However, this model was slightly different from other models as it sought to incorporate stages from identified models discussed in the literature review section, that is, from Bose (2008), Fuld & Company (2003, 2006/2007, 2008/2009), Bouthillier and Shearer (2003) and Pellissier and Nenzhelele (2013). Thus, the stages used in evaluating identified CI software packages in this study are identification of CI needs, information collection, information sorting, capturing and storing, information analysis and intelligence communication. Overall, since this model is an integrated one, it incorporated the advantages of all the models identified through literature review and used in the previous research studies such as information storage which is absent in other model except the Bouthillier and Shearer (2003) model.

The second framework was adapted from the British Standard software quality characteristics, while the third framework was adapted from the characteristics required from software packages which handle Big Data called Additional Features. To our knowledge, as at the time of writing this article, no other CI software evaluation has been carried out using the second and third framework adapted for this study.

For the second framework used, the British Standard software quality characteristics are function suitability, performance efficiency, usability, reliability, security, maintainability, portability and compatibility. Although each of these characteristics has their sub-characteristics, it was only the characteristics and their respective sub-characteristics which were relevant for this study that were used in evaluating the identified CI software packages. The first characteristic – function suitability – was taken care of by the first framework i.e. the use of the integrated CI model, while other characteristics were tested using various criteria. For instance, in testing the security characteristics of these software packages, the packages were tested to see whether they use passwords, and if so their allowed password strengths. They were also tested to evaluate whether they kept a log of user activities, and if so, how effective this log is.

Lastly, for the third framework, Big Data as a concept is not just a trendy topic, but one that seems to have come to stay as data sets seem to become more complex and voluminous in structure, with the need for them to be accurate especially when information which is required to put an organization at an advantage is needed from these data sets. In evaluating the software packages based on these criteria, features such as content management, data sourcing and inputting, dissemination, self-service access and collaboration were tested. Collaboration in particular is very important as it included checking if the software packages could perform functions which required collaboration via social networking.

In summary, using an integrated CI model for testing the software packages was a useful approach as it incorporated all the advantages of identified CI models mentioned earlier. In addition, extending the frameworks used to British Standards and Additional Features was also advantageous as the integrated CI software model on its own could not have covered these characteristics and features.

Limitations and recommendations for further research

During the process of carrying out the research, the software vendors were not very responsive in giving access to their software packages, and not all vendors provided information about the cost of their software packages. Also, for the software packages evaluated through video demonstrations and recorded webinars, it was not possible to evaluate them based on all the evaluation criteria of the CI Cycle, British Standard and Additional Features.

For possible future research, it would be useful for the evaluation to be extended to the full commercial version of these software packages as well as engaging users of these software packages in order to gain qualitative feedback on the performance of these software packages.

Final thoughts

This study although limited in scale has demonstrated that no one CI software package has the capability to fully support all the stages in the CI Cycle, all the requirements of the British Standards or the features which make it possible to handle large data sets. Therefore, in order to choose a CI software package for an organization, after identifying the CI needs of the organization, CI practitioners within the organization need to determine which of the CI stages can be performed without the use of a software and then choose which one best suits their needs based on the stages of the CI Cycle, the British Standards and the Additional Features. Additionally, since it is not always possible for software vendors to showcase the full features of software in its trial version, it could be helpful if the published marketing literature or video demonstrations of these software packages show those features which pertain to CI and which practitioners really require. For instance, since information analysis has been identified as the most important stage of the CI Cycle, it will be worth showcasing the information analyses capabilities of the software packages in the trial version, while the intelligence dissemination capabilities can be seen in the video demonstrations.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Note

References

Ahituv

Zif

Machlin

(1998) Environmental scanning and information systems in relation to success in introducing new products. Information and Management 33: 201–211.

Anon (2013) What is big data, anyway? Public CIO 11(1): 6–7.

AWARE (n.d) Competitor Analysis – A Brief Guide: The Basic Principles Of Competitive Intelligence. [Online]. Available at: http://www.marketing-intelligence.co.uk/resources/competitor-analysis.htm (accessed 06 March 2016).

Bayandin

Kretov

(2012) A competitive intelligence monitoring system for small businesses. Scientific and Technical Information Processing 39(4): 236–239.

Bercik

(2014) Introducing the FreePint Topic Series: Next-Generation Competitive Intelligence. [Online]. Available at: https://web.jinfo.com/go/blog/71289 (accessed 27 February 2016).

Bose

(2008) Competitive intelligence process and tools for intelligence analysis. Industrial Management and Data Systems 108(4): 510–528.

Bouthillier

Shearer

(2003) Assessing Competitive Intelligence Software: A Guide to Evaluating CI Technology. Medford: Information Today.

British Standards Institute (2011) ISO/IEC 25010:2011: Systems and Software Engineering – Systems and Software Quality Requirements and Evaluation (SQuaRE) – System and Software Quality Models. Switzerland: International Standards Organisation.

Bryman

Bell

(2011) Business Research Methods, 3 rd ed. Oxford: Oxford University Press.

10.

Fuld

(2012) Competitive Intelligence Early Warning for Threats and Opportunities [video]. Fuld Company. Available at: https://www.youtube.com/watch?v=QdOhjGnEOLw#t=48 (accessed 11 November 2016).

11.

Fuld & Company (2003) Leveraging the Web: A Review of Sixteen Software/Technology Offerings in the Competitive Intelligence Arena. Cambridge: Fuld & Company.

12.

Fuld & Company (2006/2007) Intelligence Software Report 2006/2007: Technology Risk and Reward. Cambridge: Fuld & Company.

13.

Fuld & Company (2008/2009) Intelligence Software Report 2008-2009. Cambridge: Fuld & Company.

14.

Fuld & Company (n.d.) What is Competitive Intelligence? [Online]. Available at: http://www.fuld.com/company/what-is-competitive-intelligence/ (accessed 12 February 2016).

15.

Gainor

Bouthillier

(2014) Competitive intelligence insights for intelligence measurements. International Journal of Intelligence and CounterIntelligence 27(3): 590–603.

16.

Intelligence Plaza (2014) Features to Pay Attention to When Selecting Intelligence Software [online]. Available at: http://www.intelligenceplaza.com/blog/features-to-pay-attention-to-when-selecting-intelligence-software/ (accessed 27 February 2016).

17.

Jadhav

Sonar

(2011) Framework for evaluation and selection of the software packages: a hybrid knowledge based system approach. The Journal of Systems and Software 84(8): 1394–1407.

18.

Jones

(2014) The Latest Visualization Techniques should be Part of the CI Professional’s Toolbox [online]. Available at: http://web.jinfo.com/go/blog/71400 (accessed 6 March 2016).

19.

Juhari

(2009) Evaluation of competitive intelligence software for MSC-status small and medium-sized enterprises in Malaysia. PhD Thesis, Loughborough University, UK.

20.

Juhari

Stephens

(2004) Timeline of historical incidences related to competitive intelligence. [Online]. Available at: http://km.lboro.ac.uk/kmci/timeline.php (accessed 17 January 2014).

21.

Keiser

(2013) Competition among competitive intelligence platforms. Online Searcher 37(1): 16–21.

22.

Limacher

(2014) The Importance of Competitive Intelligence in the Biotech Industry. [Online] Available at: http://web.jinfo.com/go/blog/71433 (accessed 06 March 2016).

23.

LINFO (2006) Freeware Definition [online] Available at: http://www.linfo.org/freeware.html (accessed 18 January 2014).

24.

McKinsey Global Institute (2011) Big Data: The Next Frontier for Innovation, Competition, and Productivity. San Fransisco: The McKinsey Global Institute.

25.

Ohlhorst

(2013) Big Data Analytics. New Jersey: John Wiley & sons Inc.

26.

Pellissier

Nenzhelele

(2013) Towards a universal competitive intelligence model. South African Journal of Information Management 15(2): 1–7.

27.

Rouach

Santi

(2001) Competitive intelligence adds value: five intelligence attitudes. European Management Journal 19(5): 552–559.

28.

Scher

(2009) Open source and freeware-strategic resources for IS pedagogical endeavours. Information Systems Education Journal 7(69): 1–9.

29.

Stonebraker

(2013) What does ‘Big Data’ mean? Communications of the ACM 56(9): 10–11.

30.

Taylor

(1986) Value-Added Processes in Information Systems. Norwood: Ablex Publishing Corporation.

31.

Teo

TSH

Choo

(2001) Assessing the impact of using the Internet for competitive intelligence. Information and Management 39(1): 67–83.

32.

Underwood

(2002) Competitive Intelligence. Oxford: Capstone.