Abstract
In recent years, with the rapid development of IoT technology, hundreds of millions of IoT devices have been manufactured and applied, and the subsequent IoT attacks have become more and more severe. The complex and diverse architecture of IoT devices, coupled with the lack of security development specifications by IoT device manufacturers, and the widespread misuse and abuse of code, lead to the proliferation of IoT vulnerabilities. The conventional IoT vulnerability detection scheme is expensive to operate, and the implementation technology is complex, which is difficult to be fully promoted. This paper proposes a lightweight IoT firmware vulnerability detection scheme based on homology detection. The processing is converted into a feature vector, which effectively reduces the platform dependence. Combined with the database technology, the storage and retrieval efficiency is increased, and the same-origin vulnerability detection is realized by calculating the cosine similarity of the vector. The experimental results show that this scheme can effectively identify the vulnerabilities in firmware.
Introduction
The Internet of Things is regarded by people as the third revolution in the information technology industry after computers and the Internet. In the scenario of the ubiquitous Internet of Things, people and things cross the constraints of time and space and are tightly connected. According to the analysis of McCarthy [21], by 2025, the total potential economic impact of the Internet of Things (IoT) will be between 3.9 trillion and 11.1 trillion U.S. dollars each year. However, in the process of vigorous development of the Internet of Things, the existing security mechanisms are difficult to cope with the increasing security requirements, leading to endless security problems in various application scenarios. A large number of devices are vulnerable to malicious code threats or illegal control or even triggers. Large-scale security incidents. From 2016 when the famous Mirai worm used IoT devices to cause a large-scale denial of service attacks [1], to the recent smart speakers being used by attackers to eavesdrop on user privacy [13], IoT security threats continue to emerge along with technological development.
Detecting security threats in time or taking defenses in advance is an important means to counter threats, but the characteristics of the Internet of Things system [3] determine that the implementation of perfect security protection faces many challenges. In recent years, the industry has tried to apply various well-applied automated vulnerabilities mining methods for general-purpose computing systems to IoT devices, but the application effects have been limited by the complex problems of IoT devices.
Complex heterogeneity of the underlying hardware platform
A wide range of types, numerous vendors, and large differences in software and hardware implementation are major features of current IoT devices. The complexity of the underlying hardware platform is a huge obstacle to the security analysis of IoT devices. The previous solutions for X86 and X64 cannot be directly adapted.
Strong specificity, the source code is not open
Most IoT devices are custom-developed by manufacturers based on their functional characteristics. Different developers have different coding styles and specifications. Even with the same network protocol and hardware interface, developers from different manufacturers have different functions, which hinders general-purpose detection solutions.
Limited operating environment
For cost considerations, IoT devices will reduce unnecessary components, and the debugging interface will be closed when leaving the factory, making it difficult for dynamic debugging to directly act on the device, even if the dynamic analysis is deployed to the actual physical through methods such as binary rewriting and static instrumentation. In the equipment, its analysis performance is restricted by the lack of hardware CPU computing power, which affects the analysis progress and results.
The IoT device program is different from the general PC program
IoT devices interact with the outside through peripheral devices, such as wifi, Bluetooth, etc., and feedback to users through APP or Web management interfaces. The users send requests to the device through these interfaces. After the device receives the request, it processes the request and distributes it to specific applications and The server returns to the foreground user after background processing. The transmission, distribution, storage, and calculation of these data are all black boxes for users, which increases the difficulty of equipment analysis.
The current research on power systems [17,19,20] is different from our scenario, and this paper focuses more on the security of firmware, although there are many differences between IoT systems and traditional PC systems, they all need to run software in essence. Software customized for IoT devices is usually called firmware, which is a program written into storage media such as EEPROM or Flash, it is generally understood as solid software. Like traditional software, the firmware in IoT devices also has defects, and almost all device firmware contains vulnerabilities. This is mainly due to the “customized” production model in the field of IoT device manufacturing and the separation of research and production. However, the entire system is not solidified when equipment is produced. Hardware manufacturers manufacture hardware facilities and attach hardware drivers. Equipment manufacturers directly purchase hardware or technical solutions from hardware manufacturers to develop development boards with specific functions.
Based on the research of existing firmware vulnerability detection technology, this paper proposes a vulnerability detection scheme based on homology analysis. The experimental results show that this scheme can effectively detect the vulnerabilities in the firmware.
Related work
Effective detection of the aforementioned firmware security defects is the key to ensuring the security of IoT devices. Automated firmware security defect detection can find defects before the firmware is released, effectively avoiding a large number of losses; on the other hand, it can realize a large number of firmware analyses in a short period, saving developers time and related financial resources.
Static analysis technology
Static analysis refers to the analysis of firmware code products (such as source code, abstract syntax tree, intermediate representation, etc.) without running the firmware system, and matching with typical firmware defect patterns to achieve the purpose of detecting potential security defects.
DTaint [7] proposed taint analysis of firmware, which can reduce the amount of code analyzed and improve the efficiency of analysis. DTaint does not need the firmware source code, uses the firmware executable file as input, converts it into an intermediate representation and analyzes it. Supports pointer analysis and indirect call analysis, adds support for different architectures, and improves scalability. However, due to the large differences in firmware of different system types, DTaint currently only supports Linux-based firmware.
Saluki [12] improved the accuracy of taint analysis by “running” code snippets. Saluki implemented a custom interpreter to “run” the IR extracted from the binary file, without the need for runtime environment support, to get the exact path Sensitive and context-sensitive data disseminate information. Saluki supports binary executable files of ARM, x86 and x86-64 architectures, and can take into account common desktop software and firmware. Saluki has also implemented customizable security guidelines, enabling taint analysis to support more types of vulnerabilities.
Linux-based firmware is often composed of multiple components, and each component is an executable file. Many analyses perform taint analysis on each executable file separately, which can easily cause a large number of false alarms. KARONTE [24] considered the interaction and data flow between different components of the firmware when analyzing the firmware, which greatly reduced the false positives of taint analysis.
BootStomp [23] proposed a static analysis method for the boot program in the mobile phone, combined with the symbolic execution-based multi-label taint analysis method, it can be found that the memory damage defect and boot program unlocking caused by the use of the attacker-controlled storage in the boot program are greatly reduced. The false positives of pure static stain analysis work.
Symbolic execution technology
Symbolic execution technology originated in the mid-1970s. The basic idea is to use symbolic variables to replace actual input, drive the program to simulate execution, collect the execution path constraint set, call the solver to solve the constraints to generate test input, and then explore the program space for Found program defects. Symbolic execution uses symbolic values to represent specific variables. Compared with fuzz testing, it can pass complex judgments more easily. It has a wide range of applications in ordinary software testing.
Davidson et al. [8] proposed a symbolic execution tool FIE based on the symbolic execution engine KLEE, which realized effective defect detection of firmware programs based on MSP430 series microcontrollers. For small firmware programs (less than 100 lines of code), FIE uses state pruning technology to avoid path explosions, and uses memory fuzzing technology to improve symbolic execution efficiency. At the same time, it adds support for interrupts and external inputs, and can analyze all paths of the program. FIE is an earlier study of symbolic execution methods in firmware testing, but it can only target specific architectures, and can handle fewer lines of code and more false positives.
Based on FIE, Hernandez et al. [16] realized the symbolic execution analysis framework FirmUSB for USB firmware by adding support for Intel 8051 MCU. FirmUSB combines static analysis and symbolic execution to check the device firmware to determine its ability to generate potentially malicious behaviors by acting as a semantic query engine. In addition, FirmUSB analyzes the binary firmware by converting it into an intermediate representation, without the need for firmware source code.
Oleksandr et al. [3] proposed a KLEE-based symbolic execution tool to analyze the dangerous memory references of the SMM interrupt handler in the BIOS. The method is to use S2E to explore the execution path through the SMM interrupt handler, and find that the interrupt handler tries to access the path of the memory outside the SMRAM protected area of the SMM code.
Yan et al. [25] proposed to use symbolic execution technology to find authentication bypass defects in binary firmware, and implemented the analysis framework Firmalice. Firmalice uses a new type of identity authentication bypass function, based on the attacker’s ability to determine the input required to perform privileged operations. If the attacker only needs to analyze the firmware to obtain the necessary input to drive the firmware to perform the privileged operation, the authentication mechanism is explained. Can be ignored or bypassed. In addition, the model allows the inference of complex backdoor procedures. Firmalice’s symbolic execution part refers to KLEE, and uses program slicing technology to enhance its scalability.
The symbolic execution-based analysis is one of the main methods for firmware security defect detection. Most of them are based on KLEE, with different focuses. Existing tools such as FIE, FirmUSB, etc. usually introduce support for specific hardware architectures for their own research goals. Inception’s work eliminates platform differences to a certain extent and greatly improves the scalability of tools. And they all added support for peripheral inputs and interrupts, and adopted certain methods to avoid path explosion and improve symbol execution efficiency. However, most of the existing symbolic execution methods require source code. Only certain tools such as FirmUSB and Firmice can directly use binary firmware as input. Therefore, future work needs to study how to use binary firmware as input for analysis and support more architectures.
Fuzzing test technology
Fuzzing technology first appeared in 1990. Miller used fuzzing to find vulnerabilities in UNIX systems. Fuzzy testing technology sends a large number of valid or invalid inputs to the running program by running the program in a real or virtual environment, and observing the behavioral characteristics of the program during operation, matching the behavioral characteristics of typical defects to achieve the purpose of detection. According to the firmware running environment, the existing fuzz testing technology can be divided into fuzz testing based on real equipment and based on simulated execution environment.
Sara et al. [2] proposed a runtime verification tool to detect concurrency defects in FreeRTOS. Their tool is based on the FreeRTOS running log extracted by Tracealyzers for analysis, checking the running time of the task to detect whether it meets the characteristics of concurrent defects. This tool can detect Deadlock Bug, Starvation Bug and Suspension Bug.
IoTFuzzer [6] found that IoT terminal devices are often accompanied by a mobile application that interacts with them, so based on the mobile application and real terminal devices, a black box fuzzing test tool was developed to detect memory corruption defects in firmware programs. The mobile application can be analyzed to deduce the fields that need to be mutated in the unknown protocol, generate effective fuzzing test cases, and overcome the encryption problem in network communication. IoTFuzzer uses a heartbeat mechanism to detect whether the device is abnormal.
Zhang et al. [29] proposed a tool ConfDiagnoser that combines static analysis and dynamic analysis to diagnose a single configuration defect. ConfDiagnoser takes Java programs and configuration items as input, collects runtime characteristics of the programs, and matches them in the database to find defects. ConfDiagnoser has four limitations: The tool only focuses on key-value configuration; the diagnosis of configuration defects is limited to the defects of one configuration item; it does not support non-deterministic defects; the validity depends on the existence of similar and correct programs in the database implement.
Chen et al. [5] realized the first system FIRMADYNE to simulate and automatically analyze the firmware of commercial Internet of Things based on Linux in the simulator. This work can simulate the Linux system and completely separate it from the hardware for simulation. FIRMADYNE uses binwalk for unpacking and runs in QEMU simulation. Various dynamic analysis methods can be easily extended on FIRMADYNE. FIRMADYNE realizes the simulation out of the hardware to a large extent, but is limited by the limited ability of QEMU. There are few firmwares that can run successfully, and it can only target Linux-based firmware. Firm-AFL [30] implements a high-performance gray-box fuzzing test for Linux-based IoT firmware based on FIRMADYNE. This is the first gray-box fuzzing test system for firmware, and it improves the efficiency of firmware simulation execution.
Xu et al. [27] proposed a method to detect potential configuration defects through simulation before software release, and realized the configuration defect detection software PCHECK. PCHECK takes IR as input, and uses configuration interface specifications and system initialization phase comments as auxiliary inputs to identify configuration items.
Muench et al. [22] found through experiments that different firmwares behave differently against memory-related defects. Many firmware systems have no obvious performance when encountering memory-related defects and can continue to run, which poses new challenges for defect detection. The author proposes to monitor the use of memory to determine whether there are defects, but this method can only be carried out in the simulator at present. In addition, a large number of other implementation flaws in reality have different manifestations. The fuzzing test based on the actual execution environment has limited hardware resources and an incomplete debugging interface, which can obtain less firmware operation information, and the defects that can be detected are limited.
Homology detection technology
The main objects of homology detection for firmware include entities such as firmware, executable files, program source code, functions, etc., extract the characteristics between the target detection object and the vulnerability object, establish a homology detection model, and express the two based on the similarity algorithm. It is a commonly used technology in firmware vulnerability detection to determine whether there is a vulnerability.
Song et al. [26] summarized the homology detection of firmware, and divided the homology detection technology into text, token and structure according to different intermediate forms. The homology detection process of non-source binary files is introduced, and the mainstream methods are compared.
He et al. [15] extracted the numerical and structural features of the vulnerability function, and used the clone selection algorithm to detect firmware homology vulnerabilities, which improved the recall rate of vulnerability detection and reduced the dependence of the objective function on the detector.
Sun et al. [14] designed a large-scale firmware vulnerability detection method, which is based on the features of firmware functions and graph embedding networks, and uses the embedding vector to calculate the similarity of functions, which can realize automatic detection and reduce labor costs.
Eschweiler et al. [10] pre-analyzed function-level features, and used features such as arithmetic and number of instructions to establish a graph matching model for vulnerability detection, but there is still an overhead bottleneck in the process of semantic feature extraction. feng et al. [11] encodes the control flow chart into a special vector for graph embedding through the code base, and uses locality-sensitive hashing to improve retrieval efficiency, but the detection results are affected by the quality of the code base.
In recent years, researchers have begun to combine methods such as deep learning and generative adversarial networks to study the problem of vulnerability detection [4,9].
ASTERIA [28] measures the semantic equivalence of a function based on the deep learning AST coding method, and improves the detection accuracy by learning the similarity between the two vectors measured by the semantic representation of the function. Vuldeepecker [18] converts semantically related code fragments into vectors for deep learning and achieves a low false negative rate.
The method of homology detection has gradually developed from coarse-grained to fine-grained, and the performance overhead is also gradually increasing, which makes it difficult to withstand large-scale detection, and lacks lightweight applications that take into account speed and accuracy. Compared with traditional vulnerability detection methods such as static analysis, symbolic execution, and fuzzing test, homology detection does not need to run firmware, has less dependence on environment configuration, and supports multi-architecture analysis; The existing homology detection methods are prone to the problems of low detection accuracy at the file level, and high performance overhead at the code level. The lightweight homology detection method proposed in this paper can eliminate architectural heterogeneity, and we made a trade-off between accuracy and overhead.
Experimental method
The overall architecture of the system is shown in Fig. 1, and we will explain its detection process around this figure.

System structure.
First of all, we need to understand that there are two ways for equipment manufacturers to produce products:
Develop firmware and debug it by themselves, and finally package the factory and hand it over to the supplier for purchasing;
The equipment manufacturer only solidifies a small section of the boot program, and the operating system, file system, etc. are first handed over to the supplier. It is constructed by third-party subcontractors and then handed over to suppliers for purchasing.
Manufacturers usually use the second approach, which leads to a phenomenon: different equipment manufacturers may choose the same subcontractor, and the software of the same equipment manufacturer may be developed by multiple subcontractors.
There is no uniform standard for the development tools, development kits, or libraries provided by subcontractors, devices of different brands may run the same or similar firmware and contain the same third-party libraries, and devices of the same brand may have multiple subcontractors Vulnerabilities caused by third-party libraries, therefore, embedded devices are often affected by some known vulnerabilities, the same vulnerability may affect different device manufacturers, and even an updated firmware will have disclosed vulnerabilities. This makes the vulnerability detection of embedded device firmware a lot of repetitive work. The analysis of only a specific model of device or firmware of a certain version cannot represent the security status of all firmware in the series. It is necessary to find a reasonable solution for embedded device firmware. The vulnerability analysis method of the firmware of the type device can effectively identify the correlation between the firmware-dependent library and the development library, to determine the homology of similar open-source codes, and then repair the vulnerabilities in the firmware more thoroughly.
The third-party libraries in embedded devices mainly include shared libraries and common toolsets. Homologous libraries refer to different shared libraries or general software (open source tools or a vendor-specific software collection) formed from a common open-source code base ancestor through a process of divergent evolution. In embedded systems, commonly used shared libraries are glibc and uClibc. Their dynamic link forms are stored in the “/lib” directory, and the file names are often “libc.so” and “ld-uClibc.so”. Compiled and written dynamically linked programs will use them at runtime. Common open-source tools include BusyBox, a lightweight toolset that integrates common commands for embedded systems, and small tools such as Boa and Httpd, which are used as web servers.
The compiler will not discard string constants during compilation, they can be used as fingerprints in the homology analysis of third-party libraries under different architectures. The implementation process is: extract the binary programs and dependent libraries contained in the firmware and search for internal identification characters. string, debugging information, and the name, version number, and other information of the dependent library, and compare the extracted identification information with the strings stored in the database. If the string matching a third-party library in a database reaches a certain threshold, it is considered homology.
Aiming at the above problems, this paper proposes a lightweight vulnerability detection scheme based on code similarity. First, obtain IoT firmware, unpack and analyze to obtain a large number of system files, extract binary executable files and dependent library files in the firmware package, and perform reverse analysis to extract function-level features. The extraction of features should ignore the differences caused by different platforms. Even if the same source code is compiled for different platforms, the assembler is disassembled and presented. Its instruction set, registers, addressing modes and functions. There are also great differences in the internal control flow graph, which requires that the extracted features can ignore the differences caused by different platforms, and the bitstream, instruction set, or register names are heavily dependent on the platform, so the selected features cannot be based on these objects. According to our experiments, it is found that all constant strings, internal identification strings, debugging information, names, version numbers and other information of dependent libraries are platform-independent. The representative parameters are recorded, and the name of the function that appears in the string and the name of the parent function that calls the function are recorded. All the above features are saved as a feature vector, which is used as the detection item for similarity matching.
The contents saved in the vector are, file name, string, parameter name, function name, parent function name (may contain multiple).
Firstly, the above-mentioned standardization process is performed on the firmware files that have exposed vulnerabilities, and the feature vectors are saved in the database. When the similarity matching is performed, the files to be tested are standardized, and the cosine similarity calculation is performed with the vectors saved in the database.
When the operation result reaches a certain threshold, it is considered to be homologous.
Take the CVE-2013-6026 vulnerability as an example of the test target of this experiment. The trigger location of this vulnerability is in the alpha_auth_check function under the webs file of the D-Link router, which contains a constant string xmlset_roodkcableoj28840ybtide. When the user logs in to the router management page using HTTP, As long as the User-Agent value is set to the above-mentioned backdoor value, the login authentication process can be bypassed and the router management authority can be directly obtained.
The location of the vulnerability is shown in Fig. 2.

CVE-2013-6026 vulnerability location.
The parent function of the vulnerability function alpha_auth_check is shown in Fig. 3, and the name of its parent function is extracted as the feature of the vector.

Alpha_auth_check parent function call diagram.
First, enter the D-Link DIR100 firmware and file information that has identified vulnerabilities into the vulnerability database, and then use other firmware outside the vulnerability database to calculate the cosine similarity through analysis and feature extraction, and judge whether there is a vulnerability according to the numerical results.
Similarly, taking CVE-2018-18708 as an example, this vulnerability is a Tenda router buffer overflow vulnerability. The overflow occurs when the sub_C24C0 function calls the strcpy() function. Through the backtracking of the vulnerability parameters and the analysis of the function call relationship showing in Fig. 4.
In the formSetMacFilterCfg() function, no boundary check is performed when assigning the parameter deviceList, resulting in buffer overflow. The vulnerability pseudocode is shown in Fig. 5.

sub_C24C0 function call relationship.

formSetMacFilterCfg pseudo code.
Referring to the EXP core code of the CVE-2018-18708 vulnerability, as shown in Listing 1, it is confirmed that the trigger location of the vulnerability is also in setMacFilterCft.
We built a vulnerability database, which contains a variety of different vulnerability types from multiple manufacturers, and then imported the firmware to be detected into the similarity detection model, and output the possible vulnerabilities according to the matching results. Table 1 shows the vulnerability similarity detection results, which are very consistent with the officially reported vulnerabilities, which proves that our method is effective.

EXP core code for CVE-2018-18708 vulnerability
Firmware vulnerability detection experiment results
This paper firstly analyzes the current vulnerability detection technologies for IOT firmware, including static analysis technology, symbolic execution technology, and fuzzing testing technology. Then, based on the existing technology and our experimental findings, a lightweight vulnerability detection scheme based on homology analysis is proposed. Select constant strings, internal identification strings, debugging information, and dependent library names, version numbers and other information as features, and convert the feature values into feature vectors. The contents saved in the vector are: file name, string, parameter name, The name of the function where it is located, the name of the parent function, the detailed information of the public vulnerability is studied, the process of the vulnerability is reproduced, and the vector value of the firmware with the vulnerability is saved in the vulnerability database. When detecting the vulnerability of the firmware to be tested, first extract the feature vector of the firmware to be tested, and perform a cosine similarity calculation with the vector value of the standard vulnerability in the database, and detect whether there is a vulnerability in the firmware according to the calculation result. The experimental results show that this scheme can effectively detect the vulnerabilities in firmware.
The method in this paper also has great limitations. The supported vulnerability types are relatively limited, the number of vulnerability samples is not rich enough, and the firmware detection process lacks a complete classification and optimization process. These are areas that need to be improved in future research.
Footnotes
Acknowledgement
This work was supported by the State Grid Shandong Electric Power Company Science and Technology Project Funded “Research on Smart IoT Active Security Protection and Simulation and Detection Technology for Energy Internet” (5206002000V5).
Conflict of interest
None to report.
