OPEN ACCESS
National defense science and technology (NDST) provides the driving force and guarantee for the building of a strong army and a prosperous country. Considering the threats to NDST security, this paper explores deep into the security evaluation and prewarning mechanism of NDST. Firstly, an evaluation index system was constructed for NDST security, in the light of the factors affecting NDST security. Next, a matterelement extension model (MEEM) was established based on entropy weight (EW), drawing on industrial security theory and operations research. The established EWMEEM was applied to evaluate the NDST security of China in 20102017. Based on the evaluation results, a prewarning mechanism was built to promote the future development of China’s NDST.
national defense science and technology (NDST), matterelement extension model (MEEM) based on entropy weight (EW), industrial security, prewarning mechanism
National defense science and technology (NDST) is a national strategic industry that leads hightech development. This industry plays an important role in the modernization of national defense and economic construction. For a long time, the NDST in China operates within a selfcontained system, which severely restricts innovation. As China steps up structural reform, the NDST gradually opens to domestic and international markets. Security becomes the top priority in the healthy and sustainable development of the NDST.
The NDST security mainly refers to the health and sustainability of the NDST under the fair environment of economic and trade, which ensures the overall stability and coordinated development of economy and society [1]. The existing studies on the NDST mainly focus on technical security, failing to explore deep into the security evaluation or prewarning mechanism of the NDST [2].
In the field of NDST security, Baark [3] was the first to sort out China’s policies on NDST security from 1949 to 1989. Alexander [4] discussed NDST security from the angle of technical intensity, and demonstrated his ideas with the intellectual property rights and technical security of Russia. From the angle of globalization, National Research Council [5] illustrated the US regulations on homeland security and S&T import and export. Chen and Sun [6] put forward legal countermeasures for the common issues of NDST security. Meng and Han [7] put forward NDST security countermeasures to solve the challenges of China’s NDST in the context of the Internet and big data. From the perspective of security guarantees, Sun et al. [8] summarized the influencing factors of NDST security, and designed the strategies to improve NDST in the following three dimensions: industrial level, talent reserve, and operation system.
The theories on prewarning mechanism have been mainly applied in food, automobiles, ecology, and finance [912]. The most popular prewarning methods include fuzzy comprehensive evaluation (FCE), and backpropagation neural network (BPNN) [13, 14]. The FCE classifies the evaluation objects by fuzzy mathematics. On the upside, the FCE could overcome the difficulty in quantifying qualitative problems. On the downside, this method loses lots of information in the weighting process, making the final weights unscientific. Focusing on local minimization, the BPNN often suffers from overfitting and low prediction accuracy [15]. Therefore, the common prewarning methods are too limited to reflect the dynamic changes or the multilayer and multidimensional state of NDST security [16].
For NDST security, the prewarning mechanism must involve various attributes that are not compatible with each other. Based on matterelement extension model (MEEM), this paper introduces the entropy weight (EW) method to establish an NDST security evaluation model, and applies the model to assess the security level of NDST in 20102019. The proposed model effectively solves the multiattribute and incompatibility problems of NDST evaluation, reduces the uncertainty of subjectively determined weights, and improves the evaluation system. The research results help to expand the application scope of the EWbased MEEM.
The remainder of this paper is organized as follows:
·Section 2 identifies the influencing factors of NDST security; ·Section 3 establishes an evaluation index system and sets up the EWbased MEEM for NDST security evaluation;
·Section 4 verifies the proposed index system and evaluation model through empirical analysis;
·Section 5 puts forward the conclusions.
The previous studies have shown that NDST security has two meanings: the internal security of NDST, and the external impacts on NDST security (e.g. the impacts from politics, economy, and military). This paper fully explores the influencing factors of NDST security from two dimensions: internal subjects and external environment.
The internal subjects mainly fall into two aspects, namely, industrial scale and technical capacity. The specific indices include NDST output increase, NDST R&D increase, growth rate of the number of NDST scientific research projects, growth rate of the number of NDST masters and doctors, and growth rate of employment in NDST and relevant industries.
The external environment mainly involves two indices: NDST export as a proportion of the total export of equipment, and NDST pollution emissions.
The MEEM is a formal model based on extension theory and matterelement theory. The core idea of the MEEM is to transform contradictory, incompatible evaluation indices into compatible ones, facilitating subsequent evaluation. The three elements of MEEM are matterelement model, extension set and correlation function.
The logical cells of the MEEM are matterelements: R=(N, C, V), where N is the name of the thing; C is the name field of feature; V is the value. The extension set extends the logical value from [0, 1] to [∞, +∞], indicating the change in the nature of things. The correlation function expresses the quantitative transform of the nature of things from the perspective of quantitative analysis.
According to the three elements, the NDST security evaluation and prewarning mechanism were transformed into a formal model. The correlations between the evaluation indices were established, using extension set and correlation function. In this way, a multilevel multiindex model was constructed for overall evaluation and prewarning of NDST security.
3.1 Determining the classic domains, joint domains, and target matterelements
(1) Matterelement
The first step is to set up the evaluation index system of NDST security. Suppose the security evaluation has m levels and n influencing factors. Then, the ndimensional matterelement for NDST security can be expressed as:
$R=\left( {{N}_{j}},{{C}_{i}},{{V}_{jl}} \right)=\left[ \begin{matrix} {{N}_{j}} & {{C}_{1}} & {{V}_{j1}} \\ {} & {{C}_{2}} & {{V}_{j2}} \\ {} & \vdots & \vdots \\ {} & {{C}_{\text{m}}} & {{V}_{jm}} \\ \end{matrix} \right]$ (1)
where, N_{j} is the j levels of NDST security (j=1,2, …, m); C_{i} is the ith feature of security level (i=1,2, …, n); V_{ji} is the value range of feature C_{i} under each security level.
(2) Classic domains and joint domains
According to the levels of NDST security levels, the matterelement model R_{oj} at the jth security level can be expressed as:
$\begin{align} & {{\text{R}}_{\text{o}j}}=\left( {{N}_{oj}},{{C}_{i}},{{V}_{oj}} \right)=\left[ \begin{matrix} {{N}_{oj}} & {{C}_{1}} & {{V}_{oj1}} \\ {} & {{C}_{2}} & {{V}_{oj2}} \\ {} & \vdots & \vdots \\ {} & {{C}_{n}} & {{V}_{ojn}} \\ \end{matrix} \right] \\ & \text{ }=\left[ \begin{matrix} {{N}_{oj}} & {{C}_{1}} & \left\langle {{x}_{oj1}},{{y}_{oj1}} \right\rangle \\ {} & {{C}_{2}} & \left\langle {{x}_{oj2}},{{y}_{oj2}} \right\rangle \\ {} & \vdots & \vdots \\ {} & {{C}_{n}} & \left\langle {{x}_{ojn}},{{y}_{ojn}} \right\rangle \\ \end{matrix} \right] \\ \end{align}$ (2)
where, N_{oj} is the NDST security of the jth level; V_{ojk}=<x_{ojk},y_{ojk}>(j=1,2, …, m; k=1,2, …, n) is the value range of the kth factor under the jth security level.
Based on the classic domains, the matterelement model in the joint domains R_{p} can be derived from the value range of each characteristic index:
$\begin{align} & {{\text{R}}_{\text{p}}}=\left( {{N}_{p}},{{C}_{i}},{{V}_{p}} \right)=\left[ \begin{matrix} {{N}_{p}} & {{C}_{1}} & {{V}_{p1}} \\ {} & {{C}_{2}} & {{V}_{p2}} \\ {} & \vdots & \vdots \\ {} & {{C}_{n}} & {{V}_{pn}} \\ \end{matrix} \right] \\ & \text{ }=\left[ \begin{matrix} {{N}_{p}} & {{C}_{1}} & \left\langle {{x}_{p1}},{{y}_{p1}} \right\rangle \\ {} & {{C}_{2}} & \left\langle {{x}_{p2}},{{y}_{p2}} \right\rangle \\ {} & \vdots & \vdots \\ {} & {{C}_{n}} & \left\langle {{x}_{pn}},{{y}_{pn}} \right\rangle \\ \end{matrix} \right] \\ \end{align}$ (3)
where, N_{p} is all the levels of NDST security; V_{pk}=<x_{pk},y_{pk}> is the value range of C_{i} under all security levels.
(3) Matterelement matrix
By the definition of target matterelement, a matterelement matrix was established for NDST security according to the features of the relevant evaluation indices, that is, the data or result of each factor in the index system were expressed as a matterelement matrix:
${{R}_{m}}=\left( {{N}_{m}},{{C}_{i}},{{V}_{mn}} \right)=\left[ \begin{matrix} {{N}_{m}} & {{C}_{1}} & {{V}_{m1}} \\ {} & {{C}_{2}} & {{V}_{m2}} \\ {} & \vdots & \vdots \\ {} & {{C}_{\text{m}}} & {{V}_{mn}} \\ \end{matrix} \right]$ (4)
where, N_{m} is the mth NDST security level; C_{k} is the kth index affecting NDST security (k=1, 2, ……, n); V_{mk} is the value of the kth influencing factor at the mth security level.
3.2 Construction of EWbased MEEM
(1) Correlation function
After modelling the matterelement and setting up the matterelement matrix, it is necessary to evaluate the NDST security by correlation function. In the previous subsection, the value ranges of the factors affecting NDST security were determined: V_{ojk}=<x_{ojk},y_{ojk}> is the value range of the kth index x_{k} under the jth security level; V_{pk}=<x_{pk},y_{pk}> is the value range of the kth index x_{k} under all security levels; V_{j} is the value of the kth index; V_{ojk}⊂V_{pk}. On this basis, the correlation function of the kth index of type i matterelement relative to the jth security level can be described as:
${{r}_{ij}}\left( {{v}_{ik}} \right)=\left\{ \begin{matrix} \frac{\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)}{\left {{v}_{ojk}} \right},{{v}_{ik}}\in {{v}_{ojk}} \\ \frac{\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)}{\rho \left( {{v}_{ik}},{{v}_{pk}} \right)\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)},{{v}_{ik}}\overline{\in }{{v}_{ojk}} \\ \end{matrix} \right.$ (5)
The proximities between value V_{k} and intervals V_{ojk} and V_{pk} can be respectively expressed as:
$p\left( {{v}_{k}},{{v}_{ojk}} \right)=\left {{v}_{k}}\frac{{{a}_{ojk}}+{{b}_{ojk}}}{2} \right\frac{1}{2}\left( {{b}_{ojk}}{{a}_{ojk}} \right)$ (6)
$p\left( {{v}_{k}},{{v}_{pk}} \right)=\left {{v}_{k}}\frac{{{a}_{pk}}+{{b}_{pk}}}{2} \right\frac{1}{2}\left( {{b}_{pk}}{{a}_{pk}} \right)$ (7)
(2) Weighting by EW method
The concept of entropy comes from information theory. The entropy measures the dispersion of indices. The smaller the entropy, the greater the dispersion, and the more important the index. Here, the EW method is introduced to weight the evaluation indices of NDST security:
Step 1. Construct a judgment matrix R based on the security levels and evaluation indices:
R=(r_{ij})_{(y×n)} (i=1, 2, …, y; j=1, 2, …, n) (8)
Step 2. Normalize the judgement matrix R.
Step 3. Determine the entropy of each index:
${{H}_{j}}=\frac{1}{\ln y}\sum\limits_{i=1}^{y}{{{f}_{ij}}\ln {{f}_{ij}}}\left( i=1,2,\cdots ,y;j=1,2,\cdots ,n \right)$ (9)
${{f}_{ij}}=\left( 1+{{r}_{ij}} \right)/\sum\limits_{i=1}^{y}{\left( 1+{{r}_{ij}} \right)}$ (10)
Step 4. Calculate the EW of each index:
$w={{\left( {{w}_{j}} \right)}_{l\times n}}=\frac{1{{H}_{j}}}{n\sum\limits_{j=1}^{n}{{{H}_{j}}}},\text{ }and\text{ }\sum\limits_{j=1}^{n}{{{w}_{j}}}=1$ (11)
(3) Correlation of NDST security
The correlation between matterelement N_{i} and security level t can be calculated by:
${{k}_{it}}\left( {{N}_{i}} \right)=\sum{{{w}_{j}}}{{r}_{it}}\left( {{v}_{ij}} \right)$ (12)
The security level of matterelement N_{i} can be determined by:
${{k}_{ito}}=\max \left\{ {{k}_{it}}\left( {{N}_{i}} \right),t=1,2,\cdots s \right\}$ (13)
3.3 Workflow of our evaluation model
As shown in Figure 1, the NDST security was evaluated in the following steps: First, the evaluation index system was established based on the factors affecting the NDST security; Second, the classic domains, joint domains, and target matterelements were identified, according to the features of NDST (formulas (1)(4)); Third, the correlation function of evaluation indices were computed by formulas (5)(7); Fourth, the index weights were determined by the EW method; Fifth, the correlation between each matterelement and security level, and the security level of the matterelement were obtained by formulas (12) and (13); The evaluation results are used to establish a prewarning mechanism.
Figure 1. Workflow of NDST security evaluation by EWbased MEEM
To evaluate the current state of China’s NDST security, this paper applies the EWbased MEEM to assess the NDST security based on the relevant data in 2017, and establishes a prewarning mechanism based on the evaluation results.
4.1 Evaluation index system
Considering the influencing factors of NDST security, an evaluation index system for NDST security (Table 1) was constructed in the principles of systematicity, dynamicity and quantification [17].
4.2 Classic domains, joint domains, and target matterelements
According to the experience of security classification in the field of industrial security evaluation, the NDST security was divided into five levels: strongly secure N_{01}, slightly secure N_{02}, neutral N_{03}, slightly insecure N_{04}, and strongly insecure N_{05}. The division was carried out under the relevant policies and rules of China’ State Administration of Science, Technology and Industry for National Defense [18, 19]. The value ranges of specific indices are listed in Table 2.
Table 1. Our evaluation index system
Primary index 
Secondary indices 
Tertiary indices 
NDST security 
Market share 
NDST export as a proportion of the total export of equipment X_{1} 
Environment 
NDST pollution emissions X_{2} 

Scale 
NDST output increase X_{3} 

NDST R&D increase X_{4} 

Technical capacity 
Growth rate of the number of NDST scientific research projects X_{5} 

Growth rate of the number of NDST masters and doctors X_{6} 

Growth rate of employment in NDST and relevant industries X_{7} 
Table 2. The value ranges of indices under each security level
Index Level 
X_{1} 
X_{2} 
X_{3} 
X_{4} 
X_{5} 
X_{6} 
X_{7} 
N_{01} 
<0,20> 
<0,10> 
<0,30> 
<0,8> 
<0,20> 
<0,20> 
<0,5> 
N_{02} 
<20,40> 
<10,30> 
<30,90> 
<8,16> 
<20,40> 
<20,40> 
<5,15> 
N_{03} 
<40,60> 
<30,50> 
<90,180> 
<16,24> 
<40,60> 
<40,60> 
<15,25> 
N_{04} 
<60,80> 
<50,75> 
<180,270> 
<24,32> 
<60,80> 
<60,80> 
<25,40> 
N_{05} 
<80,100> 
<75,100> 
<270,360> 
<32,40> 
<80,100> 
<80,100> 
<40,60> 
Based on the value ranges in Table 2, the classic domain matrix and joint domain matrix can be obtained by formulas (1)(3):
$R_{01}=\left[\begin{array}{lll}N_{01} & X_{1} & \langle 0,20\rangle \\ & X_{2} & \langle 0,10\rangle \\ & X_{3} & \langle 0,30\rangle \\ & X_{4} & \langle 0,8\rangle \\ & X_{5} & \langle 0,20\rangle \\ & X_{6} & \langle 0,20\rangle \\ & X_{7} & \langle 0,5\rangle\end{array}\right], R_{02}=\left[\begin{array}{ccc}N_{02} & X_{1} & \langle 20,40\rangle \\ & X_{2} & \langle 10,30\rangle \\ & X_{3} & \langle 30,90\rangle \\ & X_{4} & \langle 8,16\rangle \\ & X_{5} & \langle 20,40\rangle \\ & X_{6} & \langle 20,40\rangle \\ & X_{7} & \langle 5,15\rangle\end{array}\right]$
$R_{03}=\left[\begin{array}{ccc}N_{03} & X_{1} & \langle 40,60\rangle \\ & X_{2} & \langle 30,50\rangle \\ & X_{3} & \langle 90,180\rangle \\ & X_{4} & \langle 16,24\rangle \\ & X_{5} & \langle 40,60\rangle \\ & X_{6} & \langle 40,60\rangle \\ & X_{7} & \langle 10,25\rangle\end{array}\right], R_{04}=\left[\begin{array}{ccc}N_{04} & X_{1} & \langle 60,80\rangle \\ & X_{2} & \langle 50,75\rangle \\ & X_{3} & \langle 180,270\rangle \\ & X_{4} & \langle 24,32\rangle \\ & X_{5} & \langle 60,80\rangle \\ & X_{6} & \langle 60,80\rangle \\ & X_{7} & \langle 25,40\rangle\end{array}\right]$
$R_{05}=\left[\begin{array}{ccc}N_{05} & X_{1} & \langle 80,100\rangle \\ & X_{2} & \langle 75,100\rangle \\ & X_{3} & \langle 270,360\rangle \\ & X_{4} & \langle 32,40\rangle \\ & X_{5} & \langle 80,100\rangle \\ & X_{6} & \langle 80,100\rangle \\ & X_{7} & \langle 40,60\rangle\end{array}\right], R_{P}=\left[\begin{array}{ccc}N_{P} & X_{1} & \langle 0,100\rangle \\ & X_{2} & \langle 0,100\rangle \\ & X_{3} & \langle 0,360\rangle \\ & X_{4} & \langle 0,40\rangle \\ & X_{5} & \langle 0,100\rangle \\ & X_{6} & \langle 0,100\rangle \\ & X_{7} & \langle 0,60\rangle\end{array}\right]$ .
After that, the matterelements for NDST security evaluation were determined by the calculation formula of each index and formula (4). The matterelement matrix of 2017 was computed based on the basic data on NDST obtained from the NDST Industrial Analysis Report and China Statistical Yearbook:
$R_{2017}=\left[\begin{array}{ccc}N_{P} & X_{1} & 82.28 \\ & X_{2} & 85.34 \\ & X_{3} & 261.46 \\ & X_{4} & 33.42 \\ & X_{5} & 83.43 \\ & X_{6} & 75.25 \\ & X_{7} & 45.67\end{array}\right]$.
4.3 Correlation calculation
The basic data on NDST in 2017 were imported into the EWbased MEEM. Then, the correlations between each index and the security levels were obtained by formulas (5)(7). The results are listed in Table 3 below.
4.4 Index weighting based on EW method
The entropy weight (Table 4) of each index relative to each security level in 2017 was derived by formulas (8)(11).
4.5 NDST security level
After obtaining the correlation and entropy weight of each index relative to each security level, the weighted correlations between each index and the security levels in 2017 were computed by formulas (12) and (13). The weighted correlations in other years were determined in a similar manner. The computed results are recorded in Table 5 below.
As shown in Table 5, China’s NDST security was relatively good in 20102012, but dropped to the neutral state in 20132015. In 20162017, the NDST became slightly insecure. The NDST security exhibited a gradual decline in the research period. Despite the major progress in technical innovation, the NDST in China is not very secure, judging by factors like environment and scale. This is mainly attributable to the growing number of unsafe and unstable factors, as the NDST pursues economic benefits and marketoriented development in the information age.
The NDST is crucial to national security, and affects the economy, politics, and military of China. Facing the slightly insecure state, it is imperative to design a prewarning mechanism to ensure the stable and healthy development of NDST in the long run.
Table 3. Correlations between each index and the security levels in 2017
Correlation Level 
K_{j}(v_{1}) 
K_{j}(v_{2}) 
K_{j}(v_{3}) 
K_{j}(v_{4}) 
K_{j}(v_{5}) 
K_{j}(v_{6}) 
K_{j}(v_{7}) 
N_{01} 
0.339 
0.391 
0.430 
0.503 
0.608 
0.683 
0.508 
N_{02} 
0.345 
0.330 
0.305 
0.425 
0.578 
0.478 
0.523 
N_{03} 
0.214 
0.321 
0.014 
0.214 
0.354 
0.154 
0.324 
N_{04} 
0.036 
0.137 
0.039 
0.136 
0.102 
0.220 
0.182 
N_{05} 
0.124 
0.021 
0.209 
0.024 
0.134 
0.234 
0.104 
Table 4. Entropy weight of each index relative to each security level in 2017
Entropy weight Level 
w_{j}(v_{1}) 
w_{j}(v_{2}) 
w_{j}(v_{3}) 
w_{j}(v_{4}) 
w_{j}(v_{5}) 
w_{j}(v_{6}) 
w_{j}(v_{7}) 
N_{01} 
0.173 
0.163 
0.164 
0.17 
0.136 
0.123 
0.071 
N_{02} 
0.113 
0.179 
0.197 
0.166 
0.107 
0.136 
0.102 
N_{03} 
0.182 
0.152 
0.088 
0.127 
0.153 
0.107 
0.191 
N_{04} 
0.321 
0.176 
0.058 
0.089 
0.108 
0.119 
0.129 
N_{05} 
0.235 
0.176 
0.115 
0.117 
0.132 
0.088 
0.137 
Table 5. Weighted correlations between each index and the security levels in 20102017
Weighted correlation Level 
K_{j}(2010) 
K_{j}(2011) 
K_{j}(2012) 
K_{j}(2013) 
K_{j}(2014) 
K_{j}(2015) 
K_{j}(2016) 
K_{j}(2017) 
N_{01} 
0.249 
0.168 
0.418 
0.481 
0.341 
0.228 
0.501 
0.481 
N_{02} 
0.017 
0.137 
0.219 
0.409 
0.306 
0.326 
0.315 
0.409 
N_{03} 
0.283 
0.290 
0.295 
0.121 
0.149 
0.034 
0.205 
0.249 
N_{04} 
0.341 
0.198 
0.311 
0.249 
0.419 
0.212 
0.034 
0.111 
N_{05} 
0.572 
0.211 
0.412 
0.342 
0.512 
0.169 
0.193 
0.112 
Security level 
Slightly secure 
Slightly secure 
Slightly secure 
Neutral 
Neutral 
Neutral 
Slightly insecure 
Slightly insecure 
4.6 Prewarning mechanism for NDST security
Considering the relatively low security of NDST, this paper sets up a prewarning mechanism for NDST security based on the results of EWbased MEEM, in an attempt to safeguard the stable and orderly development of NDST.
The prewarning system aims to predict the influencing factors and future trend of NDST market, evaluate and diagnose the overall state of NDST security, and judge the change trend of the security level, enabling the competent authorities to take effective precautions.
The effectiveness of the prewarning mechanism depends on the reasonability and reliability of the evaluation system of industrial security. Hence, the prewarning criteria for NDST security were developed based on the results of EWbased MEEM (Table 5). With the aid of the criteria, the competent authorities could act effectively according to specific security state and actual conditions, thereby improving the NDST security and ensuring the safe operation of the country.
This paper mainly explores the security evaluation of China’s NDST, and proposes a security evaluation model and a prewarning mechanism based on EW method and MEEM. First, the influencing factors of NDST security were analyzed, and an evaluation index system was constructed for NDST security. Next, the EW method was introduced to weight the indices, and the EWbased MEEM was established. After that, the proposed model was applied to evaluate the NDST security of China in 2017. The empirical results show that China’s NDST security changed from slightly secure to neutral, and to slightly insecure, exhibiting a gradual decline. Based on the evaluation results, a prewarning mechanism was built for NDST security, which effectively coordinates the “going global” strategy and security of NDST. The research results shed new light on the prewarning of NDST.
[1] Yan, H.N. (2016). Research on the evolution path of the civilmilitary integration of innovation ability in national defense industry. Science & Technology & Economy, 29(2): 1620. https://doi.org/10.3969/j.issn.10037691.2016.02.004
[2] Stadelmann, D., Portmann, M., Eichenberger, R. (2015). Military careers of politicians matter for national security policy. Journal of Economic Behavior & Organization, 116: 142156. https://doi.org/10.1016/j.jebo.2015.04.001
[3] Baark, E. (1994). China's science and technology policy: 19491989. The China Quarterly, 140: 11671168. https://doi.org/10.1017/S0305741000053133
[4] Alexander, V. (2004). Innovation management in Russia (problems of strategy and science and technology security). Books of Alexander Varshavsky  Socionet
[5] National Research Council. (2009). Beyond'Fortress America': National security controls on science and technology in a globalized world. National Academies Press. https://doi.org/10.17226/12567
[6] Chen, S.P., Sun, J. (2017). Discussion on the legislation system construction of national defense science and technology security guarantee under the background of greater civilmilitary integration. Law Science Magazine, 38(5): 1927.
[7] Meng, F.S., Han, B. (2016). The national defense science technology and industry independently controllable development research under the new situation. Scientific Management Research, 34(2): 14.
[8] Sun, D.M., Lu, S., Sun, Y.Y. (2019). Research on countermeasures for science and technology security problems in national defense science and technology industry. In 2019 3rd International Conference on Education, Economics and Management Research, pp. 3942. https://doi.org/10.2991/assehr.k.191221.010
[9] Xiang, Y.Q., Zhang, Q.H., Han, P., Chen, M.Y. (2014). Research of earlywarning expert system for security of grain storage based on uncertain inference. Applied Mechanics and Materials, 536: 437442. https://doi.org/10.4028/www.scientific.net/AMM.536537.437
[10] Yousefi, A., HadiVencheh, A. (2010). An integrated group decision making model and its evaluation by DEA for automobile industry. Expert Systems with Applications, 37(12): 85438556. https://doi.org/10.1016/j.eswa.2010.05.021
[11] Shi, M.K., Peng, Y., Li, E.D., Wen, C.H. (2005). Study on early warning of ecological security in ethnic agricultural regions of the upper reaches of the pearl river an analysis on actual examples of 22 counties within the boundary of Guizhou. China Population Resources and Environment, 15(6): 5054. https://doi.org/10.3969/j.issn.10022104.2005.06.012
[12] Bussiere, M., Fratzscher, M. (2006). Towards a new early warning system of financial crises. journal of International Money and Finance, 25(6): 953973. https://doi.org/10.1016/j.jimonfin.2006.07.007
[13] Soomro, Z.A., Shah, M.H., Ahmed, J. (2016). Information security management needs more holistic approach: A literature review. International Journal of Information Management, 36(2): 215225. https://doi.org/10.1016/j.ijinfomgt.2015.11.009
[14] Spanos, G., Angelis, L. (2016). The impact of information security events to the stock market: A systematic literature review. Computers & Security, 58: 216229. https://doi.org/10.1016/j.cose.2015.12.006
[15] Chen, J.F., Hsieh, H.N., Do, Q.H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28: 100108. https://doi.org/10.1016/j.asoc.2014.11.050
[16] Jin, W., Li, Z.J., Wei, L.S., Zhen, H. (2000). The improvements of BP neural network learning algorithm. WCC 2000  ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000, Beijing, China, pp. 16471649. https://doi.org/10.1109/ICOSP.2000.893417.
[17] Yu, J.K., Liu, S.Q. (2013) A study on safety evaluation and early warning mechanism for China’s marine transportation industry. China Maritime Safety, (3): 2225. https://doi.org/10.3969/j.issn.16732278.2013.03.016
[18] Zhang, W.Q., Zhang, H. (2018). Measurement of regional industrial structure optimal under the restriction of resources and environment. s Statistics & Information Forum, 33(3): 8086. https://doi.org/10.3969/j.issn.10073116.2018.03.012
[19] Ning, G.Y., Zhao, J.F. (2016). Research on industrial safety evaluation model under the restriction of resources and environment. Proceedings of the 2016 International Conference on Education, Management, Computer and Society, pp. 319321. https://doi.org/10.2991/emcs16.2016.156