Entropy Weight-Based Matter-Element Extension Model for Security Evaluation and Prewarning Mechanism of National Defense Science and Technology

Entropy Weight-Based Matter-Element Extension Model for Security Evaluation and Prewarning Mechanism of National Defense Science and Technology

Zhe Xu Binghui ZhangMing Sun Guojian Pan 

Department of National Defense Economy, Army Logistics University of PLA, Chongqing 401311, China

Department of Economic Management, No. 960 Hospital, PLA, Jinan 250031, China

Department of Quartermaster & Procurement, Army Logistics University of PLA, Chongqing 401311, China

Corresponding Author Email: 
binghui2018@162.com
Page: 
279-284
|
DOI: 
https://doi.org/10.18280/ijsse.100216
Received: 
10 November 2019
|
Accepted: 
7 February 2020
|
Published: 
30 April 2020
| Citation

OPEN ACCESS

Abstract: 

National defense science and technology (NDST) provides the driving force and guarantee for the building of a strong army and a prosperous country. Considering the threats to NDST security, this paper explores deep into the security evaluation and prewarning mechanism of NDST. Firstly, an evaluation index system was constructed for NDST security, in the light of the factors affecting NDST security. Next, a matter-element extension model (MEEM) was established based on entropy weight (EW), drawing on industrial security theory and operations research. The established EW-MEEM was applied to evaluate the NDST security of China in 2010-2017. Based on the evaluation results, a prewarning mechanism was built to promote the future development of China’s NDST.

Keywords: 

national defense science and technology (NDST), matter-element extension model (MEEM) based on entropy weight (EW), industrial security, prewarning mechanism

1. Introduction

National defense science and technology (NDST) is a national strategic industry that leads high-tech development. This industry plays an important role in the modernization of national defense and economic construction. For a long time, the NDST in China operates within a self-contained system, which severely restricts innovation. As China steps up structural reform, the NDST gradually opens to domestic and international markets. Security becomes the top priority in the healthy and sustainable development of the NDST.

The NDST security mainly refers to the health and sustainability of the NDST under the fair environment of economic and trade, which ensures the overall stability and coordinated development of economy and society [1]. The existing studies on the NDST mainly focus on technical security, failing to explore deep into the security evaluation or prewarning mechanism of the NDST [2].

In the field of NDST security, Baark [3] was the first to sort out China’s policies on NDST security from 1949 to 1989. Alexander [4] discussed NDST security from the angle of technical intensity, and demonstrated his ideas with the intellectual property rights and technical security of Russia. From the angle of globalization, National Research Council [5] illustrated the US regulations on homeland security and S&T import and export. Chen and Sun [6] put forward legal countermeasures for the common issues of NDST security. Meng and Han [7] put forward NDST security countermeasures to solve the challenges of China’s NDST in the context of the Internet and big data. From the perspective of security guarantees, Sun et al. [8] summarized the influencing factors of NDST security, and designed the strategies to improve NDST in the following three dimensions: industrial level, talent reserve, and operation system.

The theories on prewarning mechanism have been mainly applied in food, automobiles, ecology, and finance [9-12]. The most popular prewarning methods include fuzzy comprehensive evaluation (FCE), and backpropagation neural network (BPNN) [13, 14]. The FCE classifies the evaluation objects by fuzzy mathematics. On the upside, the FCE could overcome the difficulty in quantifying qualitative problems. On the downside, this method loses lots of information in the weighting process, making the final weights unscientific. Focusing on local minimization, the BPNN often suffers from over-fitting and low prediction accuracy [15]. Therefore, the common prewarning methods are too limited to reflect the dynamic changes or the multi-layer and multi-dimensional state of NDST security [16].

For NDST security, the prewarning mechanism must involve various attributes that are not compatible with each other. Based on matter-element extension model (MEEM), this paper introduces the entropy weight (EW) method to establish an NDST security evaluation model, and applies the model to assess the security level of NDST in 2010-2019. The proposed model effectively solves the multi-attribute and incompatibility problems of NDST evaluation, reduces the uncertainty of subjectively determined weights, and improves the evaluation system. The research results help to expand the application scope of the EW-based MEEM.

The remainder of this paper is organized as follows:

·Section 2 identifies the influencing factors of NDST security; ·Section 3 establishes an evaluation index system and sets up the EW-based MEEM for NDST security evaluation;

·Section 4 verifies the proposed index system and evaluation model through empirical analysis;

·Section 5 puts forward the conclusions.

2. Influencing Factors of NDST Security

The previous studies have shown that NDST security has two meanings: the internal security of NDST, and the external impacts on NDST security (e.g. the impacts from politics, economy, and military). This paper fully explores the influencing factors of NDST security from two dimensions: internal subjects and external environment.

The internal subjects mainly fall into two aspects, namely, industrial scale and technical capacity. The specific indices include NDST output increase, NDST R&D increase, growth rate of the number of NDST scientific research projects, growth rate of the number of NDST masters and doctors, and growth rate of employment in NDST and relevant industries.

The external environment mainly involves two indices: NDST export as a proportion of the total export of equipment, and NDST pollution emissions.

3. Methodology

The MEEM is a formal model based on extension theory and matter-element theory. The core idea of the MEEM is to transform contradictory, incompatible evaluation indices into compatible ones, facilitating subsequent evaluation. The three elements of MEEM are matter-element model, extension set and correlation function.

The logical cells of the MEEM are matter-elements: R=(N, C, V), where N is the name of the thing; C is the name field of feature; V is the value. The extension set extends the logical value from [0, 1] to [-∞, +∞], indicating the change in the nature of things. The correlation function expresses the quantitative transform of the nature of things from the perspective of quantitative analysis.

According to the three elements, the NDST security evaluation and prewarning mechanism were transformed into a formal model. The correlations between the evaluation indices were established, using extension set and correlation function. In this way, a multi-level multi-index model was constructed for overall evaluation and prewarning of NDST security.

3.1 Determining the classic domains, joint domains, and target matter-elements

(1) Matter-element

The first step is to set up the evaluation index system of NDST security. Suppose the security evaluation has m levels and n influencing factors. Then, the n-dimensional matter-element for NDST security can be expressed as:

$R=\left( {{N}_{j}},{{C}_{i}},{{V}_{jl}} \right)=\left[ \begin{matrix}   {{N}_{j}} & {{C}_{1}} & {{V}_{j1}}  \\   {} & {{C}_{2}} & {{V}_{j2}}  \\   {} & \vdots  & \vdots   \\   {} & {{C}_{\text{m}}} & {{V}_{jm}}  \\ \end{matrix} \right]$           (1)

where, Nj is the j levels of NDST security (j=1,2, …, m); Ci is the i-th feature of security level (i=1,2, …, n); Vji is the value range of feature Ci under each security level.

(2) Classic domains and joint domains

According to the levels of NDST security levels, the matter-element model Roj at the j-th security level can be expressed as:

$\begin{align}  & {{\text{R}}_{\text{o}j}}=\left( {{N}_{oj}},{{C}_{i}},{{V}_{oj}} \right)=\left[ \begin{matrix} {{N}_{oj}} & {{C}_{1}} & {{V}_{oj1}}  \\   {} & {{C}_{2}} & {{V}_{oj2}}  \\   {} & \vdots  & \vdots   \\   {} & {{C}_{n}} & {{V}_{ojn}}  \\ \end{matrix} \right] \\  & \text{      }=\left[ \begin{matrix}   {{N}_{oj}} & {{C}_{1}} & \left\langle {{x}_{oj1}},{{y}_{oj1}} \right\rangle   \\   {} & {{C}_{2}} & \left\langle {{x}_{oj2}},{{y}_{oj2}} \right\rangle   \\   {} & \vdots  & \vdots   \\   {} & {{C}_{n}} & \left\langle {{x}_{ojn}},{{y}_{ojn}} \right\rangle   \\ \end{matrix} \right] \\ \end{align}$              (2)

where, Noj is the NDST security of the j-th level; Vojk=<xojk,yojk>(j=1,2, …, m; k=1,2, …, n) is the value range of the k-th factor under the j-th security level.

Based on the classic domains, the matter-element model in the joint domains Rp can be derived from the value range of each characteristic index:

$\begin{align}  & {{\text{R}}_{\text{p}}}=\left( {{N}_{p}},{{C}_{i}},{{V}_{p}} \right)=\left[ \begin{matrix}   {{N}_{p}} & {{C}_{1}} & {{V}_{p1}}  \\   {} & {{C}_{2}} & {{V}_{p2}}  \\   {} & \vdots  & \vdots   \\   {} & {{C}_{n}} & {{V}_{pn}}  \\ \end{matrix} \right] \\  & \text{     }=\left[ \begin{matrix}   {{N}_{p}} & {{C}_{1}} & \left\langle {{x}_{p1}},{{y}_{p1}} \right\rangle   \\   {} & {{C}_{2}} & \left\langle {{x}_{p2}},{{y}_{p2}} \right\rangle   \\   {} & \vdots  & \vdots   \\   {} & {{C}_{n}} & \left\langle {{x}_{pn}},{{y}_{pn}} \right\rangle   \\ \end{matrix} \right] \\ \end{align}$           (3)

where, Np is all the levels of NDST security; Vpk=<xpk,ypk> is the value range of Ci under all security levels.

(3) Matter-element matrix

By the definition of target matter-element, a matter-element matrix was established for NDST security according to the features of the relevant evaluation indices, that is, the data or result of each factor in the index system were expressed as a matter-element matrix:

${{R}_{m}}=\left( {{N}_{m}},{{C}_{i}},{{V}_{mn}} \right)=\left[ \begin{matrix}   {{N}_{m}} & {{C}_{1}} & {{V}_{m1}}  \\   {} & {{C}_{2}} & {{V}_{m2}}  \\   {} & \vdots  & \vdots   \\   {} & {{C}_{\text{m}}} & {{V}_{mn}}  \\ \end{matrix} \right]$          (4)

where, Nm is the m-th NDST security level; Ck is the k-th index affecting NDST security (k=1, 2, ……, n); Vmk is the value of the k-th influencing factor at the m-th security level.

3.2 Construction of EW-based MEEM

(1) Correlation function

After modelling the matter-element and setting up the matter-element matrix, it is necessary to evaluate the NDST security by correlation function. In the previous subsection, the value ranges of the factors affecting NDST security were determined: Vojk=<xojk,yojk> is the value range of the k-th index xk under the j-th security level; Vpk=<xpk,ypk> is the value range of the k-th index xk under all security levels; Vj is the value of the k-th index; VojkVpk. On this basis, the correlation function of the k-th index of type i matter-element relative to the j-th security level can be described as:

${{r}_{ij}}\left( {{v}_{ik}} \right)=\left\{ \begin{matrix}   \frac{-\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)}{\left| {{v}_{ojk}} \right|},{{v}_{ik}}\in {{v}_{ojk}}  \\   \frac{\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)}{\rho \left( {{v}_{ik}},{{v}_{pk}} \right)-\rho \left( {{v}_{ik}},{{v}_{ojk}} \right)},{{v}_{ik}}\overline{\in }{{v}_{ojk}}  \\ \end{matrix} \right.$       (5)

The proximities between value Vk and intervals Vojk and Vpk can be respectively expressed as:

$p\left( {{v}_{k}},{{v}_{ojk}} \right)=\left| {{v}_{k}}-\frac{{{a}_{ojk}}+{{b}_{ojk}}}{2} \right|-\frac{1}{2}\left( {{b}_{ojk}}-{{a}_{ojk}} \right)$          (6)

$p\left( {{v}_{k}},{{v}_{pk}} \right)=\left| {{v}_{k}}-\frac{{{a}_{pk}}+{{b}_{pk}}}{2} \right|-\frac{1}{2}\left( {{b}_{pk}}-{{a}_{pk}} \right)$         (7)

(2) Weighting by EW method

The concept of entropy comes from information theory. The entropy measures the dispersion of indices. The smaller the entropy, the greater the dispersion, and the more important the index. Here, the EW method is introduced to weight the evaluation indices of NDST security:

Step 1. Construct a judgment matrix R based on the security levels and evaluation indices:

R=(rij)(y×n) (i=1, 2, …, y; j=1, 2, …, n)           (8)

Step 2. Normalize the judgement matrix R.

Step 3. Determine the entropy of each index:

${{H}_{j}}=\frac{-1}{\ln y}\sum\limits_{i=1}^{y}{{{f}_{ij}}\ln {{f}_{ij}}}\left( i=1,2,\cdots ,y;j=1,2,\cdots ,n \right)$           (9)

${{f}_{ij}}=\left( 1+{{r}_{ij}} \right)/\sum\limits_{i=1}^{y}{\left( 1+{{r}_{ij}} \right)}$          (10)

Step 4. Calculate the EW of each index:

$w={{\left( {{w}_{j}} \right)}_{l\times n}}=\frac{1-{{H}_{j}}}{n-\sum\limits_{j=1}^{n}{{{H}_{j}}}},\text{ }and\text{ }\sum\limits_{j=1}^{n}{{{w}_{j}}}=1$           (11)

(3) Correlation of NDST security

The correlation between matter-element Ni and security level t can be calculated by:

${{k}_{it}}\left( {{N}_{i}} \right)=\sum{{{w}_{j}}}{{r}_{it}}\left( {{v}_{ij}} \right)$           (12)

The security level of matter-element Ni can be determined by:

${{k}_{ito}}=\max \left\{ {{k}_{it}}\left( {{N}_{i}} \right),t=1,2,\cdots s \right\}$          (13)

3.3 Workflow of our evaluation model

As shown in Figure 1, the NDST security was evaluated in the following steps: First, the evaluation index system was established based on the factors affecting the NDST security; Second, the classic domains, joint domains, and target matter-elements were identified, according to the features of NDST (formulas (1)-(4)); Third, the correlation function of evaluation indices were computed by formulas (5)-(7); Fourth, the index weights were determined by the EW method; Fifth, the correlation between each matter-element and security level, and the security level of the matter-element were obtained by formulas (12) and (13); The evaluation results are used to establish a prewarning mechanism.

Figure 1. Workflow of NDST security evaluation by EW-based MEEM

4. Empirical Analysis

To evaluate the current state of China’s NDST security, this paper applies the EW-based MEEM to assess the NDST security based on the relevant data in 2017, and establishes a prewarning mechanism based on the evaluation results.

4.1 Evaluation index system

Considering the influencing factors of NDST security, an evaluation index system for NDST security (Table 1) was constructed in the principles of systematicity, dynamicity and quantification [17].

4.2 Classic domains, joint domains, and target matter-elements

According to the experience of security classification in the field of industrial security evaluation, the NDST security was divided into five levels: strongly secure N01, slightly secure N02, neutral N03, slightly insecure N04, and strongly insecure N05. The division was carried out under the relevant policies and rules of China’ State Administration of Science, Technology and Industry for National Defense [18, 19]. The value ranges of specific indices are listed in Table 2.

Table 1. Our evaluation index system

Primary index

Secondary indices

Tertiary indices

NDST security

Market share

NDST export as a proportion of the total export of equipment X1

Environment

NDST pollution emissions X2

Scale

NDST output increase X3

NDST R&D increase X4

Technical capacity

Growth rate of the number of NDST scientific research projects X5

Growth rate of the number of NDST masters and doctors X6

Growth rate of employment in NDST and relevant industries X7

Table 2. The value ranges of indices under each security level

Index

Level

X1

X2

X3

X4

X5

X6

X7

N01

<0,20>

<0,10>

<0,30>

<0,8>

<0,20>

<0,20>

<0,5>

N02

<20,40>

<10,30>

<30,90>

<8,16>

<20,40>

<20,40>

<5,15>

N03

<40,60>

<30,50>

<90,180>

<16,24>

<40,60>

<40,60>

<15,25>

N04

<60,80>

<50,75>

<180,270>

<24,32>

<60,80>

<60,80>

<25,40>

N05

<80,100>

<75,100>

<270,360>

<32,40>

<80,100>

<80,100>

<40,60>

Based on the value ranges in Table 2, the classic domain matrix and joint domain matrix can be obtained by formulas (1)-(3):

$R_{01}=\left[\begin{array}{lll}N_{01} & X_{1} & \langle 0,20\rangle \\ & X_{2} & \langle 0,10\rangle \\ & X_{3} & \langle 0,30\rangle \\ & X_{4} & \langle 0,8\rangle \\ & X_{5} & \langle 0,20\rangle \\ & X_{6} & \langle 0,20\rangle \\ & X_{7} & \langle 0,5\rangle\end{array}\right], R_{02}=\left[\begin{array}{ccc}N_{02} & X_{1} & \langle 20,40\rangle \\ & X_{2} & \langle 10,30\rangle \\ & X_{3} & \langle 30,90\rangle \\ & X_{4} & \langle 8,16\rangle \\ & X_{5} & \langle 20,40\rangle \\ & X_{6} & \langle 20,40\rangle \\ & X_{7} & \langle 5,15\rangle\end{array}\right]$

$R_{03}=\left[\begin{array}{ccc}N_{03} & X_{1} & \langle 40,60\rangle \\ & X_{2} & \langle 30,50\rangle \\ & X_{3} & \langle 90,180\rangle \\ & X_{4} & \langle 16,24\rangle \\ & X_{5} & \langle 40,60\rangle \\ & X_{6} & \langle 40,60\rangle \\ & X_{7} & \langle 10,25\rangle\end{array}\right], R_{04}=\left[\begin{array}{ccc}N_{04} & X_{1} & \langle 60,80\rangle \\ & X_{2} & \langle 50,75\rangle \\ & X_{3} & \langle 180,270\rangle \\ & X_{4} & \langle 24,32\rangle \\ & X_{5} & \langle 60,80\rangle \\ & X_{6} & \langle 60,80\rangle \\ & X_{7} & \langle 25,40\rangle\end{array}\right]$

$R_{05}=\left[\begin{array}{ccc}N_{05} & X_{1} & \langle 80,100\rangle \\ & X_{2} & \langle 75,100\rangle \\ & X_{3} & \langle 270,360\rangle \\ & X_{4} & \langle 32,40\rangle \\ & X_{5} & \langle 80,100\rangle \\ & X_{6} & \langle 80,100\rangle \\ & X_{7} & \langle 40,60\rangle\end{array}\right], R_{P}=\left[\begin{array}{ccc}N_{P} & X_{1} & \langle 0,100\rangle \\ & X_{2} & \langle 0,100\rangle \\ & X_{3} & \langle 0,360\rangle \\ & X_{4} & \langle 0,40\rangle \\ & X_{5} & \langle 0,100\rangle \\ & X_{6} & \langle 0,100\rangle \\ & X_{7} & \langle 0,60\rangle\end{array}\right]$ .

After that, the matter-elements for NDST security evaluation were determined by the calculation formula of each index and formula (4). The matter-element matrix of 2017 was computed based on the basic data on NDST obtained from the NDST Industrial Analysis Report and China Statistical Yearbook:

$R_{2017}=\left[\begin{array}{ccc}N_{P} & X_{1} & 82.28 \\ & X_{2} & 85.34 \\ & X_{3} & 261.46 \\ & X_{4} & 33.42 \\ & X_{5} & 83.43 \\ & X_{6} & 75.25 \\ & X_{7} & 45.67\end{array}\right]$.

4.3 Correlation calculation

The basic data on NDST in 2017 were imported into the EW-based MEEM. Then, the correlations between each index and the security levels were obtained by formulas (5)-(7). The results are listed in Table 3 below.

4.4 Index weighting based on EW method

The entropy weight (Table 4) of each index relative to each security level in 2017 was derived by formulas (8)-(11).

4.5 NDST security level

After obtaining the correlation and entropy weight of each index relative to each security level, the weighted correlations between each index and the security levels in 2017 were computed by formulas (12) and (13). The weighted correlations in other years were determined in a similar manner. The computed results are recorded in Table 5 below.

As shown in Table 5, China’s NDST security was relatively good in 2010-2012, but dropped to the neutral state in 2013-2015. In 2016-2017, the NDST became slightly insecure. The NDST security exhibited a gradual decline in the research period. Despite the major progress in technical innovation, the NDST in China is not very secure, judging by factors like environment and scale. This is mainly attributable to the growing number of unsafe and unstable factors, as the NDST pursues economic benefits and market-oriented development in the information age.

The NDST is crucial to national security, and affects the economy, politics, and military of China. Facing the slightly insecure state, it is imperative to design a prewarning mechanism to ensure the stable and healthy development of NDST in the long run.

Table 3. Correlations between each index and the security levels in 2017

Correlation

Level

Kj(v1)

Kj(v2)

Kj(v3)

Kj(v4)

Kj(v5)

Kj(v6)

Kj(v7)

N01

-0.339

-0.391

-0.430

-0.503

-0.608

-0.683

-0.508

N02

-0.345

-0.330

-0.305

-0.425

-0.578

-0.478

-0.523

N03

-0.214

-0.321

-0.014

-0.214

-0.354

-0.154

-0.324

N04

0.036

0.137

0.039

0.136

0.102

0.220

0.182

N05

-0.124

-0.021

-0.209

-0.024

-0.134

-0.234

-0.104

Table 4. Entropy weight of each index relative to each security level in 2017

  Entropy weight

Level

wj(v1)

wj(v2)

wj(v3)

wj(v4)

wj(v5)

wj(v6)

wj(v7)

N01

0.173

0.163

0.164

0.17

0.136

0.123

0.071

N02

0.113

0.179

0.197

0.166

0.107

0.136

0.102

N03

0.182

0.152

0.088

0.127

0.153

0.107

0.191

N04

0.321

0.176

0.058

0.089

0.108

0.119

0.129

N05

0.235

0.176

0.115

0.117

0.132

0.088

0.137

Table 5. Weighted correlations between each index and the security levels in 2010-2017

Weighted correlation

Level

Kj(2010)

Kj(2011)

Kj(2012)

Kj(2013)

Kj(2014)

Kj(2015)

Kj(2016)

Kj(2017)

N01

-0.249

-0.168

-0.418

-0.481

-0.341

-0.228

-0.501

-0.481

N02

0.017

-0.137

-0.219

-0.409

-0.306

-0.326

-0.315

-0.409

N03

-0.283

-0.290

-0.295

-0.121

-0.149

0.034

-0.205

-0.249

N04

-0.341

-0.198

-0.311

-0.249

-0.419

-0.212

-0.034

0.111

N05

-0.572

-0.211

-0.412

-0.342

-0.512

-0.169

-0.193

-0.112

Security level

Slightly secure

Slightly secure

Slightly secure

Neutral

Neutral

Neutral

Slightly insecure

Slightly insecure

4.6 Prewarning mechanism for NDST security

Considering the relatively low security of NDST, this paper sets up a prewarning mechanism for NDST security based on the results of EW-based MEEM, in an attempt to safeguard the stable and orderly development of NDST.

The prewarning system aims to predict the influencing factors and future trend of NDST market, evaluate and diagnose the overall state of NDST security, and judge the change trend of the security level, enabling the competent authorities to take effective precautions.

The effectiveness of the prewarning mechanism depends on the reasonability and reliability of the evaluation system of industrial security. Hence, the prewarning criteria for NDST security were developed based on the results of EW-based MEEM (Table 5). With the aid of the criteria, the competent authorities could act effectively according to specific security state and actual conditions, thereby improving the NDST security and ensuring the safe operation of the country.

5. Conclusions

This paper mainly explores the security evaluation of China’s NDST, and proposes a security evaluation model and a prewarning mechanism based on EW method and MEEM. First, the influencing factors of NDST security were analyzed, and an evaluation index system was constructed for NDST security. Next, the EW method was introduced to weight the indices, and the EW-based MEEM was established. After that, the proposed model was applied to evaluate the NDST security of China in 2017. The empirical results show that China’s NDST security changed from slightly secure to neutral, and to slightly insecure, exhibiting a gradual decline. Based on the evaluation results, a prewarning mechanism was built for NDST security, which effectively coordinates the “going global” strategy and security of NDST. The research results shed new light on the prewarning of NDST.

  References

[1] Yan, H.N. (2016). Research on the evolution path of the civil-military integration of innovation ability in national defense industry. Science & Technology & Economy, 29(2): 16-20. https://doi.org/10.3969/j.issn.1003-7691.2016.02.004

[2] Stadelmann, D., Portmann, M., Eichenberger, R. (2015). Military careers of politicians matter for national security policy. Journal of Economic Behavior & Organization, 116: 142-156. https://doi.org/10.1016/j.jebo.2015.04.001

[3] Baark, E. (1994). China's science and technology policy: 1949-1989. The China Quarterly, 140: 1167-1168. https://doi.org/10.1017/S0305741000053133

[4] Alexander, V. (2004). Innovation management in Russia (problems of strategy and science and technology security). Books of Alexander Varshavsky - Socionet

[5] National Research Council. (2009). Beyond'Fortress America': National security controls on science and technology in a globalized world. National Academies Press. https://doi.org/10.17226/12567

[6] Chen, S.P., Sun, J. (2017). Discussion on the legislation system construction of national defense science and technology security guarantee under the background of greater civil-military integration. Law Science Magazine, 38(5): 19-27. 

[7] Meng, F.S., Han, B. (2016). The national defense science technology and industry independently controllable development research under the new situation. Scientific Management Research, 34(2): 1-4. 

[8] Sun, D.M., Lu, S., Sun, Y.Y. (2019). Research on countermeasures for science and technology security problems in national defense science and technology industry. In 2019 3rd International Conference on Education, Economics and Management Research, pp. 39-42. https://doi.org/10.2991/assehr.k.191221.010

[9] Xiang, Y.Q., Zhang, Q.H., Han, P., Chen, M.Y. (2014). Research of early-warning expert system for security of grain storage based on uncertain inference. Applied Mechanics and Materials, 536: 437-442. https://doi.org/10.4028/www.scientific.net/AMM.536-537.437

[10] Yousefi, A., Hadi-Vencheh, A. (2010). An integrated group decision making model and its evaluation by DEA for automobile industry. Expert Systems with Applications, 37(12): 8543-8556. https://doi.org/10.1016/j.eswa.2010.05.021

[11] Shi, M.K., Peng, Y., Li, E.D., Wen, C.H. (2005). Study on early warning of ecological security in ethnic agricultural regions of the upper reaches of the pearl river an analysis on actual examples of 22 counties within the boundary of Guizhou. China Population Resources and Environment, 15(6): 50-54. https://doi.org/10.3969/j.issn.1002-2104.2005.06.012

[12] Bussiere, M., Fratzscher, M. (2006). Towards a new early warning system of financial crises. journal of International Money and Finance, 25(6): 953-973. https://doi.org/10.1016/j.jimonfin.2006.07.007

[13] Soomro, Z.A., Shah, M.H., Ahmed, J. (2016). Information security management needs more holistic approach: A literature review. International Journal of Information Management, 36(2): 215-225. https://doi.org/10.1016/j.ijinfomgt.2015.11.009

[14] Spanos, G., Angelis, L. (2016). The impact of information security events to the stock market: A systematic literature review. Computers & Security, 58: 216-229. https://doi.org/10.1016/j.cose.2015.12.006

[15] Chen, J.F., Hsieh, H.N., Do, Q.H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28: 100-108. https://doi.org/10.1016/j.asoc.2014.11.050

[16] Jin, W., Li, Z.J., Wei, L.S., Zhen, H. (2000). The improvements of BP neural network learning algorithm. WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000, Beijing, China, pp. 1647-1649. https://doi.org/10.1109/ICOSP.2000.893417.

[17] Yu, J.K., Liu, S.Q. (2013) A study on safety evaluation and early warning mechanism for China’s marine transportation industry. China Maritime Safety, (3): 22-25. https://doi.org/10.3969/j.issn.1673-2278.2013.03.016

[18] Zhang, W.Q., Zhang, H. (2018). Measurement of regional industrial structure optimal under the restriction of resources and environment. s Statistics & Information Forum, 33(3): 80-86. https://doi.org/10.3969/j.issn.1007-3116.2018.03.012

[19] Ning, G.Y., Zhao, J.F. (2016). Research on industrial safety evaluation model under the restriction of resources and environment. Proceedings of the 2016 International Conference on Education, Management, Computer and Society, pp. 319-321. https://doi.org/10.2991/emcs-16.2016.156