Regional innovation systems efficiency analyses and evaluation: DEA approach implementation

The innovative transformation is a necessary condition for sustainable economic development. The study reveals an assessment and comparative analysis of the Regional Innovation Systems’ (RIS) performance in the Russian Federation using Data Envelopment Analysis (DEA). The DEA model under the Variable Return to Scale (VRS) assumption, focused on output parameters, is used to estimate the relative technical efficiency of regions based on several input and output parameters. Based on the obtained results, a rating of regions was compiled: four groups of regions were identified depending on their technical efficiency level. It was revealed that the leading regions by innovative development level are assessed by the DEA somewhat differently. A comparative analysis of the innovation systems performance at the regional and federal levels allowed us to identify the most and least effective subjects of the Russian Federation, federal districts and economic regions. The main conclusion is that less than a third of the Russian regions use their production capabilities as efficient as possible, the remaining regions can significantly improve the way they use the available resources. The results of the study might be used in making managerial decisions at the country, federal districts and regions levels in order to develop measures and mechanisms for improving the efficiency of regional innovation systems.


Introduction
In recent decades Regional Innovation Systems (RIS) have attracted significant attention of both scientists and policymakers due to their high economic role. It is believed that RIS are the reason for increasing competitiveness and productivity of a region [1] and the driving forces for knowledge economy development. More innovative and efficient manufacturing technologies attract additional investments to the regions, contributing to the surge in their competitiveness. The failure of the regions to develop the maximum possible number of new technologies is explained by the inefficient RIS management [2]. In this regard, the evaluation of RIS effectiveness has become particularly relevant.
An empirical assessment and explanation of an economic system performance is a very difficult task, regardless of the analytical context. It is important not only to determine the measurement parameters or key indicators, but also to measure the performance of the system as a whole, which requires both a quantitative evaluation and a qualitative one, based on a normative scale "better -worse". There is an agreement in the literature regarding the lack of suitable indicators not only for comparative analysis of RIS performance [3], but also for an in-depth assessment of specific systems characteristics [4]. The use of modern mathematical models and methods, in particular Data envelopment analysis or DEA for short, for RIS assessments seems to be relevant. The purpose of the study is to evaluate and to analyse RIS effectiveness using DEA modelling. As a result of DEA method application, one integral indicator for each subject of the Russian Federation in the sample was obtained. Based on the integrals value, the regions were ranked and grouped by their technical efficiency levels. The most and least effective regions are identified, the discrepancy between the technological capabilities of the regions and their RIS performance is shown, and, finally, the need for more efficient use of available resources in most regions of the Russian Federation is revealed.
The scientific contribution of the study is to use DEA as a methodology for evaluating RIS effectiveness based on several input and output parameters and to conduct a comparative analysis of RIS performance in 85 subject of the Russian Federation, 8 federal districts, as well as in the European and Asian parts of Russia.
Efficiency in the use of system resources is indicated by the degree to which inputs are generating high returns, or that the bottom line does not reflect the volume of investments.

Approaches to measure RIS performance
Research on the level of scientific and innovative activity of the regions, as well as on the development of indicators and methods for evaluating it has become widespread in Russia and abroad. The most well-known ratings are the following: The World Economic Forum and the International Institute for Management Development ratings, the World Bank Knowledge Economy Assessment, the Global Innovation Index, the Innovation Capacity Index, European Innovation Scoreboard, Innobarometer etc. Among the Russian studies, it is necessary to highlight the following ratings: Russian Regional Innovation Development Ranking, compiled by the Higher School of Economics [5], the Association of Innovative Regions of Russia ranking, the National Association for the Development of Innovations and Information Technologies, the assessment of the level of innovative development of various regions made by the Institute of Economics of Russian Academy of Sciences, etc.
These methods as well as the indexes and ratings based on them are resource-oriented and evaluate the scientific and innovative potential of the socio-economic system. As a result, regions with a large amount of resources receive a higher rating. However, this fact does not mean that the competitiveness of such regions is higher and that their innovation systems are better developed. It is considered that R&D, innovation and the amount of available resources are the decisive factors of the competitiveness of the region. While the identification of these resources is important, it is even more crucial to understand how effectively they are used. Moreover, it is also not obvious that the regions with the highest income level (value added, GRP, etc.) are the most effective ones [6].
The RIS efficiency depends on the degree to which the input resources generate a high return. The fact that the final results do not reflect the amount of investment invested, in contrast, reveals the inefficiency. In addition, there are a few problems associated with the regions heterogeneity, the multidimensional nature of regional innovation systems and differences in the applied efficiency criteria.
The Regional Innovation System (RIS) can be defined as a system of interconnected institutions for the creation, possession and transfer of knowledge, skills and artefacts that define new technologies [7] or as a combination of many regional parameters for creating an innovation-friendly environment [8]. From this point of view, RIS is technically an efficient converter of inputs and outputs. Since we are dealing with scientific and technical indicators to illustrate the regional results of innovation activity, RIS can be characterized by the effectiveness of the cost-benefit ratio. In the analysis of the public sector, all three defining elements of efficiency (inputs, outputs and their functional relationship) are subject to serious conceptual and measurement biases [9]. Therefore, in the RIS analysis, we are dealing with many inputs and outputs, which may be heterogeneous, and sometimes even incomparable.
Applied researchers are increasingly interested in analysing the technical efficiency of regional economies, which is expressed in the degree of approximation of the level of production to the maximum output, achieved through the available resources, or through the greatest return on the technologies in use and on financial, labour and other costs. The approach based on the identification of the so-called efficient frontier has become popular [10]. The Data envelopment analysis (DEA) and Stochastic Frontier Analysis (SFA) are used to assess the efficient frontier. The DEA method, proposed as a tool for measuring production efficiency in 1957 by Farrell [11] and improved by Charnes, Cooper, Rhodes and Bunker [12,13], is considered as one of the most popular nonparametric approaches to measuring technical efficiency in the field of spatial economics.

Data Envelopment Analysis (DEA)
DEA is a relatively new "data-driven" approach to evaluating the performance of a set of peer objects called Decision-Making Units (DMUs) that convert inputs to outputs. The definition of the DMU is universal and flexible, which allows to use the DEA approach in assessing the performance of various entities in a wide variety of contexts in many countries. DEA methods are widely used (mainly by foreign researchers) to analyse enterprises in various industries: banking, transport, health, education, ICT, space, oil, agriculture, and other sectors. Moreover, using the DEA methods it is possible to assess the effectiveness of public administration, national and regional innovation systems [see for example [14][15][16].
The DEA measures the relative efficiency of the DMU, which have an economic interpretation as a production capability frontier and a management interpretation as a multicriterion decision-making method (MCDM). Initially, the concept of DEA as a measure of Pareto efficiency was proposed by Farrell [11]. The relative efficiency of DEA is consistent with the extended Pareto-Koopmans definition: full (100%) efficiency is achieved by the DMU if and only if none of the parameters of its inputs or outputs can be improved without worsening others. This avoids the need to specify weights reflecting the relative significance of various input or output parameters when aggregating them. Moreover, it reveals the formal relationship between inputs and outputs.
The DEA model links the above definition to other definitions of efficiency used in engineering, economics, and business. The DMU is ineffective if its technical efficiency (θ) is less than one (θ <1). The DEA is a classic application of linear programming techniques to compare the performance of multiparameter objects. The DMU efficiency is defined as a weighted ratio of results (input) and resources used (output). Evaluation of the efficient frontier is carried out by repeatedly solving the linear programming problem for each assessed object. When a more efficient DMU is added to the sample, the efficiency of all DMUs will change.
Orientation of the DEA methodology towards frontiers, rather than central tendencies, as in statistical regression, allows to identify relationships that remain hidden from other methods of analysis. Furthermore, it is quite useful to apply this methodology in cases of the complex (often unknown) nature of the relationship between multiple inputs and outputs involved in the DMU. This does not require explicitly formulated priori assumptions and variations with different types of models. In most management decisions or social sciences, theoretically possible efficiency levels are not known. Therefore, in determining relative efficiency, the emphasis is shifted to empirically available information, hence the name of the method.
Within the DEA methodology framework, several problems associated with measuring the performance of the systems are solved at once. Firstly, the problems of the efficiency standard and the indicators aggregation are eliminated (each of the inputs and outputs is a separate dimension of the production possibilities space, and there is no need to bring them to a single form). Secondly, the opportunity of measuring the DMU efficiency as the distance from a point in multidimensional space to the efficient frontier occurs. Thus, the DEA has the following advantages: it does not require to set the weights of the input and output parameters or to formulate hypotheses about the functional relationships between the inputs and outputs; it allows working with a large number of input and output parameters.
The DEA is a well-established approach to assessing RIS [17]. The general parameters of the RIS assessment identified in previous studies include revenue, R&D expenditures, sales, innovative products, patents, publications, profits, the number of researchers and technicians. These parameters depending on the formulation of the problem, are used as inputs or as outputs [18].
Initially, the DEA was proposed by A. Charnes, V.V. Cooper and E. Rhodes as a method of comparative assessment of efficiency, but later the researches started to use the method to group (to cluster) objects in terms of their efficiency level. In this study, the results of the DEA are used to group regions according to their technical efficiency.

Data and methodology
The methodological basis of the study is the general theory of economic growth and systems analysis. In terms of theory, the study is based on the results of accumulated foreign and Russian experience in the assessment of innovative development at the federal and regional levels. The DEA method was used as a methodology for assessing the relative technical efficiency of regions based on several input and output parameters. The inventive activity of the regions was evaluated in accordance with their ability to develop advanced manufacturing technologies and to launch innovative products at a given level of production and information and communication technologies, human capital and R&D expenditures.

DEA models
At the first stage of applying this method, it is assumed that each of the analysed units (DMU) has a set of input and output parameters. For the -th DMU, they are represented by column vectors and , respectively. Then, for all n objects, is a matrix of input parameters of dimension × , is a matrix of technologically possible (maximum attainable) output × dimensional parameters.
Using the duality theory, we get a mathematical programming problem: where is a scalar and is a constant of × 1 dimensional vector [19].
The value ≤ 1 obtained in solving the research problem is a measure of the -th object efficiency. DMUs for which = 1 are effective and are on the efficient frontier, forming a piecewise linear function. DMUs for which < 1 are ineffective and are below the efficient frontier. The degree of inefficiency is determined by the distance between a point and the efficient frontier: the longer the distance between the point linked to a given DMU and the efficient frontier, the less effective the DMU is [19].
The above model, aimed at achieving a given production volume with a minimum level of resource consumption, is called an input-oriented model. Model 1 assumes that there are constant returns to scale. In order to take into account variable returns to scale, it is necessary to add in this Model a restriction on the sum of the weight coefficients : Due to this restriction the efficient frontier becomes convex. Output-oriented models can be built in a similar way. In this case, the main goal is to achieve the maximum result (output) at a given level of input resources. Output-oriented model assumes constant returns to scale: Output-oriented model assumes variable returns to scale: Since the goal of models 3 and 4 is a proportional increase in the values of the output parameters, the value of the variable calculated on them will be greater than or equal to one ( ≥ 1). In order to get more common form of the efficiency indicator , lying in the range from zero to one, it is necessary to use the reciprocal of [19].
DEA estimates the efficient frontier reflecting the maximum possible level of production in a selected group of regions. The efficient frontier assessment is carried out by repeatedly solving the linear programming problem for each evaluated object.  Let's illustrate the concept of efficiency and the DEA method under variable returns to scale assumption and with the use of two parameters for RIS: one input (R&D expenditures) and one output (number of advanced manufacturing technologies developed). The coordinate system is the following: stands for the "Input" and stands for the "Output", the CRS graph is plotted (see Fig. 1).
Regions 1 and 3, which are at the efficient frontier, perform very well ( = 1). Regions 2 and 4, which are below CRS, are ineffective ( < 1). The improvement of the efficiency of regions 2 and 4 requires achieving the efficient frontier through the cost cutting in the input-oriented model (to point B 'for DMU2), or through the surge in productivity in the output-oriented model (to point C' for DMU4).
The efficient frontier remains the same when the model is oriented towards both input and output, as it is shown on Fig. 1, the orientation of the model only affects the direction of projection of ineffective points onto this frontier. The choice of a method for increasing efficiency (orientation of the model) is carried out by the researcher and depends on the object under study: for instance, for government entities with a fixed budget and social goals, cost cutting is not an applicable strategy for increasing efficiency.

DEA algorithms
The algorithm for applying the DEA method involves the sequential execution of the following stages:

• Selection of input and output indicators
Using study findings, RIS indicators that measure the resource base, technological and innovative potentials of the regions were used as input parameters: the use of information and communication technologies (ICT), incl. personal computers, servers, local computer networks, cloud services, the Internet; the use of advanced manufacturing technologies (AMT); the cost of digital transformation, implementation and use; domestic expenditures on R&D; the number of staff engaged in R&D; the number of patent applications filed in the Russian Federation (see Table 1). As for the output parameters, they are based on the RISs performance measures: the number of developed AMTs, coefficient of inventive activity, the volume of innovative products, gross regional product (GRP)

• Model selection
In the study it is relevant to perform an analysis under the Variable Return to Scale (VRS) assumption, i.e., BCC-Output DEA model. BCC-model reflects the efficiency of DMUs performance (management and business-processes quality).

• Raw data
The calculations use the official data from Rosstat and from Rospatent for 2019. The use of a single database for the research simultaneously ensures the reliability of information and the consistency of the methodological and methodical statistical base for all indicators and all objects of the study. Moreover, such approach allows to expand further the analysis and to monitor continuously the performance of Russian regional innovation systems.

• Data analyses and results interpretation
Excel, IBM SPSS Statistica and OSDEA GUI software were used to carry out calculations, data processing and evaluation. As a result, a comprehensive table was formed. It reflects the level of RIS efficiency, and the regional classification based on this parameter.

Research results and discussion
The analysis revealed that for 2019 only 26 regions of the Russian Federation have a technical efficiency coefficient equal to one. They are efficient in terms of the use of available resources and therefore serve as benchmarks for other regions. The rest of the regions are characterized by irrational use of resources (the efficiency level is less than one). Regions differ significantly in respect of technical efficiency -the coefficient varies from 1 to 0.216. The regional frequency distribution by technical efficiency coefficient is shown on Fig. 2. Based on the results mentioned above, the regions were ranked by the level of relative (technical) efficiency (see Fig. 3, 4). Less than a third (30.6%) of the regions are technically efficient. Only 5 regions (the Moscow Region, the Novgorod Region, the Sverdlovsk Region, the Murmansk Regions and the Krasnodar Territory) are close to high efficiency level (1 < θ ≤ 0.9), other regions lag behind to varying degrees. Only 12.9% of regions use more than 75% of the available resources, a third of regions (32.9%) use more than half of their resources, but less than 75%, almost a quarter of regions (23.5%) use less than half of the available resources. The greatest lag is demonstrated by the Altai Republic and the Tver Region (θ ≤ 0.25). The efficiency of innovation systems in more than half (55%) of the regions turned out to be below the average for the Russian Federation.  In order to conduct a profound analysis, the DEA results are compared with the rating of innovative development of Russian regions made by f the Higher School of Economics [7]. Top regions by innovative development level are assessed by the DEA somewhat differently, and vice versa (see Fig.3).
The leaders in terms of technical efficiency are not only the largest agglomerations and innovatively active regions (Moscow, St. Petersburg, Novosibirsk Region, Tyumen Region and other regions, as well as the Republic of Tatarstan), but also the lagging regions in respect of innovative development (the Republic of Tyva, the Republic of Adygea, the Republic of Crimea , the Sakhalin Region, the Orenburg Region and other regions, as well as the Yamalo-Nenets, Khanty-Mansi and Chukotka Autonomous Okrugs, etc.). Their resource base is much lower than that of other regions with less efficient RIS, but the efficiency of their use is higher, thus, even a slight increase in the quantity of available resources will lead to better innovative activity results.
The average value of the technical efficiency coefficient (θ) in the Russian Federation is equal to 0.717, the minimum value is 0.219, the standard deviation is 0.235, that points to a heterogeneous group of regions and significant differentiation. In accordance with the level of technical efficiency, all regions were divided into four groups: 1) the most efficient regions (leaders) (θ = 1); 2) relatively efficient regions, (1 > θ ≥ 0.75); 3) less efficient regions (0.75 > θ ≥ 0.5); 4) inefficient regions (θ < 0.5) (see Fig. 4). Source: Authors' estimations.
The first group consists of the capital and another 25 effective regions. The second group includes 9 regions, which level of technical efficiency is lower by less than 25 %. The third group is the largest one, it consists of 30 regions, whose technical efficiency is lower by 50 % than that of the leaders. Finally, the fourth group includes 20 regions with the lowest technical efficiency coefficient, which is half as much as that of the leading regions or even lower.
Efficient innovative regions are unevenly distributed over the territory of the Russian Federation. Seven of them are located in the Volga Federal District (the Republic of Mari El, the Republic of Mordovia, the Republic of Tatarstan, the Kirov Region, the Orenburg Region, the Penza Region and the Samara Region), four of them -in the Central Federal District (the Moscow Region, the Belgorod Region, the Lipetsk Region and the Yaroslavl Region), in the Ural Federal District (the Khanty-Mansi Autonomous Okrug, the Yamalo-Nenets Autonomous Okrug, the Tyumen Region and the Chelyabinsk Region) and in the Far Eastern Federal District (the Sakha Republic (Yakutia), the Chukotka Autonomous Okrug, the Sakhalin Region, the Jewish Autonomous Region). Only one innovative region is located in the North -Caucasian Federal District (the Republic of Dagestan).
Comparative analysis of the RIS performance at the federal districts level revealed that the highest concentration of effective regions (leaders) is observed in the Ural Federal District and the neighbouring Volga Federal District, the lowest concentration is in the North Caucasian Federal District (see Fig. 5).

Share of inefficient regions
Technical efficiency Share of leading regions In federal districts with a high concentration of the most efficient regions, the share of ineffective regions is simultaneously low, thus, the highest average technical efficiency is observed in the Ural Federal District, the Volga Federal District and Southern Federal District. In the North Caucasus and the Siberian Federal Districts, the share of leading regions is lower than the average for the Russian Federation and the share of inefficient regions is higher, as a result, these federal districts show the lowest average indicator of technical efficiency.
A comparative analysis of the relative region efficiency revealed that RIS in Asian part of Russia are, on average, more effective than those in European part and in the Russian Federation as a whole (see Table 2).
The number of effective regions in Asian part of Russia is one and a half times greater than those in European part. Moreover, this figure is 30.1% higher than that of the average for Russia. The concentration of effective regions is higher in Asian part of Russia, and ineffective regions are located mostly in European part of Russia. In general, the analysis showed that for a given level of input parameters, the productivity of most Russian regions can be boosted without increasing the number of production factors in use. For instance, with an improvement in the use of available resources, the value of the output indicators in Sverdlovsk region can be increased by 2.4%, and in the Tver region -by 4.6 times. The average efficiency level of 0.717 showed that it is possible to increase the efficiency of Russian RIS by 28.3 % without changing the amount of available resources, i.e. the productivity of most regions can be significantly improved.

Conclusions and future research direction
The effectiveness of regional innovation systems is of great importance, since it is crucial for the regions not only to have qualified human resources and a high level of investment, but also to get real results from these resources that ensure sustainable economic growth.
The analysis revealed that the regional innovation systems of the Russian Federation differ significantly in terms of technical efficiency -the efficiency coefficient varies from 1 to 0.216. Only 30,6% of the regions included in the analysis, proved to be technically efficient. Almost a quarter of the regions use less than half of the available resources. More than half of the regions fell below the average level for the Russian Federation. Thus, for a given level of input parameters, the productivity of most regions can be significantly improved.
Moreover, the distribution of technically efficient regions is uneven across the territory of Russia. The highest concentration of effective regions is observed in the Ural Federal District and the neighbouring Volga Federal District, the lowest concentration of leading regions is in the North Caucasian Federal District. In Asian part of Russia, the number of the most effective regions and the average level of technical efficiency are significantly higher than those in the European part and in the Russian Federation as a whole. Top regions by innovative development level are assessed by the DEA somewhat differently, which emphasizes the importance of the technical efficiency evaluation for the regional innovation systems. The application of the DEA methodology at the regional level for RIS performance analysis is an effective tool that allows to solve a wide range of research problems: to monitor RIS performance, to identify and to classify technically efficient and inefficient regions, to find a quantitative measure of their technical efficiency (or inefficiency), to find out the examples of the so-called best practice, to indicate effective goals for each region and to reveal the best ways to achieve them. The main limitation of this study is that the results could differ depending on the distribution of the assessment parameters for input and output.
The proposed algorithm for assessing RIS performance can be applied to a different set of territories (in the context of federal subjects, districts, economic regions, etc.) and different types of regions, considering their industry specialization. The application of the proposed approach within the scope of separate regions and federal districts allows to identify the most effective ones and to determine the need for managing input and output indicators. Analysis of the characteristics of leading regional innovation systems permits the identification of the determinants of their success. This will have a positive impact on regional and municipal authorities' coherence in developing and implementing the innovative and socio-economic development strategies. Funding