Development of methodology for evaluation of innovative IT-projects

. Innovative business development based on a digital transformation is the urgent task for companies. It is able to ensure the implementation of innovation strategy and digital transformation of business. Due to the high development speed and the IT-market updating, the problem is lack of methods of rational choice of the IT-project. In practice, companies often implement IT-projects without reasonable selection and ranking, which leads to a high proportion of failed innovative IT-projects. As a result, the company does not receive a commercially successful product or service that stands out to the consumer among competitors. The evaluation of innovative IT-projects is based on the financial indicators analysis of the expected results from the implementation. The evaluation of the strategic coherence of the project is ignored. The research approaches the methodical of ranking innovative IT-projects in companies. The method consists in a comprehensive analysis of the impact of the proposed project results in various aspects of the company (strategic, environmental, organizational and technological). The study highlights the features of innovative projects in the field of information technology, a comparative analysis of methodological approaches to the evaluation of innovative projects, modified the model of acceptance of IT-innovations. of The The the factors that the adoption of IT-innovations to the reverse when IT-innovations in the company not take root". The practical results of such studies are models, frameworks and recommendations that allow to evaluate the possibility of adopting an innovation.


Introduction
The digitalization of the economy continues to gain momentum. According to McKinsey's forecast the share of digital business in the world GDP will be 34% by 2020 [1][2][3][4][5][6][7][8][9][10]. Businesses are trying to become digital as quickly as possible -this is partly due to the high speed of the emergence and development of "breakthrough" technologies, as well as the desire to be closer to the consumer, in whose life every day becomes more "digital". Technologies that create new markets and/or change the ratio of values in the current market, making old products uncompetitive, as their advantages lose their value. For example: artificial intelligence, Internet of things, blockchain, nano materials, etc. [6]. There is a term for such an organization change: "digital transformation" is a fundamental transformation of the company's products and services, its business model with the help of information technologies. Companies that use new digital technologies to ensure consistency at all levels of the organization consistently implement a common strategy, providing end users with competitive products and services with distinctive features.
An integral part of digital transformation is the use of new innovative technologies, for example, for companies such as Huawei [2], Luxoft [3], Samsung [4], companies in the tourism industry [5], etc. It is noteworthy that today it is information technology that is most often the source of innovation -at least once in 1-1.5 years, innovations in the form of new technologies and products based on them appear on the IT-market [7]. However, not all companies manage to competently introduce new IT-technologies. Thus, in PwC's global study, experts assessed the digital maturity of companies from different industries and found that only one tenth of the surveyed companies belong to "digital leaders"; another half -to "digital innovators", which are characterized by the active introduction of new technologies, but without proper coordination with the company's strategy, as well as without the expertise and support of senior management in their implementation [8].
Often, the concept of "innovation" and "information technology" is equated -according to research, for most modern enterprises IT are able to ensure the implementation of their innovative strategies in the framework of digital transformation [9]. Statistics also show that only 10% of innovative IT-projects are considered successful (that is, they have a commercially successful product or service) [10]. Thus, most innovative IT-projects are recognized as a failure.
In the real world, in order to become a "digital leader", companies seek to quickly implement "breakthrough" technologies through the implementation of innovative projects and implement such projects, taking into account their importance and compliance with the company's strategy. This is especially true for medium and large companies that conduct project activities on their own and where several projects are often implemented in parallel. For such companies the ranking of the projects may become the basis for successful control system project.
Project ranking is a process that allows to increase the probability of success of strategic projects and increase the consistency of projects with strategic goals [11]. Implementation of innovative projects without prior selection and ranking deprives the company of coherence, leads them from the path of the "digital leader" in the direction of a less mature "digital innovator".
To check how well the innovation fits the company in the international practice formed the class of adoption of innovation models, however, their use is not common due to the theoretical nature. The company begins to benefit from the project not immediately after its completion, but a while after its completion.
The peculiarity of many innovative projects is their delayed impact on the company's Business, along with the fact that it is difficult to determine the exact financial results of such projects [12]. A number of foreign sources indicate the problem of the lack of connection of innovative projects with the strategic goals of the company, which is also among the top 5 reasons for the high share (80%) of failed projects [13][14]. It follows that  20 20 9 92 2 today, the issues of assessing the priority of innovative IT-projects (hereinafter InITP) are not fully developed, more in-depth research and development of methods and tools are required to provide in practice an integrated approach to ranking innovative IT-projects.
Currently, the most common approach to ranking innovative projects, including in the field of IT, is the analysis of the expected financial indicators from the implementation, for example, the analysis of the level of profitability, payback period of projects, etc. [15] that is, the impact of innovation on only one side of the Business, financial.
The aim of this study is to develop an approach to a comprehensive assessment of the impact of innovative IT-projects on the company.

Determine the characteristics of IT-innovations
Today, the term "innovation" is integral to most spheres of society. Depending on the context, the term has different meanings. Thus, in the works of Eurostat [16], Schumpeter J. [17], Anshina A. A. [18], Kozlovskaya E. A. and others [19] innovations are represented as introduction and use at the enterprise results of scientific developments; Santo B. [20], Medynsky V. G. [21], Barysheva A.V. [22] represent innovation as process of scientific development and researches without emphasis on further use and introduction; Glukhov V. V. and others [23], Twiss B. [24], the Organization of economic cooperation and development [25], Edison G. [26] allocate in the definitions the commercial component of innovations -the moment, when the research results acquire an economic component. Summarizing the studied definitions, we can distinguish the following features of the concept of "innovation": -most definitions distinguish a chain (sequence) of stages or phases, which are different manipulations with the object; -innovation does not end at the stage of creating a new technology and the physical realization of the idea; activities for marketing a new product, its withdrawal and promotion in the consumer market, stimulating demand and so on -are the part of the innovation process; -the analyzed definitions characterize the result of the final process -the implemented idea in the form of new products, services, technologies; -implementation of innovative activity requires organizational support (allocation of resources, change of organizational structure, creation of new functions at the enterprise, etc.); -the recommendations for the collection and analysis of data on innovation "Oslo Manual" [19] introduces another term directly related to innovation -diffusion (the way in which the economic importance of innovation spreads from the place of first implementation to different consumers -countries, markets, industries, enterprises). Thus, the economic value of innovation stands out.
The study highlights the features of innovation in the field of IT. The definition of "ITinnovation" is most often found in individual studies, for example, in Portnova A. [27], Weisman O. [28][29][30][31], and also in some English-language sources (at Swanson B. [12] and Grover V. [32]). Summarizing the proposed interpretations, IT-innovations can be defined as a set of technical innovations that support information exchange technologies, as a result of which information becomes an important component of the production process, changes the production and market products parameters, increasing its added value.
One of the objectives of IT-innovation is to rationalize corporate information flows and generally improve the quality of information (efficiency of its receipt, its relevance to the recipient, reliability and sufficiency), which, as a result, affects the value of products to the end user. Based on the analysis of the above-mentioned works, the following features of innovative IT-projects are highlighted: -the higher uncertainty about the success of its result. It is not known in advance what the result of such a project will be, whether it will be successful and how much; -the complexity of the customer determining of the product and the end user, which should be influenced by the result of the project; -the use of new practices (approaches to development, collection of requirements, software environments, products, etc.); -the prototyping complexity of the final project result; -the high ambiguity in the project implementation time evaluation. The initial assessment of the terms can change significantly after specification of the task to be solved, the customer and the end user, the concept of the project result, which is not clearly defined at the beginning; -the complexity of the interim project results evaluation, difficult to monitor the project implementation plan; -the positive impact on the added value of the company's product. The introduction of innovation can have a positive impact on the optimization of the company's operating activities, however, first of all, the innovation should affect the company's customers in the form of improved characteristics of the product or service.

Analysis of approaches to the evaluation of innovative IT-projects
Against the background of how innovation is becoming a priority factor in the development of the economy in many countries and accumulates experience in the management of innovative projects, there is a growing need to develop adequate ways to assess the impact of them. In a number of papers on measuring the innovative projects results, the main problems are too vague requirements for the project results and a high probability of significant differences between the expectations of the project and its results [33]. In the review work "Methodologies for innovation evaluation at the proposal stage: a comparative review" the authors divided the methods of evaluation of innovative projects into 4 groups: financial approaches, multi-criteria approaches, approaches with regard to ratios, approaches to assessing project portfolios [34].
Financial approaches were among the first due to their ability to provide project evaluation in the units of measurement most understandable for Business -money. As a result of preliminary assessment of input and output cash flows of the project, it was decided which alternative to invest in. The most common methods of financial evaluation include the method of accounting rate of return, the method of calculating the payback period of the project, the method of the reduced cost of the project, the method of the internal share of profitability.
To assess the intangible positive and negative effects of information technology, there is a need to take them into account when assessing the investment attractiveness of the project. It is difficult to compare the tangible and intangible effects, first, from the point of view of units of measurement, and secondly, from the point of view of their importance.
To bring the estimates to a single unit of measurement and ranking of estimates of importance for a particular company, a group of multi-criteria approaches was formed, in which a pool of evaluation criteria is formed before the direct evaluation of the investment proposal, they are assigned weights, and at the output of the investment proposals are compared by an integral indicator. One of the most complete methods, taking into account diverse criteria, include the method of information economy, as well as the SIESTA (Strategic Investment Evaluation and Selection Tool Amsterdam) method [34]. A prerequisite for the emergence of relative assessment approaches was the need to compare projects among themselves. In the context of IT, several indicative indices are proposed, for example, the share of income from IT-investments in the total income of the company. However, it is not necessary that the indices take into account only financial indicators -employees involved in the development and, for example, the number of business processes where improvement or the number of new services/products of the company are recorded can be compared. A significant method of this group has recently become the method of return on investment from management (ROM), which allows to evaluate the management expertise of the company.
Approaches to the assessment of portfolios have become a frequent practice when making investment decisions. They allow you to evaluate projects at different levels of the organization, considering not only the attributes of the project or project program, but also critical business indicators, in some cases, even the business model. Within this group of approaches, questions are raised not only in the key of "whether to invest in this project", but also "what important business activities we want to improve", "what IS support key business processes and what project to improve them". These approaches include the Bedel Method [35], the method of investment mapping and the method of forming an investment portfolio [34].
To select the approach to the InITP priority assessment in the study, a comparative analysis of the above methods was performed, the results are presented in Table 1. Based on the features of InITP identified earlier, a list of criteria for comparing approaches to their assessment is formed: 1) scope of the approach; (The approach uses only project data or takes into account the organization peculiarities, the impact of the project results on the business processes of other company departments, the achievement of strategic goals, etc.); 2) the uncertainty of the success of the project results; 3) the ambiguity of the project implementation time; 4) the impact of the project result on the value added of goods/services; 5) the nature of the evaluation criteria of the approach (qualitative/quantitative); 6) the nature of the results of the approach (The results of the approach are compared by the type of scales [36]: 1 -nominal scale is used to assess qualitative variables, referring the object to be evaluated to any class, 2 -order scale used to reflect the ranks and priorities, 3 -interval scale to compare the difference between the estimated objects). 7) methodological equipment of the method (Whether the methods provide recommendations for the collection of information, the calculation of intangible benefits, as well as general recommendations on the staff that is most suitable for the evaluation of projects, the procedure and timing of the evaluation, etc.). 8) the complexity of the approach. (How difficult it is to collect information for the application of the method, how obvious and visible the results obtained, how specific should be the knowledge and skills of the employee to use the method). "+" was placed in the case of a positive assessment of the approach on this criterion, "-" -in the case of a negative assessment, the symbol "+/-" indicates the presence of restrictions on the criterion under consideration. On the basis of comparison of approaches it is revealed that the group of multi-criteria approaches is of the greatest interest. Such approaches are not limited to financial valuation, which is important in the analysis of InITP, the results of which may be intangible benefits. Moreover, the multi-criteria approach allows for evaluation not only at the project level, but also to rise to higher levels (for example, the strategic objectives of the organization) through the inclusion of appropriate criteria. Methods of multi-criteria approach are more flexible regarding the uncertainty of the project implementation period. The ordinal scale of assessments allows us to obtain a comparison and ranking of projects from different areas, presented in one plane of the criteria important for a particular company, and not for the market or the industry as a whole. But in the context of innovative projects, one of the features of which is high risk, the optional risk assessment of the project in the form of a given criterion is insufficient. More acceptable is the calculation of several project scenarios, similar to the approaches to the analysis of project portfolios.

Development of an approach to ranking innovative IT-projects
With the development of IT and innovations analysis in this area in the scientific community, a-separate area of research on the "adoption" of IT-innovations. The "adoption" of IT-innovation refers to the successful implementation of IT-innovation in the company, which leads to a qualitative improvement of activities (for example, business processes) and increased efficiency [37]. The research analyzed the factors that influence the adoption of IT-innovations or lead to the reverse situation when IT-innovations in the company "do not take root". The practical results of such studies are models, frameworks and recommendations that allow to evaluate the possibility of adopting an innovation. Analysis of the such models use allowed us to identify the most common ones: a model of technology adoption [38], the model potential applications of the technology [39], a framework "Diffusion of innovations" [40], the framework of "Planned adoption" [41], a model of technological, organizational, and surrounding context (Technology-Organization-Environment model) [42].
The study of the above models allowed us to identify criteria for their comparison, which take into account the models limitations, the subject of their analysis and the results of their application: 1) type of approach; 2) binding to the size of the company; 3) the limiting stage of the life cycle of IT-innovation; 4) level of acceptance assessment; 5) strategic coherence; 6) object of analysis of IT-innovation adoption; 7) the result of applying the model.
The results of the comparative analysis are given in Table 2. to the size of the company or the stage of development of the evaluated innovation, the assessment is carried out at the level of the organization as a whole and considers its activities from different sides (3 contexts). It is worth noting that this approach provides an implicit assessment of strategic coherence.
The authors of the TOE model distinguish three contexts, processes and characteristics of which influence the success of the "adoption" of IT-innovations, namely: technological context, organizational context, surrounding context [42]. The technological context determines how IT-companies are ready to implement the technology, how the technology is mastered in the market and is feasible under the technical conditions and capabilities of the company. The organizational context describes how the size of the company, its organizational structure, communication processes, internal drivers of changes will affect the adaptation of IT-innovations. The surrounding context assesses the impact of ITinnovation on the company's position in the industry and in the environment of competitors, and also takes into account the limitations that may face the implemented IT-innovation from the state, and the necessary transformations to support it.
In the framework of the study as a feature of the implementation of InITP highlighted the need for consistency with the strategy of the company, but the original model does not allow to evaluate this relationship explicitly.
To take into account this limitation, a strategic context has been added to the model to assess the compliance of IT-innovations with the company's strategy, as well as the impact on the achievement of strategic goals. In the works on the Fleischer-Tornyatski model use, the authors reveal the contexts with the help of well-known models, frameworks and tools of strategic, organizational and it management (for example, the framework of the Porter's five forces, the value chain, Ostarwalder-Pigne canvas, PEST-analysis, SWOT-analysis, models of enterprise architecture TOGAF, Zahman, etc.).
The choice of models is given to the company or organization that evaluates the adoption of IT-innovation based on the recommendations for the study of innovation  20 20 9 92 2 companies in the document "Oslo Manual" [19] in the ranking process identified three main stages (Figure 2). Phase 1. Initiation and planning of project ranking activities. In the framework of the stage, preparatory work is carried out -in particular, the selection of InITP, with the help of Gantt charts and control events, plans and schedules for project evaluation, the appointment of responsible persons and members of the expert team is implemented using the responsibility matrix.
To determine the innovativeness of the project and its selection in the menu of candidate projects for evaluation and ranking, a number of methods and recommendations are proposed, for example, in the works of Mutanov [46], Sergeev [47], Lytnton [48], etc. This study does not consider the selection process of innovative project for this study it is accepted that already considered obviously innovative projects.
Phase 2. Conducting data collection activities to rank projects. This is currently the main work on the assessment of priority projects, which formed a modified model of the adoption of IT-innovations Fleischer-Tornyatski. The activities of the phase include the preparation of a framework for project evaluation (identification of criteria, development of an evaluation tool), data collection for evaluation and analysis of results.
The following models can be used to highlight criteria in four contexts: Evaluation by experts can take place both collectively during discussions (brainstorming) or open grading with subsequent calibration, and closed, independently of each other, in this case, the arithmetic mean of all estimates is set.
Phase 3. Analysis of ranking results. The final processing of the data and ranking of projects is the final stage, within which the goal is achieved -to obtain a ranked list (rating) of InITP. The received questionnaires are processed, for each criterion the average score for all experts is set, after which the final score is set for the candidate projects, which equally takes into account the four contexts similar to the original model.
The process of implementation of the ranking stages depends on the specifics of the company-its size, staffing, experience and maturity level of project management, internal developments in working with innovations will determine the frequency of application of the approach, the list and number of positions for the team of experts, the format of interaction during the evaluation and other organizational aspects.

Application of the developed approach in the Russian ITcompany
Testing of the developed approach was carried out in one of the leading Russian companies in the field of Internet technologies, which owns a large search system on the Internet and accompanying services. Key markets for the company are Russia, CIS countries, Turkey and Israel. Due to the significant coverage of the audience on the search portal (about 20 million unique users per month), as well as partner network sites, the company has the 92 2 ability to provide advertising space in various formats, which brings up to 96% of the company's income.
The package of services also includes advising clients on the services and providing analytical materials.
Fundamental and applied research in the field of computer science and mathematics, the results of which are further used either as a new service or a significant improvement for existing services, are conducted in a dedicated management.
The developed approach was first applied in the company in 2017. As part of the first stage, the specialists of the group of promising products formed a list of six candidate projects. Each project was assigned a responsible manager who provided all the necessary information about the project (for example, the project passport, its content, a list of resources that will be required for its implementation, expected results, etc.) and took part in activities for ranking projects. A team of experts has been formed. It was important to involve specialists from the areas corresponding to the four contexts of the IT-innovation adoption model. Thus, the team of experts involved specialists from several departments:  to assess the strategic, the surrounding contexts -the Deputy head of the division of advertising analysts, associate client service division, Deputy division agent service (division of the Commercial department);  to assess the technological context -senior specialist, senior software developer, leading analyst of the internal services support unit of the Infrastructure department;  to assess the organizational context -four heads of working groups of the Commercial department.
The schedule appointments are generated, responsible managers exposed the deadlines of provision of information on projects candidates, forms to collect information are developed.
In the second stage, a list of criteria for assessing the priority of InITP is determined on the basis of a modified model of acceptance of IT-innovations. To form the criteria of the strategic context, within which it is necessary to analyze the impact of the project on the business logic of the company, the canvas of Ostarwalder-Pigne was chosen, which describes the activities of the company on 9 blocks [49]: key resources, key partners, key activities, value proposition, consumer segments, sales channels, building relationships, cost structure, revenue structure. The completed canvas was handed over to experts who identified the problematic elements of the business model ( Figure 2). In particular, there are an element of advertising agencies in the block "key partners", sale of advertising services in the block "key activities", advertising surfaces and personal analytics in the block "value proposition", small and medium businesses in the block "consumer segments", personal manager in the block "building relationships", advertising in the block "revenue structure". Thus, the evaluated projects had to focus on the identified problem entities. The gray block indicates the problem blocks of the company under consideration. Similar to the strategic context, the experts identified the problem entities of other contexts, on their basis the questions for the questionnaire are formed. For questions of strategic context it is offered to use a qualitative scale, for technological, surrounding and organizational -a quantitative scale. The choice of scale can vary depending on the models selected for context disclosure.
To assess the impact of candidate projects on selected problem entities in the study, a questionnaire (Table 3)   How will the implementation of the project affect the segment of the value proposition "Personal Analytics"? 1.6 How will the implementation of the project affect the segment of key activities "Sale of services and solutions"? 1.7 How will the implementation of the project affect the segment of key activities "Customer Support"? 2. Surrounding context (based on SWOT-analysis) 2.1 How many of the indicated opportunities will the result of the project affect? 2.2 How many of these threats will the result of the project reduce?
2.3 How many of the identified weaknesses does the project "avoid"? 2.4 How many of the identified strengths does the project involve?
3. Organizational context (based on the motivational model of the combined approach to enterprise architecture TOGAF and Archimate) 3.1 How many the organization goals will the project impact on?
3.2 How many evaluations of the organization will the project affect?
3.3 How many drivers of the organization is the project focused on?
3.4 How many of the external stakeholders of the organization will the project affect?
4. Technological context (based on the multilayered enterprise architecture model of the combined TOGAF and Archimate approach) 4.1 How many applications components and services with which the project result will be integrated do not require radical improvement/development from scratch?
4.2 How many of those services, where the results of the project will be introduced, do not require a radical revision/development from scratch? 4.3 How many supporting business processes are affected by the project results? 4.4 How many main business processes are affected by the project results? 4.5 How many business services are affected by the project results? 4.6 How many foreign artists will the results of the project influence?
As a result of the second stage, a survey was conducted, materials and expert assessments were collected for six candidate projects in four contexts.
As part of the third stage, the received estimates were processed, the projects were given a final score and a ranked list of projects was formed. The final score for the contexts was given as a relative assessment of the project (percentage) of the maximum possible for a particular context. The final score of the project was calculated as an arithmetic mean by four criteria, based on the principle of equality of contexts, laid down in the original model of adoption of IT-innovations [42]. Next, the project was assigned a rank depending on the value of the complex relative assessment -the higher the score, the higher the rank of the project. The project estimates are presented in four contexts in the form of a radar chart, in figure 3, where the "ideal" project would have a 100% value in all contexts. To check the adequacy of the results of the developed approach, the ranks of the projects obtained by the application of the procedure were compared with the ranks of the same projects obtained from internal sources of the company (Table 4). According to the results, four of the six projects received the same rank. The ranks of the projects "Smart" clustering of clients" and "Smart" forecasting of the advertising budget, obtained from internal data and within the framework of the approach, differed in one point. Smart processing of the stream summary 52% 5 5 Smart prediction of anomalies in the company's income 48% 6 6 The study of internal information made it possible to find out that the project "Smart" clustering of customers was exposed to a lower rank, as at the time of prioritization in the company there was no clear division of customer segments. The product line did not take into account the peculiarities of different types of advertisers, there was no obvious need to customize advertising solutions. Similarly, the approach was re-applied in 2018, the results are presented in table 5. Unlike the first application, the ranks of all projects received within the framework of the approach coincided with the ranks of the internal sources of the company. At the first iteration, the deviation from the rating accepted by the company was ≈6%, at the second iteration, no deviations in the ratings were revealed. The identity of the project ratings suggests that the developed approach is close to the method used to make decisions about the ranking of projects. The application of the developed approach in 2017-2018 allowed to create a database of models and materials, to develop templates and formats for meetings and questionnaires, which will reduce the time of implementation of the approach in the future. Moreover, the approach managed to formalize partly intuitive and subjective factor of decision-making. That is, earlier the process of ranking InITP in the company was presented as a "black box". A set of candidate projects was submitted to the entrance, a ranked list of projects for implementation appeared at the exit. For the initiators of the candidate projects, the timing of the evaluation, the format of the interaction, the composition of the team that conducted the evaluation remained unknown throughout the entire process.
Presentation materials on the results of the procedure are presented at quarterly meetings of the Commercial department.
After discussing a new approach to ranking innovative IT-projects, it was decided to continue using it to collect data and confirm the adequacy of the model. At the same time, the implementation of the approach will take place in parallel with the current "intuitive" method of project pyritization -this will allow calibrating the composition of the procedure or adjusting individual steps.
It is planned that the regular application of the procedure will clarify the list of competencies of the expert team, taking into account four contexts. Based on the first iteration of the procedure, it is possible to develop requirements for the form and content of supporting documents that help experts to evaluate projects. The developed approach allows for a comprehensive assessment and ranking of innovative IT-projects based on the analysis of the impact of their expected results on the four areas of the enterprise. The approach can be used in the following situations:  the company chooses a new innovative technology among several alternatives;  the company selects innovative projects for implementation from the list of candidate projects;  the company prioritizes projects to determine the sequence of their implementation.

Conclusion
The developed approach was successfully tested in a large Russian IT-company, which allowed us to formulate practical recommendations for the application of the procedure in a Russian company.
The prospects of the study are in the conceptual development of the approach -the study of the significance of each of the four contexts in the assessment of the priority of InITP and the subsequent calibration of their weight coefficients of importance in the final score of the projects.
Another area of research is the development of procedures based on iterations of the developed approach.