Assessment of the efficiency of educational project management using neuro-fuzzy system

. The project represents the introduction of elements and methods of artificial intelligence in the work programs of disciplines in the direction of "Management". To assess the efficiency of such project management, it was proposed to use tools related to machine learning methods that include neural networks and fuzzy logic. The results of such an assessment are obtained using a neuro-fuzzy anfis (adaptive neuro-fuzzy inference system) type system, which is implemented using the MATLAB R2018b software package.


Introduction
The considerable attention is paid to the use of innovative digital technologies in education in the countries of the European Union. The report on policy models for the integration and innovative use of digital technologies in education aims to better understand how digital education policies are implemented in the EU and beyond, highlighting key enablers, success factors and barriers [1,2]. The key findings seek to inform, guide and inspire policy-makers at all government levels (local, regional, national and international) in designing new policy initiatives. The integration and innovative use of digital technologies in education has become a policy priority across Europe. The European Union and its Member States have adopted a range of measures to support the digitalisation of education. It is also a key priority for several of the Europe 2020 Strategy flagship programmes, in particular the Agenda for New Skills and Jobs, Youth on the Move, the Digital Agenda and the Innovation Agenda. The effective use of digital learning technologies and resources (DLT&R) in Education & Training (E&T) is considered a key enabler for the realisation of the educational targets of the Europe 2020 Strategy [2].
Digital technologies in education are based on the methods of artificial intelligence (AI), therefore the inclusion of AI methods in the educational programs of universities is a timely and necessary step. It is AI methods that are the cornerstone of high-tech industries. Without knowledge of modern AI tools, a university graduate cannot achieve high results in the chosen field of activity. Moreover, this axiom is applicable in any university, in any country, in any direction of study.
The authors of the article, working at the department of management of high-tech industries at a Russian university, have learned from their own experience that it is necessary to introduce methods and elements of AI into the work programs of the disciplines of the "Management" direction. The department is running an educational project, the topic of which is closely related to the above tasks. To assess the effectiveness of managing this project, the standard methods described, for example, in [3][4][5], do not fully meet the task. The purpose of this work is to develop a methodology to evaluate, preferably in quantitative form, the effectiveness of project management related to educational activities at the university.

Methods
All universities in the world show interest in evaluating the effectiveness of educational systems. One difficulty arises when indicators are used that fail to provide accurate information. For example, in USA a widely available statistic is the level of spending on education expressed on a per pupil basis. At first glance, this looks like an efficiency indicator since it provides insight into the commitment of resources (the expenditure figure) and the result (the number of students being served by the system). Critics note that this statistic has been rising over time and conclude that the system is becoming less efficient. Traditional education systems are not best equipped to cope with the changing nature of learning, the changing demands on learners and their competences, and the need for new ways of teaching and managing complexities. Numerous studies find that system-wide educational innovation is notoriously difficult to achieve. Hence it is important for EU Member States to learn from each other, and from other countries efforts, successes and failures, to understand better how digital education policies can be designed to trigger innovation and change in education. The paper proposes a method for evaluating the effectiveness of managing an educational project, the goal of which is to introduce AI methods into the work programs of disciplines of students enrolled in the direction of "Management". The main component of AI, which is proposed to enter into the program, is machine learning [6][7][8][9]. The term machine learning refers to the automated detection of meaningful patterns in data. In the past couple of decades, it has become a common tool in almost any task that requires information extraction from large data sets. We are surrounded by a machine learning based technology: search engines learn how to bring us the best results, anti-spam software learns to _alter our email messages, and credit card transactions are secured by a software that learns how to detect frauds. Digital cameras learn to detect faces and intelligent personal assistance applications on smart-phones learn to recognize voice commands. Cars are equipped with accident prevention systems that are built using machine learning algorithms. Machine learning is also widely used in scientific applications such as bioinformatics, medicine, and astronomy.
At the university where the authors work, for students enrolled in the direction of "Management", it is proposed to introduce such methods in the disciplines of "Risk Management", "Strategic Management", "Personnel Management", etc. The neuro-fuzzy system type ANFIS (Adaptive Network-Based Fuzzy Inference System) as proposed to assess the effectiveness of management an educational project [10][11][12][13].
This approach is essentially a rule-based fuzzy logic model whose rules are developed during the training process of the model. The fuzzy logic mechanism can be implemented using an algorithm type Mamdani or Sugeno. The main difference between the Mamdani and Sugeno output systems is the way to get a clear output. In the Mamdani algorithm, a clear output value is obtained by defuzzification the output fuzzy set. In the Sugeno algorithm, a weighted average is generated at the output. We used the algorithm Sugeno. ANFIS implements Sugeno's fuzzy inference system in the form of a five-layer direct signal neural network.

Results
We built ANFIS system to assess effectiveness of the educational project. The following four features were taken as input attributes: the level of teacher training (X1); quality of teaching materials (X2); equipment and software (X3); financing (X4). The values of the input features are simulated on the basis of a 10-point scale (1-10 points). The output parameter is the effectiveness of the project, which is determined by an expert on a 100point scale (0-100 points). The volume of data drawn was 20 samples, of which the first 14 were used as a training sample, and the remaining samples as a test sample. The loaded training and test data is shown in Figure 1. The solution of the problem is performed in the program MatLab R2018b. Two options are possible for choosing such a system when generating a fuzzy inference system: -option lattice partition (Grid partition). Here, the membership functions of fuzzy terms are evenly distributed inside the data universe.
-option of subtractive clustering (Sub.clustering). With this technique, the number of clusters is determined at run time using the original data distribution.
The second option was chosen due to the simpler structure of the system. The formed ANFIS system is shown in Figure 1. As can be seen from Figure 2, the system has 5 layers: the first, consisting of 4 nodes, corresponds to the input parameters; the second one determines the number of membership functions for each variable; the third is the number of rules formed (14 in total); the fourth is the output values corresponding to each rule; fifth -system output.
The test results of the system are shown in Fig. 3, where there are 6 pairs of points: test data and the results obtained by the ANFIS system. The system operation is illustrated in Figure 4 (a fragment of the "View Rules" window is shown): when you enter four values of the vector of input features, the system evaluates the effectiveness of this project variant at 58 points on a 100-point scale.

Discussion
Machine learning will be steadily adopted in a variety of educational technology tools. Consequently, when digitizing education, it is necessary to evaluate its effectiveness. Selected parameters affecting the effectiveness of the educational process may be different. The authors only demonstrated one of the possible approaches to the formation of such an assessment. Nevertheless, the proposed methodology forms a quantitative characteristic of the effectiveness of the education process.

Conclusion
Thus, the paper shows that a hybrid neuro-fuzzy ANFIS system can be used to evaluate the effectiveness of a project. The scope of observations and, in particular, the test sample, does not seem to be large enough to produce fully reliable conclusions. However, the authors saw their task differently, namely: to identify the possibilities of new techniques in solving traditional problems.