Evaluation of the execution of government contracts in the field of energy by means of artificial intelligence

. The methodology for evaluating the execution of government contracts in the energy sector by means of machine learning is presented. The signs describing performers and customers in the public procurement system were identified, the risks of fulfilling contracts on the part of customers and performers were identified, the main categories for compiling a dataset were identified and a dataset was assembled. Data problems are described, and ways to fix these problems are described. The problem of classifying the execution of government contracts is solved, a software package for intelligent forecasting of the execution of government contracts is described


Introduction
Timely and high-quality execution of contracts concluded by a state or municipal customer in the context of global economic challenges deserves special attention [1]. Currently, a number of ministries, departments, state-owned enterprises periodically face the problem of late execution of state contracts, or not fulfilling the contract at all [2]. This entails significant risks, both for an individual enterprise/department, and in some cases for the state as a whole. This is especially important for the energy sector, since the safety and viability of not only state institutions, but also all branches of production depends on it.
To assess risks, it is necessary to analyze the activities of companies collectively from all sides. There is not enough information about existing obligations and concluded contracts. Selective consideration of complaints and fragmentary inspections do not contribute to the effectiveness of the existing procurement control process, which indicates the need for a modern automated control system using artificial intelligence. Three parties participate in eprocurement: the bidder (customer), bidders (suppliers) and the operator of the electronic trading platform (ETP). Each of the parties has its own powers, while it does not have the right to interfere in the activities of the other, because otherwise the risk of manipulation during the procurement increases sharply, as a result, there is a violation of the order of selection of the counterparty. In the model of regulation of public funds expenditures, it is required to take into account all possible parameters of each of the participants in electronic procurement.

Identification of risks in the contract system
To identify economic and corruption risks, customers in the contract system can be identified by the following parameters: 1) A procurement schedule that includes the needs for the year ahead: what is being purchased, in what time frame and in what volume; 2) The fact of violation of the procurement rules established earlier; 3) Access to the content of other people's applications; 4) Restrictive logistics requirements (different delivery locations, lifting to the floor, etc.); 5) The requirement of documents that are optional before delivery (a certificate for a batch, etc.); 6) Requirements for a specific technical solution -the terms of reference in this case are drawn up for a pre-determined supplier, and for other applicants it will be just a vague text; 7) Amendments to the draft contract; 8) The absence in the contract of a proper procedure for control of execution, examination, acceptance; 9) Sampling of the contract volume; 10) Termination by partial execution; 11) "Temporary storage" -in case of failure of delivery terms by "your" supplier; 12) Penalties; 13) Acceptance of products (acceptance of nonconforming, "payment" for acceptance); 14) "Payment" for the "correct" interpretation of the "managed" terms of the contract; 15) Acceptance of products not delivered; 16) Evasion of the requirement to pay penalties, fines and penalties; 17) A significant number of non-competitive methods of procurement, that is, in the form of procurement from a single supplier (contractor, contractor); 18) "Regular" procurement participants do not participate in a specific purchase; 19) The same natural (legal) person acts as a supplier (contractor, contractor); 20) Procurement participants "unexpectedly" withdraw their applications.
Currently, the bulk of inspections are carried out by customer organizations, while potential threats may come from suppliers. In addition to essential information, such as the company name, contact coordinates, declarations that the company is not bankrupt and there are no tax arrears, suppliers of goods and services can be classified as follows: 1) The date of accreditation on the electronic platform -the recent registration of the organization is in doubt (a few weeks or months before the date of the announcement of the auction); 2) The number of employees of the company authorized to participate in the procurement of EDS; 3) Getting into the register of unscrupulous suppliers. As of the end of the third quarter of 2022, the specified register contains 1,164 registry entries in respect of 1,029 procurement participants.
Additionally, you should pay attention to the following verification criteria: 1) Experience in performing works that are the subject of the competition; 2) The same natural (legal) person acts as a supplier (contractor, contractor); 3) Lack of the required number of specialists of the required qualification level for the execution of the contract; 4) Lack of direct contacts with counterparties; 5) Absence in the staffing of the organization of the person responsible for accounting (chief accountant); 6) Contracts with the counterparty contain conditions that are not typical of common practice, etc.
In addition, it seems suspicious if the participants in the procurement are legal entities with the following characteristics: 1) Creation of a "mass" registration at the address; 2) Insignificant (minimum) size of the authorized capital; 3) The absence of equipment and other material resources for the execution of the contract on the right of ownership or other legal basis.
Certain verification principles can also be applied to the ETP operator: 1) In order to create the appearance of competition, procurement participants are physical (legal) persons who are objectively unable to fulfill a potential contract, etc. 2) Procurement participants who are uncertain as a supplier (contractor, contractor) are involved as subcontractors; 3) Unjustified rejection of applications; 4) Targeted work with suppliers (phone calls, etc.); 5) Documents that are not necessary for the production of this type of work in favor of the customer are requested from the procurement participants; 6) Artificial prolongation of the contract term; 7) Several independent lots are combined into one. Such a trick is used to limit competition, since not every bidder will be able to fulfill several unrelated conditions at once.
Based on the statistics of violations in recent years, as well as in the light of recent observations that customers are less likely to post data on procurement procedures, and suppliers are less likely to participate in them, the need for tools for automated detection of abuses when concluding state contracts does not require proof. Thus, by analyzing the listed parameters, the intelligent system could identify unscrupulous participants in electronic procurement. Table 1 shows the main categories and their transcripts for compiling a dataset.

Development of an information system for the execution of state contracts
The information system assumes the following elements: -a database of contracts for training models and subsequent analysis; -a set of criteria for successful fulfillment/ non-fulfillment of the contract; -database of counterparties; -transaction data of counterparties; -counterparty cluster analysis module; -contract cluster analysis module; -semantic contract analysis module; -module for building predictive models of contract execution; -web user interface.
The training quality of machine learning models significantly depends on the training data set. For training, it is planned to use databases of a specific department for which it is planned to develop an information system, as well as data that is publicly available on the EIS Procurement portal [3]. Information about counterparties is available on specialized information resources, for example, in the Spark system [4]. Data unloading from external systems is possible with the help of a parsing system that can be developed in Python c using a specialized library Beautiful Soup [5]. The following clustering algorithms can be used to implement cluster analysis modules: hierarchical, k-means and a specialized Scikit-learn library. The semantic text analysis module is implemented using a specialized OpenNLP automatic natural language processing framework. Predictive models are developed based on the following machine learning methods. The analysis of Table 2 shows that all models give excellent results on the test sample. Despite the fact that contract dates have been removed from the input attributes. This allows us to judge that the dataset has useful information for predicting "dangerous" contracts on the principle of execution for much more than a year. In the future, it is necessary to use other data, since the definition of dangerous contracts as an outlier by the terms of execution may not always be correct. The proposed information system will make it possible to qualitatively predict the execution of the contract, calculate the probability of failure of deadlines and the reasons for non-fulfillment of the contract, which can be useful for analysts, service managers, government officials.
The article was prepared as part of the state assignment of the Government of the Russian Federation to the Financial University for 2023.