On the possibility of measuring the level of articulatory skills

The article dwells upon the possibility to assess quantitative evaluation of the level of articulatory skills. The rapidity and correct pronunciation of letter combinations, as well as the placement of verbal accents in the course of reading a coherent text, are subject to quantitative assessment. A procedure is proposed in which the subject reads aloud a standard text, and then a text, which includes a number of unfamiliar words (quasi-words). As a result, the subject’s pronunciation competence is estimated quantitatively (on a 100-point scale). The validity of the proposed procedure is checked by testing groups of subjects with varying levels of language training, as well as by data from a similar survey in Russian. Recommendations are given on the use of the procedure described. Besides, the experiment allowed us to point out the common mistakes made by the students. These mistakes concerned the rules of reading some letters and letter combinations, placing the primary and secondary stresses in polysyllabic words, as well as tertiary stresses in compound nouns. We considered both phonological mistakes which lead to misunderstanding, and non-phonological ones which distort words and make them difficult to understand in the process of communication.


Introduction
Phonetic aspect is part and parcel of the process of learning a foreign language. Lack of adequate pronunciation skills leads to errors in speech and accordingly makes communication difficult. The formation of pronunciation skills is a long and complex process that begins with mastering the ability to articulate individual letters and letter combinations, after which the learners move on to words, acquiring the basic rules of verbal accents, and further on -to pronounce whole phrases, i.e. learning to use the correct models of intonation, norms of pause, logical accent, etc.
For each stage of this process, there are numerous exercises of the appropriate type, that allow, with a certain degree of success, to achieve the final goal -fluent speech, as close as possible to the speech of native speakers of the language being studied.
As A. Cruttenden [5] notes, because of the pride of place given to grammar nowadays, there has been a tendency to attach lesser and lesser importance to the teaching of pronunciation. However, phonetics mustn't be undervalued and ignored in teaching any foreign language. It is necessary to start training pronunciation since the initial stage of learning a foreign language, because it is rather difficult to correct pronunciation mistakes afterwards, as we know from our own experience. The due approach to teaching the English phonetics to the Russian students will help correct common mistakes in the English pronunciation.
As we know, there are two types of pronunciation mistakes: phonemic and nonphonemic. Phonemic mistakes are worse. They occur when some phoneme is replaced by another one, producing word confusion. Non-phonemic mistakes take place when an allophone of a phoneme is replaced by another allophone of the same phoneme or by a sound that seems similar to the given sound but does not actually exist in that language. Such mistakes do not cause word confusion, but if they are numerous, they make utterances difficult to understand because of a foreign accent.
It is only usual that any learning process involves a stage of control of the acquired knowledge. In our case, it is important for the teacher to know what is the level of the student's phonetic competences, whether he is making the necessary progress, how much he differs from other students in the group, etc. Until now, such assessments were purely subjective -with all their inherent disadvantages (for more details see N.D. Galskova et al. [4]). In this paper, an attempt is made to obtain a quantitative assessment of the level of pronouncing skills in English. The object of measurement will only be the ability to master sound-letter correspondences and to set verbal accents while reading aloud a coherent text.
At an early stage of learning, reading aloud in a foreign language is slow, and is characterized by a significant degree of uncertainty, numerous errors. As a certain amount of sound-letter correspondences becomes mastered, a sufficient vocabulary is acquired, and sufficient experience is accumulated, reading becomes faster and relatively error-free. In fact, these two parameters (speed and number of errors) have laid the foundation of our quantitative approach to assessing phonetic skills.
A skill is usually defined as an ability to do something automatically due to training. The automatism is manifested in that actions are performed subconsciously. Accordingly, the articulatory skill is, apart from other things, the ability to pronounce individual sounds, syllables and words.
Thus, the aim of this paper is to present a procedure which enables to assess quantitative valuation of the articulatory skill.

Research Material
As you know, in English there are a number of specific difficulties in articulating some sounds, for example, sonant [ ŋ ] in the position between vowels (singing [ˈsɪŋɪŋ], bringing It is a common place when Russians unvoice final sonants, which is the result of interference of their Russian articulation skills and leads to misrepresentation of meaning. Generally speaking, while pronouncing English consonant phonemes (and vowels, too), Russians are evidently inclined to replace them with Russian phonemes which sound similarly. According to A.A. Reformatsky [6], the main difficulty in learning the pronunciation of a foreign language is "not mastering the new articulatory skill, but dissenting from one's own." Some letters are read differently in different contexts (cf. An especial problem is the pronunciation of letter combinations. The difficulty here is that the pronunciation of many of them is not the sum of the articulation of individual letters they consist of (i.e. igh [aɪ], rh [r]). Moreover, the same combination can be read differently even in the same positions, i.e. ch: [ʧ]  pronunciation of words with such letter combinations and that requires considerable experience. Fortunately, there are a number of letter combinations that follow certain rules.
So, speaking of letters and letter combinations, obtaining phonetic skill consists, on the one hand, of memorizing the pronunciation of words (the larger the vocabulary, the less likely it is to make a phonetic error) and knowledge of the rules of reading.
Finally, it is worth reminding, that in English (as in Russian, too), when uttering a polysyllabic word beside articulating sound combinations one should think or the word stress. The place of stress is an individual characteristic of every word. Any word of a language proves to be meaningful only if it is perceived as the unity of its phonemic composition and accent -a unity that is obligatory for all the speakers of the language in question.
In most cases, the place of stress in the word has to be memorized, but one may trace out some regularity here. For example, in two-syllable words, the stress is most often placed on the first syllable, and in many multi-syllable words, it is placed on the second syllable from the beginning or on the third one from the end (most often these positions coincide). These rules follow a trend, but are not absolute, so they are usually not taught directly, but are learned by students in the course of assimilating speech experience.
The procedure proposed in this paper allows you to evaluate not the quality of the utterance of individual words, but the level of articulatory skill as a whole in the course of reading a coherent text.

Method of evaluating articulatory skills
The procedure for measuring the articulatory skill begins with the student being asked to expressively read aloud, at his or her habitual pace, a small fragment of a coherent text in English (text 1). The time is recorded.
Then the student is given a text of exactly the same length (text 2). Some words in this text are replaced by so-called quasi-words. These are invented, artificial words; the student sees them for the first time, which means that he or she has never heard them pronounced (cf. lidge, spalk, palagerial, etc.).
If the subject has a fairly advanced articulatory skill, he or she will be able to read these words without mistakes and, more importantly, do it relatively quickly. Here, again, time is recorded.
The difference of the reading rate of these two texts forms the basis of the proposed procedure for evaluating the articulatory skills. Expressiveness of reading is not taken into consideration.
In addition, at the end, the subject receives a list of all quasi-words from text 2 for reading, and the number of errors is recorded.
As a result, for each subject we have at our disposal data on three parameters: the reading time of text 1 and text 2, as well as the number of phonetic errors made in the course of reading the list of quasi-words. To determine the level of articulatory skill, the following indicator R art is proposed: According to the formula the value of indicator R art depends on the following factors: -time of reading standard text 1 (the less time spent, the higher the indicator); -the difference between time t 1 and time t 2 (the smaller the difference, the higher the indicator); -number of phonetic errors: 2 points (2 seconds) were deducted for each error in the control text.
What will be the result of the "ideal" subject with this approach? First, since he/she has developed perfect skills of sound-letter correspondences, he/she is quite fluent in reading a standard text and equally fluently reads text 2 (i.e., his t 1 = t 2 ). In the group of subjects we interviewed (100 teachers and students of a language university), t min for text 1 was 26 sec. We will consider it ideal, so for our virtual ideal subject t 1 = t 2 = t min = 26.
Secondly, the "ideal" subject will not make mistakes when reading the list of quasiwords, i.e., he will get M 2 = 0.
Taking account of this data, we get the following R art value for our "ideal subject": Brief comment. Subject A is close to the ideal. He has read the standard text quickly enough (almost at the minimum speed of 26 seconds); he has a very small difference in the reading time of the two texts (less than two seconds), and he has made only 1 mistake when reading the list of words. Accordingly, his R art is close to the maximum possible value.
In contrast, subject B even a standard text reads quite slowly, but the main thing is that she has a large difference in reading time between the two texts. Text 2 was read with great difficulty, she spent much time for hesitation, and sometimes lost time on self-correction (first she read a quasi-word in one way, then in some other one). Accordingly, her R art index is significantly lower than that of subject A.
To check the validity of the procedure just described we have carried out an experiment with a representative group of subjects. Experiment 1.
Subjects. 100 adults with obviously different language grounding took part in Experiment 1. The group of subjects comprised school teachers of English, as well as university lecturers and students of different academic years (including those who were studying English as a second foreign language).
Material. As we have said above, a standard text was given for reading aloud at the first stage. This was the first paragraph of the story "The Wizard of Oz" (it contained 500 characters with spaces).

Text 1
Dorothy lived in the midst of the great Kansas prairies with Uncle Henry, who was a farmer, and Aunt Emily, who was the farmer's wife. Their house was small. It had only one room that contained an old cookstove, a cupboard for the dishes, a table, four chairs, and the beds. There was no garret and no cellar, except a small hole in the ground. They called it a cyclone cellar. The family could go there in case one of those great whirlwinds arose mighty enough to crush any building in its path.
For stage 2 we composed a text of the same volume that contained 30 quasi-words.

Text 2
When I was a moster, my famlet and I were once whoring in line to buy chatmots for the jammit. In front of the pooze woat was another fander with phitigious wrendles which were kambering their potent sorants. They were bewightly dennering about bracks, illectics, and other hotts that they would pusk the komlet. When we stood in the tissionary we could delletly see a shist lidge on every side. The sun had pongingly baked the palagerial vint into a lassing spalk and the wind had smooked it, too.
When selecting quasi-words for text 2, we tried to test the most typical rules for reading letters and letter combinations (see table 1). Experimental procedure. The experiment was conducted with each subject individually.
After reading text 1, he or she was given the following instruction: "Now we will give you another text. It consists of grammatically and syntactically correct phrases, but includes a number of words that are unfamiliar to you. Although you do not know the meaning of these words, you will be able to read them making use of the rules of articulation of letters and letter combinations that you know. For example, recall the famous phrase 210, 19030 (2020) E3S Web of Conferences ITSE-2020 https://doi.org/10.1051/e3sconf/202021019030 of L. Shcherba: «Глокая куздра штеко будланула бокра и кудрячит бокрёнка» (Glokaja kuzdra shteko budlanula bokra and kudrachit bokrjonka). It is difficult to grasp the meaning of the phrase, but it is quite unlikely that you won't be able to read it aloud, laying reasonable stresses (glókaja kúzdra shtéko, etc.). So try to read the text aloud in an ordinary manner, correctly articulating the words and observing the desired pause and intonation." For the purpose of training the subjects were granted with a piece of Russian text that contained both familiar and unfamiliar words.
The reading time of text 1 and text 2 was recorded. After this, the subject was asked to articulate the entire list of 30 quasi-words included in text 2 (the number of errors was recorded).
Thus, each subject was characterized by three individual indicators: the rate of reading a standard text, the rate of reading a text with quasi-words, and the number of mistakes when reading a list of quasi-words.
In the course of reading the students made several common mistakes concerning the following rules of reading: 1) the 4-th type of vowel reading; 2) the letter combination "vowel + consonant + l or r". In this case the vowel must be read according to the 1-st (alphabetic) type of reading: "acre" [ᶦeIkə], "cable" [keIbℓ]; 3) a vowel which occurs before the letter combination "ld" or "nd" is pronounced according to the 1-st type of reading: "gold" [ɡəʊ ℓd], "blind" [bℓaInd]; 4) a vowel before the letter combination "rr" is read by the 2-nd closed type: "hurry" [ᶦhᴧrı], "carry" [ᶦkaerı]; 5) the vowel "a" before the letter combinations "st", "sk" must be read by the 3-rd type: "past" [pɑ:st], "task" [tɑ:sk]; 6) the vowel "o" before "m", "n", "v" and "th" is pronounced as When placing a word stress, the students made the following typical mistakes: 1) they did not always distinguish two-syllable words where the position of stress on the first syllable characterizes nouns, while its position on the second syllable characterizes verbs. This applies to phonological mistakes, for example: "protest" [ᶦprəʊ test] («протест») -"to protest" [prəᶦtest] («протестовать»), "conduct" [ᶦkɒndkt] («поведение») -"to conduct" [kənᶦdkt] («вести себя»), etc. The reason for such mistakes is that differentiating and delimitative functions of word stress are secondary for Russian, although the free placement of stress does result in the existence of accentual minimal pairs: муᶦка -мукаᶦ [7]; 2) the difficulties associated with the correct setting of accents in polysyllabic words are complicated by the presence of the secondary stress in the accentual structure of the English language, the incorrect use of which is unacceptable. Although, the incorrect use or omission of the secondary stress does not lead to phonological mistakes, it distorts words and makes them difficult to understand. The most common mistake of this type is omitting the secondary stress in polysyllabic words, since similar words in Russian have only one stress. As a result of this mistake, the sound appearance of the word is often distorted. For example, instead of [prəˌnnsıᶦeɪʃn] the Russian-speaking people tend to say [prənənsıᶦeɪʃn], instead of [ˌdemənᶦstreɪʃn] -[dImnᶦstreɪʃn], etc. [5]; 3) we can also note that the Russian-speaking students often omit the tertiary stress in compound nouns. This also greatly distorts words on account of unacceptable vowel reduction, for example, [ᶦbℓaekbəd] instead of [ᶦbℓaek˳bɔ:d]; 4) in addition, it is quite common to transfer the strong stress in three-syllable words from the antepenultimate syllable to the last syllable of a word, for example, [rIkɡᶦnaɪz] instead of [ᶦrekəɡnaɪz]. The reason for this mistake is that the accent in Russian tends to the second half of a word.
To avoid all these mistakes, it is necessary to keep in mind the main trends and rules for placing accents in the English words. Stress is one of the most important components of the sound structure of a word, and the placement of stress is an individual characteristic of a separate word. Misplacing the accent position may lead either to difficulties in understanding the meaning of the word (in this case, we deal with non-phonological mistakes), or to misunderstanding, when one word is taken by the interlocutor for another one, which is an accentual phonological error [1].
As L.A. Telegin notes [10], stress is not just an acoustic-articulatory phenomenon, but a language factor, a component of the formal structure of significant language units, and correct stress in a word is a necessary condition for mutual understanding.

Results and Discussion
For each group of subjects, the average of the R art indicator was obtained. The results are presented in table 2. 2nd year university students (study English as a second foreign language) 9 37.5 As expected, the highest values of R art were achieved in groups 1 and 2. It is natural to assume that university lecturers and undergraduates are quite proficient in articulatory skills. Although their average R art (about 60) is very far from ideal, it can be considered a "reference point". Note that we are talking about the average value; individual variation, of course, took place (from the minimum 43.5 to the maximum 91.5). This is also notable. Not all English speakers articulate equally well. Group 3 is somewhat (though slightly) inferior to groups 1 and 2, and here, too, there is a significant data variation. This result seems to indicate that phonetic skills are formed fairly quickly, which is not true for lexical or, say, grammatical skills. Somewhat alarming results were shown by Group 4. They are inferior to those of students'. A third of the teachers have a R art within the range 34.3 to 45.0! It reveals professional inefficiency of a large amount of our school teachers.
Finally, the results of group 5 are very telling. Students in this group had been learning English for only one year, and we would hardly expect them to acquire a high-level articulatory skill (one of the students has an "anti-record" indicator -17.7).
The described above data allow us to assent to the validity of the R art indicator. Another evidence of validity is provided by the detailed analysis of group 3 results (table 2).
Students in this group make up a 1st year study group. We asked the teacher who gives a practical course in phonetics in this group to assess the level of phonetic competence of each student. To make it easier, we asked to sort out the six most advanced students and the six poor ones. Then we calculated the average R art for each of these subgroups and obtained the following results: the average R art in the group of advanced students was 62.7, while the group of poor students had the average 40.9. As you see, the difference is quite significant. Our indicator elicits it unequivocally, and, therefore, fully reflects the opinion of the teacher and, if necessary, can successfully replace it.
To make sure that our R art indicator is valid, the experiment described above was reproduced on the material of the Russian language (experiment 2). It is natural to assume that the phonetic skill in the native language of an adult subject is fully formed, and his/her R art index will in any case be higher than that for the English language.
The texts chosen for experiment 2 were similar in their parameters to the English texts (table 3). Please note that although the texts are almost identical in their total length (number of characters), they differ in the number of words and -even more so -in the average length of quasi-words (in Russian they are significantly longer). The last fact makes reading Russian texts more difficult. However, we expect that even with this handicap, the R art indicators for the Russian texts will be higher. The experimental procedure of experiment 2 (instructions, recording of results, etc.) exactly repeated the procedure of experiment 1.
A total of about 20 subjects from different groups participated in experiment 2, but we were particularly interested in those who read texts in both languages (9 people). Comparison of the data obtained for this group gave the following result: the average value of the RPA indicator for Russian was 70.5, and for English -only 55.9. The difference might have been even greater if it hadn't been for one thing. When working with a normal Russian text, almost all subjects introduced an element of expressiveness to their reading (those norms of intonation and pausing that we mentioned earlier). This, of course, increased the reading time and biased the R art indicator. Expressive reading in English occurred only occasionally.
While comparing R art values for Russian and English we picked up some interesting result. We divided the group of 9 subjects who read texts in both languages into two  As we see, even in their native language, the subjects differ quite significantly in the level of their phonetic skill. According to table 4, there is a significant correlation between R art values in Russian and English. In this connection, it is worth researching a more general problem of linguistic ability, but this will require a more extensive experimental material.
In any case, this problem remains outside the scope of the claimed topic of this paper (some ideas on the problem are discussed elsewhere [11,12]).

Conclusion
The procedure just described allows us to obtain a reasonable quantitative assessment of the level of articulatory skills and may be recommended for practical use. It proves to be quite easy to use and may be of interest to teachers of English who are engaged in training English pronunciation. Of course, in each case, teachers can choose their own experimental texts and their own set of quasi-words that check the phonetic rules they need. The R art indicator can be very useful when checking the assimilation of a specific phonetic material, including the most typical phonetic mistakes of the Russian-speaking audience [10,11]: non-alveolar articulation of consonants [t], [d], poor differentiation between similar consonant sounds [w-v] or [ŋ-n], etc. Besides, one can focus on checking the success in assessing certain trends and rules that help determine the placement of verbal stress [2]. All problems of this kind are solved by selecting the appropriate quasi-words inserted in the experimental text.
However, two requirements should be met. First, the standard and the experimental texts must be of the same length. Secondly, you should define the t min for the text that was selected as standard. Its value will depend both on the text's parameters (length, language style, etc.) and on the level of language proficiency of the group of subjects evaluated. By the way, for Experiment 2, we set t min = 28 seconds (this is slightly longer than in Experiment 1 -due to the specific difficulties of reading the Russian text, which we mentioned above).
In other respects, the procedure for testing and calculating the R art value may be realized the way we described in the paper.