Frequency of use. What do the indicators frq1, frq2 and LL-score mean in the dictionary of meaningful vocabulary

Frequency Corblant M., who defined hypoglycemia as a blood glucose concentration of less than 30 mg% (1.67 mmol/l) in the first 72 hours of life, found it in 4.4% of all live births. In 1971, Lubchenco L. O. and Bard N., using the Corblant M. criteria, identified hypoglycemia in newborns with greater

I wrote a funny PHP script. I ran all the texts on the Spectator through it to check the language. In total, 39,110 different word forms are used in the texts. How many different ones exactly? words- quite difficult to determine. To get at least somehow closer to this figure, I took only the first 5 letters of the word and compared them. The result was 14,373 such combinations. It would be a stretch to call this the “Spectator” vocabulary.

Then I took the words and examined them for the frequency of repetition of letters. Ideally, you need to take some kind of dictionary, to complete the picture. You cannot run texts, you only need unique words. In the text, some words are repeated more often than others. So, the following results were obtained:

o - 9.28%
a - 8.66%
e - 8.10%
and - 7.45%
n - 6.35%
t - 6.30%
p - 5.53%
s - 5.45%
l - 4.32%
in - 4.19%
k - 3.47%
n - 3.35%
m - 3.29%
y - 2.90%
d - 2.56%
I - 2.22%
s - 2.11%
b - 1.90%
z - 1.81%
b - 1.51%
g - 1.41%
th - 1.31%
h - 1.27%
yu - 1.03%
x - 0.92%
f - 0.78%
w - 0.77%
c - 0.52%
sch - 0.49%
f - 0.40%
e - 0.17%
ъ - 0.04%

I advise those who go to the “Field of Miracles” to memorize this table. And name the words in that order. So, for example, it would seem that such a “familiar” letter “b” is used less often than the “rare” letter “s”. We must also remember that a word has more than one vowel. And that if you guessed one vowel, then you need to start following the consonants. And besides, the word is guessed precisely by its consonants. Compare: “**a**i*e” and “sr*vn*t*”. In both cases, the word is “compare”.

And one more consideration. How did you learn English? Remember? E pen, e pencil, e table. What I see is what I sing about. What’s the point?.. How often do you say the word “pencil” in normal life? If the task is to teach how to speak as quickly and efficiently as possible, then you need to teach accordingly. We analyze the language and highlight the most commonly used words. And we start learning from them. To more or less speak English, only one and a half thousand words are enough.

Another pampering: to form words from letters randomly, but taking into account the frequency of occurrence, so that it looks like normal words. In the first ten “random” four-letter words, “donkey” popped up. In the next fifty - the words “rushing” and “NATO”. But, alas, there are a lot of dissonant combinations, such as “bltt” or “nrro”.

Therefore - the next step. I divided all the words into two-letter combinations and began to combine them randomly (but taking into account the frequency of repetition). Steel in large quantities will produce words similar to “normal.” For example: “koivdiot”, “voabma”, “apy”, “depoid”, “debyako”, “orfa”, “poesnavy”, “ozza”, “chenya”, “rhetoria”, “urdeed”, “utoichi”, “stikh”, “sapot”, “gravda”, “ababap”, “obarto”, “eleuet”, “lyarezy”, “myni”, “bromomer” and even “todebyst”.

Where to apply... there are options. For example, write a generator of beautiful branded playful names. For yoghurts. Like, “memoliso” or “utororerto”. Or - the generator of futuristic poems "Burliuk-php": "opeldiy miaton, linoaz okmiaya... deesopen odesson."

And there is one more option. Need to try...

Some statistics on the use of Russian words:

• The average word length is 5.28 characters.
• The average sentence length is 10.38 words.
• The 1000 most frequent lemmas cover 64.0708% of the text.
• The 2000 most frequent lemmas cover 71.9521% of the text.
• The 3000 most frequent lemmas cover 76.5104% of the text.
• The 5000 most frequent lemmas cover 82.0604% of the text.

After the note I received this letter:

Hello Dmitry!

After analyzing the article “Language will bring you to Kyiv” and the part where you describe your program, an idea arose.
The script you wrote seems to me to be intended absolutely not for “Field of Miracles” to a greater extent, but for something else.
The first most reasonable use of the results of your script is determining the order of letters when programming buttons for mobile devices. Yes, yes - it is in mobile phones that all this is needed.

I distributed it in waves ()

The following is the distribution by buttons:
1. All letters from the first wave go to 4 buttons in the first row
2. All letters from the second wave are also on the remaining 4 buttons in the same first row
3. All letters from the third wave go to the remaining two buttons
4. 4.5 and 6 waves go to the second row
5. 7,8,9 waves go to the third row, and the 9th wave goes completely (despite the seemingly large number of letters) to the third row of the 9th button, so that the 10th button is left for all sorts of punctuation marks ( period, comma, etc.).

I think everything is clear as it is, without detailed explanations. But still, could you process with your script (including punctuation marks) the following texts:

And then post the statistics? It seemed to me? that the texts reflect our modern speech as much as possible, and yet we both speak and write SMS.

Thank you very much in advance.

So, there are two ways to analyze the frequency of repetition of letters. Method 1. Take a text, find unique (non-repeating) word forms in it and analyze them. The method is good for building statistics based on words in the Russian language, and not on texts. Method 2. Do not look for unique words in the text, but go straight to counting the frequency of repetition of letters. We get the frequency of letters in Russian text, and not in Russian words. To create keyboards and other things, you need to use exactly this method: texts are typed on the keyboard.

Keyboards should take into account not only the frequency of letters, but also the most persistent words (word forms). It’s not so difficult to guess which words are the most commonly used: these are, firstly, official parts of speech, because their role is to serve always and everywhere, and pronouns, whose role is no less important: to replace any thing/person in speech (this, he, she). Well, the main verbs (to be, to say). Based on the results of the analysis of the texts listed above, I received the following “popular” words: “and, not, in, that, he, I, on, with, she, how, but, his, this, to, a, all, her, was, so, then, said, for, you, oh, at, him, me, only, for, me, yes, you, from, was, when, from, for, still, now, they, said, already, him, no, was, her, to be, well, nor, if, very, nothing, here, herself, so that, to herself, this, maybe, that, before, we, them, whether, were, is, than, or, her” and so on.

Returning to keyboards, it is obvious that in the keyboard the letter combinations “not”, “what”, “he”, “on” and others should be as close to each other as possible, or if not close, then in some optimal way. It is necessary to conduct research into exactly how the fingers move across the keyboard, find the most “comfortable” positions and place the most commonly used letters in them, without forgetting, however, about letter combinations.

The problem, as always, is one: even if it is possible to create a Unique Keyboard, what will happen to the millions of people who are already accustomed to qwerty/ytsuken?

As for mobile devices... Probably it makes sense. At the very least, the letters "o", "a", "e" and "i" must be exactly on the same key. Punctuation marks in order of frequency of use: , . - ? ! " ; :) (

- — Topics information protection EN word usage frequency … Technical Translator's Guide

Y; frequencies; and. 1. to Frequent (1 digit). Monitor the frequency of repetition of moves. Required part of planting potatoes. Pay attention to your pulse rate. 2. The number of repetitions of identical movements, oscillations in what direction. unit of time. Hours of wheel rotation. H... encyclopedic Dictionary

I Alcoholism is a chronic disease characterized by a combination of mental and somatic disorders resulting from systematic alcohol abuse. The most important manifestations of A. x. are altered endurance to... ... Medical encyclopedia

CAPTURE- one of the specific terms used in Russian hook records. non-linear polyphony, characterized by a developed subvocal polyphonic structure and a sharp dissonance of the vertical. Singing implementation of the term in the present day. time has not been studied... Orthodox Encyclopedia

Stylostatistical method of text analysis- is the use of mathematical statistics tools in the field of stylistics to determine the types of language functioning in speech, patterns of language functioning in different spheres of communication, types of texts, specific functionalities. styles and...

Portioned flavored snus, mini portion Snus is a type of tobacco product. It is crushed moistened tobacco, which is placed between the upper (less often lower) lip and gum... Wikipedia

Scientific style- presents scientific. the sphere of communication and speech activity related to the implementation of science as a form of social consciousness; reflects theoretical thinking, appearing in a conceptually logical form, which is characterized by objectivity and abstraction... Stylistic encyclopedic dictionary of the Russian language

- (in specialized literature also patronymic) part of the family name, which is assigned to the child by the name of the father. Variations of patronymic names can connect their bearers with more distant ancestors, grandfathers, great-grandfathers... ... Wikipedia

Common use, applicability, prevalence, applicability, marketability, generally accepted Dictionary of Russian synonyms. usage noun, number of synonyms: 10 generally accepted (11) ... Synonym dictionary

Reasoning- - functionally semantic type of speech (see) - (FSTR), corresponding to the form of abstract thinking - inference, performing a special communicative task - to give the speech a reasoned character (to arrive logically at a new judgment or ... ... Stylistic encyclopedic dictionary of the Russian language

I would like to warn you that the information presented in this article is somewhat outdated. I did not rewrite it so that later I could compare how SEO standards change over time. You can find up-to-date information on this topic in new materials:

Hello, dear readers of the blog site. Today’s article will again be devoted to such a topic as search engine optimization of websites (). Previously, we have already touched on many issues related to such a concept as.

Today I want to continue the conversation about internal SEO, while clarifying some points raised earlier, as well as talk about what we have not yet discussed. If you are able to write good unique texts, but do not pay enough attention to how they are perceived by search engines, then they will not be able to make their way to the top of search results for queries related to the subject of your wonderful articles.

What affects the relevance of text to a search query?

And this is very sad, because in this way you will not realize the full potential of your project, which can turn out to be very impressive. You need to understand that search engines for the most part are stupid and straightforward programs that are not able to go beyond their capabilities and look at your project with human eyes.

They will not see much of everything that is good and necessary in your project (that you have prepared for visitors). They only know how to analyze a text, taking into account many components, but they are still very far from human perception.

Therefore, we will need to at least temporarily climb into the shoes of search robots and understand what they focus their attention on when ranking various texts for various search queries (). And for this you need to have an idea about, for this you will need to read the article provided.

Usually they try to use keywords in the page title, in some internal headings, and also distribute them evenly and as naturally as possible throughout the article. Yes, of course, highlighting keys in the text can also be used, but you should not forget about over-optimization, which may result.

The density of keys in the text is also important, but now this is rather not a desirable factor, but, on the contrary, a warning - one should not overdo it.

Determining the density of occurrence of a keyword in a document is quite simple. In fact, this is the frequency of its use in the text, which is determined by dividing the number of its occurrences in the document by the length of the document in words. Previously, the position of the site in the search results directly depended on this.

But you probably understand that it will not be possible to compile all the material only from the keys, because it will be unreadable, and thank God this is not necessary. Why, you ask? Yes, because there is a limit to the frequency of using a keyword in the text, after which the relevance of a document for a query containing this keyword will no longer increase.

Those. It will be enough for us to achieve a certain frequency and we will thus optimize it as much as possible. Or we will overdo it and fall under the filter.

It remains to solve two questions (and maybe three): what is this maximum density of keyword occurrence, after which it is already dangerous to increase it, and also to find out.

The fact is that keywords highlighted with emphasis tags and enclosed in a TITLE tag have more search weight than similar keywords simply appearing in the text. But recently, webmasters have begun to use this and have completely spammed this factor, which is why its importance has decreased and can even lead to the ban of the entire site due to abuse of strong points.

But the keys in TITLE are still relevant, it’s better not to repeat them there and not try to cram too much into one page title. If the keywords are in the TITLE, then we can significantly reduce their number in the article (and therefore make it easy to read and more suitable for people, not for search engines), achieving the same relevance, but without the risk of falling under the filter.

I think that everything is clear with this question - the more keys are enclosed in accent and TITLE tags, the greater the chance of losing everything at once. But if you don’t use them at all, then you won’t achieve anything either. The most important criterion is the naturalness of introducing keywords into the text. If they exist, but the reader doesn’t stumble over them, then everything is great.

Now it remains to figure out what frequency of use of a keyword in a document is optimal, which allows you to make the page as relevant as possible, and will not entail sanctions. Let's first remember the formula that most (probably even all) search engines use for ranking.

How to determine the permissible frequency of using a key

We have already talked about the mathematical model in the article mentioned just above. Its essence for this particular search query is expressed by one simplified formula: TF*IDF. Where TF is the direct frequency of occurrence of this query in the text of the document (the frequency with which words appear in it).

IDF is the inverse frequency of occurrence (rarity) of a given query in all other Internet documents indexed by a given search engine (in the collection).

This formula allows you to determine the correspondence (relevance) of a document to a search query. The higher the value of the product TF*IDF, the more relevant the document will be and the higher it will rank, all other things being equal.

Those. it turns out that the weight of the document for a given query (its compliance) will be greater, the more often the keys from this query are used in the text, and the less often these keys are found in other Internet documents.

It is clear that we cannot influence the IDF, except by choosing another request for which we will optimize. But we can and will influence TF, because we want to grab our share (and not a small one) of traffic from Yandex and Google search results on user questions we need.

But the fact is that search algorithms calculate the TF value using a rather cunning formula, which takes into account the increase in the frequency of keyword use in the text only up to a certain limit, after which the growth of TF practically stops, despite the fact that you increase the frequency. This is a kind of antispam filter.

Relatively long ago (until about 2005), the TF value was calculated using a fairly simple formula and was actually equal to the density of occurrence of the keyword. The results of calculating relevance using this formula were not entirely liked by search engines, because they pandered to spammers.

Then the TF formula became more complicated, such a concept as page nausea appeared and it began to depend not only on the frequency of occurrence, but also on the frequency of use of other words in the same text. And the optimal TF value could be achieved if the key turned out to be the most frequently used word.

It was also possible to increase the TF value by increasing the text size while maintaining the percentage of occurrence. The larger the towel with the article with the same percentage of keys, the higher the document will rank.

Now the TF formula has become even more complicated, but at the same time, now we do not need to bring the density to the value when the text becomes unreadable and search engines will impose ban on our project for spam. And there is no need to write disproportionately long sheets now either.

While maintaining the same ideal density (we will determine it just below from the corresponding graph), increasing the size of the article in words will improve its position in the search results only up to a certain length. Once you have the ideal length, further increasing it will not affect relevance (more precisely, it will, but very, very little).

All this can be seen clearly if you build a graph based on this tricky TF (direct occurrence frequency). If on one scale of this graph there is TF, and on the other scale - the percentage of the frequency of occurrence of the keyword in the text, then we will get the so-called hyperbole as a result:

The graph, of course, is approximate, because few people know the real TF formula that Yandex or Google use. But qualitatively it can be determined optimal range, in which the frequency should be located. This is approximately 2-3 percent of the total number of words.

If you consider that you will also enclose some of the keys in accent tags and the TITLE heading, then this will be the limit after which a further increase in density may be fraught with a ban. It is no longer profitable to saturate and disfigure the text with a large number of keywords, because there will be more minuses than pluses.

What length of text will be sufficient for promotion?

Based on the same assumed TF, one can plot its value versus length in words. In this case, you can take the frequency of keywords constant for any length and equal, for example, to any value from the optimal range (from 2 to 3 percent).

What is noteworthy is that we will receive a graph of exactly the same shape as the one discussed above, only the length of the text in thousands of words will be adjusted along the x-axis. And from it it will be possible to draw a conclusion about optimal length range, at which the almost maximum TF value is already achieved.

As a result, it turns out that it will be in the range of 1000 to 2000 words. With a further increase, relevance will practically not increase, and with a shorter length it will drop quite sharply.

That. We can conclude that in order for your articles to rank high in search results, you need to use keywords in the text with a frequency of at least 2-3%. This is the first and main conclusion that we made. Well, the second thing is that now it is not at all necessary to write very voluminous articles in order to get to the Top.

It will be enough to exceed the threshold of 1000 - 2000 words and include 2-3% of keywords in it. That's all - that's it recipe for the perfect text, which will be able to compete for a place in the top for low-frequency queries, even without using external optimization (purchasing links to this article with anchors that include keys). Although, rummage around a little Miralinkse , GGL, Rotapost or GetGoodLink, you can, because it will help your project.

Let me remind you once again that you can find out the length of the text you wrote, as well as the frequency of using certain keywords in it, using specialized programs or using online services that specialize in their analysis. One of these services is ISTIO, about working with whom I talked.

Everything I said above is not one hundred percent reliable, but very similar to the truth. In any case, my personal experience confirms this theory. But the algorithms of Yandex and Google are constantly undergoing changes, and few people know how it will be tomorrow, except those who are close to their development or developers.

Good luck to you! See you soon on the pages of the blog site

You might be interested

Internal optimization - keyword selection, nausea check, optimal Title, content duplication and linking under LF
Keywords in text and headings
How keywords affect website promotion in search engines
Online services for webmasters - everything you need to write articles, search engine optimization and analyze its success
Methods for optimizing content and taking into account the theme of the site during link promotion to minimize costs
Yandex Wordstat and semantic core - selection of keywords for a website using statistics from the online service Wordstat.Yandex.ru
Anchor - what is it and how important are they in website promotion?
What search engine optimization factors affect website promotion and to what extent?
Promotion, promotion and optimization of the site yourself
Taking into account the morphology of the language and other problems solved by search engines, as well as the difference between high-frequency, mid-range and low-frequency queries
Site trust - what is it, how to measure it in XTools, what influences it and how to increase the authority of your site

The dictionary includes the most common words of the modern Russian language (2nd half of the 20th – beginning of the 21st centuries), equipped with information about the frequency of use, statistical distribution by text and genre, and by the time of creation of texts. The dictionary is based on the texts of the National Corpus of the Russian Language with a volume of 100 million words. More information about the history of frequency dictionaries of the Russian language and the methods of creating the “New frequency dictionary of Russian vocabulary” dictionary can be read in.

The concept of the dictionary was developed and prepared for publication by O.N. Lyashevskaya and S.A. Sharov, the electronic version was prepared by A.V. Sannikov. The authors express their gratitude to V. A. Plungyan, A. Ya. Shaikevich, E. A. Grishina, B. P. Kobritsov, E. V. Rakhilina, S. O. Savchuk, D. V. Sichinava and other participants of the NCRY seminar, who took part in the discussion of the principles of creating the dictionary. We thank O. Uryupina, D. and G. Bronnikovs, B. Kobritsov, as well as employees of Yandex LLC A. Abroskin, N. Grigoriev, A. Sokirko for their assistance at various stages of collecting and computer processing of the material.

How to find a word in the dictionary?

The two main sections of the dictionary are a list of words, ordered alphabetically and by overall frequency of use in the corpus. All words are given in their original (initial) form: for names this is the nominative case form (for nouns, as a rule, the singular form, for adjectives - the full masculine form), for verbs - the infinitive form.

The alphabetical list contains 60 thousand of the most frequent word forms. To find information about the desired word, go to the section, select the first letter of the word and find the desired word in the table. To quickly find a word, you can also use the search box, for example:

Word: great

In this way, you can find information not only about a specific word, but also about a group of words that begin or end in the same way. To do this, in the search window, use an asterisk (*) after the typed sequence of letters (“all words starting with...”) or before the string of letters (“all words ending with...”. For example, if you want to find all words starting with re-, type in the search box:

Word: re*

If you want to find all words ending with – just a little, type in the search box:

Word: *no

In the frequency list of lemmas, words are ordered by the overall frequency of use in the corpus of the modern Russian literary language. The frequency list includes 20,000 of the most commonly used lemmas.

To find information about the desired word, go to the section and find the desired word in the table. To search for information about individual words, it is best to use the quick word search window.

Why can't I find a word in the dictionary even though I can find it in the corpus?

This could be due to several reasons. Firstly, the word may have low frequency (for example, only 3 occurrences in the corpus) or be used only in texts written before 1950. Secondly, a word can appear many times, but in one or two texts: such lemmas were deliberately excluded from the dictionary. Thirdly, we cannot exclude that there was an error in the automatic determination of the original form or part-speech characteristics of the word, or that the word was erroneously attributed as a proper noun. The site presents a “test” version of the frequency dictionary, and we are going to continue working to clarify its lexical composition.

What information can you get about the use of a word?

In the dictionary you can get the following information about the use of a word in the corpus: