Nova beseda Frequency Lexicon was compiled from the Nova beseda text corpus at the Fran Ramovš Institute of Slovenian Language with hyphen characters unified and with leading and trailing non-breaking spaces deleted.
Unlike most other Slovenian corpora Nova beseda texts were pre-processed before inclusion. Typos and words with supefluous hyphens, originating from false line joinings were corrected and parts of texts in foreign, non-Slovenian language were marked-up and excluded from the lexicon.
The corpus contains 318 million tokens, mostly wordforms. It is available for search through the web page http://bos.zrc-sazu.si/a_beseda.html, where wordform search is reached by selecting "word seach" in the right hand side "What to do?" column. On the mentioned web page the corpus structure is also explained.
The lexicon is UTF-8 coded, has 2,251,151 lines, each containing the following 2 data fields, tab separated:
-
token, Slovenian: pojavnica.
The vast majority of tokens are wordforms, also included are numbers and selected multiword units such as URLs, e-mail addresses, place names like New York, car plates, ID numbers.
-
frequency, Slovenian: pogostnost.
The sum of all frequencies is 318,170,212.
