Office of Academic Resources
Chulalongkorn University
Chulalongkorn University

Home / Help

AuthorBaayen, R. Harald. author
TitleWord Frequency Distributions [electronic resource] / by R. Harald Baayen
ImprintDordrecht : Springer Netherlands : Imprint: Springer, 2001
Connect tohttp://dx.doi.org/10.1007/978-94-010-0844-0
Descript XXII, 335 p. online resource

SUMMARY

This book is an introduction to the statistical analysis of word frequency distributions, intended for linguists, psycholinguistics, and researchers workยญ ing in the field of quantitative stylistics and anyone interested in quantitative aspects of lexical structure. Word frequency distributions are characterized by very large numbers of rare words. This property leads to strange statistiยญ cal phenomena such as mean frequencies that systematically keep changing as the number of observations is increased, relative frequencies that even in large samples are not fully reliable estimators ofpopulationprobabilities, and model parameters that emerge as functions of the text size. Special statistical techniques for the analysis of distributions with large numbers of rare events can be found in various technical journals. The aim of this book is to make these techniques more accessible for non-specialists. Chapter 1 introduces some basic concepts and notation. Chapter 2 describes non-parametricmethods for the analysis ofword frequency distributions. The next chapterdescribes in detail three parametricmodels, the lognormal model, the Yule-Simon Zipfian model, and the generalized inverse Gauss-Poisson model. Chapter 4 introduces the concept of mixture distributions. Chapter 5 explores the effectofnon-randomness inword use on the accuracy of the nonยญ parametric and parametric models, all of which are based on the assumption that words occur independently and randomly in texts. Chapter 6 presents examples of applications


CONTENT

1 Word Frequencies -- 1.1 Introduction -- 1.2 The frequency spectrum -- 1.3 Zipf -- 1.4 The quest for characteristic constants -- 1.5 The lognormal distribution -- 1.6 Discussion -- 1.7 Bibliographical Comments -- 1.8 Questions -- 2 Non-parametric models -- 2.1 Basic concepts -- 2.2 The Urn model -- 2.3 The Structural Type Distribution -- 2.4 The LNRE zone -- 2.5 Good-Turing estimates -- 2.6 Interpolation and Extrapolation -- 2.7 Discussion -- 2.8 Bibliographical Comments -- 2.9 Questions -- 3 Parametric models -- 3.1 Introduction -- 3.2 LNRE models -- 3.3 Evaluating Goodness of Fit -- 3.4 Parameter estimation -- 3.5 A comparative study -- 3.6 Comparing Lexical Measures Across Texts -- 3.7 Discussion -- 3.8 Bibliographical Comments -- 3.9 Questions -- 4 Mixture distributions -- 4.1 Introduction -- 4.2 Expectations, variances, and covariances -- 4.3 Examples of mixture distributions -- 4.4 Morphological Productivity -- 4.5 Discussion -- 4.6 Bibliographical Comments -- 4.7 Questions -- 5 The Randomness Assumption -- 5.1 The Randomness Assumption -- 5.2 Adjusted LNRE models -- 5.3 Discussion -- 5.4 Bibliographical Comments -- 6 Examples of Applications -- 6.1 Distributional properties of the lexicon -- 6.2 Morphological productivity -- 6.3 Authorship and Style -- 6.4 Beyond word frequency distributions -- 6.5 Some practical guidelines -- A List of Symbols -- B Solutions to the exercises -- C Software -- D Data sets


Linguistics Artificial intelligence Computational linguistics Psycholinguistics Statistics Linguistics Computational Linguistics Statistics general Psycholinguistics Artificial Intelligence (incl. Robotics) Theoretical Linguistics



Location



Office of Academic Resources, Chulalongkorn University, Phayathai Rd. Pathumwan Bangkok 10330 Thailand

Contact Us

Tel. 0-2218-2929,
0-2218-2927 (Library Service)
0-2218-2903 (Administrative Division)
Fax. 0-2215-3617, 0-2218-2907

Social Network

  line

facebook   instragram