Critiquing the Critics: Statistical Analysis of Music Critics' Rating Distributions as a Measure of Individual Refinement

In order to evaluate rating behavior among modern music critics, a previously compiled database that randomly sampled 352 different critics' ratings of more than 15,000 albums was utilized. Potential quantitative markers of rating refinement were explored by analyzing frequency distributions of album ratings. As a group, critics' ratings were found to be roughly normally distributed, but individual critics were found to vary widely in rating behavior, as evidenced by both visual inspection of histograms and related statistics, such as skewness, kurtosis, z-score ranges, and Kolmogorov-Smirnov calculations. Precision remains an area in need of improvement among all critics, with the rating scales varying between only 10-20% precision, and ceiling effects with somewhat negatively skewed distributions occurring among some critics.