Decades ago, soon after the 100-point scoring scheme became a popular way to evaluate wine, a glossy wine magazine awarded a 100-point score to an exalted First Growth Bordeaux.
As a former math major in college and professional skeptic, I was already cynical about rating the quality of a wine by numbers — especially if more than one evaluation system is used at the same time. The magazine’s policy made no sense to me. It was almost paradoxical. (See “The Unexpected Hanging,” by Martin Gardner.)
The magazine said its evaluation had been done by five judges, that the evaluation was totally blind (the judges didn’t know the identity of the wines), and that the final score represented the cumulative opinion of all judges.
I immediately saw problems. The magazine asking us to accept the premise that three or more of the five judges had independently come to the conclusion that one unknown wine was worth 100 points — and that the other two judges had rated it at least 99 points.
That inconsistency proved that some of the problems associated with wine judging may not always be as evident as they should be.
The late UC Davis Prof. Maynard Amerine, in his 1976 book “Wines, Their Sensory Evaluation,” outlined several wine-evaluation fallacies when people are rating several wines.
Most people seem to prefer ranking systems to give them a frame of reference as to what a wine expert, or an expert panel, considers great.
Even UC Davis sees the need for some form of ranking, and for decades has used a 20-point system to evaluate student-produced wines. This system was later adapted to analyze commercial wine. Decades ago, I used a form of this for my own purposes. (I never published the results.) It was merely a shorthand method for determining how to write about a particular wine.
The first 12 numbers were used for various flaws. No. 13 was reserved for wines I believed to be from a winemaker who was unclear on the concept (overly oaked Pinot Gris, a sweet “Brut” sparkling, etc.).
A 14 was OK, nothing special. I wouldn’t recommend it because it wasn’t outstanding. In my mind, many such wines are very expensive.
Wines rated 15 to 20 were good enough to recommend.
— 15s were wines I’d drink, usually with a specific food (Chianti with tomato-y pasta, etc.)
—15.5 is the number Australian show judgings use for a bronze medal. I’d recommend a 15.5, especially if the price were right.
—16 was for exemplary examples of the breed; 17 was for a wine that Australian show judgings would award a silver medal. It’s a wine I’d buy if the price were reasonable.
—17.5 to 18: wines that were nearly perfect.
—18.5. This is a gold medal wine in the Aussie show system, a wine of near perfection, worth buying.
—19 and 20: gradations of greatness.
The greatness of the Australian show judging system is that all judges are professionals with at least a decade of judging experience. They understand the parameters and honor wines coming from every sub-district of the continent.
Most American wine judges think of themselves as professionals, but I learned how professional the Aussie system is on my first judging experience there in 1992. It was humbling.
American judging systems often have rules that seem logical, but have no particular justification. Judging by price is, for example, ludicrous, not to mention pointless. I’m also suspicious of judging where 80% of the wines get medals.
Judging systems that accumulate raw scores from the judges, add them up, and divide by the number of judges depend on every judge understanding uniqueness. This never happens.
Averaging scores reduces the final score of each wine based on the vote of the weakest judge, who normally clumps scores in the middle. (Amerine’s “Error of Central Tendency.”)
This has the effect of dragging down the score of a great wine from a gold-medal candidate to a silver or lower. This is often the case when a panel has six or seven judges. In such settings, great judges’ voices are muted by bad judges’ ill-formed opinions.
Also, events where a majority of mediocre judges can overrule a smaller number of exceptional judges make no sense.
The Australian show system is so professional because of the high skill level of all senior judges (three per panel) and the additional skills of the three associate judges, who are effectively senior judges in training.
The best wine competitions are those that focus on wines of a delimited area, which is why I love to judge competitions in New York, Michigan, the Pacific Northwest, Mendocino County, Sonoma County, and the Sierra foothills.
The regional distinctiveness one sees in such areas ought never be discounted. Quality judges honor such distinctions. The only time such competitions collapse into a morass is when judges dismiss regionality – which can actually be one of wine’s endearing charms.
This has occurred to me twice when judging in Canada. I adored many of the wines because of their distinctive qualities, but several local judges disparaged their own products. They were witnessing gold, but acting as if it were pyrite.
Visionary judges who have traveled widely recognize uniqueness and honor it.
Wine of the Week: 2014 Silverado Vineyards Merlot, Mount George ($40): You need no score to identify the dramatic quality of this wine from a long-time producer that understands regional distinctiveness, balance, structure, and varietal personality. Some people might prefer the 2015 vintage of this wine (it is excellent), but almost 18 months after its release, this earlier vintage has a couple of additional bonus points. Not only did the extra time in the bottle add happily to its varietal aroma (which includes tea, olive, dry and fresh cherry), because it’s not as fruit-driven as the 2015, the price is more reasonable than it was when released. Mt. George is a great cool location for Merlot! For an added treat, decant for at least an hour. Occasionally discounted.
Of all the grape varieties that claim preeminence in this world, disheveled, tattooed, unshaven, sandal-wearing Pinot Noir seems to be the least likely candidate for stardom.
Dan Berger lives in Sonoma County, where he publishes “Vintage Experiences,” a subscription-only wine newsletter. Write to him at email@example.com. He is also co-host of California Wine Country with Steve Jaxon on KSRO Radio, 1350 AM.
Be the first to know
Get local news delivered to your inbox!