Reimagining Cupping Competition Scoring, Part 1: The problem.
The blog was originally posted on Daily Coffee News on June 12, 2017.
Since 1987, there has been a cupping competition at the annual Kona Coffee Cultural Festival. In this competition, 50 to 80 farms compete to be recognized as the best in Kona. Like other coffee competitions, a handful of judges cup through the coffees a number of times to determine the winner.
Over the years, the scoring system used for the competition has evolved and has recently settled on the fairly commonplace 100 point, Specialty Coffee Association scoring system. While the SCA scoring system is familiar and revered by many in the specialty coffee industry, it is imperfect.
I want to be clear, there is no perfect scoring system for a competition — this rings true for any product, not just coffee. They all have two major imperfections.
- No system defines what a winning item would taste like. This lack of definition prevents judges from scoring accurately as they have no standard of excellence to compare to. Moreover, competitors never know what to submit if they don’t know which taste is actually competitive. Without a definition of quality, every competition score is based somewhat on the whims of the judges.
- Scoring in general, including in competitions, is very subjective. When the preferences and expectations of individual judges influences the scoring, the composition of judges can be more important than the items being judged.
The value of a 100 point scoring system for a competition is that it is a familiar concept to people and, more importantly, it is very easy to figure out who wins.
For the upcoming festival in November, Daylight Mind Coffee Company was asked to host and run the cupping competition. While this is exciting in and of itself, it affords us the opportunity to try to invent an improved competition scoring system. We hope to address the flaws inherent to 100 points systems and offer something that is objective and locally adaptable to every competition.
As we move along in this adventure, we’ll be sharing our methodology and findings through a series of posts. We hope that readers here will be eager and willing to offer us feedback and criticism to help us improve the design. We aspire to transparency every step of the way, both to our peers and to the competition participants. We know we won’t get it perfect the first time, but if we don’t start, we’ll never finish.
Feel free to comment with your thoughts below or email them directly to Shawn Steiman and stay tuned for Part 2: “Defining the Winning Coffee.”