Does a sample roast affect a coffees final score? We would argue it does, and that “cupping through the roast” when evaluating coffee is a skill only a very few hold. We would also say that your sample roasts might be the most important roasts you do.
This post unpacks this premise and examines how an experienced group of coffee professionals at the Nordic Roaster Forum scored the same coffees when roasted differently.
Why does a sample roast matter more than other roasts? First, because it usually establishes a large buying decision. Second, because we have observed that how a sample is roasted directly effects how it is scored and perceived.
There are varying levels of forgiveness or importance placed on roast quality at different stages. The importance likely depends on who you are speaking with – a customer will probably think the production roast quality is the most important, while a producer may say the same about the sample roast.
We will say that at the purchasing stage is where the most importance should be placed on the roast.
The sample roast often sets up a purchasing decision for a business.
A customer buying a bag of roasted coffee to take home should not receive a poor roast, of course. But they are investing £7-16/$15-25 and a week or two of consumption.
A business is spending thousands of dollars, and that coffee becomes a product it sells and bases its reputation and income on. Similarly, a producer is capitalising on their season’s work.
If we are looking for great coffees, and paying for quality in green coffee, the roast should be excellent so as to showcase the coffee accurately.
The analogy that comes to mind is around the Olympic sports that have judges assign scores. An athlete spends years preparing for their event, and they expect that the stage will be the same for each competitor. If their course/mats/track/pool are different for each competitor, it’s not fair. If we are the judge, and we want to find great coffees/pay fairly for a coffee, then we should be roasting that coffee excellently, every time.
Sample roasting is not profiling a coffee.
There are 3 categories of roasts that we like to discuss at IKAWA:
Nordic Roaster Forum 2018 Workshop: Testing This Theory
In 2018, we had the pleasure of hosting a workshop in Tim Wendleboe’s cupping lab at the Nordic Roaster Forum. We took the opportunity to have some fun and set up a cupping with two parts: a “pick one” section, where we had a group of 2 or 3 coffees and we asked “which coffee would you buy?”. And a section where we asked cuppers to assign scores.
What we didn’t tell our attendees and cuppers was that across all the cuppings there were only a handful of coffees on the table, and each coffee had been roasted a number of different ways using profiles from our Online Roast Profile Library.
For example, here is the responses on the Google Form for set 2:
10 out of 15 people chose cup 3 over cup 4. Both cups were the same coffee but roasted differently.
In the next group:
Again, 10 out of 15 people chose one particular cup (cup 7) but it was the same coffee in each cup; roasted differently. While 15 people isn’t a statistically significant sample, but it is consistent with what we’ve observed elsewhere, and those 15 people are very accomplished coffee professionals and are representative of people making coffee buying decisions
In the second half of the cupping we asked cuppers to score the coffees. It was again a blind cupping, and they did not know that there were only 3 coffees used across all 7 sets.
For example, we had two roasts of a beautiful Parainema from Santa Barbara in Honduras.
Average score:
Cup 1 : 84.7
Cup 2: 83.7
1 point difference is large, but not huge.
Averages can hide the true story – each individual cupper had an average difference between cups of 1.5 points – a few people has zero difference, and some were 3 points different..
Here we see more differentiation which is significant to buyers.
This is the same coffee roasted with two good profiles. The fact that some people might give a coffee 3 points less based on a roast is significant.
Looking at the next set, it’s even more interesting. We roasted a Guji from Ethiopia 3 different ways, and used 3 different styles of roast. All profiles were from very accomplished coffee professionals from different countries.
Average Scores:
Cup 3: 85.5
Cup 4: 82.9
Cup 5: 85.3
This is of course interesting, but the scores are again magnified when we look at the difference in each cuppers scores:
What we observed was that although the averages were not massively different between Cup One an Cup Three, there was a large difference in score for each cupper.
This is by no means definitive or statistically significant, but it does align with what we’ve observed and recognised from our personal experiences: a certain roast will let a coffee shine through better than others.
There are people that buy coffee for a living and cup thousands of coffees that have become very good at knowing how a coffee will taste, almost regardless of the roast.
However, that skill is thousands of hours of investment and for most of us, not a point where we would get to soon. A good roast will let people find the best coffee, and more importantly, reward good coffees.
This is the first part of a small series we’re writing up about our Nordic Roaster Forum workshops and our small investigation into sample roast profiles and how they affect scores and purchasing decisions.
