Geek Critique Score Analysis - Even Geekier

Started by Kid Montana, 09/24/2018 12:32 PM

Previous topic - Next topic

Kid Montana

Back in February of 2017 I posted an analysis of all the geek critique scores and provided some commentary on what I'd observed.  I have revisited that post with another year and a half's of geek critiques and some new charts and analysis for your enrichment.   I excluded some cigars with only one or two reviews from the individual metrics, but those cigars are included in the aggregated data/analysis.

Some facts and figures:
* The total number of reviews grown from 932 to 1031 total reviews across 221 different cigars.  
* The lowest average score with more than 3 reviews goes to the Montecristo Pilotico Pepe Mendez Toro, reviewed in March, 2018, with only 68.75 points average.  This low average was due entirely to a terrible 13 point score earned due to a major defect the reviewer experienced.  

The worst of the worst, the bottom 10 (includes cigars with 1 or 2 reviews).
1. Cooperstown Stinger   60.00
2. Cooperstown Grand Slam   67.00
3. Montecristo Pilotico Pepe Mendez Toro   68.75
4. Cuba Aliados (Core Line) Toro Extra   69.80
5. Di Fazio Picoso Doble Robusto   70.20
6. Dunhill Aged (formerly Dunhill Dominican) Condados   72.25
7. JFC Tobacco La Flor De Cuba   73.80
8. Evil Genius Cigars White Chapel Belicoso   74.60
9. Alec Bradley Black Market Filthy Hooligan (2013)   75.80
10. Leccia Tobacco Luchador El Gringo Frog Splash   75.80

* The highest average score has not changed and goes to the Sindicato Cigars Sindicato Toro Maduro reviewed way back in Feb, 2015, with an average of 94.2.
The Best of the Geek Critiques (includes cigars with only 1 or 2 reviews)
1. Sindicato Cigars Sindicato Toro Maduro   94.20
2. Paul Stulac White Blinding Light Torpedo   93.20
3. Partagas (Cuban) Serie D No. 4   93.00
4. Don Pepin Garcia (My Father Cigars) My Father (Rosado) No. 4   92.60
5. Drew Estate Herrera Esteli Corona   92.40
6. Vegas Robaina Famosos   92.20
7. Arturo Fuente Anejo Reserva No. 50   92.00
8. AKA (American Kick Ass) Nth Degree Apex   91.80
9. Valeroso Primavera Regalo   91.60
10. Crowned Heads La Imperiosa Double Robusto   91.40

* The cigar with the most consistent scores was the Illusione MJ12 (The Majestic 12) Maduro with an average of 88.66 (88, 89, 89).
The top 10 most consistently scored cigars (lowest standard deviation of scores) with corresponding score average:
1. Illusione MJ12 (The Majestic 12) Maduro   88.66666667
2. Drew Estate Undercrown Shade Corona   88.8
3. Leaf & Bean Leaf and Bean by Noel   90.4
4. Camacho Liberty 2015   90.2
5. Crowned Heads La Imperiosa Double Robusto   91.4
6. Sindicato Cigars Sindicato Toro   89.6
7. 1502 Black Gold Toro (Box Pressed)   87.5
8. Valeroso Primavera El Estado   89.6
9. Ave Maria (Core line) Crusader   91
10.  La Flor Dominicana N.A.S.   86

* The cigar with the least consistent scores was also the Montecristo Pilotico Pepe Mendez Toro, likely due to the 13 point scoring throwing off the standard deviation.  
The least consistently scored cigars:
1. Montecristo Pilotico Pepe Mendez Toro   68.75
2. JFC Tobacco La Flor De Cuba   73.80
3. Olman Premium Cigars Olman Toro   78.50
4. Di Fazio Maduro Doble Robusto   78.40
5. CyB (formerly Cuenca y Blanco) Lonsdale Club   77.80
6. Dunhill Aged (formerly Dunhill Dominican) Condados   72.25
7. Cuba Aliados (Core Line) Toro Extra   69.80
8. Pinar del Rio Liga Especial Reserva Superior Salomon   81.00
9. Alec Bradley Black Market Filthy Hooligan (2013)   75.80
10. CAO Flathead V660 Carb   83.80

What is interesting is the most consistently rated cigars averaged 89 points, while the LEAST consistent only averaged 76 points.   If you see a lot of variability in a cigar's Geek Critique results, odds are its not very good or not going to be popular with most people.  

The overall distribution of cigar ratings is a fairly classic Poisson distribution, and is not normal.  
* The overall average is 86.11.
* The most frequently given score is 90 with 101 reviewers assigning this score.
* The standard deviation for any one review is 7.414 points.  So the 1sigma score range is 78.7 - 93.5 points.  By implication, a cigar with a single review within this range can be expected to be "about as good as any other cigar in this review data set."  Cigars reviewed above 93.5 points could be considered "significantly" better than average.  


Thanks for reading, if you'd like to peek at the raw data, check out my badly organized and formatted spreadsheet with all 1031 entries!
https://docs.google.com/spreadsheets/d/11nVO64DxHQ6VMQGJQp9f9EN10MVl_JjOLUauwni5xHk/edit?usp=sharing


I'd rather have a cigar...

05Venturer

WOWZERS Diddy, you bored today?
Good info tho
Kent
 Guru of Cynicism & Cigars

"Pump the brakes; you take your shirt off but leave your sunglasses on?" "What sort of backwards !@#$ing pageantry is that?" "You going to fight with those shades or play pokerstars.com?"

benchjockey

You learn more by listening then you do from talking.

Brlesq

Interesting data.  Thanks for putting the time in.

I was surprised to see the Evil Genius Cigars White Chapel Belicoso on the "worst of the worst" list.  RDF and I recently discovered these this past Spring while visiting Famous Cigar Shop, and they were on the front counter being touted as a "new cigar made by a local guy".  They were so good that we split a bundle!  Maybe the recipe from 2014 was changed, or maybe these are the original cigars with 5 years of age on them and they have dramatically improved.  Either way, we both loved them! :confused:
Bruce
Chief Enabler 
Guru of Decorum & Sarcasm


Hey! How come Habana is written on here with a Sharpie ?!?

A day without whiskey is like . . . just kidding, I have no idea!


appollo


nirab

Dan the Man with a Plan for us geeks to Understand!! Well done my brother! :biggrin:  :bigthumbs:
The Punk Rock Guru of Meditation and Lending a Hand

DON'T believe EVERYTING you THINK...
it's COOL to Be a GEEK!

''Any time you have an opportunity to make a difference in this world and you don't, then you are wasting your time on Earth.''-Roberto Clemente

"When I do good, I feel good. When I do bad, I feel bad. That is my religion." - Abraham Lincoln

"It is possible to live happily in the here and now. So many conditions of happiness are available - more than enough for you to be happy right now." -

Thich Nhat Hanh

"There is a principle which is a bar against all information, which is proof against all arguments, and which cannot fail to keep a man in everlasting ignorance - that principle is contempt prior to investigation."

The Cigar Vault
512-361-3289
in Beeuudaful Buda, Texas!

 

                                                               



Chefjohn

That is an awesome analysis Dan. Thanks for taking the time to put that together.
Artificial intelligence is no match for natural stupidity.

Kid Montana

#7
One of the things I like the most was when cigars got really consistent ratings, and what that rating is.  Take for example the LFD NAS, it was consistently disliked compared to some others.   :lmao:


I'd rather have a cigar...

klamm143

Great info - thanks for doing the work.
Kevin R. Lamm

It is what it is - and these things too shall pass. It may pass like a kidney stone - but IT WILL PASS.

Mautrak

That is very interesting.  Thank you.

I know when I do a geek critique that I individually score each area and continue to make notes and adjust the number as I enjoy the cigar.  So maybe about an inch in I start to assign numbers and then tweak those as I go.  But when I get to the very end I add up the numbers to see what the final score is.  I've done that on cigars in the past and thought, "82? I enjoyed it more than that."  And then I go back and see if I can adjust numbers upward.  And the reverse is true too.  I've had some high final numbers and thought that the cigar just wasn't that spectacular and then I go back to see if I should lower some scores.

Deener27

Part of issue with the weighted Geek Critique scoring is that it is weighted heavily toward construction, burn etc.  Something that has been debated for a while now.  Not that there is anything wrong with the Geek Critique model but it seems easy to skew the numbers.

I would love to work through some numbers pulling some of the either scores out and see what it looks like regarding flavor only.


"Sometimes you eat the bear, and sometimes the bear, well, he eats you." The Stranger

"Judge Smails: You know, you should play with Dr. Beeper and myself. I mean, he's been club champion for three years running and I'm no slouch myself.

"You may run like Mays, but you hit like !@#$." ~Lou Brown in Major League

"Looks like you have El Hosel, the laterals—you know, the S-word" ~ Romeo in Tin Cup

Beegerply

Well done Dan, thanks for taking the time to do this.  :biggrin:
Byron

Guru of small balls and big sticks
--------------------------------------------------

"If there are no cigars in heaven I shall not go" Mark Twain

Brlesq

QuoteMautrak - 9/24/2018  10:01 PM

That is very interesting.  Thank you.

I know when I do a geek critique that I individually score each area and continue to make notes and adjust the number as I enjoy the cigar.  So maybe about an inch in I start to assign numbers and then tweak those as I go.  But when I get to the very end I add up the numbers to see what the final score is.  I've done that on cigars in the past and thought, "82? I enjoyed it more than that."  And then I go back and see if I can adjust numbers upward.  And the reverse is true too.  I've had some high final numbers and thought that the cigar just wasn't that spectacular and then I go back to see if I should lower some scores.

Yeah, I do this too.  I'm not a fan of the way we weight numbers on certain categories.  For example, why is "pre-light" 1-20 points?  It's just a gauge of your first impression that lasts only a few seconds, and many of the attributes you are scoring in that category are also the same attributes under "construction".  I'd be a fan of capping 10 total points for "pre-light" and give the other 10 points to "flavor and aroma".  Just my 2 cents
Bruce
Chief Enabler 
Guru of Decorum & Sarcasm


Hey! How come Habana is written on here with a Sharpie ?!?

A day without whiskey is like . . . just kidding, I have no idea!


Kid Montana

QuoteBrlesq - 9/25/2018  6:03 AM

QuoteMautrak - 9/24/2018  10:01 PM

That is very interesting.  Thank you.

I know when I do a geek critique that I individually score each area and continue to make notes and adjust the number as I enjoy the cigar.  So maybe about an inch in I start to assign numbers and then tweak those as I go.  But when I get to the very end I add up the numbers to see what the final score is.  I've done that on cigars in the past and thought, "82? I enjoyed it more than that."  And then I go back and see if I can adjust numbers upward.  And the reverse is true too.  I've had some high final numbers and thought that the cigar just wasn't that spectacular and then I go back to see if I should lower some scores.

Yeah, I do this too.  I'm not a fan of the way we weight numbers on certain categories.  For example, why is "pre-light" 1-20 points?  It's just a gauge of your first impression that lasts only a few seconds, and many of the attributes you are scoring in that category are also the same attributes under "construction".  I'd be a fan of capping 10 total points for "pre-light" and give the other 10 points to "flavor and aroma".  Just my 2 cents

Agreed.  Flavor 50 points,  construction, burn, appearance 50 points.


I'd rather have a cigar...

Mautrak

Sorry to send your thread off on a tangent, Daniel.  But I tend to agree with you and Bruce that maybe some changes are in order, and that seems to be supported by your analysis / number crunching.  The tough thing then, and it should probably start in a new thread so this doesn't get totally jacked, would be comparing old to new.  There would end up being a pre-scoring system change group of reviews, and a post-scoring system change.  

Food for thought.  And you started it Daniel!

Kid Montana

#15
QuoteMautrak - 9/25/2018  3:01 PM

Sorry to send your thread off on a tangent, Daniel.  But I tend to agree with you and Bruce that maybe some changes are in order, and that seems to be supported by your analysis / number crunching.  The tough thing then, and it should probably start in a new thread so this doesn't get totally jacked, would be comparing old to new.  There would end up being a pre-scoring system change group of reviews, and a post-scoring system change.  

Food for thought.  And you started it Daniel!

Well yes and no.  I think the scoring needs to be revised, but I don't think it will make a difference.  People tend to think, 80/100 is only "average" and not necessarily great.  90/100 is the bottom of the really good.  You'll see similar distributions in academic scoring.   I'm willing to bet you'd see a similar distribution of scores for cigars reviewed by Cigar Aficionado.   I'll try to see if I can't scrape some data and do a comparison.


I'd rather have a cigar...

junglepete

QuoteBrlesq - 9/25/2018  8:03 AM


Yeah, I do this too.  I'm not a fan of the way we weight numbers on certain categories.  For example, why is "pre-light" 1-20 points?  It's just a gauge of your first impression that lasts only a few seconds, and many of the attributes you are scoring in that category are also the same attributes under "construction".  I'd be a fan of capping 10 total points for "pre-light" and give the other 10 points to "flavor and aroma".  Just my 2 cents

Could not agree more :bigthumbs:
Guru of Frugality

"It is your decisions, not your conditions that shape your life."  ~Tony Robbins~

CoreyMacLeod

No Signature Necessary.

Cfickter

QuoteKid Montana - 9/25/2018  7:04 PM  
QuoteMautrak - 9/25/2018  3:01 PM  Sorry to send your thread off on a tangent, Daniel.  But I tend to agree with you and Bruce that maybe some changes are in order, and that seems to be supported by your analysis / number crunching.  The tough thing then, and it should probably start in a new thread so this doesn't get totally jacked, would be comparing old to new.  There would end up being a pre-scoring system change group of reviews, and a post-scoring system change.    Food for thought.  And you started it Daniel!
Well yes and no.  I think the scoring needs to be revised, but I don't think it will make a difference.  People tend to think, 80/100 is only "average" and not necessarily great.  90/100 is the bottom of the really good.  You'll see similar distributions in academic scoring.   I'm willing to bet you'd see a similar distribution of scores for cigars reviewed by Cigar Aficionado.   I'll try to see if I can't scrape some data and do a comparison.

Don't disagree Dan, while the scores may stay in balance with historical trends, the score would better reflect the reason we all smoke cigars and that is flavor/taste.

I am one who wouldn't mind seeing the ranking weights change.   My suggestions would be
Pre-light 10 - I don't that many of us would offer a cigar to critique knowing it was going to look bad, or even send a bad one out
Lighting and Burn 20 - burn I find more frustrating than construction, so I added 5 pts here
Construction 20 - I took 10 away from here, mostly because of the same point I made regarding pre-light, but also I tend to have much fewer construction issues vs burn
Flavor and Aroma 50 - Need to be the heavyweight of the rankings

 

Guru Master of the Minions

Alcohol, Tobacco, & Firearms should be a convenience store, not a department of our government!

Gunga galunga ... gunga, gunga-lagunga." - Carl Spackler

Education is important, cigars are importanter!

I like long walks, especially when they're taken by people who annoy me






   
Privacy Policy     Terms of Service
Copyright © 2007-2024 Cigar Geeks, Inc. All rights reserved.