It's that time again, one of the best events every year, the Memorial is kicking off the disc golf season. Annually I have to make my point about the inflated ratings for the Open and the deflated ratings for the Female and Senior divisions due to the way the course plays.

A 59 by a woman is so much different than a 59 by an Open player. Even though the hole distances are the same the Women and Senior, and even the GM's have to go the long way around the water on some holes and the Open players generally go over the water and are able to develop better scoring chances.

If you just calculate the ratings per each division independently instead of the whole tournament as a group, the ratings will be much more accurate. Sure it's great for the Open players, but the Women are getting hosed for their excellent play....HUMOR ME CHUCK!!! :)

I was waiting on this post all day!

We already did that back in 2002 and the Am pool generated the same ratings as the Pros at Fountain and at Vista even though the average rating of the pools was at least 75 points apart. Here was another test on the 2008 data that I actually wrote up for Kevin's benefit:

Courses Providing �Unfair� Challenges?
Before showing the correlation results for the Pro Worlds, Kevin McCoy brought up an issue I�d like to address. His concern was whether courses like Fountain and Vista with several water carries might naturally depress the scores and potential ratings of lower level players who might have to play around the water on those holes. We had tested this idea about 5 years ago during the Memorial and found the lower rated players had no evident problem shooting their ratings.

The following shows how well each Pro division played in comparison to their ratings on the two courses. MPS and FPM are not shown since they played different tees.
..................MPO..MPM..MPG..FPO
Vista (2Rd avg)..+6.2 -0.8 -7.4 -9.5
Fountain (1Rd)..+6.3 -5.5 -20.2 +5.3

There�s no question that the Open players fared alright on these layouts in comparison. However, consider that 10 points is slightly more than one throw on these courses. It appears the GMs in particular gave up a few shots more at the Fountain than any other group. These results are in line with what we�ve found before in terms of differences between a group of Open players who have the same average ratings as a group of Masters and older players plus top women. On courses with terrain balanced between open and wooded holes, both groups shoot the same total scores.

However, if you look at scores on specific holes, the Open players do better on the open holes and the women and older players do better on the wooded holes by a few tenths of a shot. When you have open courses with a lot of power shots it shouldn�t be surprising that the Open players fare a little better relative to their ratings compared with the women and older players. What should be surprising is that the data shows it really isn�t as much difference as you might expect considering the number of carries on both of those courses.

Here's the preliminary scoop from yesterday at the Fountain. The numbers shown below are the average increase or decrease in the average round ratings that whole division earned unofficially.
MPO: -0.9
FPO: +5.1
MPM: +4.3
MPG: -4.2
MPS: -6.7
Looks like the Women and Masters did just fine dealing with those water carries.

Chuck,

Have you seen a pic of our new Chuck? He's Chaw-lie, Boston style to me.

Hook me up with the numbers? Just humor me and give me two spreadsheets of ratings. 1 being total field which we all can see on the schedule page and then 1 with ratings within the division....the girls are getting hosed. And 5 or 6 of the top rated rounds happen at this event every year....I was a Euclidean Geometry flunk out in college but Comeon.
The correlation of high ratings for men and low ratings for women has to do with the courses playing 1400 feet longer for the girls.

Chuck I have a bigger theory on the discrepancies the Rating System and I would like to discuss ways to come up with different variables that make the system more accurate.

The numbers above are the ratings differences at Fountain in round 1. The Women and Masters got better ratings by those amounts on average compared with Open, GM and Sr GM.

The average rating of pro women who played all three rounds is 895.4. They received average ratings of 898.3, 900.1 & 893.9. They are ahead on ratings so far for three rounds. Compare this with Open with an average of 982.5. They have earned average ratings of 980.9, 976.7 & 989.4. They are slightly down on earned ratings as a group after three rounds.

Kev, your premise just isn't borne out with the actual data, at least comparing FPO with MPO. MPM and MPG are also up slightly after three rounds. Only the four Sr GMs are down after three rounds, having big trouble at Fountain today, primarily one player. But as a small group, they are going to have bigger swings than any larger group from round to round. If anything, we should feel sorry for those Open players who feel like they need to take the big risks around water and get penalties as a result that hurt their ratings.

Duh.... of course it averages out with all the water OB, the high scores are high and the low scores are lower due to stroke and distance.

What do you think the numbers should be? The ratings system for 10 years now consistently produces round ratings (within normal statistical variance) that average the ratings of the propagators and that's what we see here. Your implication that certain player groups are advantaged or disadvantaged in some way on some courses with special conditions doesn't hold water (if you will).

Sorry for the slow response, kids seem to take more of my time than anticipated :D

I'm going to give you some numbers since you are a mathematician and you base everything on the numbers and numbers don't lie.

Fisrt is a list of 5 NT's from last year, I left out The Oklahoma event due to severe wind and the Hambrick due to rain. It shows the top 5 players ratings and what their avg. rating to finish 1st through 5th. Other than the Majestic 13pt difference the average is 3 pts over existing rating.


Now 5 years of the Memorial *(2007 men and women 2nd rd rated separately) If you throw out 2007 which ratings were done separately between divisions, the average MPO margin of the top 5 is 18.9 points higher than their rating. The numbers prove that the distance shots carrying water is increasing the ratings numbers in the MPO

I am proposing that the rating be determined by each division within themselves when there is a fair number of participants like Majors and NT's

The first fallacy in the numbers is only looking at the top 5 players. That would only be valid if the exact same player pool played all of the events which isn't the case. Ratings are based on the whole field of propagators so that would be a better comparison. But the biggest factor in why the numbers vary has to do with the type of course(s) and their SSA. The Majestic and the Masters are on courses with 27 holes although the equivalent 18-hole SSA at the Majestic is the lowest among the events listed besides the Memorial. Thus, the wider ratings gap you saw at the top. The lower the SSA, the wider the dispersion of ratings from top to bottom by the end of the event among a propagator field that's the same size and average rating. In general, a course that's more open but has tough hazards (Winthrop, Vista, Fountain) will produce a wider spread of ratings than a course with the same SSA that's more wooded.

Chuck you do a great service for disc golf.

I felt my ratings were a bit low for playing in the open
division. I finished way at the bottom so I am not a great player.
Tons of OB, couple of windy days, long waits between holes.
It was a battle.

When I play locally at pitch and putt courses I think round
ratings are generous but not these courses.

A 41 is ridiculous at Fountain. I think the rating for
that round should be 1100. It is nearly flawless.

I do see the logic in separating ratings based on divisions
in major or NT events. Although, the smaller divisions would
not have the diversity that the larger divisions have.

A 41 is ridiculous at Fountain. I think the rating for
that round should be 1100. It is nearly flawless.
It will likely end up close to the 1109 unofficial rating.

I will start with saying Chuck, Roger and crew have devised a fantastic membership benefit with the ratings service. It has been tweaked here and there over the years. Most players I meet all have some weird stories of various anomalies. It seems the biggest factor is that the TD report form is too simple and not enough weight is calculated into weather or tougher courses. My opinion and we all have them.

What we need is just like the technical standards of the disc and the target, a comprehensive review of how the ratings are devised and how the process can be improved and more accurate. Each year we are weighting more and more on these ratings with everything from sponsor bonuses, player of year and now a PDGA Major.

We will need some super math and stats heads on the review team. Every time I talk with Chuck about ratings he makes my head spin. I think that is his strategy;) He has devised the calculations better than an over the counter hedge fund and we know how those have ended up, he,he.

--Why only a few years ago 1020 was elite world class, now 1030 even 1040?

--Why is it on TD reports, when you enter Nikko and Climo into tournaments they didn't play and give them actual shot tournament rounds, everyone's ratings go up?

--Why is the CSUMB course literally a ratings cash cow meanwhile Mike Olse blisters East Metro Park in Texas and seems shorted?

CSUMB MPO winning AVG (1071 in 2010) (1054 in 2009) (1038 in 2008)

M Olse kills a SSA 65 course with an amazing 55 (1062):confused:

We all could give examples, any math heads want to review?

Biscoe? Devine?

We shot very well at Metro this weekend in "best score" and shot a 53 with the next best being a 57! Basically Olse played better weeks before by himself than 95% of the doubles teams. We went into "best shot" hoping to shoot in the 40's but only shot a 52 which was still a hot score.

Basically Olse shot as good as one person can and in poor conditions at that! Beat a 999 round by 9 and get a 1060!?

alright CHUCK.... 1st and 3rd rounds at memorial at fountain hills are the same?!?! BOOOOO... the wind conditions were dramatically different for these rounds affecting scores by at least 4-6 strokes on average if not more... 30+ mph winds when you are on holes 5-10 is killa... I went +10 on the first round from holes 1-9, and +2 during 3rd round on front 9; 8 strokes difference and wind played a serious factor, and same ratings for both rounds, no way jose....

Sorry but no difference. We broke out the SSAs for every day and even separated Advanced from all other divisions in the Ams and Open from all other pro divisions at both the GCC and Memorial. Variances were all within expected normal variations (<2) so the rounds were combined on the same layout for SSAs.

so morning and afternoon rounds are not calculated differently based on weather conditions even though I see it says that weather is a factor in determining round ratings? The ssa for those cards who played in less than 15 mph winds for the majority of their round versus the ssa for those cards who played in constant 20-35 mph winds is what? *speaking of first round AMS on Thursday.
I think the morning rounds were all calm versus those who started after Noon? The best score on my card was a +8 and from my own experience, this looked like a 950-960 rated round considering the wind we had, and it's only a 912?
I am only seeking more information how this rating system works and why it worked out the way it did. I was under the impression that weather during a card's play would impact ratings, not comparing apples and oranges within same division; low winds vs high winds in same division.....
my perspective, looking at players who played in the afternoon 1st round vs their scores in 3rd round.
Player 1 - improved 9 strokes
player 2 - improved 11 strokes
player 3 - improved 9 strokes
player 4 - improved 6 strokes
player 5 - improved 14 strokes

Just looking for more information; not complaining at all... ;)
Did any of the top 10 players who all had under 60 their first round, play after 1230PM during first round? Curious... :cool::p

at the AMS St. Patrick's classic, I played the first round (27 holes) in 5.5hours in the rain. OB water lines tripled and disc slippage was at a near 100%. If only I was able to play in the sunshine during the morning. Oh well, it's the breaks we get playing at different times. Most people suggest mornings are better to play, as the winds pick up in the afternoons. In this case, the rain came out full power in the afternoon. Just happens that way.

2009 Memorial last year was the same way as the 2010, morning players got the break, afternooners played in heavy winds. Now if only they could turn off that darn fountain during play. My car got punished, and so did certain players on the East side of the course when the fountain poured.

We calculated SSAs separately for the morning and afternoon every day on each course for the GCC and the Memorial. We would have kept them separate for the ratings calculations IF the difference in SSA was greater than 2.0 either in morning versus afternoon OR one day versus another day on the same layout. All differences were less than 2.0 SSA (~15 to 18 rating points depending on course).

The normal variance between SSAs from morning to afternoon on a day where the wind (or lack of) is identical, both rounds can be up to 2.0 with the same propagators. So unless, the SSA values differ by more than 2.0, we don't know whether the variance is caused by wind or just normal statistical variation. If it's less than 2.0, we then combine all scores and produce ratings such that everyone who played that layout gets the same rating for the same score.

The problem with tee time rounds is not in the ratings but in the fairness of the format itself. No one gets paid at an event based on their ratings but based on actual scores. Is it fair that some in a division get low wind and some high wind in the same division? In ball golf, they sort of deal with that by keeping the same groups together for the first two rounds and start them early one day and later the next day so in theory they get roughly equal wind conditions. We haven't done that in disc golf in our tee time events with large divisions. So if there's a complaint, it should be about the competition format fairness, not the ratings. If the competition format were more fair, then the ratings would also average out more fairly for the event, too.

great answer... am a bit disappointed that the ssa was less than 2.0 when comparing morning players and afternoon players, but if it was calculated that way then I can't complain. Keep it up.

I'll point out, I've never been a fan of tee time rounds, either to play them or to rate them. But enough people like them so they're here to stay. I've already mentioned the fairness issue when the weather varies significantly over the time period the same large division is on the course either Advanced or Open. Let's say the wind all of a sudden jumps from nothing in the morning to a value after lunch that would produce a 3.6 difference in SSA if there were two shotgun rounds played on the same layout.

Now, if tee time rounds are played on that same day instead, you can have some players who get none of the high wind, some that get less than half the holes with high wind, some with more than half high wind and some with all high wind. If that happens in the same division, we can't really calculate the ratings fairly unless we break it down into maybe 20 player groups at a time. TDs just don't collect that grouping info for us to do it. If that tee time round was two divisions and we did SSAs separately, we still might not even see a difference in SSAs more than 2.0 because the one group had at least some players getting high winds for part of their rounds and the other group would have at least some players with lower winds for part of their rounds.

So instead of say SSAs of 50.0 and 53.6 if the rounds were done as shotgun starts, we get morning and afternoon rolling start SSAs of maybe 51.0 and 52.6 that are only 1.6 apart. Even if the wind was wicked for the last groups of the day, there wasn't enough difference in SSAs to break them out separately. So everyone gets and SSA of 51.8 even thoug the early morning players really played in 50.0 SSA conditions and the late players mostly played in 53.6 SSA conditions. The good news is that at least using the average SSA value, the worst error for some players is only 1.8 SSA or about 16-18 rating points.

great answer... am a bit disappointed that the ssa was less than 2.0 when comparing morning players and afternoon players, but if it was calculated that way then I can't complain. Keep it up.

Just remember "There are three kinds of lies: lies, damned lies, and statistics." :)