Course Design Validation

One of the first steps in disc golf course design process is to establish the skill levels a design will be developed for. One of the last steps is to validate whether that design turned out to be appropriate for those skill levels. Designers have created many courses over the years where the intended skill levels were at least loosely defined before starting – broad terms like pro versus am were considered. It’s gotten even better since ratings came on the scene ten years ago. This made defining skill levels a little more precise using ratings ranges like Gold, Blue, White, Red and Green for different sets of tees and pins.

However, only a small percentage of course designers over the past 30 years have made the effort to validate whether their designs actually provide appropriate challenges for players in specific skill range(s). In all fairness, design validation is a pretty new concept developed in the past 8 years. It also involves doing a bit of math. Members of the Disc Golf Course Designers (DGCD) group have access to a utility spreadsheet called the Hole Forecaster which will help do the math part of the design validation process. For everyone else, this article will show how to do it at a basic level.

Design Validation Math

The math part of design validation analyzes individual hole scores thrown during tournament or league play (where hopefully PDGA rules are followed) by players with ratings in the intended skill range for the design. The primary purpose is to review whether acceptable scoring spreads have been achieved or to discover unusual scoring distributions. In general, the wider the scoring spread on a hole – a good mix of different scores – the better the hole does to allow those playing better to excel. The worst case is a hole where virtually everyone in a skill range shoots the same score. This hole becomes meaningless for competition when everyone in the division tends to shoot the same score. It makes no difference whether that hole is played or not to determine winners in that division.

Calculating Scoring Distributions

The process to calculate scoring distributions is straightforward. Gather scorecards from players in the ratings/skill range to be evaluated. Typically these are players with ratings within +/- 35 ratings points of the mid-point of the skill range being evaluated. The PDGA defines four skill levels of Gold (1000), Blue (950), White (900) and Red (850>) with the mid-point rating shown in parentheses. Enter the players’ names, ratings and hole scores into a spreadsheet as shown.

Create rows that simply total the number of 2s, 3s, 4s, etc. scored on each hole. Then, using those values, create rows that calculate the percentages of each score on each hole. In addition, include a row that calculates the scoring average and average rating of the players. Make sure the average rating of all the players in the evaluation falls within about 10 points of the target mid-point for the ratings range being evaluated. If necessary, delete some of the players to bring the overall ratings average in range.

WR Jackson Course Example

In this example from the 2007 Hall of Fame Classic, John Houck was trying to determine how well the holes played for Gold level players (which is who the course was designed for). As a side note, since these were gold level players with an average rating of 997 close to 1000, their overall scoring average of 68.1 is essentially the Scratch Scoring Average (SSA) for the course.

The first red flag to look for in a scoring spread evaluation is a hole where one score occurs more than 70% of the time. Note the red numbers for holes 3, 7 and 9. A distribution with a score of 3 occurring more than 70% of the time is unfortunately fairly common on open holes when the designer hasn’t done this validation process to see and correct the problem. When a value of 70% occurs on wooded holes like the Jackson, it’s more surprising since wooded holes tend to spread scores no matter what length they are.

In the case of holes 7 and 9, you might expect that top pros would eat up holes in the 375' range, as John did. We all know that most can throw that far. But do they really rack up the birdies at that distance? As you can see, the statistics show that it's just not happening on these holes. Interestingly, this was predicted by the Hole Forecaster before this event (which Houck had a hand in developing) but Houck was curious whether the new, longer discs had changed our original forecasting factors. The low percentage of bogeys indicates that the fairways are wide enough and fair enough. It's just that 375' is still a tough distance to consistently get up and down in 2, even for the world's best players using newer plastic.

Since this event, Houck has been looking at tweaking the design on these holes and a few others so look for changes when the staff at the IDGC has time to address them. Note that the scoring spreads for 14 of the other 15 holes were appropriate for gold level players as Houck had planned in his designs with averages close to the intended par values for those holes. Both holes 2 & 3 were planned par 4s but the pins were installed much shorter than where Houck had planned. The result was tweener holes (averages around 3.5) which will get improved.

Other Course Examples

The following table provides more information on Jackson hole 9 along with some other holes where the distributions might provide some insight. Blue level players also played the Jackson at that HOF Classic. As expected, they shot even more 3s than the gold level players on hole 9 since that 375’ length is beyond their max design range, plus that hole is a slow turnover to the right which adds a little more difficulty for most righties.

The skill level a particular hole level was designed for is shown in dark bold type like Gold. For example, hole 1 at Vista del Camino which many have played at the Memorial shows one of the best distributions for a gold level par 3. A scoring average near 2.7 is ideal because it means about 1/3 of the players are shooting better than the others. Holes where roughly 1/3 score better set the table for the better players to excel overall. The added bonus on this hole is the scary water hazard players have to tee over. This hazard helps produce the small percentages of 4s & 5s. Hazards can be a bonus for the distributions on par 3 holes as long as they are not too punitive. This can be seen with higher bogey and double bogey percentages.

Steady Ed hole 7 designed for Blue level shows one of the better distributions for a tough par 4 hole. The first segment of this dogleg is slightly uphill over a small ridge. The second section dogs to the right but bends around to the left along a moderately wide corridor. There’s no OB but players get in trouble misjudging angles and how far to throw their tee shot. Here’s an example where a tough par 4 hole for blue level also works as an “easy” par 4 hole for gold level.

Typically, the scoring average for gold level will be 0.2 to 0.4 less than blue level on a hole. Steady Ed hole 11 shows how this can be a problem for certain par 4 holes when both blue and gold level divisions are playing the same layout. This pin placement has an okay distribution as an easy par 4 for blue level but is not even a par 4 for gold level. This can be frustrating for gold level players playing well because it’s really hard to birdie and most just play safe for their 3 and move on.

A new longer pin placement in a different direction will hopefully work better for tournament play where the scoring for blue might average 3.8 and gold around 3.5. That’s not necessarily ideal for gold but at least it can provide some separation between the top and bottom half of the field. In addition, the current placement will be lengthened maybe 45 feet.

Adjusting length is the primary tool a designer has to correct scoring distribution problems. If a tee is not yet permanent, it might be the easiest to change. If the tees are already permanent, then changing pin placements or adding alternates might be the better option if possible. The rule of thumb for design is every 25 feet added in the woods or 35 feet added in the open adds 0.1 to the scoring average. Shortening by those lengths reduces the scoring average by 0.1.

Avoiding “Par 2”

Hole 2 at Kaposia is an example where the scoring average for gold level has been consistently around 2.4. The course was really designed for blue level with an average around 2.7 which is just fine. However, Kaposia gets used for tournaments like Worlds and NTs. The hole is pretty much landlocked so moving the cement tee would be difficult and there’s not really room for an alternate pin for big events. Although it hasn’t been tried, this hole might be improved for tournament play with some creative OB lines.

Dealing with creative OB lines is the essence of the USDGC, home of the last hole in the table - #17. I went thru the evolution of this hole in issue 5 of Flying Disc Magazine in 2008 so I won’t address that here. However, it shows the bizarre distribution where there are many more 2s and 4s on a par 3 hole than 3s. This scoring distribution more than anything else has lead to its redesign and incorporating new concepts like buncrs. So this hole is still scary but the score of 3 has now become the most common score in the distribution even though 2s and 4 or more are still significantly in the mix.

Super Gold Level Distributions

Even though the best players in the world enter the USDGC, design guidelines for their “super gold” skill level (those with ratings over 1025) are only useful for a limited number of top level events with final 9s where only a handful of super gold players are throwing. Most of the time, designers don’t have a way to test that temporary (usually) design for scoring spread before showtime. However, scoring analysis can still help if the event will be there again the following year.

John Houck and I are always looking to design super gold holes likely to have at least one or two players out of the group shoot a different score than the others so there are still chances for scoring movement in the Finals. For example, at least one Open player shot a different score on each hole of the Final 9 at Pro Worlds 2007 which came down to the last hole. In comparison, the top Open Women, who are solid blue level players, shot the same score on five of the nine holes because several of the holes were not in their appropriate range to potentially provide scoring spread. There’s always this trade-off between using the same course for the top women and all other pro and am divisions for the Finals at Worlds for simplicity versus using different tees so their layout provides the same opportunities for scoring swing as Open.

Balance the Averages

Once you’ve determined the scoring spreads for your holes, even if most are OK, there’s still an overall balance issue to consider.

Having two to three times as many holes with scoring averages in the 2.5-2.8, 3.5-3.8 and 4.5-4.8 range as those with averages in 3.1-3.4 and 4.1-4.4 range provides a better balance for competition. Holes in the 3.1-3.4 and 4.1-4.4 range primarily help separate the bottom third of the players from the top two thirds. However, for league and tournament play we want more holes to help separate the top third from the bottom two thirds. Holes with scoring averages in the 2.5-2.8, 3.5-3.8 and 4.5-4.8 range are much better at doing that.

Summary

If the course design validation step has not been completed, is the course design really done? If the designer didn’t establish design goals with the course owner before starting, what benchmarks can the designer use to guide their design process or the course owner use to determine whether the job was done well? There are hundreds of popular courses that could be just as popular if they were also designed more consistently for specific skill levels. Popularity and designing appropriately for skill levels are not in opposition as design goals. But even a good scoring spread on a hole doesn't guarantee it's produced by skill versus flukey factors.

As you can see, doing scoring spread analysis to help validate your courses is a little extra work. But it’s a necessary final step to say a course design has been completed in a professional manner. The unfortunate thing is many players may never notice the subtleties behind any changes you make to improve the course based on this information. Players react positively more to the course aesthetics than anything else. If the scoring spread indicates changes might improve a hole, try to make them in a way that doesn’t diminish the aesthetics, if any, for the hole. Fortunately, simply changing the lengths to improve spreads rarely messes with the overall look of the hole and sometimes improves it. Time to get out your calculator.