Ranking Part 2: Adjusting for Difficulty

ARDF is conducted in nature. So, like golf, mountain bike racing, or skiing, the venue influences the game’s difficulty level. Navigating in a sparse forest is easier than in jungle-like undergrowth. Running on flat smooth terrain doesn’t require as much effort as climbing steep rocky hills. Some ARDF courses are going to be more difficult than others, and that will always be the case.

So when a competitor scores highly on a difficult course it says more about his/her fitness and skills than acing an easier course. And because an ARDF score is derived from one’s finish time relative to other competitors on the course, prevailing against stiff competition is also a better indicator of an athlete’s skills and fitness level.

Venues across the USA have diverse terrains, and course difficulty can vary widely. So if we are going to compare scores achieved in competitions held at different venues those scores need to be adjusted accordingly. Ideally, the scoring system would be designed to handle adjustments automatically. In orienteering this is accomplished by calculating scores relative to the average times of the top finishers. Because orienteering competitions consistently have a sufficient number of highly-skilled entrants, the top performances establish a good baseline by which to compare the subordinate finishers.

But many ARDF categories at USA events lack highly-competitive entrants and sometimes contain only a single competitor. So the question arises: how to adjust ARDF scores to make them comparable from one event to the next?

Ideal Time

One proposal is to utilize the concept of ideal time ( $T_{i}$ ). Ideal time is the expected finish time for an elite ARDF athlete traversing a course with the following results:

Finds each assigned transmitter in optimum order following the shortest practical route based on the competition map;
Unaffected by chance. E.g., doesn’t find any flags off-cycle, makes no wrong turns, etc.;
Runs at a speed that is realistic for his/her age and gender category taking into account the running conditions present along the route followed.

Note that ideal time is not YOUR ideal time, nor is it the ideal time of any of the competitors in the competition. Ideal time represents what a healthy elite athlete should be able to accomplish without anything unpredictable happening.

There isn’t currently a software program capable of calculating ideal time, so it is a manual process. Although it is a manual process, it is not very difficult, and the process can be documented in detail in order to allow almost anyone to perform the calculation. Calculating ideal time involves spending some time analyzing the competition map, plotting the shortest practical route from start to finish along the optimum order of transmitters, and estimating a realistic speed of movement along that route. A unique ideal time is assigned for each course for each competition held during an ARDF event.

Once calculated, the ideal times represent the results that an elite athlete in each age/gender category could have achieved. Ideal time can be applied to the scoring approach presented in Part 1. By doing so, the scores of all competitors who found all the assigned transmitters will be adjusted, making them more reflective of what they would have scored had an elite athlete actually participated.

Adjusting for Competition

Calculating ideal time and inserting it into the race results has the effect of adjusting all competitors’ scores to reflect what they would have been had an elite competitor actually participated. This will be the case even for categories in which only a single competitor participated. Utilizing ideal time in this manner doesn’t require any special formulas, and the adjusted scores still reflect the correct finish order.

Examples

Consider the M40 results from a Classic 80m competition in which the ideal time was calculated to be 60 minutes (3600 seconds)and there were five transmitters for the M40 competitors to find.

(Example 1) M40 competitor “A” found all five transmitters and finished with a time of 75 minutes (4500 seconds). He had the best overall time (i.e., $T_{1} = T_{A}$ ) amongst all M40 competitors who found all five transmitters. Without putting ideal time into the results, competitor “A” would have a score of:

$S_{A} = (5 + T_{1}/T_{1})\cdot 16.7 = 6 \cdot 16.\overline6 = 100.0$ points

Including ideal time ( $T_{i}$ ) in the results adjusts the score of competitor “A” as follows:

$S_{A}^i = (5 + T_{i}/T_{A})\cdot 16.7 = 5.8 \cdot 16.\overline6 = 96.\overline6$ points

(Example 2) M40 competitor “B” who found only four of the five assigned transmitters. No adjustment for ideal time would be calculated for this competitor. Ideal time is only calculated for a course that includes all the assigned transmitters. Those who find fewer transmitters will not receive adjusted scores.

(Example 3) Let’s look at a very slow M40 competitor “C” who found all five transmitters but required the full time limit of 3 hours (10800 seconds). His unadjusted score would be:

$S_{C} = (5 + T_{1}/T_{C})\cdot 16.7 = 5.41\overline6 \cdot 16.\overline6 = 90.2\overbar7$ points

Adjusting for ideal time gives:

$S_{C}^i = (5 + T_{i}/T_{C})\cdot 16.7 = 5.\overline3 \cdot 16.\overline6 = 88.\overline8$ points

Adjusting for Course Difficulty

The value calculated for ideal time will be greater for difficult courses and lesser for easier courses. Therefore, by comparing the ideal time calculated for an actual course with the ideal time calculated for a standard reference course, a unitless ratio can be derived. That ratio can be used to adjust competitor scores to account for the course’s difficulty.

So long as the standard reference is consistent the adjustment will be fair, since it will affect all competitors’ scores identically. So, for both consistency and simplicity, a reference of one hour is suggested for Classic and Foxoring courses, and fifteen minutes for Sprint, since those are the generally-used length targets used by most course designers. This reference can be named the ideal reference time $(T_{i}^R)$ and uses the same time units (seconds) as ideal time.

The unitless ratio to be used for adjusting for course difficulty is calculated as follows:

(1) $\begin{equation*}\gamma = T_{i} / T_{i}^R\end{equation*}$

Where:
$\gamma$ is the unitless course difficulty factor,
$T_{i}^R$ is the ideal reference time, and
$T_{i}$ is the ideal time.

The more difficult a course is the greater its ideal time. If the course designer made a course so that elite competitors complete it in the targeted amount of time (1 hour Classic/Foxoring or 15 minutes Sprint) then $T_{i}^R \approx T_{i}$ so that $\gamma \approx 1$ . And as a course is made more difficult the value of $\gamma$ increases above $1$ , and less difficult courses will have $\gamma$ values less than $1$ .

$\gamma$ only applies to those factors affecting a competitor’s overall time. Therefore, only the time component of a competitor’s score should be multiplied by $\gamma$ . $\gamma$ should not be applied to the component of a score derived from the number of transmitters found.

When $\gamma$ is multiplied by the time component of a competitor’s course score adjusted for the ideal time, the result is the adjusted score ( $S_{c}^{adj}$ ) that can be compared apples-to-apples to any other adjusted score for the same type of competition and age/gender category. Since transmitter count is not involved in the computation of the adjusted score, it is fine to rescale the score without regard to the number of transmitters. So the formula for adjusted score is simply:

(2) $\begin{equation*}S_{c}^{adj} = 100 \cdot \gamma \cdot T_{i} / T_{c}\end{equation*}$

Where:
$S_{c}^{adj}$ is the adjusted score for competitor $c$ ,
$\gamma$ is the course difficulty factor,
$T_{i}$ is the ideal time for the course, and
$T_{c}$ is the finish time for competitor $c$ .

Inserting the right side of Equation 1 for $\gamma$ into Equation 2 gives the following equation for $S_{c}^{adj}$ that allows us to skip the calculation of $\gamma$ :

(3) $\begin{equation*}S_{c}^{adj} = 100 \cdot T_{i}^2 / (T_{i}^R \cdot T_{c})\end{equation*}$

Where:
$S_{c}^{adj}$ is the adjusted score for competitor $c$ ,
$T_{i}$ is the ideal time for the course,
$T_{i}^R$ is the ideal reference time,
$T_{c}$ is the finish time for competitor $c$ .

Example

(Example 4) Consider the M40 results from a Classic 80m competition in which the ideal time ( $T_{i}$ ) was calculated to be 60 minutes (3600 seconds).

With the information above, calculate the adjusted score ( $S_{c}^{adj}$ ) for Example 1. First, select the appropriate ideal reference time:

$T_{i}^R = 3600$ seconds (For a Classic competition.)

Use $T_{i}^R$ , $T_{i}$ , and $T_{c}$ to calculate the adjusted score for competitor “A” from Example 1. Using Equation 3:

$S_{A}^{adj} = 100 \cdot T_{i}^2/(T_{i}^R \cdot T_{A})$

$= 100 \cdot (3600)^2 / (3600 \cdot 4500) =$ 80.0 points

Other Comments

Because ideal time assumes no chance events, it is possible that some elite competitors will achieve finish times shorter than ideal time. When that happens in the scoring system described in Part 1, those athletes will receive scores greater than 100.0 after the ideal time is applied to the overall results. Such outcomes are acceptable and should be interpreted as indicating that some transmitters were probably found off-cycle.

The scoring methodology described in these posts does not attempt to adjust for differences between the various competition formats, competitor age, or gender. Therefore, adjusted scores must only be compared or combined with other adjusted scores for the same competition format and age/gender category.

There have been some questions regarding Equation 3 and its use of the square of ideal time ( $T_{i}^2$ ). The ideal time is squared in the calculation of adjusted score ( $S_{A}^{adj}$ ) because it is being used to compensate for two separate and independent factors: the strength of the competition, and course difficulty. So it is necessary for it to be applied twice.

Note that the adjusted score contains only a time component and therefore isn’t affected by transmitter count. The effects of transmitter count were accounted for in the calculation of ideal time and adjusted score. So it is reasonable to combine (e.g., average) and compare adjusted scores within the same age/gender category from competitions of the same format having differing numbers of transmitters.

Experimentation with the formulae above is encouraged. Their use over time will no doubt reveal improvements to this scoring approach.

Summary

Because ideal time assumes the technical skills and fitness level of an elite athlete, it can be used to adjust for the absence of strong competition at a particular event. Because ideal time takes into account the actual course lengths and terrain, it can be used to adjust for the effects of the venue on competitor scores. Because the process of calculating ideal time can be documented and repeated by trained individuals, it can be applied objectively and precisely.

But, ideal time is not an ideal solution. It would be better to have a sufficient number of elite competitors at each ARDF competition to utilize orienteering’s scoring approach. But until the day of highly-competitive USA ARDF contests arrives, the use of ideal time as presented above is a reasonable approach allowing results to be compared between events having different levels of difficulty, even when few (or no) elite competitors were in attendance.

The approach described in this writing will not impact the order of finish for any event, but it achieves two important goals:

It allows competitors to see how they would have scored relative to elite competitors like those they might find at a World Championships.
It provides a way to fairly compare scores from events held at different venues and presenting different levels of difficulty.

Number 2 above will be examined more closely in Part 3.

3 thoughts on “Ranking Part 2: Adjusting for Difficulty”

Jerry Boyd says:

2020-06-13 at 2:31 am

This concept makes sense to me. Now just having some free time to look this over visiting the different blog posts (part 1, 2 and 3) and studying it in detail.

Also looked at the sample URL sent in a recent email that brings up the web page that links to a google spreadsheet and lets me compare my standing with others. I like it.
The stated ideas and concepts here makes sense to me.

To my knowledge this is the first tool that give the US ARDF community in the US the ability (in an automated way) that allows each competitor to conduct analysis where they can compare there standing with others.

This is a very good start, just imagine how powerful this could be that after each event the event data gets uploaded with hours of the last finish and just by going to a URL you get all this extracted data. I can see this tool s an aid with helping people to improve there performance.

As with any concept as we apply it in field use we well get lessons learned from field use that will allow us to improve the process and address any issues as they occur.

Jerry
Charles says:

2020-05-13 at 11:17 pm

A flat open course favors quickness. But when there are fewer route decisions to make, less thought is required. So even if one’s mind isn’t so quick, there is less thinking required when the route choices are few. I’m not convinced that flat and open poses a major obstacle for most people.
Joseph Huberman says:

2020-05-13 at 1:58 pm

“So when a competitor scores highly on a difficult course it says more about his/her fitness and skills than acing an easier course. ”
I disagree with this statement in the sense that the two courses emphasize different skills. A rugged steep course favors physical ability. An obstructed thick venue favors good route choice. A flat open area favors quick thinking. So ‘easy’ vs ‘difficult’ is not as simple as it seems on first thought.

Comments are closed.