Thursday, 6 April 2017

Summary of an Analysis of Scoring Systems for UKAC Contests

Summary of an Analysis of Scoring Systems for UKAC Contests. Issue 1.

A team of volunteers from both inside and outside the RSGB VHF Contest Committee have been working to analyse scoring systems for the Tuesday evening UK Activity Contests. This report gives the output from that study and is background information to feed into the RSGB VHF Contest Committee Rules Consultation for 2017. That consultation, which will follow in the next week, will provide a formal means for providing feedback.

The report is broken into two pieces - this shorter summary of the analysis and conclusions, and a much more detailed report for those people who are interested in looking at the subject in more depth.

Details of Study.
The objective we set ourselves were to come up with a scoring system which,
- Will be recognised as fair by the vast majority of entrants across a wide geography
- Is straightforward enough to be easily understood and explained (and coded)
- Works for all bands from 6m through SHF
- Works for all sections of the contest (Open, Restricted and Low Power)

The primary rule in taking part in this analysis was to put any pre-conceptions behind us, and to use the real data from the newly published open logs of 2016 to tell us what was actually happening, and what would happen if we tried different schemes.

As a result of this work, we believe that we have come up with a proposal for a fairer scoring system for these contests, but in the process we have been surprised by much of what we uncovered, and by how inaccurate some of the speculation on the behaviour of different scoring systems was.

What constitutes fairness has been a critical element of the work. We have adopted the concept of a Potential Score for a square. From the data in real logs showing what stations were active in the contest, the relative score that a station in each of the UK1 squares could achieve if they had equivalent skills, equipment and take-off is calculated. A correction is applied for different types of scoring systems (bonuses or multipliers), and then the closer that the resulting scores are across the whole UK for a particular scoring system, the fairer it is.

We haven’t tried to compensate for differences in station’s equipment, operating skill, or take-off. We don’t think that we should compensate for equipment and skill – these are hard won factors which contesters develop through experience. It would be nice to compensate for take-off, but we haven’t been able to determine a sufficiently reliable way of doing this with public domain data.

What we have tried to compensate for is the differences in the numbers of QSO partners which are accessible to a particular station to work in the contest based on their location and the actual way that activity is spread around the country.

To make sure that the theoretical analysis of this new scoring system doesn’t create unexpected results, we also rescored the real contest logs against each scoring system.
This is to make sure that, [throughout this document, where UK is used – read UK & Crown Dependencies] we’re not changing the basic structure of these highly successful and popular contests, but at the same time we are helping the most disadvantaged stations.

We repeated this analysis for each band from 6m to 23cm, and for Open, Restricted and Low Power, and in some cases for multiple months of logs.

The detailed analysis is described in detail in the main report, but a brief summary is provided here.
- The amount of activity outside the UK, and the impact on scores is much less than commonly assumed, even from the East Coast. UK activity represents 96% of the QSOs made across the UK as a whole on 2m.
- M5 is broadly more favourable to stations in the North of the country, while M7 is more favourable to stations in the South, but both schemes exhibit equivalent levels of bias when measured across the country as a whole.
- Multiplier systems generally make scoring less fair than 1 pt/km or bonus based scoring because they exaggerate the differences between high and low activity areas.
- The people who are most disadvantaged by both M5 and M7 (relative to 1 pt/km) in these contests are those in Central and Northern Scotland. By the time that you venture that far North, activity is very thin indeed.
- Bonus based scoring systems provide a means to reward the most disadvantaged stations better than multiplier based schemes.

A New Proposal – B2
Based on this analysis, we are proposing a new bonus based scheme (B2) with the first contact in each square adding a bonus of 500, 1000, or 2000 points as shown in Figure 1 below.