About control item settings in SecureMarker
A control item in SecureMarker is a candidate response that has been marked to a high standard and is used to ensure examiners are marking fairly and within accepted tolerances. You can create control items and set control item settings in the Quality Control screen.
There are three ways you can use control items to monitor the quality of marking:
- Daily competency run – Show examiners control items daily at the start of their marking session.
- Ongoing marking – Present examiners with control items during marking.
- Rolling review – Review the control items an examiner has marked historically.
This article explains the settings available in order to configure control items as part of the daily competency run, ongoing marking, and rolling review.
In this article
Navigating to the Quality Control screen
You can configure control item settings in the Quality Control screen.
Exam-level control item settings
To access exam-level control item settings, select Item Defaults.
Item-level control item settings
To access item-level control item settings, select an item or item group and choose Examiner Competency Settings.
About the daily competency run
You can set up a daily competency run to present examiners with control items when they start marking an item for the first time. This ensures the quality of marking on a daily basis before examiners begin to mark live responses.
Refer to the following table to learn about the settings available.
| Setting | Description |
|---|---|
| Number of CIs for daily examiner competency run | Defines the number of control items examiners see when they start marking an item. |
| Number of CIs to be marked within tolerance | Defines the number of control items in the daily competency run that examiners must mark within tolerance. |
| Maximum cumulative error allowed | Defines the maximum number of marks an examiner can allocate out of tolerance across all control items in the daily competency run before they are suspended. |
About ongoing marking rules
You can set up ongoing marking rules to present examiners with control items during live marking. This allows you to ensure the quality of marking is as consistent as possible.
Refer to the following table to learn about the settings available.
| Setting | Description |
|---|---|
| Probability of presenting CI |
Defines the probability that each response presented to an examiner is a control item. Enter a value between 0 and 100.
TIP: If you do not want any control items to appear during marking, set this value to 0.
|
| Maximum consecutive number of non-CIs that can be presented |
Defines the maximum number of consecutive responses that are not control items that an examiner can receive before they must see a control item. |
| Maximum consecutive number of CIs that can be marked out of tolerance | Defines the maximum consecutive number of control items that an examiner can mark out of tolerance before they are suspended from marking. |
About the rolling review
As part of the ongoing marking rules, you can set up a rolling review. This allows you to review examiner performance historically on control items to ensure the quality of marking is as consistent as possible.
Refer to the following table to learn about the settings available.
| Setting | Description |
|---|---|
| Number of CIs in rolling review |
Defines the total number of control items in the rolling review.
EXAMPLE: If you enter a value of 10, the rolling review will check an examiner’s performance on the last 10 control items they marked.
|
| Maximum number of CIs that can be marked out of tolerance in rolling review | Defines the maximum number of consecutive control items that an examiner can mark out of tolerance before they are suspended. |
Further reading
To learn more about setting up control items, read the following articles: