The Items Delivered report in Reporting displays a detailed analysis of how each item has performed in a test form. This includes information like the average mark achieved by candidates, item exposure, P value, R value, and discrimination index.
This article explains how to navigate and interpret and Items Delivered report, including how to perform a raw data extract.
In this article
Navigating to an Items Delivered report
To view an Items Delivered report, go to Reporting > Analytical Data.
Choose a subject in the Analytical Data screen and select the number in the Test Forms column to view the Test Forms report.
To review how items in a test form performed, select the number in the Items Delivered column.
About the Items Delivered report table
The Items Delivered report grid lists all items in the chosen test form. Each row is dedicated to a single item. Refer to the following table for information on each column.
|Item Position in Test||
Displays the item’s position in the test form. This is not applicable if items are randomised or chosen by dynamic rules.
NOTE: By default, the table is sorted by Item Position in Test in ascending order.
|Item Name (ID)||Displays the item’s name. The item’s unique ID code is displayed in brackets.|
|Mean Mark / Max Mark Available||Displays the average mark achieved on the item and the item’s maximum possible mark.|
|Item Exposure||Displays the number of times the item has been shown to candidates in a test.|
|Attempted||Indicates how many candidates attempted the item as a percentage.|
|Viewed Only||Indicates how many candidates only viewed the item (and did not attempt a response) as a percentage.|
|Not Viewed||Indicates how many candidates did not see the item as a percentage.|
|Avg. Response Time (Seconds)||Displays candidates’ average response time on the item in seconds.|
Displays the item’s probability value.
A P value is a statistical representation of item difficulty calculated by dividing the number of times an item has been answered correctly by the number of times the item has been attempted. P values are represented as a decimal number between 0 and 1. The closer the P value is to 1, the easier the item is considered to be.
If an item is answered correctly nine times out of ten, it will have a P value of 0.9. This could be considered an easy item.
If an item is answered correctly five times out of ten, it will have a P value of 0.5. This could be considered a medium item.
If an item is answered correctly two times out of ten, it will have a P value of 0.2. This could be considered a hard item.
Displays the item’s R value (also known as the Pearson product moment correlation coefficient).
The R value measures the correlation between the item score and the test score. R values range between -1 and +1.
A positive R value (0.1 to 1) indicates a positive correlation between candidates’ item scores and test scores.
A neutral R value (0) indicates no correlation between candidates’ item scores and test scores.
A negative R value (-1 to -0.1) indicates a negative correlation between candidates’ item scores and test scores.
IMPORTANT: R values are only populated for non-dichotomously scored items. Non-dichotomously scored items can have multiple outcomes. Some examples of non-dichotomously scored items would be an Essay item worth ten marks and a Multiple Response item with four correct answer options worth four marks, where one mark is awarded for each correct answer option.
Displays the item’s point biserial correlation coefficient. This is a special version of the Pearson R value that only applies to dichotomously scored items. Point biserial values range between -1 and +1.
A positive point biserial value (0.1 to 1) indicates a positive correlation between candidates’ item scores and test scores.
A neutral point biserial value (0) indicates no correlation between candidates’ item scores and test scores.
A negative point biserial value (-1 to -0.1) indicates a negative correlation between candidates’ item scores and test scores.
IMPORTANT: Point biserial values are only populated for dichotomously scored items. Dichotomously scored items only have two outcomes (correct and incorrect). An example of a dichotomously scored item would be a Multiple Choice item worth one mark.
Displays the item’s discrimination index (DI).
A discrimination index indicates the extent to which an item is able to successfully discriminate high-performing candidates from low-performing candidates.
If more high-performing candidates get an item right than low-performing candidates, the item positively discriminates and has a DI between 0 and 1. An item with a positive DI generally supports for the assessment qualities of the test.
If more low-performing candidates get an item right than high-performing candidates, the item negatively discriminates and has a DI between -1 and 0. An item with a negative DI can compromise the integrity of the test.
Indicates whether the candidate’s performance on the item contributed to their total mark for the test.
NOTE: You are informed if an item has been used as both a scored and non-scored item in multiple test forms during the specified date range.
Displays the item’s version number.
If multiple item versions have been delivered to candidates, you are able to view each version’s data. By default, the Items Delivered report displays the most recent version.
To view more information about an item, choose the subject and select Item Information .
The Item Information dialog displays the selected item’s ID number, who it was last saved by, when it was last used in a test, how often it has been unattempted, its parent centre, subject, and subject owner.
About the Quintile Analysis panel
The Quintile Analysis chart divides candidates into five performance brackets known as “quintiles” (quintile 1 being the highest performing bracket and quintile 5 being the lowest). Each quintile is given an average P value. The chart allows you to see the spread of difficulty matched to performance level across the whole candidate pool.
About the Distractor Analysis panel
The Distractor Analysis chart displays analytical data for Multiple Choice, Multiple Response, and Either/Or answer options. Candidates are divided into three performance groups (High, Middle, and Low), with each group’s responses charted against the relevant answer option.
About the Candidate Comments panel
If candidate comments have been enabled on the test form, you can view any comments candidates have left on items in the Candidate Comments panel along with the candidate’s name and reference.
Select Export as CSV to download all candidates’ comments in spreadsheet format.
Filtering the Items Delivered report table
By default, the Items Delivered report displays data from all items used in the test form in the last 30 days. To change the data range, select Filter .
To learn more about the available filters, refer to the following table.
|Last Used||Determines the date range for the Items Delivered report. Choose whether the grid displays data for items used in the Last 7 Days, Last 30 Days (default), Last 180 Days, or a Custom Date Range.|
|Hide items where fewer than 50 tests taken||
Only shows items that have been used at least 50 times.
NOTE: Psychometric statistics may not be reliable for items that have been used less than 50 times.
|Scored Items||Displays scored items on the Items Delivered report.|
|Non-scored Items||Displays non-scored items on the Items Delivered report.|
|Item Name or ID||Finds items based on their names or ID numbers.|
Exporting Items Delivered reports
You can export two reports from the Items Delivered screen.
Exporting a report as a CSV
To export the data in the Items Delivered report table to a spreadsheet, select Export as CSV .
Raw Data Extract
To export all available information attached to the items (including data not in the Items Delivered table such as tags, item relationships, and each candidates’ response), select Raw Data Extract .
Select the information you want to include in the Raw Data Extract dialog. Select or clear ALL to include or remove all options, respectively. Select Export as CSV to download the raw data spreadsheet.
|Column||Description||Data type||Character limit||Example|
|Candidate Tags||Displays any tags that are associated with the candidate.||Character||N/A||CandidateTag_Gender: Male|
|Completed Date||Displays the date the candidate completed the test.||Date||N/A||06/05/2019 12:37:00|
|Keycode||Displays the candidate’s unique keycode for the test session.||Character||12||27ZRZV6P|
|First Name||Displays the candidate’s first name.||Character||50||John|
|Last Name||Displays the candidate’s last name.||Character||50||Doe|
|Reference||Displays the candidate’s unique alphanumeric reference code.||Character||300||BIO3451A|
|Gender||Displays the candidate’s gender.||Character||1||
|DOB||Displays the candidate’s date of birth.||Date||N/A||01/01/1990|
|Subject||Displays the test’s parent subject.||Character||100||Biology|
|Centre||Displays the test’s parent centre.||Character||100||Test Centre A|
|Test||Displays the test’s name.||Character||100||Biology KS3|
|Test Form||Displays the test form’s name.||Character||200||Biology KS3 - Autumn 2019|
Displays the mark the candidate achieved for the test form.
This reflects any changes in Re-marking. It is not possible to view re-marked item’s original marks in the Raw Data Extract. You can view original marks on an item-by-item basis in Audit.
Displays the total marks available for the test form.
Displays the grade the candidate achieved.
Displays the item’s name.
Displays the mark achieved by the candidate on the item.
Displays the total mark available for the item.
Displays the response given to the candidate for the item.
A (Multiple Choice)
Blood vessels (Short Answer)
Displays the item’s correct answer(s) or answer combinations.
A (standard marking)
A,5|B,3|C,2|D,0 (weighted marking)
AB,2|CD,3|AC,1|ABC,2|A,1 (combination marking)
Displays the time the candidate spent viewing the item (in seconds).
Displays the item’s order in the test form.
Displays the question type.
Displays any Learning Outcome tags assigned to the item.
Understanding the physiology of humans
Displays any Unit tags assigned to the item.
Displays any Keyword tags assigned to the item.
Displays any custom tags assigned to the item.
Indicates whether the mark the candidate achieved on the item contributed to their overall score on the test.
Indicates whether the item was computer or human marked.
Displays any enemy relationships the item has.