LOFT troubleshooting

Linear-on-the-fly testing (LOFT) lets you create a unique test for each candidate using items based on domain and psychometric data. This style of testing ensures item exposure is optimised across your bank of items, meaning that individual items are never overexposed or underutilised.

This article provides a typical troubleshooting process for any issues that may be encountered when using the LOFT/Adaptive screen to create LOFT tests.

If you have tried the steps outlined in this article and are still having issues with LOFT, contact support@surpass.com.

In this section

1. Find your test form

Find your test form in the LOFT/Adaptive screen and select Edit Test Form.

2. Download and check rules for each section

For each section of your test form, select Download Rules to download the rules for each section. Check that the rules are formatted correctly in the CSV file. For more information, read Uploading psychometric targets to a LOFT test form.

3. Create a new test form

Create a new test form with just one external optimiser section. Read Creating test forms and Adding sections to a test form for more information.

4. Upload rules to new test form

Find your new test form in the LOFT/Adaptive screen, select Edit Test Form, and upload your downloaded rules from one section of your original test form to your new test form’s single section. Choose your corresponding item pool from the drop-down list.

Simulate your new LOFT test form. For more information on simulating a LOFT test form, read Simulating a LOFT test form. Schedule and sit your test form to see if there are any issues.

Repeat this process for each section in your original test form to try and identify which section is causing the issue. If you are unable to identify and fix the issue, create a new, duplicate test form that contains the same amount of sections as your original test form, and continue troubleshooting below.

5. Find and download item pools for each section

Navigate to the Item Pools screen.

Find the item pool(s) used in your test form. Select Edit.

Select Download Item Pool to download your item pool. Download each item pool used in your test form.

6. Check item pools

Determine how many items are in your item pool for each section. Check this number against your downloaded rules CSV to ensure there are enough items in the pool to conform to your rules.

EXAMPLE: If your item pool contains 10 items, ensure your rules specify up to and including 10 items for the section.

If there are not enough items in the pool, amend your item pool and repeat the process of simulating, scheduling, and sitting your test form to confirm you have fixed the issue. If you are still unable to identify and fix the issue, continue troubleshooting below.

7. Check enemy relationships in item pools

Determine if there are any enemy items for each section by checking the Enemy Items column in your downloaded item pool CSV files. If any items have an enemy item, check to see if the enemy item is also in the item pool; compare the Item Name column with the Enemy Items column.

EXAMPLE: If your item pool contains 10 items and the rules for the section specifies 10 items are required, an enemy relationship present in the same item pool will result in just 9 items being pulled.

If there are not enough items in the pool due to enemy relationships present, amend your item pool and repeat the process of simulating, scheduling, and sitting your test form to confirm you have fixed the issue. If you are still unable to identify and fix the issue, contact support@surpass.com.