OSCAR - QC Setup

Quality Control (QC) Setup is made up of five different QC types.  Below is a brief description of each QC type

1. Exemplar - Ability to make sets of "example" responses available to scorers.  Exemplars are made available through the OSCAR resources tab and can be accessed during practice, qualification, and operational scoring. 
2. Practice - Creation of practice sets that allow scorers to apply scores to responses and receive a report/feedback based on the scores they applied vs. the predetermined true score. 
3. Qualification - Creation of a set of responses with a "passing" requirement that must be met before scorers can begin scoring operational responses. 
4. Validity - During operational scoring, scoring supervisors configure validity papers into a scorers’ workflow. Validity papers are papers for which the “true score” is already known. The paper is embedded into the workflow of each scorer and the scorer is expected to get the score “correct.” Validity papers are identified and annotated by expert staff as a part of the range-finding process and are used throughout operational scoring
5. Calibration -  Often in conjunction with validity papers, calibration sets are created and delivered to individual scorers when validity thresholds are missed or to deliver to an entire scoring team when drift begins to occur. When setting up a calibration set users have the ability to deliver the calibration as “Blind” and determine if the specific set should be auto delivered or not. Calibration sets can be ‘auto-delivered’ which means it’s used as a part of validity corrective action; automatically sent once a scorer reaches a predetermined threshold. If auto-deliver is turned off, the administrator can manually deliver the calibration set to an individual or scoring team.