Admission council helps in statistical assessment of LSAT – India

The Law School Admission Council (LSAC) helps law schools in various countries evaluate the critical thinking skills of their students. Sanchayan Bhattacharjee speaks to Nazia Rahman, member of LSAC’s psychometrics team about using data to analyse and evaluate student performances.

• How is an exam statistically analysed? Can you give an example?

The statistical support of an exam spans the entire process from test creation to score reporting. In assembling a test form, statistical characteristics, such as question difficulty, are used to ensure that the overall difficulty of a test form to be administered conforms to predefined specifications. After the test has been administered, further statistical analyses are carried out to ensure that individual test questions and the test form as a whole performed as expected. Ultimately, statistical analyses are used to generate final test scores.

• How can statistical analysis of an exam help:

Test Givers?

The statistical analysis of test data results in test scores that are more reliable, valid, and fair. Therefore, test givers can have greater confidence in the decisions they make based on the scores that are produced as a result of these analyses.

Test Takers?

Like test givers, test takers can feel confident in the precision and fairness of their test scores.

• What is the Item Response Theory (IRT) which is used in the assembly of Law School Admission Test (LSAT) India forms?

It is a sophisticated mathematical model that can be employed at every stage in the testing process.

• How has this model helped the testing process?

IRT allows for a high degree of precision in the estimation of psychometric characteristics, such as difficulty, at both the question and exam levels. This precision allows for better control over how exams are assembled and how scores are reported, resulting in high quality assessments.

• Is this process a purely quantitative analysis? Or is there a qualitative component as well?

There is a qualitative component to the analysis of test questions and test forms. At LSAC, staff members with advanced degrees in areas such as logic, English, and linguistics review question and test form statistics for LSAT—India. In particular, these experts look for any questions that performed differently from what was expected. As a result of this review, if a question is determined to be unfair in any way to any segment of the test-taking population, it will be removed from the scoring of the test. Due to the stringent review process that occurs prior to the test, such removal of a question has never occurred for LSAT—India.

• How can schools, colleges and other educational institutions benefit from such statistical analyses? How reliable is the data interpretation?

Schools, colleges, and educational institutions can have greater confidence in the test scores produced through these stringent statistical analyses, and ultimately in the decisions they make based on the resulting test scores. All statistical analyses of test data carried out are subject to stringent quality control procedures making it extremely reliable. In addition, by the standards of the global psychometric community, the statistical reliability of the test scores produced for the LSAT—India is extremely high.