India should make its high stakes tests reliable, valid and fair in order to make its education system internationally respected, says the author.
An internationally respected education system is a crucial factor in India’s ability to compete on the world stage.
Indeed the perceived integrity of educational assessment practices is vital for India’s education system, national economy, and international image. For without credibility, qualifications are of little value – and the respect allotted by other countries to the qualifications earned through an entire nation’s education system can be affected significantly.
Focusing on the way exams are developed - and especially on the integrity of those exams – could provide a further boost to India’s global reputation.
India is emerging as a knowledge-based economy and human capital is becoming its major strength. Yet only one in ten of 17-23 year olds are in the higher education system, according to a recent survey, highlighting the severe inadequacies of India’s infrastructure for the delivery of education. This highlights the need for upgrading the educational infrastructure to optimise India’s potential in today’s globalised, knowledge-driven world.
Over 2,000 years ago, China held exams as part of the selection process for civil services, and similar tests are still being used today for university admissions and to select employees. These “high-stakes exams” used for university admissions or professional qualification tests have significant consequences for those who take them and organisations that sponsor them. Globalisation and the digital age mean that the high-stakes exams of today have different and more sophisticated demands placed upon them.
The growing importance of knowledge as a commodity and the rapid pace of technological advancements, along with globalisation and the increasing significance of qualifications, require an educational assessment process that promotes meritocracy. Test providers must work to design and build exams according to industry best practices that are valid, reliable and fair.
Validity means the inferences made from test scores are meaningful, reflecting critical aspects of relevant knowledge or skills. For example, doing well on a university admissions test should mean someone is more likely to do well at university. Reliability refers to consistency and reproducibility, and a fair exam is one that is equitable for all test takers regardless of their background or social status. Exam scores should be equivalent for test takers who have an equivalent ability level.
Modern-day test-sponsors have many tools at their disposal to develop and deliver exams which are valid, reliable, and fair. These include modern-day measurement science and computer-based testing (CBT). When developing computer-based tests, for example, test sponsors’ subject matter experts collaborate with psychometricians to design and build tests according to industry best practices.
Psychometricians are measurement experts who analyse data on candidate performance as well as exam and question performance. Question difficulty is measured by the proportion of test takers who are able to provide the right answer. Question discrimination gauges how well the question distinguishes between more or less able test takers. Psychometricians perform a process called test equating that enables test results to be comparable across time and across different versions of the examination. This is an important process for increasing the reliability and fairness of exams. For example in a medical exam, a doctor should not be able to pass simply because the version of the exam he or she took was easier than other versions. Psychometricians also work with test sponsors to establish a passing standard reflecting the required competence necessary to meet the exam’s purpose. Many organisations that sponsor high-stakes exams across the world have shifted from pencil-and-paper testing to CBT.
Since exam papers do not have to be physically shipped to and from testing locations, CBT eases the administrative burdens encountered by test sponsors making it much easier for testing organisations to compete on a global stage. And people who cannot afford the time or money to take a test in a city centre on a particular day have the opportunity to visit a local CBT test centre as exams are made available on-demand throughout the year or available within pre-determined blocks of time.
Exam security is vital to fairness. In a CBT environment, exams can be constructed in ways so that all test takers not taking the exam at the same time do not get the same questions. Measurement science – and the work of the highly skilled psychometricians – is the key to ensure that each and every test taker receives a fair exam, one that is equivalent from both a content and statistical perspective. The test equating process is used so that the exam result is determined by the ability level of test takers, and not by the version of the exam they receive. Some test sponsors opt for the cheaper option of ignoring psychometric processes, but this is risky in terms of security and candidate fairness.
The use of computers in everyday life around the world is now much more entrenched than when computer-based testing first became popular. Jobs in today’s economy require computer literacy, and CBT provides a means of taking an exam that is more similar to how people work and study today. The level of computer literacy required for most tests is very basic but practice opportunities and tutorials can also be provided.
A December 2012 report by McKinsey and Company, Inc. found that India has the third-largest number of internet users worldwide. The report also acknowledged that weak infrastructure keeps India’s internet penetration low but improvements in its internet ecosystem would enable better capitalisation through economic contributions from the internet.
India’s education, assessment and professional qualification systems must prepare to continuously meet the human capital demands of today’s globalised knowledge-based economy. CBT has become the benchmark delivery method for high-stakes exams. The flexibility and ease of access provided by CBT mean a greater ability to compete in the globalised economy. High-stakes exams should be valid, reliable and fair, and this is critical if India is to continue the upward trajectory of its international standing in today’s ever-shrinking and computerised world.
The author is test development strategist, Pearson VUE
