Item Response Theory


Usually the training on Machine Learning and Data Science focuses on supervised methods. Unsupervised methods are often neglected apart from perhaps some mention of k-means or other simple clustering methods. Nevertheless there are lots of powerful techniques that should be part of the toolbox of any competent Data Scientist. One of theses techniques are Latent Variable models[1]. These models are somehow related with Partial Component Analysis (PCA) but unlike it, they provide a predictive model[2]. A common example could be IQ tests, in which some accessible data (the results from some tests) are used to measure some “latent” variable which is not directly measurable, the IQ. This latent variable just summarizes the correlation pattern between observable variables, so it is important not to ascribe any causal meaning to it, but it can still be very useful to predict future outcomes that were not directly observed.


One interesting application of this latent variable model is Item Response theory[3]. Originating in psychology, IRT addresses test results within a population, particularly binary response tests in educational settings. From the results of these tests one tries to infer the skill of the student on a specific subject.
The fundamental concept is that the probability of a student selecting the correct answer to a question depends not only on their skill but also on the question’s difficulty. This relationship can be easily modeled with a Poisson distribution.
Later one can add additional parameters including the question’s discriminative power (logistic curve slope) and a minimum success probability (accounting for random guessing). Once the model is established, estimation of different parameters can be easily done using standard statistical techniques like Maximum Likelihood or MCMC if you are follow a Bayesian approach. The results of this analysis can be used not only to determine the skills of individual students, but also to look at effects of educational policies, compare populations and so on. This kind of analysis are amenable to hierarchical modeling, which is achievable in a Bayesian setting where model layers align with demographic variables such as e.g. schools, districts, and countries.

This flexible model extends well beyond the simple setting. It readily accommodates additional factors like response time, randomized tests (where questions are concealed to protect student anonymity), and more. Again, the remarkable point is that this approach gives us access to a variable (skill) to which do we do not have direct access and thus we can not apply the usual supervised learning techniques.

The model is naturally most popular in psychology and pedagogy but it is easy to find applications in different contexts. For instance, tests could be brand-related questionnaires, skills could represent affinity with a brand. Or in sales we could try to estimate the effectiveness or productivity of salespersons in different stores and so on.
通常、機械学習やデータ・サイエンスのトレーニングは教師ありの手法に焦点を当てています。教師なし手法は、k-meansや他の単純なクラスタリング手法についての言及を除けば、軽視されがちです。とはいえ、有能なデータサイエンティストのツールボックスの一部となるべき強力なテクニックはたくさんあります。そのひとつが潜在変数モデル[1]です。これらのモデルは部分成分分析(Partial Component Analysis, PCA)と関連していますが、PCAとは異なり予測モデル[2]を提供します。一般的な例としては、IQテストがあり、いくつかのアクセス可能なデータ(テストの結果)が、IQという直接測定できない「潜在」変数を測定するために使用されます。この潜在変数は、観測可能な変数間の相関パターンを要約しているだけなので、それに因果的な意味を与えないことが重要ですが、それでも直接観測されなかった将来の結果を予測するのに非常に有用です。








[1] Bartholomew, D. J., Knott, M. and Moustaki, I. Latent Variable Models and Factor Analysis (2011), Wiley.

[2] Shalizi, C. [Advanced Data Analysis from an Elementary Point of View]( ADAfaEPoV/).

[3] Fox, J.-P. Bayesian Item Response Modeling (2010), Springer.




メールアドレスが公開されることはありません。 が付いている欄は必須項目です