Sunday, 1 December 2013

A method for calculating item misplacement

I have been struggling to think of a way to calculate and compare the difference in the ordering of items between different Mokken scales, eg different samples or between small and large samples from the same population. The problem is that you need a reference point and that should be the definitive ordering of the items; something that we can only know in theory but, in practice - unless we sample the whole population - we can't know.  So, what about a very large sample - in the region of 10,000 - that we can assume gives the ordering in the real population?

So, we have the reference point, how do we compare the ordering in sub-samples.  We can count the number of times an item is not in its usual place in the reference ordering, seems sensible, but the problem is that when one item is out of place there must be at least two items out of place as another one will be moved in the hierarchy. The misplacement of any single item means that another is automatically misplaced but we do not know which one was misplaced first. The solution is simple, calculate the total number of items misplaced and subtract 1; therefore, the number of misplaced items - compared with a reference sample - is 'm - 1' where 'm' is the number of misplaced items. The '- 1' accounts for the item that has to move for any misplacement of items to occur. In this way it should be possible to compare different samples from the same population or from a very large sample and to study, for example, the effect of taking different samples or a series of samples of the same size or a series of sample of a different size.

Admittedly, the point I am making is quite minor.  However, whether or not you take the '-1' correction into account any comparisons will be the same but - while the minimum number of misplaced items is clearly zero - then it is more sensible to have the next smallest number as '1' and then the sequence continues up to 'n-1' where 'n' is the number of items in the scale.  Therefore, for a scale of 'n' items the maximum number of misplaced items is 'n-1' because if all the items are displaced then 'n = m'; again accounting for the fact that if all items are misplaced then the minimum number of misplacement that needed to occur was 1.