Old versions of Stata

When upgrading to a new version of a software, I am used to uninstalling the old version first because most software require this or I am concerned that  different versions installed will result in conflicts. So, I do the same when I install newer versions of Stata. If I already have the new one, why will I need the old ones?

Then came an answer today! I was trying run Tristan Zajonc’s -openirt-* help file examples to see how the program works. I was using version 11.1. The program returned an error, which I thought unlikely  because the examples come from the program’s help file (although this is not always a guarantee…). I tried the other examples and the same error occurs. Just to be sure, I opened openirt.ado to see which version the program was set to so that I can run the same examples using this version and see if it works. One hiccup—I have no other versions currently installed! I had to install version 10 all over again and update it to 10.1 as -openirt- was set to run in version 10.1. -openirt- ran smoothly in version 10.1. Problem solved; lesson learned.

I remember a discussion about this on Statalist, here is the thread.

*-openirt– (Zajonc 2010) estimates Item Response Theory models for dichotomous data.

12 Responses

  1. I’m trying to run a principal component analysis in Stata 11.1. It runs the factor analysis and allows me to rotate and promax. Then when I try to save the factor scores by the ‘score’ command I get the following error:

    Return code 301 last estimates not found; You typed an estimation command such as regress without arguments or attempted to perform a test or typed predict, but there were no previous estimation results.

    I am a Stata novice and have only used it to run some very basic analysis. Factor analysis is pretty basic too. So is the score function I’m trying to execute. And I have spent the last two hours searching the internet to figure out this error. I’m unable to find an answer.

    Since our tech staff recently upgraded this computer to Stata 11.1 from an older version, I’m wondering if that is the problem. Help, please.

  2. This is an admirably clear and full discussion.

    I distrust fragile models, those that are difficult to fit, or to fit reproducibly. Such difficulty is not an indictment of any particular estimation method. Such difficulty often, but not always, means that you need a better estimation method. So, I don’t think I disagree here.

  3. Nick is right, this is not a Stata issue. Rather it is because openirt relies on a compiled program for speed reasons — something that in hindsight is probably too much pain to be worth it. I have tested the compiled program on linux, OSX, and 64-bit Windows. It’s very possible that I need to compile another version for 32-bit Windows. Unfortunately, these are the only machines I had easy access too. StataCorp has the advantage of being able to easily compile on a larger set of platforms.

    Nick’s point about the random number generator is taken. I’ve uploaded a new version that has a seed() option. I’d prefer to take the random number generator state from Stata directly, but that’s more work than I have time for at the moment. This means that the seed is set every time that openirt is run, rather than once, which is not ideal for simulation studies. That’s why I randomly set the seed — also a bad idea but one that works ok in practice.

    More broadly, this software uses MCMC to estimate the IRT model. It is therefore inherently random and you should expect slightly different results across model runs if the seed is different or if you let the state continue to evolve. If the results are very different you should be concerned. It could be a sign that the chain is not converging, although this is not a problem I’ve had. In general, run a much longer chain than the default when doing your final analysis. And make sure two runs with different seeds match up.

    The Monte Carlo nature of the estimation may be perceived as “fragile” by some. I don’t think that’s fair. Standard ML estimation of IRT models have significant problems. They often to not propagate uncertainty from the parameter estimates to the ability estimates. They cannot be used directly to estimate potentially interesting quantities such as the population standard deviation. And, perhaps most importantly, in my experience there seems be a significant downward bias in the maximum likelihood ability estimates in finite samples when performance is poor. The asymptotic approximation for the standard error is also completely wrong. The sampling distribution for the ability estimates is highly non-normal in finite samples at poor ability levels. Bayesian methods solve all these problems.


  4. Nick, thank you very much for your feedback. Your absolutely right! I executed my do-file again and I did get close BUT different estimates. I think I have read something about random number seed in Scott Long’s book. Will check it out…

    On the Stata version, I also got confuse as to whether it has something to do with the problem or not. -openirt- now works both in Stata 10.1 and 11.1 in my machine (Windows 64-bit), but it fails to work in another machine (Windows 32-bit)… Whatever the case, it looks like there’s a problem. I have already e-mailed Tristan Zajonc, the author. I will also send him your feedback.

    Again, thanks.

  5. I had a quick look at -openirt-.

    It appears to use a different random number seed each time it is run in a particular session. If true, this might explain the lack of reproducibility. It also seems that a model that fits some times and not others is inherently fragile. I distrust programs in which you do not have the possibility of direct control over random number seeds — although a clone of -openirt- could be written to provide this.

    I thus doubt that Stata version is in any sensethe issue here. I very much doubt whether the fact that you have two different versions of Stata installed is at all problematic for your purpose.

    I have not looked at the -openirt- documentation.

  6. Sorry, I forgot to mention that the -openirt- package must first be installed:
    net install http://www.people.fas.harvard.edu/~tzajonc/stata/openirt/openirt


  7. I can’t run your code.

    sysuse naep_children

    does not work for me.

  8. When I ran the following yesterday using v11.1:
    sysuse naep_children, clear
    openirt, id(id) item_prefix(item) ///
    save_item_parameters(“items.dta”) ///

    It returned:
    Selected items: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
    22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
    Setting up response data…
    Setting up parameter data…
    Default model: Three parameter logistic (3PL)
    OpenIRT could not estimate your model. Please check the specified options and try again.
    This was not the case when I used v10.1. What’s interesting is when I tried it just now using the same v11.1 I used yesterday, it went perfectly well. Is it because I already have v10.1 installed?

    To check, I have uninstalled v10.1, rebooted the computer, and ran the same examples using v11.1. This time the program was executed.

    So what went wrong? It might just be a matter of rebooting the computer (which I didn’t try yesterday)?

  9. The short answer to Shiel is that ado files usually _are_ supported by later versions — so long as they are written properly. One longer answer I’ve already written out as the FAQ just cited.

    Stata does sometimes withdraw features, but usually they are not features that one should miss.

    I didn’t understand either precisely why -openirt- didn’t work in Stata 11. Mitch does not seem to spell that out.

  10. Here is a link (http://www.stata.com/support/faqs/lang/version2.html) to the FAQ “I have a certain version of Stata and have come across a user-written program that is apparently written for a later version of Stata. What are my options?” (Nick Cox 2009). I found this link via a very recent post on Statalist.

  11. but why is that? Seems not efficient… Shouldn’t ado-files supported by old Stata versions autmatically be supported by later versions?

Leave a Reply