Posted on 3 January 2011 by Mitch Abdon
Vacation is finally over. Besides HBO, Discovery, and Animal Planet, I have spent my break reading Scott Long’s The Workflow of Data Analysis Using Stata (Workflow). Except for the frequent “YOU’RE GUILTY” flags that flash before me in almost every page, I definitely enjoyed the Workflow. It got me to think of how I use to do things and, more important, the Workflow showed me how to improve them or how to do things right.
Since it is the time for New Year’s resolutions—the list of things you promise yourself and hope that you actually accomplish half of the list—I made a short list of my own:
1. Create a proper Research Log for every project. My work notes are everywhere. If I am asked what I did for a certain task, I consult my incomprehensible hand written notes in my physical notebook and my comments and do-files, data files, or other e-files in my computer. Most of the time, I could find what I am looking for, but only after wasting so much time.
The Workflow (p. 41) provides an example of a Research Log. One look at this snapshot of a research log will give so much information on what has been done and when, which files does which tasks, and more. I can only imagine how much time I should have saved if I had something similar to this.
2. Document now, not later! This is rephrasing Long’s law of documentation (p.34): “It is always faster to document it today than tomorrow.” Reading this part of the book reminds me of a recent (re)work that I and a colleague did before the Christmas break. We had to rerun all the do-files we wrote for a certain paper because we cannot replicate some of the tables. Two days of work gone because of poor documentation.
3. Follow the Workflow’s posting principle (p.125): “Once a file is posted, it should never be changed.” So G-U-I-L-T-Y!
4. Never name a file “final” (p.137). I actually have a file named final.dta and another final_new.dta and another final_new2.dta. What a mess!
5. Practice using Stata’s -datasignature- feature (p.139).
6. Backup and archive properly. The Workflow devotes a whole chapter (ch. 8 ) on protecting files. When some of my do-files were corrupted for reasons only my computer knows, I had to re-type them (my only back up were the ones I printed). Shame.
So help me God.
Happy New Year everyone! =)