Don't feed the Stata

Today, I set out to test how the amount memory allocated to Stata affects efficiency. The "set mem ___" command sets the amount of RAM to allocate to Stata. I remember reading something at some point that mentioned allocating too much memory can slow the program down but I figured it would be better to see for myself. Sure enough, it did.

The experiment was much like the previous ones. I did five trials of a program with 900m versus 450m allocated. I ran programs with 900m, then 450m, 450m, 900m, 900m, 450m, and so on so that one allocation wasn't always running before the other. I ran programs of varying lengths for each trial. Here are the results:

Trial #. 900m time, 450m time
1. 7:26, 6:22
2. 2:12, 2:02
3. 1:56, 1:40
4. 5:33, 4:43
5. 6:52, 6:16

About a 12% reduction in times by cutting the memory allocation in half. Don't feed Stata too much memory or it will get bloated and slow down.

Fun with Stata: Batch Mode (Part II)

Contrary to my initial findings, running Stata in batch mode does not seem to make programs run faster afterall. I ran the same experiment as last time, alternating between running the same program in batch mode and interactive mode. The main difference this time is that the program is longer. Here are the results:

Trial #. batch time, interactive time
1. 4:57, 4:27
2. 7:29, 5:33
3. 5:47, 8:15
4. 9:14, 5:48
5. 8:38, 7:30

The average batch time was 7:13 while the average interactive time was 6:19 so, if anything, batch mode seems to be slower for longer programs. Darn! The experiment was done under realistic conditions rather than "clean" conditions since I had to use the computer for other things while running the tests (which is probably why there is a lot of variation in the times) so this isn't a conclusive result but it's enough for me to stop using batch mode.

The most interesting result here is how much variance there is in times. I didn't expect this to be the case since I have a dual-core processor which makes it easier to run multiple applications at the same time. The large variance is actually good news since it means there's potentially a lot of room for figuring out how to make Stata run faster. Furthermore, the best lead for trying to reduce times is in figuring out why it was so fast in first trial!

Fun with Stata: Batch Mode

The other day, Nick tipped me to the fact that you can run Stata in batch mode on Windows (and Macs). At first glance, this didn't seem very useful for me because it's not much of a bother to have the interactive window open. After a little more thought, I wondered if it wouldn't make the program run faster. I have little theoretical basis for why this might be the case (perhaps having to display output sucks up Stata's resources) but I've been rather obsessed with trying to run programs efficiently lately. (Having a program that take days to run will do this to you.)

So I experimented: Run the program in interactive mode. Record time. Repeat for batch mode. 10 trials. To my surprise, the program consistently ran faster in batch mode. The average interactive time was about 1:20 and the average batch time was 1:05. A substantial 15 second (19%) improvement. The next question is to see how results look for longer programs. If you only save fifteen seconds on a ten minute program, it's just not worth it. But if it continues to save 19%, then it just may be. Stay tuned for this experiment.

And, of course, if you're running in batch then you might want make the leap to a fancier text editor like Emacs with the Stata mod. Not fun to get started but it looks like it may pay off in the long-run.

In the meantime, to run a program in batch mode, you just have to:
1. Go to Run.
2. Type "C:\Program Files\Stata9\wstata" /b do "DIRECTORY\PROGRAMNAME.do"