Posted on 25 September 2015 by Mitch Abdon
, coined by Joseph Schumpeter in Capitalism, Socialism, and Democracy
, refers to the process by which new innovations kill old inefficient products or processes. But we are not talking about that but instead, of destroying data to create more useful information. By destroying
, we mean altering the data currently loaded in memory with no undo
button to rely to. When you load or open data into Stata, Stata stores the data in your machine’s RAM. Any changes made, therefore, are not permanent or saved in your hard drive until you call on
, but still be careful that you do not overwrite your raw data files.
Filed under: Basic functions, Data Management | Tagged: collapse, contract, proc freq, SAS, tab, table, tabulate, uber | 1 Comment »
Posted on 8 September 2015 by Mitch Abdon
The UN Comtrade
is the largest repository of disaggregated trade statistics. It offers free access to detailed annual trade data starting from 1962 and monthly trade data from 2010. Free access is limited to 50,000 records per query. This limitation is relaxed in some cases
but the API (as of date) only allows a maximum of 50,000 records per query for all
The UN Comtrade data extraction API (currently beta version) to access the database is publicly available. How can we exploit this to download Comtrade data directly from Stata?
Filed under: Basic Programming, Books and websites | Tagged: ado, API, comtrade, comtradedl, import delimited, UN Comtrade | Leave a comment »
Posted on 1 September 2015 by Mitch Abdon
In yesterday’s post
, one of the needed cleaning was to extract date and day of week from the string:
. list date in 1/5
| date |
1. | Date: August 31, 2015 at 1:42:41 PM GMT+8 |
2. | Date: August 24, 2015 at 12:36:55 PM GMT+8 |
3. | Date: July 27, 2015 at 2:51:27 PM GMT+8 |
4. | Date: July 20, 2015 at 2:45:43 PM GMT+8 |
5. | Date: July 20, 2015 at 2:07:49 PM GMT+8 |
Filed under: Basic functions, Data Management | Tagged: daily(), dates, dow(), strpod(), substr | Leave a comment »
Posted on 31 August 2015 by Mitch Abdon
I tried Uber in late May and since then it has been 131 Uber rides covering 1,200 kilometers and 80 hours on the road. Uber (and GrabTaxi) has eliminated the wait under the heat (and rain) and the dealing with the assholeness of most taxi drivers here in Metro Manila. But what I love most about Uber, apart from their customer service, is the data they send. Trip receipts are automatically sent as soon as the trip has ended. These do not only show how much I am charged but include time, distance, fare disaggregated by time and distance, and many more. GrabTaxi receipts, on the other hand, only show amount paid and manually encoded by drivers.
Filed under: Basic functions, Basic Programming, Data Management, Graphics | Tagged: findval, graphs, traffic, uber | 2 Comments »
Posted on 21 July 2015 by Mitch Abdon
qualifer with time-series data is tricky. Until you meet
. Let us use quarterly German macro data, lutkepohl2
, from Stata website to illustrate.
Filed under: Basic functions, Graphics, Time series | Tagged: display, if, q(), tin(), tsline, tsset | Leave a comment »
Posted on 12 March 2015 by Mitch Abdon
In a comment to the previous post
destring complication: negative numbers
, Nick Cox pointed out “the most important advise” in using
unless you are absolutely sure that you are right or are willing to do things again if you made the wrong decision. The
option is there for a purpose.”
In addition, his comment point to simpler solution than using regular expressions.
Filed under: Basic functions, Data Management | Tagged: charlist, destring, real(), regular expressions, tabulate | 1 Comment »
Posted on 10 March 2015 by Mitch Abdon
Less than 2 hours flight…
In a Stata training, one of the students wondered why after importing an Excel file of financial indicators into Stata some were read as strings. A quick browse at the data indicates the presence of hyphens (“-“) and that these were used in different ways: one to indicate a negative number and another to indicate a missing observation.
How do we convert these variables to numeric as
returns an error?
Filed under: Basic functions, Data Management | Tagged: charlist, destring, regexm, regular expressions | 2 Comments »
Posted on 10 February 2015 by Mitch Abdon
How do we import data from all sheets in a number of Excel files? Each Excel file has a different number of sheets with names of no discernible pattern, but (thankfully) each sheet has the same structure: the first observation is in the same row and the columns correspond to each other. An example is the set of 17 Excel files of census data of barangays
(villages) that was provided to me. Each Excel file corresponds to one region and within each file are sheets corresponding to the province in the region. How do we consolidate all sheets in all files into one data file?
Filed under: Basic functions, Basic Programming, Data Management | Tagged: excel, foreach, fs, import, import excel | Leave a comment »
Posted on 18 January 2015 by Mitch Abdon
clean results window
in Stata thirteen, typing
Filed under: Basic functions | Tagged: clear screen, cls | Leave a comment »
Posted on 8 January 2015 by Mitch Abdon
In the Philippines, there has been an emerging demand in the public sector for data-driven analysis of various govenrment programs (yes, emerging.. don’t ask why.. but the leadership has a lot to do with this.. bless them). I am lucky enough to be invited to introduce Stata to some government agencies (and some private institutions) and how it can be used with their data. Often the the question where to get a copy of Stata pops up. So, for reference: here is the sole distributor of Stata in the country:
Filed under: Books and websites | Tagged: distributor, Philppines, Stata | 2 Comments »