Thursday, February 27, 2014

Bigfoot module for LyX

For the lazy, here is a module for support for the bigfoot package for LyX (outlined here). Find your User directory from Help > About LyX and then put the file in "<User directory>/layouts/". Then do Tools > Reconfigure.

Friday, January 24, 2014

PDFs that allow toggling between images

Authors sometimes want to their documents to be able to display different images in different settings. For instance, color charts often become indecipherable when printed in gray-scale so a separate version with patterns might be preferable. One simple way to achieve this in PDFs is with Optional Content Groups. See this PDF for a working example. It has a link that a user can click and it will toggle images in the PDF. The file was produced using LyX with the tikz and ocg-p LaTeX packages. This zip contains source materials for those who would like to create your own. OCGs don't use JavaScript so this method should be fairly portable.

Saturday, December 21, 2013

Combating projection bias

One manifestation of projection bias is where you think that your views won't change with time. For example, young people often think they will never want to be married or have children. I think a neat idea for a website would be to summarize from long-running surveys to see which opinions or beliefs change with age and which do not. You could even break out the data by subgroup so that someone could see, for example that "of the people with similar education and income levels who held my view, X% changed their mind". This might help when thinking about long-term commitments such as living location, jobs, and partners.

Wednesday, September 11, 2013

Automated Statistical Reports in MS Office

I prefer LyX to Microsoft Office when writing academic works, but sometimes MS Word/Powerpoint is necessary. My one requirement for the workflow is that manual fiddling isn't required. This is a bit challenging because Office products can't show PS/PDF material inline. You can include an EPS as a graphic which can print OK (if you print to a PDF or have a PS printer), which is fine for Word, but will only show an unavoidably ugly "preview" inline which doesn't work for PowerPoint. For presentations, you should output to a raster format (like PNG) or for tables you can possibly output to EMF.

Generating EPS/PNG files from graphics is built into most stats software (R, Stata), so the only hard part is converting tables. I'll assume you can generate these a tex and start there (so as to minimize differences between Office and Latex generated reports). Scripts below were tested on Windows 7 with Cygwin and I had placed the programs in step four in my path.
  1. Export a table to mytable.frag.tex.
  2. Wrap that tex fragment in a minimal "standalone" document.
    • >echo \documentclass[varwidth=true, border=10pt]{standalone} > mytable.tex
    • >echo \begin{document} >> mytable.tex
    • >cat mytable.frag.tex  >> mytable.tex
    • >echo \end{document} >> mytable.tex
  3. Make your table into a "standalone" PDF (the PDF will be just the size of the table). Your TeX distribution will automatically download the 'standalone' package.
    • >pdflatex mytable.tex
  4. Convert the PDF to the output required.
    • >pdftops -eps mytable.pdf mytable.eps
      • Could also do this in Ghostscript like below.
    • >pstoedit -f emf mytable.pdf mytable.emf
      • This will change the fonts on you. If you have Greek letters this conversion might fail in which case you need to add -adt which turns letters into polygons (not perfect but OK). You can also convert with Adobe Illustrator which will also do font conversion.
    • >gswin64c -dSAFER -dBATCH -dNOPAUSE -sDEVICE=pnggray -r600 -sOutputFile=mytable.png mytable.pdf
There are two options for inserting. The first is the easiest uses absolute paths for images so won't work well with sharing the files among other users (e.g. in Dropbox) or moving the project folder.

  • Use normal image insertion but use the dropdown next to "Insert" and select "Insert and Link". Some VBA scripts are available that will convert the absolute path to relative paths (e.g. here).
  • Insert pictures using field codes. Insert -> Quick Parts -> IncludePicture. Then put in a relative path like pics\pic1.png and select "Data not stored with document" (and you probably want to check resizing both horizontally and vertically). If you want to convert to pictures (if you send it out), then remove the "Data not ..." by removing "\d" from the field code, and make a formatting edit (like going to Format Picture and then changing the Layout wrapping style).

Now when the linked files are updated by programs, they will be updated in Office. If the document is open then select an image (or using select all) and press F9.

Edit 2016-01-22: Expanded part about relative paths to images.

Also note that emfs might not display correctly on Mac Powerpoints (same for MathType equations). You can print pptxs to pdfs but then you will lose any animations.

Thursday, September 05, 2013

Modifying Stata library routines

I was recently not understanding why some error messages were being generated by mata's optimize(). I found that it was quite easy to modify the routines to include some debugging print-outs. Broadly we want to create modified routines that mask the existing ones. Here's the basic steps:

  1. Find the mata file (in my case deriv.mata) in the Stata folder, copied it to your own directory, and make changes. 
  2. Turn it into a personal library. See the basic template at -help mata_mlib-.
  3. Have the personal library called instead of the builtin routines. The -mata mlib index- command from the previous step outputted the order in which the libraries are searched for a function.

Enjoy your modified version. When you're all done you can use -mata mlib index- again to get the default ordering.

Wednesday, June 26, 2013

Strategic voting and Oxford-style debates

In determining which side wins a debate, one variant of  Oxford-style debates has people vote about a proposition before and after hearing the debate. The side that sways the most voters towards theirs wins. For example, if an audience votes initially that they are (60% For, 20% Against, 20% Undecided) and afterward they are (65% For, 30% Against, 5% Undecided) then the Against side wins (they change 10% toward their side while the other only swayed 5%). This system, though, incentivizes dishonesty in the first round. If I believe I am unlikely to change my mind during the debate, I should vote opposite my current view in the initial vote.

To see if strategic voting is a problem, one could take a debate's audience and have half of the people's votes be noted but not counted towards the winner. If the counts of the two groups differ, then something might be going on.

The harder question is how do you incentivize truth telling for those that switch.
Edit: One way maybe would be to just have a vote at the end but have a prediction market (for the final vote) be going throughout the debate. The winner of the debate is the one that moved the market price the most. 

Wednesday, July 18, 2012

The alphabet soup of fried chicken places


There is a wide variety of fried chicken restaurants in Dhaka, Bangladesh. There's so many that someone mentioned trying to find for each letter of the alphabet. So here's a list of "_FC".

A: American,
B: Best
C: Cripsy, California, CP (unsure what CP stands for)
D: Dhaka
E: 
F: Fortuna
G: Golden, Good
H: Happy
I:
J:
K: Kentucky, Kok Kok
L: Lime's
M: Macdanald?
N: (in Mohakhali, but unsure what N stands for)
O:
P:
Q:
R:
S: Southern (not sure they still exist...)
T: Taitan
U:
V: Vhoot (means "ghost" in Bengali)
W: (in Banani but unsure what W it stands for)
X:
Y:
Z:

Not bad at over 50% of the letters!

Tuesday, July 17, 2012

DC metro station interior information

On my normal WMATA routes, I (like many other commuters) know where to get on the train so that I end up next to the exit at my destination (this is possible because the trains almost always stop at the front of the platform). But if I'm going to an unfamiliar station I can't shorten the walk at the destination station by walking a bit at my departure station. Knowing the location on the platform of the escalators/stairs/elevators could save time at those stations which only have a single exit on one end. I imagine this type of information would be especially useful for those with impaired mobility. WMATA comes close to providing this info by allowing one to find out about the status of escalators/elevators. It provides and accompanying description (e.g. "Escalator between street and mezzanine") but it's sadly not specific enough.

I've been told that WMATA does have this information digitally so hopefully they'd be able to provide it. Otherwise, the information could be user-gathered. The Washington Post did that with their Metro cell phone service map which had a publicly-editable custom map on Google Maps. Another option would be to have user edit OpenStreetMap. Or possibly one of the smartphone apps (which would benefit from this info) could provide and option for user submitted information). Anyone want to champion one of these options?

(And for the smartphone app developers out there, could you integrate the direction of station exits like StationMasters does? It's a small point but kind of nice.)

Sunday, July 15, 2012

Crowdsourcing journal/data information

In academia, researchers often work with and learn from data and methods that have been used before. I think are new projects that could fill existing gaps in this process and therefore speedup the learning and research process by reducing redundancy. I'll highlight three here. If they already exist, please let me know. If they don't, hopefully someone could start the project and allow crowd-sourcing of this information (could be as simple as setting up a wiki and collecting links to places where some of this data already exists):

  1. Data scripts: Publicly available datasets are often not in the best of shape. They often need to cleaned-up, labeled, linked to other data sources, or processed in standard ways. Additionally, there may be external information about the quality or other facts that should be documented and understood by researchers. Documentation and scripts (in multiple languages) would be the goal here. See for example asdfree.com.
  2. Study replication: Researchers over try to replicate existing studies in order to understand a method, for a class project, or to see conduct extensions. As many authors do not contribute accompanying code for working with the data, there is a lot of reverse-engineering that has to be done. This would not only save time, but disseminate important information about the implicit assumptions in papers.
  3. Typo corrections: Ever puzzled over an equation in a journal article and gone through the trouble of finding that there is a type in it? Published material is never perfect, and unless it is a large mistake, authors normally don't post corrections. But noting small (non-controversial) corrections could still save a lot of time. Obvious spelling mistakes are not worth the time to correct, but even if conclusions aren't overturned it is still helpful to correct intermediate steps.
    Edit: PubPeer seems to provide this function.

Saturday, July 14, 2012

Smartphone app for verbal confidential agreements

A professor recently cautioned a graduate-student class not to share research ideas too liberally as sometimes professors steal the ideas of students. It is helpful to get input on ideas, however, so using a mechanism such as an NDA would seem appropriate in preventing this. I would imagine that most of the most vulnerable type of interaction is face-to-face meetings. In this case what would be nice is to audio-record a quick declaration of confidentiality and then the subsequent conversation. If an IP lawyer made a short script that could be read and recorded then a smart-phone could be created that would play the "this a confidential conversation" audio and then start recording. Any takers?

Wednesday, July 11, 2012

StatWeave in LyX

While it's pretty easy to integrate R into LyX documents (using Sweave or knitr) I hadn't found any native way of integrating Stata code. I've hacked StatWeave (which allows Stata, R, and a bunch of other stat languages) to work in my LyX (in Windows 7 with LyX 2.0.4) and thought that others might like this as well. Here's the steps:

  1. Copy statweave.module to the "Resources/layouts" of your user directory (See HelpAbout LyX).
  2. Open LyX and go to  Tools→Reconfigure
  3. Restart LyX and go to  Tools→Preferences
  4. On the left choose File HandlingFile Formats.
  5. Click the New... button and then fill in  Format: StatWeave, check Document format, Short Name: statweave, Extension: swv. Then Apply.
  6. On the left choose File HandlingConverters
  7. Using the drop-down menus, create a new one from "StatWeave" to "LaTeX (pdflatex)" with the command "statweave --target tex $$i" (make the changes and then click "Add" in the upper right). Then Save.
Once you get that setup you should be able to compile: stata-test-bq.lyx. Notes:
  • LyX disables the enter key from inserting a line-break in insets (anyone who knows about this please let me know why or a work around). You can paste them, though, so copying several lines of code into a Stata Code Chunk insert works fine (if you are writing it in LyX then just copy a line break from somewhere and paste as needed). 
  • If you don't want to use the custom insets you can always just use plain TeX code (e.g. You can always just use TeX code inserts \begin{Statacode} ... \end{Statacode} or \Stataexpr{...})
  • The StatWeave manual is helpful for the non LyX part.
  • When configuring your Stata executable do not include "do " before "%codefile%".
  • You may need to edit the MiKTex path given the default ones are for version 2.6.
  • If you want to use relative paths for input files then you can pass the directory of the LyX file to Stata as an environment variable. In the converter replace the statweave command with "set orig_PWD=$$r && statweave --target tex $$i" (then ModifySave). Then in Stata you can change do that directory with -global orig_PWD =trim("`:environment orig_PWD'")"- (it will have the final slash). Don't cd around in the program or StatWeave wont' know where it's graphics files are for conversion.
Suggestions very appreciated.

Update: I've made a newer version the just focuses on Stata and allows for code block options. You can easily show figures now right-clicking in the block, choosing opts, and then putting in a string like fig, height=4.5in, width=9in, dispw=4in

Monday, May 31, 2010

Exchange for book site credits

I use several online book-exchange sites (e.g. PaperBack Swap and BookMooch). I find, however, that often my demand for books and my available credits will be on different sites. What would be great is if there was a credit-exchange for these kinds of scrip (like a foreign exchange market). On this new website, users could register their book-exchange site usernames and then put in orders for trading the different types of credits. It could operate on varying levels of trust. In all of the level, the minimum requirement is that the book-exchange sites allow people to transfer credits.

  1. If the credit-exchange was completely trusted, it could provide the settlement, logging into the book-exchange sites and transferring the credits simultaneously. This requires giving the credit-exchange one's passwords to the other sites.
  2. If the credit-exchange was partially trusted it could provide escrow services. Once a trade was matched, the two parties would transfer the credits to accounts held by the credit-exchange. Once both parties had transferred, the credit-exchange would complete the trade.
  3. If the credit-exchange wasn't trusted, the site would just inform the matched parties of the trade details and let them settle it themselves. For this latter to work well, the book-exchange sites would probably also have to provide messaging services and some form of feedback/reputation to discourage reneging.

Surveying the current available book-exchange sites (discounting "swap" sites that don't use credits) the two largest sites do allow transferring credits, so this idea is feasible. And it would be good for the sites involved as it would stimulate more usage (regardless of how the "exchange rate" was). Anyone want to try to set this up? For the economists out there, could you use insider access to data on trades in a clever way?

Book exchange sites
Site Credit price Credits per book Coverage Transferable credits Private messaging Reputation/ Feedback
PaperBack Swap $3.45 1 USA Yes Yes No
BookMooch NA 1 Intl Yes ("smooches") Yes Yes
Bookins $0.60 varies (3-8) USA No (user names hidden) No No
What's on my bookshelf $2.00 varies (1-2) Intl No Yes Yes
Readers United $0.50 varies (5-30) Intl No Yes Yes
Title Trader Yes ($3-4) varies (1-2) Intl No Yes Yes
2swap NA 1 USA No Yes No

Update: After searching the forums of PBS and BM, it appears that there are only a few people who ask for trades, so the market is probably fairly small.

Friday, April 27, 2007

Russian books a bit easier

One difficulty I've heard of for the casual reader of many Russian novels, such as those by Dostoyevsky, is the multiplicity of names for the same characters. Well, with a list of names for each character, free translations (from Gutenberg), and some find-and-replace, I've made versions of Crime and Punishment and Brothers Karamazov where each character is only referred to by one name. Of course it looses something with the simplification, but if confusion of the names is a real problem maybe someone will find this useful.