Custom bibliographies

I've recently encountered in more than one occasion a publisher that will ask for references in a particular format (If there are three or less authors then all should be mentioned in the references but only one +"et al." in the text unless there are two, and if there are more than three then it must be the first three authors then "et al." in the references and only one +"et al." in the text).

This is the "prefered format" (as of now) in ADS, but I haven't found a bibliography style file (.bst) for Bibtex that meets that criteria. If you try to change your favorite .bst file, you'll find that it uses an odd language, and some people online might have some suggestions on how to change it, but I couldn't easily find how to edit it.

Instead I found a mention on an elusive package called makebst. After a couple of broken links I found it (via here).

Download it, unzip it, then run:

latex makebst.tex

It will then ask a great amount of questions for every possible situation. In my humble opinion this should be a file provided by the people in charge of editing the publication. But this information will be useful for those writing their own book (e.g. Thesis) and need their BibTeX output in a certain way.

As for the makebst.zip sadly I can't seem to be able to attach a file to this post, and as nothing is eternal, I don't know where to place it where it will be more reliable than the link already provided.

Compiling LaTeX in gedit with one key

This might be useful for a very small niche, but just in case anyone else needs it, well I hope you land here.

The scenario is simple: You like using the popular gedit text editor for writing your latex presentations and papers (as I write this gedit is the default text editor in ubuntu and many other linux distributions related to GNOME). To see changes fast you want to have a keypress that will update the pdf file which you may have open in your favorite viewer (and love how you don't have to do anything for it to refresh automatically).

If you go to Tools > Manage External Tools, you can run your own commands by pressing certain keys. Using the Enviroment Variables that can be found in the documentation, you can write a simple little bash script that will run both latex and dvipdf with only one press key:

#!/bin/sh
TEXNAME=$GEDIT_CURRENT_DOCUMENT_NAME
DVINAME=${TEXNAME%".tex"}".dvi"
latex $TEXNAME
dvipdf $DVINAME

Note that to change the extension from .tex to .dvi I used the string DVINAME=${TEXNAME%".tex"}".dvi" thanks to root45 at askubuntu.com for this.
I was first compelled to use: ${$TEXNAME:0:${#texname}-3}"dvi" but this substitution is not accepted by gedit (although it's perfectly valid in bash).

So in the end this is how your external tool should look like:

After that well define the shortcut key, and the applicability only to LaTeX files. You may want to set the output to be displayed in the "Bottom Pane" to see errors when there are some. I just prefer to save screen space and if it doesn't update the document then I run these commands on a terminal to troubleshoot.

LaTeX: ! Undefined control sequence

This short entry is to address a common annoyance which I've found difficult to tackle and to which I haven't found a straight answer online.

When using BibTeX reference copy/pasted from the ADS it will often have special "control sequences" for journal names.

The error comes when trying to compile and the error is similar to this:

! Undefined control sequence.
l.10 \newblock {\em \aap

where in your case \aap might be changed by the offending citation's journal.

This can be fixed by creating the definition or simply by removing the control sequence (\aap), but the whole point of LaTeX is the use of templates to standardise publications without having the author worry much about it. In this particular case I also wanted something as general and standard as possible... there might be a better answer to what I found, but for the time being this seems to fulfil those requirements good enough.

I could not find this article until after the fact, but this summarizes the solution: Journal Abbreviations used in the ADS BibTeX entries. And what it says there is you should install the AASTeX package, but in case you don't want to do that the simplest solution is to save the aas_macros.sty file to your project folder and include them in your main .tex file with the line:

\usepackage{aas_macros}

To do the proper thing and install the AASTeX package, you can follow the wikibooks guide on Installing Extra Packages or consulting the README file in the package distribution, I don't go into the details of package installation as it depends greatly on the software you use to write and compile LaTeX.

Making long exposure images with star trails out of a timelapse with python

So I stumbled upon this image online (thank you jannne):

Which is simply amazing. Author said it was simply stacked frames from this video.

Although that image was amazing enough, it still might leave some of us wanting something of a higher resolution, so that's when I decided to spend an important amount of time on this. First, if you want to do exactly the same thing (or almost the same with another set of images), the original videos and photos can be found in this great NASA page¹ (right hand column, where it says High Resolution).

Note that the method used here can be also applied to a normal session of astrophotography where instead of leaving the shutter open and discarding the possibility of doing an amazing timelapse video, one might just take many images and join them together at will later. This will also save you from the risk of your image being overexposed by an unexpected source of light.

This time I went with python mostly due to it's very laid back nature (this is of course a leisure project). Take into account that this process deals with a number of images in the hundreds and a total size in the Gigabytes range (of course more if you happen to have RAW images).

In Python working with pictures, in particular joining two of them is rather easy. In this case the lighten² function of the ImageChops module of the Python Image Library(PIL) works very well.

import Image
from PIL import ImageChops #for lighten function
im1=Image.open("1.jpg")
im2=Image.open("2.jpg")
finalimage=ImageChops.lighter(im1, im2)
finalimage.save("1and2.jpg","JPEG")

(for those completly new to python, you can both run "python" in a terminal and type in each one of these commands, you can put then in a .py file and run it "python lighterpixels.py" or with the proper header at the first line of the file #!/usr/bin/python put it in a shell file so it can be executed this way: "./lighterpixels.sh", this is in linux, for other OSes it's similar)

Also, for those unfamiliar with python, in order to read a directory you can use os.listdir. This is:

import os

files = os.listdir("HighRes/") #create a list with all the filenames in the HighRes/ folder

There are better ways to only get the *.jpg files in order to avoid errors when this tries to read non-jpg files, or to allow your program to accept input from the command line, but I'm trying to keep this simple, so feel free to be more elegant and add more versatility to your programs.

So when put together all of the code you need is:

from PIL import ImageChops
import os, Image
files = os.listdir("./HighRes")
finalimage=Image.open("./HighRes"+files[0])
for i in range(1,len(files)):
  currentimage=Image.open("./"+files[i])
  finalimage=ImageChops.lighter(finalimage, currentimage)
finalimage.save("longexpo.jpg","JPEG")

This will take some time as it is hundreds of images so you might want to add a progress bar to that or have some patience with it, but beauty of it is that it can be run in any computer with enough RAM to open two images at a time (hmmm... it could even be ported to a smartphone...)

So here's the final product:

High resolution [4256x2832] version (with hosting courtesy of xHaZxMaTx's imgur pro account) is here: http://i.imgur.com/FpIs2.jpg

Now, note that the lighter function only compares and gives out the lightest pixels in every image, so this is good for our moving point sources to make streaks. However if you pretend to make a dark object lighter and sharper by combining many images, then this is not the function we want. User urquan over at reddit suggested loading the images into 64-bit numpy arrays, and working with those. Here's his final code:

from PIL import ImageChops, Image
import os, numpy

imagedir = "HighRes"
gamma = 2.2

def getData(file):
   image = Image.open(file)
   data = numpy.asarray(image, numpy.float32)
   data = ((data/255) ** (gamma)) * 255 # invert gamma
   return data

files = os.listdir(imagedir)
imagedata = getData(imagedir+"/"+files[0])
count = len(files)
for i in range(1, count):
   print "Processing %d/%d - %s" % (i, count-1, files[i])
   currentdata = getData(imagedir+"/"+files[i])
   imagedata = imagedata + currentdata
   del currentdata

# Normalize values
imagedata *= 255.0/numpy.max(imagedata)
# Gamma correction
imagedata = ((imagedata/255) ** (1/gamma)) * 255

finalimage = Image.fromarray(imagedata.astype(numpy.uint8))
finalimage.save("allblended.png")

This method is slightly more complicated and slower but is more correct and you don't loose information with it. However PIL is not very good at handling anything other that 8-bit pixels, so I suggest using uquan's method but then using something like pyFITS to save it and edit the final result with some visual editing program like GIMP. But I might do that some other time.

Another option to test out your results before exporting them to a lossy format (jpg, png, etc) is to type in one by one these commands into python (running python in a terminal) and playing with it before giving it to PIL. For example, multiplying all pixel values times two:

imagedata = imagedata*2
finalimage = Image.fromarray(imagedata.astype(numpy.uint8))
finalimage.show()

the show() command will show you the result but you will still have the information correctly stored in the imagedata array. so if you don't like it you can keep on altering it. Another suggestion I have is that instead of normalizing to the maximum value of the pixels, is to do it as a function of where the 99 percentile of the population is at... but hey, let's keep this simple.

¹Images used for this project are courtesy of the Image Science & Analysis Laboratory, NASA Johnson Space Center. They can be found here.

²I first tried using PIL's blend function, but since this is a low light picture and PIL isn't very good with high precision numbers, jolly swagman at stack overflow suggested I use the lighter function instead.

Plotting a grid on a projected gnuplot graph (e.g. Mollweide)

Following the first post on how to plot a map on gnuplot, and then how to plot it with a Mollweide projection (all in the same post), I decided to dedicate a post on how to do a grid overlay.

Mostly because I first thought the best idea would be to simply plot a file with data points only on the meridians and parallels. This is a bit of a kludge and doesn't allow you to easily add styles to the lines.

The obvious solution: to plot the function that describes the lines of our grid. But one really nice thing I stumbled upon (in this nice blog called Gnuplot tricks) is using the "for" loop to plot a function n times:

f(x,i)=i*x

plot [-pi:pi] for [i=1:10] f(x,i)

Otherwise you'd have to plot each one of the lines. This way you only have to issue two or four commands (depending if you want a different pattern for a "minor grid").

We'll use the Mollweide projection parametric functions that we copied from Roberto's blog(the ones mentioned in my previous post). Now I almost lost my head trying to express these parametric expressions as normal functions so I could just write "plot y(x)" (the parametric ones depend on other variables and give you an expression of x,y pairs instead of a function y(x) ).

Enter the command: set parametric. This will allow you to do the same as with columns in a data file, which is to plot on function (y=f(t)) against another (x=(g(t))¹ .

So this is the code for a Mollweide grid:

set parametric

plot [-pi:pi] for [i=-5:5] mwx(pi*i/10,t),mwy(pi*i/10,t) with line linetype -1, \

for [i=-5:5] mwx(t/2,pi*i/5),mwy(t/2,pi*i/5) with line linetype -1

Note that we use "with linetype -1" for solid black lines and "with linetype 0" for dotted black lines. Gnuplot is programmed so that the simple way is to use predefined styles for lines/points/etc. However one can specify color with the somewhat obscured command "with lines linecolor rgb '#000000'" or abbreviated "w l lc rgb '#000000'".

Another very important thing to note is that the range of the dummy variable t is only set at the beginning of the plot command and cannot be issued multiple times (cannot be issued after the comma). I considered using multiplot for overriding this but it was a very bad idea when dealing with the map previously plotted.

Oh right, you'll probably want the map to be behind those lines, but the range of the plotted functions' dummy variable (t) must be issued right after the plot command, so in order to do the full plot of map and grid this is the final plot command:

plot [-pi:pi] "gnuplot.dat" using (mwx(bconv($2),lconv($1))):(mwy(bconv($2),lconv($1))):($3/1E7) with points palette, \
for [i=-2:2] mwx(pi*i/6,t),mwy(pi*i/6,t) w l lt 0,\
for [i=-3:3] mwx(t/2,pi*i/4),mwy(t/2,pi*i/4) w l lt 0, \
for [i=0:1] mwx(t/2,pi-2*pi*i),mwy(t/2,pi-2*pi*i) w l lt -1 lw 3, \
"grid-labels.dat" using (mwx($2/180*pi,$1/180*pi)):(mwy($2*pi/180,$1*pi/180)):3 with labels

I tried plotting the grid labels more elegantly but failed due to constrains in the system, so I ended up just plotting a data file that contained the coordinates and text for a label. Also note I decided to change the inner linetype to 0.

Here's the final result:

¹I tend to try in all possibilities to give a reference to what helped me, but in this case it was hours of droning with my favorite search engine until I stroke the right phrase that lead me to a gnuplot faq page that I didn't quite find explanatory but gave me the clue to search the word "parametric" (hoping for the manual but falling on this lanl webpage I keep bumping into).

Plotting a 2D map with colors for a 3rd column of values. (and Mollweide project it)

In case you're new to the scientific community, gnuplot is almost a standard tool in plotting scientific data. It's free (as in open source) very robust and extremely versatile. However this versatility might be to blame for it's lack of user-friendliness, there exist some GUIs (Graphic User Interfaces) but they're not robust and go against the concept of being able to control anything you want in your graphics (and also automating graphs, but that is for another post).

However sometimes you want to do something very specific, and this is the case for this post, I have some XYZ data in which X and Y are simple coordinates (galactic in this case) and Z is a value for each one of those points.

In my case I have a file with 196608 sets of points each with it's value, of course if you try to get any spreadsheet software to graph that it'll probably crash, make your computer slow or simply refuse to accept that many lines.

The data file looks something like this:

357.77228 52.416012 2.0093570

358.66337 52.416012 2.0057610

359.55446 52.416012 2.0027640

0.44117647 52.029727 1.9932720

Googling you might stumble upon solutions that rely on plotting "with p3dm", sadly these are not what you need, as this command is meant to plot grids and will therefore need sets of points that make up each rectangle, each separated by spaces. You'll probably get frustrated with the error "Hint: Missing blank lines in the data file? See 'help pm3d' ".

The final answer to my problem I found here on kiko's blog. Which is summarized in two lines:

gnuplot> set view map 

gnuplot> splot "test_map.dat" with points palette pt 9 

If you want to change the colors following the directions from this site(yes, they're using pm3d), you can just input the command:

set palette rgbformulae 22,13,-31

Play with those numbers all you want.

Note: I later found (at the Gnuplotting blog) a better, more pleasant gradient that is easier to modify if you know RGB hex codes (I didn't change it for this post so if you want to see it test it out):

set palette defined ( 0 '#000090',\

1 '#000fff',\

2 '#0090ff',\

3 '#0fffee',\

4 '#90ff70',\

5 '#ffee00',\

6 '#ff7000',\

7 '#ee0000',\

8 '#7f0000')

That is before splot (or after and then issue the command "replot").

And here's the final result for me (before adding titles but after tweaking xrange and yrange):

Some basic things for those new to gnuplot are:

set xrange[0:360]

set yrange[-90:90]

set title "Graph Title"

set xlabel "Latitude (b)[degrees]"

set ylabel "Longitude (l)[degrees]"

set key off

Well, I know this post (and probably many more to come) says nothing new, but in this age of over-information, I hope I blabbered on enough so that if you're looking for this specific problem it might just pop up in your online query (and also to place higher up in your search engine the place where I found the solution).

P.S.: Mollweide projection

Something very nice to look into is using a Mollweide projection, this can be done in gnuplot thanks to Roberto's function:

Just copy/paste this block of text into the gnuplot command line:

mwhigh(x) = 1.0 + -0.919061*(abs(1.0-x))**0.674635

mwmed(x) = mwhigh(x) -0.0807765 + 0.161136*x -0.0796311*x**2

mwlow(x) = mwmed(x) -3.53551e-05 + 0.000645749*x

mwst(x) = x < 0.2 ? mwlow(x) : (x < 0.9 ? mwmed(x) : mwhigh(x) )

mwt(b) = b>0 ? asin(mwst(sin(abs(b)))) : -asin(mwst(sin(abs(b))))

mwx(b,l) = 2.0*sqrt(2.0) * l * cos(mwt(b))

mwy(b,l) = sqrt(2.0) * sin(mwt(b))

and that'll mean you have the mwx and mwy functions to change your coordinates, I used them this way:

splot "gnuplot.dat" using (mwx($2,$1)):(mwy($2,$1)):3 with points palette

Note one thing: when you're just plotting a column in your data file, the number of the column will suffice (1:2:3), however, when you're using functions inside the plot or splot command you have to put the functions inside parentheses and the columns have to be called out with dollar signs ( (functionX($1):functionY($2):3 ). This took me quite a while to figure out because it gives you no errors and just generally misbehaves. Sadly I can't remember what tipped me off to this, but part of it was the gnuplot manual, which is big, but a must have reference.

You may have noticed that those equations will not work for my data since they're meant for coordinates that go from: [-pi/2,pi/2] for b and [-pi,pi] for l.

So we just need to change the coordinates adding two more functions:

bconv(b) = b*pi/180

lconv(l) = (l<180) ? l*pi/180 : (l-360)*pi/180

note that in the second one I had coordinates that went from 0 to 360, which is odd, but it was a good example of how to use conditionals.

and now we can change the plotting line to:

splot "gnuplot.dat" using (mwx(bconv($2),lconv($1))):(mwy(bconv($2),lconv($1))):3 with points palette

So the now beautiful result is:

note that for cosmetics (and since the axes mean very little now) we can remove the border, ticks and labels:

unset border

unset xtics

unset ytics

Well, I guess that's that for a long P.S. section, I hope it makes sense. If anyone wants homework: post in the comments how to overlay a couple of central axes with the correct values for longitude and latitude!

(Also feel free to ask for help or just let me know that this post somehow helped you, I'd love to know).