Adding local latex packages to texlive in ubuntu

When having custom packages to use for a document, you need to put them next your latex files when compiling. This causes problems when you need these packages for many documents. The solution is to add these packages to the latex installation. This can be done for the current user by creating a latex tree in the $HOME/texmf directory. The structure of this directory needs to follow the texmf-local described here. This directory is searched by default by the texlive installation.

Portable Meld on Windows

Often I need to compare and merge files or folders together for changes between them. Gladly, many applications exist for such a task. This can come as using the diff command line tool, sadly it is not visually pleasant. Therefore several visual interfaces to ease the comparison side by side. Examples of such are:

• kompare
• kdiff3
• meld
• beyond compare
• winmerge

A detailed comparison between the them and others can be found here. My personal favorite of these is meld (specially for folder comparison). On a Linux system, it is quite easy to set up, specially if it is deb bases because there is a package for it. A problem shows up when wanting to use meld in Windows. After some googling I found this guide showing how to install it for windows. Using this guide, I managed to get it to work, but I prefer a portable installation to take it anywhere.

Following are the steps I followed to get a portable version of meld up and running. Take care that performing the steps need administrative rights, but it can run anywhere.

1. Create a directory to contain meld binaries and its dependencies. Let us reference to it as <meld install dir>
2. Download portable python 2.6, and install it under <meld install dir>\Python26
3. Add the portable python installation to the registry. This is done by opening the portable python interpretor an running the script from here
4. Download the GKT+ 2.2 run time all in one bundle from here. Extract it to <meld install dir>\gtk+
5. Download an install the latest PyGTK libraries. These are the PyGTK, PyCairo, and PyGObject modules. This will install to the correct python based on the registry entry from step 3.
6. Download meld 1.5 from here. Extract it to <meld install dir>\meld-src
7. Create a file “<meld install dir>\start-meld.bat”. The content of this file is:

   @echo off
   set PATH=%CD%\Python26\App;%CD%\gtk+\bin
   python meld-src\bin\meld
   

And that is it, run start-meld.bat and enjoy 🙂

A DRY KISS

Absolutely Don’t Repeat Yourself and Keep It Simple, Stupid! (A DRY KISS) .

DRY and KISS are common concepts between software developers, but are very difficult to follow. Specially when you start with some idea, run with it, and it starts to layer up in complexity. Without even recognizing it goes out of control. With each of there added layers, you might find yourself repeating things without even recognizing the fact.

Drawing the very fine line between simplicity and and complexity is a very difficult task. It gets even more difficult without taking a step back to look at the big picture, the original aim, the purpose.

This is why an external review of the matter by a 3rd party is often vital. It gives perspective that helps the general good. Such a review process is often excruciating, painful, and long. But in the end of day, without it there is no real gain.

A a final note, I believe that A DRY KISS is something to try and follow in general.

Proposal: Hierarchical Unit Testing for less woes

Disclaimer: The following content is of an idea which might be already existing, so please feel free to comment if you find it in existing testing frameworks 🙂

A common problem I face during test-driven development is when having to tests (for ease i’ll call them T1 and T2). T1 and T2 test the functionality of F1 and F2 respectively. The fun starts when F2 is dependent on F1, and both tests fail. In a test suit of thousands of tests, it would be hard to see the correlation from only the report of running the test suite.

It gets worse when the dependency between tests is not direct. For example  if F3->F2->F1 (-> means “depends on”), where only T1 and T3 fail and T2 is fine. These relations need a very careful reviewer of the test results, and one who is well aware of the relations between components and their tests. This might be impossible in HUGE teams spanning many components.

But what if the testing framework was aware of these relations, and was able to report them. Such relations could be represented for example as new annotations in JUnit. With building the relation graph between tests it would be easy show that T2 might have failed because T1 failed.

So this is an open call, any one willing to try creating this, if it is not already out there?

Google instant search: A better spell checker and calculator

One of my main uses for Google search is not the searching, but has slowing move towards often queries just for the sake of spell checking. Recently Google released its instant search. With that, I can easily see my spelling mistakes as I type, no more need to press enter :).

What I found by accident and found it even more useful is using instant search as a calculator. Some of you might know the Google easter egg about the answer to life universe and everything. Google calculator has been part of Google’s search features for some time now. With the instant search, the results of the calculator is also changed live. A cool and useful example (that was the way that made me discover this) is the binomial coefficient function. Give it a try and play around with it here.

Happy lazy calculating 🙂

Layar POI service using Google App Engine

Layar is a cool handheld augmented reality application that allows to overlay “Layers” on the image seen by a handheld device’s camera. One can think of these layers as content that is seen based on your current location. This allows to overlay digital data over actual live imagery.

The Layar API depends that the source of data (Points Of Interest) is a RESTful web service, that sends an HTTP GET request, and expects back  a JSON object. The details of the GET parameters, and the required JSON object could be found here in the layar API documentation.

To create a layer, one needs to provide such a service that provides the Points Of Interest (POI). Such a service could easily be written and provided by google app engine. This article will discuss how to do so using the google app engine python SDK, and is highly depends on the getting started guide for google app engine through python. Sections 1-4,6 are sufficient for understanding of the coming content.

Handling Requests

Since requests from layar to the POI web service come in the form of GET requests, the parameters can simply be accessed in the request handling method through:

self.request.get(parameter_name)

POI Response

The POI response that layar expects is a json object. The API place many restrictions on the response. The first comes in the content type of the HTTP response. Thus, it has to be set as follows:

self.response.headers['Content-type'] = 'text/javascript; charset=utf-8'

Another restriction is in JSON object returned as a response, where almost all the fields are required by layar. To make things simpler, the object is to be represented as a python object. This could be later converted to a JSON object. The JSON object responded with could be represented as follows (as a minumum):

{'layer':'layer name', 'hotspots':list_of_POI, 'errorCode':0, 'errorString':'ok'}

Where hotspots is a list of POI object. These object could be represented as:

class POI:
    def __init__(self,poi_id,title,lat,lang):
        self.actions = []
        self.id = poi_id
        self.imageURL = None
        self.lat = lat
        self.lon = lang
        self.distance = None
        self.title = title
        self.line2 = None
        self.line3 = None
        self.line4 = None
        self.attribution = ""
        self.type = 0
        self.dimenion = 1
        self.transform = {'rel':True, 'angle':0, 'scale':1.0}
        self.object = {'baseURL': ""}

From python to JSON

As python follows a batteries all included strategy, there are libraries that convert python dictionaries to JSON objects. Even though google app engine uses python, the “json” module is not available. Thankfully, an equivalent one is found through “from django.utils import simplejson”. It could be used to convert a python dictionary (that looks frightfully like a JSON object) to a JSON object string as follows:

simplejson.dumps({'layer':'guc', 'hotspots':poi_list, 'errorCode':0, 'errorString':'ok'})

The problem with simplejson is that it only takes dictionaries or lists. Thus, this poses a problem when using the POI class mentioned earlier. Again, python comes with a rescue, where a dictionary representing the object could be obtained as follows:

poi = POI('C1','C1',29986707,31438864)
poiDictionary = poi1.__dict__

Putting the code together

Now that we have the POI object, and the conversion mechanism, we are ready to have a request handler for layar requests. An example of static POIs is as follows:

from django.utils import simplejson
from poi import POI
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

class POIHandler(webapp.RequestHandler):
	def get(self):
		self.response.headers['Content-type'] = 'text/javascript; charset=utf-8'
		#latitude and longitude is an integer will be divided by 10^6
		# so take care of accuracy after the division
		poi1 = POI('C1','C1',29986707,31438864).__dict__
		poi2 = POI('C2','C2',29986744,31439272).__dict__
		poi3 = POI('C3','C3',29986995,31438923).__dict__
		poi4 = POI('C4','C4',29987153,31439245).__dict__
		poi5 = POI('C5','C5',29986326,31438810).__dict__
		poi6 = POI('C6','C6',29986688,31438569).__dict__
		poi7 = POI('C7','C7',29986442,31438370).__dict__
		pois = [poi1,poi2,poi3,poi4,poi5,poi6,poi7]
		# final getPOI response dictionary
		d = {'layer':'guc', 'hotspots':pois, 'errorCode':0, 'errorString':'ok'}
		self.response.out.write(simplejson.dumps(d))

application = webapp.WSGIApplication(
		[('/',POIHandler),
		 ('/getPOI',POIHandler)],
		debug=True)

def main():
	run_wsgi_app(application)

if __name__=='__main__':
	main()

Updates

With the new layar 4 api coming soon, and the new features being in beta2, un update of how to add one of my favorite new features is needed. This is having different actions on the entire layar. It can be easily done by adding an action attribute to the final getPOI response dictionary (see code above). An example of the addition is as follows:

d['actions'] = [{'uri':'http://google.com','label':'action on the entire layar'}]

SVN post-commit hook cronjob

Few SVN hosting sites provide ssh access to their svn servers. Such is needed to create hooks, like a post-commit hook that sends an email with every commit. Such hooks are often required by the development team. This article discusses the creation of a cron job that polls the SVN repository to simulate a post-commit hook that sends an email with every commit.

Setting up sendEmail

sendEmail is a nice tool to send emails via command line. This is needed to script in the cronjob sending an email. On ubuntu, install it from apt-get by:

sudo apt-get install sendemail libio-socket-ssl-perl libcrypt-ssleay-perl

The latter 2 packages are needed for connecting to an SMTP server that uses a secure connection (like Gmail, which is used in the example).

The post-commit hook script

The following is a python script, that could be added as an entry in the cron table

#! /usr/bin/env python

import os,re

log_file = LOG_DIR
svn_log_seperator  = '------------------------------------------------------------------------'
svn_dir = SVN_DIR
mail_filter = '[special mail identifier in subject]'
message_file = '/tmp/commit_mail'

mail_from = MAIL_FROM
mail_to = MAIL_TO

smtp_server = "smtp.gmail.com:587"
smtp_user = SMTP_USER
smtp_pass = SMTP_PASS

def send_mail(subject,mail):
    mail_file = open(message_file,'w')
    mail_file.write(mail)
    mail_file.close()
    mail_command = 'sendEmail -f %s -t %s -u "%s" -s %s -xu "%s" -xp "%s" -o message-file="%s"' % (mail_from, mail_to, subject, smtp_server, smtp_user, smtp_pass,message_file)
    print mail_command
    os.system(mail_command)

print "analyzing old log file"

old_log_f = open(log_file)
old_log = old_log_f.read().split(svn_log_seperator)
old_log_f.close()

print "updating svn"
os.system("/usr/local/bin/svn update %s" % svn_dir)
os.system("/usr/local/bin/svn log %s > %s" % (svn_dir,log_file))
print "got new log"


new_log_f = open(log_file)
new_log = new_log_f.read().split(svn_log_seperator)
new_log_f.close()

delta = set(new_log) - set(old_log)

for commit in delta:
    l = [x for x in commit.split('\n') if x]
    details = l[0].split('|')
    subject = "%s %s %s" %(mail_filter,details[0],details[1])
    f_diff=os.popen('/usr/local/bin/svn diff -c %s %s' % (details[0], svn_dir))
    diff = f_diff.read()
    f_diff.close()
    mail = "%s\n%s" % (commit, diff)
    send_mail(subject,mail)

Perquisites before running the script

Having a working copy of the svn reposity
Running the following command once before the script:

svn log > $PATH_TO_LOG_FILE

Developer vs. User <=> Driver vs. Mechanic

Hypothesis: The Developer User relation is equivalent to the car driver mechanic relation.

Proof:
Through my few numbered years of driving in the lovely streets of Cairo, i faced car troubles every now and then. In the case of such circumstances, I head to my trusty mechanic. From here, starts the problem, when I am driving my car, and i notice a problem, it is very difficult to reproduce by the mechanic. Such a reproduction often happens when it’s too late.

Here, comes the fitting analogy, mechanics are car using experts. They manage to use cars in an expert manner, one that hides any possible problems in the car.

The same maps to the computer software users, where they see problems that developers are unable to reproduce, simply because of their computer usage habits that are so much different from that of end users.

Quod Erat Demonstrandum (QED).

Thus, the lesson to learn here is, every time you want to feel like a user, think of yourself driving your car, and weird things starting to happen.

Life: A python perspective

from people import friends

while not dead:
    try:
        something_new()
    except LifeIsToughError:
        pass

Breaking up with your code

A famous quote from The Cathedral and the Bazaar is “Plan to throw one away; you will, anyhow”. This never hits a programmer with it’s true meaning until they have to throw aways a large bit of code base that they invested time and effort in. Coders tend to build an emotional connection towards every letter of code they wrote (I know i sure do). On the other hand, needs arise where obstacles render current software “unfixable”. This confronts with the painful fact that change must be done. Taking an objective stand point in that decision is a tough decision, but facts are fact in face of improvement. In the end of the day, breaking up is tough but a fresh start would most likely end up in more bulletproof code.