Thursday 27 December 2012

Amazon EC2 Basics (Part 1.2)

To install a regular Apache version of Tomcat 7 (c.f., the version found in the Amazon repository described in Part 1):
  • If you installed Tomcat by following Part 1, stop the currently running Tomcat service:
    • sudo service tomcat7 stop
    • sudo chkconfig tomcat7 off
    • chkconfig --list (check that everything is off)
  • Get the Apache Tomcat repo and untar it into the /opt directory in your filesystem:
    • cd /opt
    • sudo wget -c (Go to the official Apache Tomcat website and get the URL of the binary version you want to work with. I have chosen apache-tomcat-7.0.34 here.)
    • sudo tar xvfz apache-tomcat-7.0.34.tar.gz
    • sudo ln -s /opt/apache-tomcat-version /opt/tomcat (This is optional. It creates a symbolic link to the directory.)
  • Create a user account to run Tomcat. If you installed Tomcat by following Part 1, then skip this step. Otherwise, run the following commands: 
    • sudo /usr/sbin/groupadd -g 91 -r tomcat
    • sudo /usr/sbin/useradd -c "Apache Tomcat" -u 91 -g tomcat -s /sbin/nologin -r -d /opt/apache-tomcat-7.0.34 tomcat
  • Change file permissions:
    • cd /opt/apache-tomcat-7.0.34
    • sudo chown -R root:apache-tomcat-7.0.34 *
    • sudo chmod -R go-w bin lib
    • sudo chmod -R g+rw conf logs temp webapps work
  • Create a start/stop script (preferably):
    • sudo vim /etc/init.d/tomcat
    • Type in:
    • #!/bin/sh
      # Baseline Apache Tomcat start/stop script
      # chkconfig: 345 80 20
      # description: Apache Tomcat
      case $1 in
              /bin/su -s /bin/sh -l tomcat -c /opt/apache-tomcat-7.0.34/bin/
              /bin/su -s /bin/sh -l tomcat -c /opt/apache-tomcat-7.0.34/bin/
              /bin/su -s /bin/sh -l tomcat -c /opt/apache-tomcat-7.0.34/bin/
              /bin/su -s /bin/sh -l tomcat -c /opt/apache-tomcat-7.0.34/bin/
      exit 0
    • sudo chmod go-w /etc/init.d/tomcat (Change permission of script file.)
  • Start Tomcat using the script:
    • sudo service tomcat start
  • You should be able to see the Tomcat manager on port 8080 of your instance! If you don't see it, make sure you change your Amazon security group permissions to allow access to port 8080. 
  • If you want to start Tomcat automatically on instance boot, type:
    • sudo /sbin/chkconfig --add tomcat
I personally chose to install the Apache version and not the Amazon version, because:
  • The Amazon version is a customised and only certain builds are available. 
  • The Amazon packages get upgraded when you update the system. This might break things.
  • Most of the help resources you find online refer to the Apache version.
(This post is based on this article.)

Monday 17 December 2012

Raspberry Pi store!

The folks at the Raspberry Pi Foundation have just released the Raspberry Pi store! There are only a couple of apps a the moment, but this is a fantastic idea and I'm really excited by the possibility of developing Python-based games for the Raspberry Pi!

Friday 7 December 2012

Smoothing edges in Photoshop

How to smoothen edges in Photoshop (something I'm going to have to do a lot for my next project):

  • Select magic wand tool 
  • Click outside of object
  • Select inverse
  • Add layer mask
  • Add gaussian blur
  • Click levels (Image -> Adjustments) 
  • Adjust to liking

Monday 3 December 2012

AngelHack Toronto : Presentation Disaster

Background :

Saturday morning, I headed off to Extreme Labs in Toronto to create an Android client for, an app which connects amateur rock musicians. Musicians can form gigs, attend auditions, form bands, etc. This was a solo project.

Day 1 :

I built the app up from ground up, coding all day and night. It was going extremely well. Here was when I made a huge mistake- I registered to present before anybody else had registered. This effectively meant that I was the first to present out of about 66 presenters.

Day 2 :

I applied the finishing touches to my app. I was to present at 2pm. The problem was, that the AngelHack guys didn't have a HDMI cable for me to connect my Android device to the screen.  All other devices (including Google TV) were accounted for. This was extremely frustrating- the only way to present would be to use the Android emulator. These emulators are extremely slow and horrendous.

Presentation :

An unqualified disaster. My app was beautiful, but I never got a chance to showcase it. I had only 2 minutes to present. The Android emulator immediately froze up, and there went my chances. I had prepared a few screenshots though, which I managed to bring up. However, 2 minutes just flew by and I had nothing to show.
Damage Control :

  • I sent a mail to the organizers requesting a second presentation due to the technical failures.
  • I sent mails to the judges apologizing for the failed Android emulator. This isn't surprising- Android emulators consume a lot of memory and are bulky. iOS emulators work beautifully well on the other hand.
  • There was a presentation break midway. I did what I had to. I walked up to the seated judges, and personally demo-ed my product on my Android device. Some of them didn't seem interested; others were more receptive. I was satisfied though- my presentation might've been an unqualified disaster, but at least they had seen my product in action.

Lessons Learnt :
  • Have backups : I should have created a powerpoint presentation for this. Although I did have the screenshots, they were not organized in any order and the judges likely got confused.
  • Don't depend on the organizers to make provisions for your product : I should've purchased an Android HDMI cable beforehand. Relying on the Android emulator is possibly the worst thing anyone can ever do.
  • NEVER be the first presenter : I "submitted" my app the first, which by default, made me the first presenter. Presenting early is probably a good thing, but never first.
Conclusion :

Kinda disappointing. I went without any sleep, built a beautiful app, but screwed up the presentation. However, I took a chance and personally demo-ed the product to the judges during the break. I felt way better after that. Anyway, I shall work on Rock88 to completion and release it in the coming few months.

Other comments :

Wifi connectivity was poor, but the organizers (Hackernest) did a great job overall.

Congrats to the winners - Airpost (Met Alam and Angelo beforehand),  Raffi (Met Ian beforehand) and Froogler (Didn't meet anyone from this team beforehand).

Presentation order :

Airpost   - 49 / 66
Raffi       - 29 / 66
Froogler  - 13 / 66

Rock88 Screenshots :

Thursday 29 November 2012

Profiling with VisualVM

Visual VM is quite a useful profiling tool that comes with the jdk (6.0 and above). You just fire it up from the command line (type 'jvisualvm', assuming you have included jdk in your classpath) and deploy your server or application. Double click your server process from the Application window and click the profiler tab. Enable settings to configure which java package to profile.

Thereafter, head over to the CPU tab. Now you are free to interact with your server or application and Visual VM will collect all the relevant data as a visual graph (e.g., identifying bottlenecks, method execution times, etc). This is clearly preferable to using System.nanoTime() calls to monitor method execution times.

Wednesday 28 November 2012

Toronto Data Science Group Meetup 2

Today's presentation topic was on Applied Machine learning, touted as a technical lecture. I was really hoping to see a rough algorithm of some sort, but no such luck. I have compiled a short list of takeaways that I gleaned from the meetup:

  • On classification : words have prior polarity, depending on the context. e.g., the word "like" has a positive polarity when used as a verb and no priority when used as an adverb (c.f., "I like fish" vs "It looks like fish").
  • On SVM's : (Case study 1) Predict how loyal a customer would be based on open ended survey results. Filter out noise and chunk words. Identify features. (Case study 2), where patients take an adaptive survey, and SVM's are used to predict best course of treatment. Also, this data is used in the prediction of insurance premium rates.
  • Other examples : prediction of box office performance, viral tweet prediction, web traffic prediction.
  • Conclusion : Data quality is key. There is no "magic" in machine learning, just hard work. SVM's are good for classification if the identified features are simple and independent. Linear SVM's work well for text classification. Deep neural networks would work better on complex and dependent features. Always start with an optimization objective in mind. On any given machine, you should only be trying to maximize a single optimization objective.

Wednesday 7 November 2012

RISC OS on Raspberry Pi

The raspberrypi foundation has now made the RISC OS, developed in 1987, available for use with the Raspberry Pi. I think I'll give it at try, just for kicks. Here's the announcement, here's a short intro, and here's a forum.

Monday 5 November 2012

Techcrunch Meetup in Toronto

I headed off to the Steam Whistle Brewery today evening to attend the Techcrunch Meetup in Toronto. There was a meandering queue outside the entrance and I spent a good half an hour in the cold before I was ushered in. It wasn't that bad though- I made a couple of acquaintances who were in the vicinity. Weirdly enough, the tickets were being checked manually instead of being scanned (each ticket was associated with a QR code).

The venue itself seemed overcrowded. Each attendee got 2 free tickets which could be redeemed for beer, which is always a good thing (the hors d'oeuvres were horrible though). There were quite a few startups who had set up booths showcasing their products. I actually liked this one called Picatic- they provide a platform for organizing crowdfunded events. Others included Uber, for online taxi bookings, and SoftLayer, a cloud hosting platform.

All in all it was a reasonably interesting meetup. I actually recognized a few familiar faces whom I had seen in previous tech events. The best part, for me, was actually listening to the startup founders describe their products. Networking in such events seems like a rather perfunctory affair to be honest (or maybe it's just me).

Friday 2 November 2012

Bingle Update

I just released an update to the paid version of my Android app, Bingle- it features a new scrabble dictionary, CSW12! This was after I received an email from a customer requesting that I add this dictionary to my app. So come on people! Download Bingle! Request for new features, and I shall try my very best to implement them. This applies to the free version as well.

Additionally, I have decided not to invest more time in developing apps for the Amazon app store for the following reasons :

  • The demand is magnitudes lesser in comparison to the Google Play Store. 
  • Amazon takes about a week (at least) to review every apk before it goes on live. 
  • You cannot redirect any links to the Google Play store from an Amazon app.
  • The Amazon developer console is completely ill designed in terms of navigation and frequently infuriates.
  • Deleting your app from the Amazon appstore is a painful affair.

Friday 26 October 2012

Microsoft Surface

So one of my coworkers bought a surface tablet recently and I got a chance to play around with it. The first thing which struck me was the touchcover integration- it clips so seamlessly. The typing itself feels weird. The keys don't depress, so you don't get that typing "feel". It still beats touch typing on the surface (its is way too wide to touch type efficiently). The kickstand is brilliant and totally inconspicuous. The hardware is top notch and it really is a beautiful piece of work.

The software is lagging though, both figuratively and literally. When you run a bunch of updates, typing becomes a hassle because the lag is almost prohibitive. I'm sure the Microsoft devs will fix such bugs though. The resolution is not as high as you'd expect and many of the apps aren't optimized for the tablet. Plus, Windows 8 RT ain't Windows 8.

Overall, its a unique piece of work. Its pretty expensive though, and I wouldn't buy the current version. Its probably worth waiting another couple of months for the next revision (which runs on Windows 8) while the software devs fix the bugs.

Monday 22 October 2012

Amazon EBS failure brings down Reddit again

Amazon did it again. Their US-East-1 Region EBS experienced "degraded performance" this morning, even affecting applications managed by my company. Makes me wonder why I chose to go with EBS for my side-project. Meh, I'll probably have a bigger reason to complain if I actually get any traffic.

Sunday 21 October 2012

Hacking health

So I attended the hacking health event yesterday. I was thoroughly exhausted by the time I actually got to the event so I couldn't have much fun. Plus, there was no break at all during the idea pitches and presentation. Nevertheless, I jotted down the projects which seemed interesting.

The event itself was well attended, given that it was held on a Friday, when most normal people would be chilling at home or at a bar. I'll admit that I was only interested in hearing about the ideas. I don't expect to actually hack an app this weekend (pretty busy this weekend). Many of the ideas dealt with some form of image processing, and many others were iPhone/iPad based.

Some of the ones I liked: pill identification using images, wound detector for diabetic patients and reducing patient waiting times.

Wednesday 17 October 2012

XBMC on Raspberry Pi

Following the instructions provided here, I was able to get XBMC running on my device (which runs on the "Raspbian" OS). XBMC is an open source media center solution. You can watch movies, view pictures and listen to music using the player. You can also choose to install "add ons" within the app (e.g., CBS News, MTV, TED Talks,  Youtube, etc) and use these to download short video podcasts directly from the servers. What I like about the player is that even when you navigate away from the video you're watching, the video still plays in the background, which is a nice touch.

The installation process itself took a really long time. I had to leave my Raspberry Pi on overnight while the code compiled. Without any overclocking, the menu navigation is somewhat laggy, but not prohibitively so (the interface looks great though). One really cool feature is to enable the XBMC web server (Go to Settings -> System -> Web Server), and use your Android or iOS device as a remote to navigate the interface (you will need to download the free XBMC app on your mobile phone).

The video playback is really smooth though, so if you're watching HD movies on your Raspberry Pi, there shouldn't be any problem. All in all, it's definitely worth installing XBMC if you can afford the storage.

Tuesday 16 October 2012

Toronto Data Science Group Meetup

I had a really enjoyable time at the Data Science Meetup at the Mozilla Toronto office where about 40 of us gathered to network and have technical discussions. Christopher gave a talk on selecting an optimization objective, which was more of a general lecture. Many of us were already familiar with the basics of machine learning, but nevertheless, the Q&A session was quite informative and the lecture was interesting. Afterwards, a whole bunch of us left for drinks. I got to meet some really intelligent people in the meetup. Overall, it was an evening well spent and I'm looking forward to our next meeting.

Monday 15 October 2012

Raspberry Pi with Neogeo

So I followed Shea Silverman's blog post, but could not get gngeo to work properly. The gngeo GUI would hang whenever a game was loaded or it was exited. Following this comment from the Raspberry Pi forums, I decided that compiling gngeo from the source would be a better option. Unfortunately, that comment doesn't explain the full process and what's required to get started. After messing around a bit, here is the complete series of things you would need to do:
  1. Install GCC4.7:
    • sudo apt-get install gcc-4.7
    • export CC=gcc-4.7
    • export GCC=g++-4.7
  2. Install SDL:
    • Run "sudo apt-get install libsdl-dev"
    • (AND/OR; you may or may not need to do the following)
    • Get the source code from here
    • The file should look something like this "gngeo-0.7.tar.gz"
    • Untar the source folder (tar -zxvf gngeo-0.7.tar.gz) and cd (change directory) to that folder.
    • On the terminal, run "./configure; make; sudo make install"
  3. Install zlib:
    • Get the source code from here
    • cd into libz and run "./configure; make; sudo make install"
    • Installing zlib may not actually be necessary. I just went ahead and installed it, following the gngeo readme file instructions.
  4. Install gngeo:
    • Get the source code from here
    • run "./configure --build=i386 --host=arm-linux --target=arm-linux --disable-i386asm --enable-cyclone --enable-drz80; make; sudo make install"
  5. Download (which is the neogeo rom) and whatever game roms you need, unzip inside /usr/share/local/gngeo:
    • cd ~/.gngeo/
    • wget (Note that this file needs to be saved as gngeorc, not gngeorc.txt)
    • You might need to unzip inside the folder "/usr/share/local/gngeo" if gngeo complains that it is missing a couple of files.
    • Run "arm-linux-gngeo -i ~/ ~/" to actually play your game.
  6. For fullscreen:
    • type "sudo vim ~/.gngeo/gngeorc"
    • change the line "effect none" to "effect scale 2x50"
    • change the line "fullscreen false" to "fullscreen true"
A couple of helpful commands you might use: scp (I used this copy roms over from my mac to the Raspberry Pi after having sshed in), wget. I managed to get metal slug x, art of fighting 3 and king of fighters 97 working.

Friday 12 October 2012

Initial tinkering on the Raspberry Pi

What an amazing device- a mini-computer, the size of a credit card, and only for CAD$43! Here are the things I've done on my Raspberry Pi so far:
  1. Installed Raspbian "Wheezy" (which is an OS) into my SD Card:
    • Here's where you can find the OS
    • Here are the instructions
  2. When you boot up, you're presented with a blue screen where you can configure your settings:
    • Configure your keyboard, expand rootfs (expand root file system to enable spare SD card space), configure the timezone
    • You can reconfigure everything later using the command "sudo raspi-config"
    • Always shutdown the device using the command "sudo shutdown -h now" to avoid accidentally corrupting your SD card
  3. Installed vim:
    • Run the command "sudo apt-get install vim"
  4. Checked out the Raspbian GUI:
    • Input the command "xstart"
    • Midori is pretty slow (you'll need a different browser to view videos online)
    • Some of the python games don't work for some weird reason
  5. I also sshed in via my mac:
    • Plugin an ethernet cable to your Raspberry Pi (or share your network using your laptop)
    • Input the command "ifconfig" to get the ip address of your Raspberry Pi
    • Input the command "ssh pi@" in your laptop, replacing the IP with the Raspberry Pi's IP
    • You'll get the message "The authenticity of the host can't be verified, etc". Type "yes".
    • You'll be asked for a password, which is "raspberry" by default
    • You should be able to access your Raspberry Pi device using your laptop!
  6. I also installed MAME and Neogeo:
    • For mame, follow the instructions here. Note that you should be placing your game roms (which is just a zip file; you shouldn't need to unzip it) in the following folder : /home/pi/.advance/rom
    • I actually had a SIGSEGV error when I installed the binaries from the given link. If that happens to you, get the MAME binary from here instead.
    • For installing Neogeo, I'll create a separate post because the whole process was far more tedious.

Wednesday 10 October 2012

My Raspberry Pi has arrived!

Quick update- I just got my Raspberry Pi today! I'm planning to install MAME and run a bunch of games. I might also integrate a custom game controller. I'll keep you posted.

Short note on Java Stacks

Always use ArrayDeque instead of Stack because it is likely to be faster than a Stack when used as a stack (also faster than LinkedList when used as a queue, but this is obvious since LinkedLists need to allocate memory for each insert, c.f., array backed ArrayDeque).

ArrayDeque is backed by ArrayLists (which are backed by resizable arrays), as opposed to Stack, which is backed by Vectors (also backed by resizable arrays). Vectors are synchronized. The unsynchronized, and thus, faster, ArrayLists have made Vectors redundant. I expect this is where the gains come from by using ArrayDeque instead of Stack.

Monday 8 October 2012


Most Java developers aren't familiar with mixins. Basically, a mixin is an abstract method. Multiple mixins can be inherited by a subclass (i.e., multiple inheritance). But I guess the key difference here is that mixins are used to collect functionality, as opposed to being specialized by their subclass. This seems like a nuanced philosophical difference.

In Ruby, you can't define instance methods within mixins, but if you include a mixin within a class, the mixin methods become instance methods of that class. The mixin essentially behaves as a superclass. Unfortunately, Java doesn't officially support mixin's or multiple inheritance, but it's something worth knowing.

Wednesday 3 October 2012

Vignette : Scrum done right

My current company is excellent at scrum. We have daily meetings, which rarely last more than 5-10 mins. Meetings are focused, and everyone is amiable and enthusiastic. There is a scrum master who manages the flow of the meeting and also checks up on deadlines.

Interestingly, my experience with scrum was not always this positive. Scrum was just this fancy buzz word, ostensibly paraded as an "agile" methodology, when in fact, it was often a painful processes consisting of endless, irregular and unfocused sessions. I personally feel that implementing scrum correctly is an important factor in maintaining the morale of employees. Done wrongly, its almost poisonous.

Any company which claims to follow agile methodologies should take note- please keep scrums short and painless.

Tuesday 2 October 2012

First day at

I decided to embrace my inner explorer and visit the "headquarters" today (they have open house Tuesdays). Basically the members of are software devs and hardware hackers who meet up and work on side projects. The meeting place (in Kensington) is a small room, embellished with all sorts of gadgets and machines. There were roughly 15 hackers in all. Most of them were working on Raspberry Pi devices, though there were a couple of teenagers who were engaging in other things (picking locks anybody?).

It was interesting. I didn't really get to talk to many of these hackers. They're the shy types, a fairly standard character trait. These things take time anyway. In any case, I intend to visit next Tuesday, and with my laptop this time. Hopefully my Raspberry Pi would've arrived too.

Tuesday 25 September 2012

Git on Amazon EC2

So here's how I'm going to be setting up deployments on my EC2 instance- I'll do a git push of my committed changes from my local machine to my remote repository (I'm hosting mine on Bitbucket; Github is great too) and just do a git pull on my EC2 instance. I should then be able to deploy a new version of my application directly. You could write a shell script to automate this (deployment).

Here's how to configure git on your instance:
  1. sudo yum install git
  2. cat ~/.ssh/ | ssh -i amazon-generated-key.pem ec2-user@amazon-instance-public-dns "cat >> .ssh/authorized_keys" (Copies your local ssh key to Amazon. This makes sure that the message "Permission denied (publickey)" doesn't show up.)
  3. git clone (You'll be prompted for your password. If you get the following message "fatal: Authentication failed", then check you might have special characters in your password. You'll need to change it.)

PostgreSQL 9.1 on Amazon EC2

Amazon's Elastic Block Storage (EBS) may not be a perfect place to host your database e.g., performance and reliability are common gripes (Reddit once went down for a couple of hours due to this) but it's pretty easy to work with and is fairly flexible. You can also save database snapshots. In any case, I think it would be an interesting experiment to setup a Postgres DB on EBS and potentially check how it scales.

EBS falls under the free tier, as long as your EBS storage is under 30 GB, even with multiple EBS "volumes" [source]. Here are the steps I took in setting up Postgres 9.1 on my EC2 instance using EBS as the datastore:
  1. Go to your Amazon online console and Click the "Volumes" link located under "Elastic Block Store". The volume which is already present is the root volume.
  2. Create a new volume. I set the size as 10GiB. Check that the availability zone is the same as the zone of your root volume.
  3. Check your new volume and click "Attach Volume" to associate it to your EC2 instance. My device was set to /dev/sdf:
  4. Mount your device (i.e., your EBS volume) to a folder: 
    • sudo su - (switches user to root)
    • yes | mkfs -t ext3 /dev/sdf (makes an ext filesystem in your device)
    • mkdir /pgdata (make some directory)
    • mount /dev/sdf /pgdata (mount your device to this directory)
    • exit (exit from root)
  5. Make yum config changes so that we're able to create our own yum repo to circumvent dependency issues:
    • sudo vim /etc/yum.repos.d/amzn-main.repo (At the bottom of the "[amzn-main]" section, after "enabled=1", add "exclude=postgresql*". This tells yum that we don't want to use an amazon repository for packages that meet the postgresql criteria.)
    • sudo vim /etc/yum.repos.d/amzn-updates.repo (Add the same exclude to the bottom of the "[amzn-updates]" section)
  6. Download the repository/key installation rpm from
    • wget
    • sudo rpm -ivh pgdg-redhat91-9.1-5.noarch.rpm
  7. Since the rpm is generated for Red Hat Enterprise Linux 5, we need to make a minor change:
    • sudo vim /etc/yum.repos.d/pgdg-91-redhat.repo
    • Make the following update:
    • ## ORIGINAL
      # baseurl=$releaserver-$basearch
      ## UPDATED
  8. Install PostgreSQL 9.1:
    • sudo yum install postgresql91-server
    • sudo rm -rf /pgdata/lost+found (postgres' initdb will fail to initialize a database cluster in /pgdata when there are files or directories present)
    • sudo chown -R postgres:postgres /pgdata (changes ownership of /pgdata to the postgres user)
    • sudo su -
    • su - postgres
  9. Configure and launch the server:
    • /usr/pgsql-9.1/bin/initdb -D /pgdata
    • vim /pgdata/postgresql.conf
    • Update the line #listen_addresses = 'localhost' to listen_addresses = '*'
    • Update the line #port = 5432 to port = 5432
    • Update pg_hba.conf (for client authentication). Navigate to the bottom of the file (vim /pgdata/pg_hba.conf) and change to:
    • TYPE    DATABASE        USER            CIDR-ADDRESS            METHOD
      # "local" is for Unix domain socket connections only
      local   all             postgres                                trust
      # IPv4 local connections:
      host    all             power_user               md5  
      # IPv6 local connections:
      host    all             all             ::1/128                 md5
  10. Start the server and create the database for "power_user":
    • /usr/pgsql-9.1/bin/pg_ctl start -D /pgdata
    • /usr/pgsql-9.1/bin/createuser power_user (press "y" when prompted)
    • /usr/pgsql-9.1/bin/psql -p 5432
    • postgres=# ALTER USER power_user WITH PASSWORD 'pwd';
    • postgres=# CREATE DATABASE foo WITH OWNER power_user;
    • postgres=# \q
  11. In your Amazon console, you need to make sure that you update your security groups and add a Custom TCP rule for the port 5432. This is to access your database remotely:
  12. The following are your database properties:
    • URL is jdbc:postgresql:// (note that you can replace with the Elastic IP of your instance)
    • Username is power_user
    • Password is pwd
You can pgAdmin to connect and manage your database remotely if you want. Other tools like DbVisualizer also provide a GUI for you to view your schema and data stored in your database (DBVisualizer even allows you to view blob data).

(This post is based on Postgres 9.0 in EC2.)

Sunday 23 September 2012

Coding in the Subway

Before joining my current company, I was determined to release my Android app, Bingle. Call it paranoia, but I didn't want any legal trouble, however remote. Just a day before starting work, I hastily tested and released my app, and headed off to visit a relative.

While on the subway, I realized that my app would throw an exception if the device wasn't connected to the internet. I immediately booted up my laptop and fixed the bug. After the train ride, I located the nearest Starbucks, connected my laptop to the wifi and released a new APK file.

Lesson learnt- don't test hastily. Also, I should've looked at my tickets on Bitbucket before releasing my app.

Friday 21 September 2012

Software Maintenance

In university, I had this eccentric lecturer who'd repeat, almost manically, "Software Maintenance is the worst. Whatever you do, don't work for the maintenance department.". Every lesson. EVERY LESSON!

One of my slacker friends, who barely attended class, was once caught daydreaming. "Slacker, I've never seen you in class before. Do you take this course?". Without missing a beat, this guy says "Maintenance department".

(I just fixed a rather infuriating bug today and was reminded of this story.)

Monday 17 September 2012

Amazon EC2 Basics (Part 2)

The next step would be to associate an elastic IP with your EC2 instance and setup an Apache server:
  1. From your Amazon console, click elastic IPs and Allocate a new address.
  2. Associate it to your EC2 instance (if you don't associate with an instance, you'll be charged).
  3. Wherever you registered your domain name (Godaddy, Namecheap, etc), you're going to have to change the Address Record (A record) to point to your IP. e.g., for Namecheap, click "Manage domains", the "All host records", and enter your EC2 IP address for both the fields as shown below:

  4. You should be able to head over to your registered address (e.g., on port 8080 now and see the tomcat manager.
  5. You might want to install Apache 2 (if you want, otherwise, you can just let Tomcat work as a web server instead of a servlet container; using Apache 2 as the server with Tomcat as the container would allow you to fiddle around with modules). In this example, we're going to be using mod_proxy to proxy requests to Tomcat. Apache listens on port 80 and Tomcat on port 8080.
  6. Installing and starting apache is simple:
    • ssh into your instance
    • type sudo yum -y install httpd
    • type sudo /sbin/chkconfig httpd on
    • type sudo /sbin/service httpd start
    • /var/www/html/ is the root web directory; httpd.conf is located at /etc/httpd/conf; mod_proxy is located at /etc/httpd/modules
    • note that after changing httpd.conf, you should restart apache using sudo /sbin/service httpd restart
  7. Before doing anything else, you need to modify your Amazon security group to open up port 80:
    • on your Amazon console, click Security Groups
    • check the box of your security group, click "Inbound", create a custom TCP rule for port 80 (port range would be 80) and add this rule
  8. Also, you might want to modify port 8080 to only accept requests from your instance for added security:
    • on your Amazon console, click Security Groups
    • check the box of your security group, click "Inbound", create a custom TCP rule for port 8080 (port range would be 8080) and under "Source", add your elastic IP e.g.,
    • if you were following Part 1 of this series, you should remove the old rule for port 8080, where the "Source" was
  9. Next, modify your httpd.conf file for any context path that you wish to forward to Tomcat. In this example, I'm forwarding / to Note that by default, mod_proxy is already configured properly in this file (otherwise, follow this link, modifying as appropriate):
    • ProxyPass         /
      ProxyPassReverse  /
  10. Also modify the Tomcat server.xml file:
    • vim /etc/tomcat7/server.xml
    • Make sure that you comment out existing lines which deal with port 8080 and instead add the following line:
    • <Connector port="8080" proxyName="" proxyPort="80"/>
  11. Restart your Tomcat and Apache, and you should be all set. Just type and you should be able to view your website.

Saturday 8 September 2012

Amazon EC2 Basics (Part 1)

(Before we begin, note that the instructions for installing Tomcat 7 is for the version that is included with Amazon's repository. For instructions for installing the official and full Apache Tomcat version, please read Part 1.2 instead.)

Here is a short description of how to get started with an EC2 Project for absolute beginners. We'll also be exploring the following stuff along the way- Tomcat 7, Apache 2, mod_proxy.
  1. Go to your Amazon EC2 console (online) and start up a new instance.
  2. Follow the steps from there, its pretty straightforward. The main step you need to take care of is when setting up the security group.
    • You'll want to create a rule for ICMP (from the drop down), select "All". This enables ping.
    • You'll want to create a rule for TCP for port 22 so that you can ssh in
    • You'll want to create a rule for TCP for port 8080 for http
  3. When you're done, you can start the instance from the online console itself.
  4. Next, right click the instance and click "Connect". You'll be able to see the ssh command which you can execute via your terminal.
  5. Run a bunch of commands
    • sudo yum update (to get the latest patches)
    • sudo yum install java-1.6.0-openjdk-devel (to get the JDK; you only have the JRE right now; if you get the full JDK, you can debug in tomcat)
    • sudo yum install tomcat7 (to install tomcat 7; located in /var/lib; conf is located in /etc/tomcat7)
    • sudo service tomcat7 start (to start tomcat)
    • sudo yum install tomcat7-webapps tomcat7-docs-webapp tomcat7-admin-webapps (to get the webapps, which aren't downloaded with the tomcat package)
  6. Now, connect to your instance on port 8080 e.g.,

  7. From here, you'd probably want to navigate to the manager app, but probably won't be able to because access to the manager app is restricted by default.
    • vim /etc/tomcat7/tomcat-users.xml (vim is my text editor of choice, you can choose any text editor, really; if you are denied permission to change the tomcat-users.xml file, you'll need to change permissions for this file using the chmod command)
    • insert the following into the tomcat-users.xml file: <role rolename="manager-gui" />
    • also insert: <user username="tomcat" password="pwd" roles="manager-gui" />
    • now you need to restart tomcat (sudo service tomcat7 restart)
    • connect to your instance on port 8080 again, click on manager and input the username and password
  8. You're all set! Just head on over to the tomcat manager and deploy your war files!
(This post is based on this article.)

Thursday 6 September 2012

The research process

My paper got published recently and I'm pretty stoked about it, especially since undergrads rarely get to publish, let alone with first authorship. I'd like to take a moment to share my thoughts on the research process that culminated in this achievement.

Initially, I had decided on a project and supervisor relatively late in the semester. It was related to the Prisoners Dilemma, and I was tasked with running an arbitrary bunch of experiments to simulate agent interactions. Those first few weeks, I really thought hard. I read voraciously. The whole time, I was really under pressure to come up with something innovative and interesting. Soon, I had a rough idea. I was pretty excited about modelling noise in these agent interactions, and some realistic form of noise, something which wasn't boring. Interactions between people are affected by things like language, culture, etc (i.e., noise), and it would've been really cool to model these types of interactions. Now I needed to formalize this idea.

This was a painful process and I spent a bunch of sleepless nights crying myself to sleep (cue laughter). One fateful dinner, my group of friends, none of them CS majors, were discussing their projects. Hearing mine, one of my friends went "Hey that cultural thingy you were blabbering about- that kinda sounds like something we learnt in business class. I can't remember, but it has a name.". I then spent the next half an hour hounding her (some might call that harassment). Finally, I had the term I was looking for- pyschical distance.

The next step was far more rewarding. I thought of modelling sound waves as communication signals between agents in this Prisoner's Dilemma like game. These sound waves would be affected by the noisy effects of Psychical Distance. This was difficult- I had never taken a course in sound engineering. So I found a professor who knew things- sound things! Anyway, turns out, he couldn't help me. Another few sleepless nights followed. Finally, I gave in and visited the library. I remember hauling 10-15 sound engineering tomes to the nearby table, poring through those books in search of the elusive equation (which would help me design a function to model psychical noise). You should've seen me- disheveled hair, unshaved beard, baggy eyed. I was surprised no one mistook me for a hobo. Incredibly, I found a bunch of equations which fit perfectly into my model! That was a great day.

Fast forward presentation day, I had a really cool project which I had poured my heart and soul into. Too bad nobody really understood much of it.

Anyway, I wasn't done yet. I soon got a job, but I was quite relentless in designing new experiments in my spare time. My supervisor, suggested that we submit a paper. The publishing itself took nearly 8 months after but boy was I happy.

The whole research process was quite rewarding, but I'd like to highlight a few things I learnt-
1. Choose a good supervisor
2. Have varied interests (my project is essentially an amalgamation of comp sci, math, and business)
3. Talk to your friends
4. Textbooks in the library aren't just for show
5. Luck does play a part sometimes

Now I've not mentioned some of the more obvious and hackneyed advice that goes into research (work hard, don't give up, etc), but, uh, yeah, there's light at the end of the tunnel friends.

Wednesday 5 September 2012

Microsoft Lumia 920 looks cool

Microsoft and Nokia recently unveiled the Lumia 920, which on face value, seems like a pretty good device. It boasts a dual core 1.5 GHz CPU, awesome camera, a nifty wireless charger (its pretty cool, but seems to add little value in my opinion), an apparently sharp display and to top it off, I personally like how the device looks. Its scheduled to be released later on this year though, presumably to avoid competing with the iPhone 5. It is a step in the right direction, but the main problem with Windows devices are apps.

Until Microsoft beef up their appstore, I personally don't see why people would go for a Microsoft device. Actually, the way I see it, its not about the number of apps, but rather, the number of good, free, apps. I really hope Microsoft are cognizant of this. The day they "fix" their appstore (and their dev framework) will be the day I'll start taking them as serious players in the smartphone market.

Saturday 1 September 2012

A little bit about Git

Its going to be difficult for me to go back to SVN. Git has made version control a cinch; I actually look forward to creating, merging and switching branches. SVN users may recall the tedious experience of working with branches.

The magic is in the "distributed" approach that Git takes. What this means is that, you can actually get the entire history of your remote repo and store it locally (with SVN, you'd only have the latest revision, and not the whole history). Changes can be made locally, without needing to go online.  It is only when you push local commits to the remote repo (so that others can fetch and merge your changes) that you need internet connectivity.

I actually remember that in my previous company, where we used SVN, sometimes, the server would go down and you'd be stuck, unable to commit and collaborate until the problem was fixed. With Git, this could've been circumvented if somebody had just pushed his copy to another server, since this copy would essentially contain the whole history of the project.

One of my favourite Git features is to stash changes. Essentially, users can choose to store their current working changes away and switch to another task immediately. They can later unstash and continue working on the older task.

Also, specific commits can be cherry picked to be applied to a different branch, partial file commits are possible- the possibilities are endless. 

The only downside is that the learning curve is slightly steep initially. This shouldn't deter developers though, as the whole experience of switching from SVN to Git is immensely rewarding.

Friday 31 August 2012

Design : Android and the back button

Users expect things to work a certain way. For example, pressing the back button on the main menu of an app should actually take you back to the home screen of your device. Unfortunately, a ton of Android game makers ignore this rule and override the back button behaviour to do nothing. Some display an Android textview to exit the app instead. Often, those that don't override the back button indulge your patience by throwing up a "helpful" dialog, "Do you really want to exit? Really? Really really?"... "YES PLEASE!" The worst offenders of this design however, do nothing. That's right, nothing.

Enter "Gears and Guts". This is a great game, but notoriously hard to actually exit unless you press the home button (this doesn't actually exit the app). The main menu screen of the game offers no way of exiting. Plus the back button is overridden to do nothing. Pressing the home button fixes this right? Not really. The app still consumes a significant amount of memory in the background (unless it is auto terminated in low memory situations). Unless I'm missing something, this is quite a shoddy implemetation.

Another annoying appp, widget rather, is the reader for Reddit, called reddit is fun. After reading an article, pressing back prompts the user with the following dialog "Really exit?". Stop nagging me, ma!

So game makers, please follow design guidelines. And for gods sake please don't use Apple's design guidelines on Android. They're two different use cases.

Thursday 30 August 2012

First Amazon Review on Bingle

Wow I guess the Amazon app store is not completely superfluous after all. Here's one review from a really satisfied customer. Its things like this that really make my day.

Friday 24 August 2012

Peer programming

I kinda don't get peer programming. Recently, I had a chance to see it in action (and even participate), though it failed to win me over. Imagine sitting alongside a fellow developer, frequently interrupting his or her train of thought.

Others assert that this might actually increase focus, as developers are less likely to stray off and log on to Facebook or whatever. Developers can learn from each other too. 

Peer programming seems like something you might use to get fresh recruits up to speed. Otherwise, I feel that the constant interruptions will probably suck all the fun out of programming. There's a time and place for everything- perhaps during the design phase, consulting and brainstorming with peers would be a good option. Once that's decided though, implementation should probably be an individual thing. Let your creative juices flow and all. Plus, when coupled with code reviews, peer programming seems wholly unnecessary.

Or perhaps I'm being overly judgemental...

Thursday 23 August 2012

Ancient API's

Some "API's" just need to die. Consider the one I recently worked with- an abstruse, and seemingly undecipherable pile of slop. This particular API exposed a UNIX-like command line interface for querying a library database. Queries were frequently piped (c.f. SQL JOIN), and almost completely undecipherable without the help of a poorly worded, massive, "manual" (password protected of course).

My specific task was to circumvent the piping functionality for a few commands, which were failing on Windows. The specific API commands were apparently designed for UNIX systems. Piping them on Cygwin was leading to rogue spawned processes. Long story short, it was decided that the piping behaviour be handled manually.

Windows imposes a 8191 character limit on its commands. I couldn't just redirect input as I pleased- input often exceeded this limit. Sending input by chunks proved viable initially, by was an O(N) solution, N being the number of chunks. This scaled poorly for popular library books.

Scrapping that, I made a custom parser for handling the API commands, where only the bare minimum was piped. This was an interesting, but tedious, hack. Without giving too much away, this was the only alternative (to give you a hint of the irritating stuff which led to this decision, consider running primitive API commands in a primitive Gnuwin32 environment, with a whole ton of other medieval restrictions). It was tough going, though worth the extra effort. It ran at O(N), N being the number of piped commands, a significant improvement, most commands being really short.

Even for the most popular library books, our queries ran under 800ms!

What a hellish ride though. I'm sure this will make a good story someday, and by good, I mean I'll probably have to stab somebody after.

Wednesday 22 August 2012

App download surges, then plunges

My Android app, Bingle, was published fairly recently. The analytics are fairly surprising- a steady, almost unwavering, increase in downloads followed by a precipitous drop. The Google Admob campaign I started might explain the initial increase, but even after withdrawing the funding, the numbers kept rising. Might be due to some residual multiplier effect. The drop is what confuses me though.