“When I was very young, my father practiced law. One of his best friends, Grant McFayden was a client. He was a perfectly marvelous man of enormous charm and integrity - just a wonderful, wonderful man.

In contrast, my father had another client who was a blowhard, overreaching, unfair, pompous, difficult man. And I must have been fourteen years old and thereabouts when I asked, “Dad, why do you do so much work for Mr. X - this overreaching blowhard - instead of working more for wonderful men like Grant McFayden?”

My father said, “Grant McFayden treats his employees right, his customers right and his problems right. And if he gets involved with a psychotic, he quickly walks over to where the psychotic is and works out an exit as fast as he can. Therefore, Grant McFayden doesn’t have enough remunerative law business. But Mr. X is a walking minefield of wonderful legal business.”

This case demonstrates one of the troubles with practicing law. To a considerable extent, you’re going to be dealing with grossly defective people. They create an enormous amount of the remunerative law business. And even when your own client is a paragon of virtue, you’ll often be dealing with gross defectives on the other side or even on the bench. That’s partly what drove me out of the profession.”

- Charlie Munger in The Poor Charlie’s Almanack

A very similar thought is in Desiderata too:

Avoid loud and aggressive persons; they are vexatious to the spirit.

I broke my phone and missed carrying a few items on my last trek. This is a short checklist for future references.

• Torch
• Pens
• Hand sanitizer
• Basic medicines
• Charge Kindle one night before
• Sleeping bag (also works as a pillow)
• Multi-tool (knife + cutter)
• Playing Cards + Werewolf cards
• Lesser clothes
• Extra socks
• More lowers and less jeans
• Extra phone and an extra pair of spectacles

Bonus: Want some ideas for places to travel? Check this.

Last week, I visited Shenzhen in China with my family for 3 days. Shenzhen is a shopping paradise with huge markets such as Dongmen and Lo Wu commercial center. Shenzhen is surrounded by hills, which give it a pleasant weather all the year round.

I compare my visit to China to that of some alien world, where the habitants have progressed as much as us (or even more), but where they don’t understand our language, they don’t accept our credit cards and where we can’t use our mobile applications.

These are few of the things I noted for our next trip to China.

Things to note before visiting China:

• Google is banned in China. So is Facebook and Twitter.
• Since Google is banned, we cannot use Android Play Store, Google Maps or Gmail.
• Credit Cards (our was International MasterCard) might not work in China.
• Sim cards from Hong Kong might not provide internet access.
• WhatsApp and WeChat work. So does iPhone.

Workarounds and preparations:

• Pre-install Baidu Translate for text translation and voice translation.
• Download offline package in Baidu Translate: There is an option in the app to download the offline package so that the translation works even when there is no internet.
• Use Uber: Ensure to change your mobile number in the profile option. The driver will probably call you before coming. Have someone else tell your current position in that case.
• Print out locations in Chinese names (especially your address of stay): It is helpful to get someone to write down the Chinese names of locations you wish to go to.
• Bring sufficient RMB (Chinese Yuan Renminbi): Since credit cards might not work in China, it is better to carry sufficient RMB. We found the best conversion rates in Tsim Tsa Tsui (in Hong Kong) market (don’t use Western Union).
• Print the emails: Since Gmail wont work in China, it is better to print or download important emails before arrival.
• Use AirBnB: We booked our stay through AirBnB. The host was extremely co-operative and always available over WhatsApp. It is good to have some local person to rely on.
• Buy a sim-card soon: The sim costs around RMB 70 with the data pack. It is a helpful to have it early.

Enjoy your trip to China.

I like to use DigitalOcean for trying out new web-applications and scripts. The cheapest plan comes at $5/month but provides only 512MB of memory. Adding a 1Gb swap memory is usually sufficient for temporary memory extensive needs. These is a short documentation on how to add the swap partition.

Adding a swap partition:

sudo dd if=/dev/zero of=/swap bs=1M count=1024
sudo mkswap /swap
sudo swapon /swap

Making the partition persistent on system reboots:

sudo vi /etc/fstab
# Add a line to tell the system to use the file
/swap   none    swap    sw  0   0

Optimize swappiness and vfc_cache_pressure for VPS needs:

sudo vi /etc/sysctl.conf
# Add the following at the bottom
vm.vfs_cache_pressure = 50
vm.swappiness = 10

The detailed article about what each of the above commands do is available here.

Image credits: @Mukesh

I attended Pycon India during last couple of days. As always, it felt great to meet so many smart programmers creating amazing things. These are my notes from various workshops, talks and interactions ¹.

In an interaction with @Gaurav, he mentioned about Behave library - “it is for behaviour-driven development. It is so intuitive that even a non techy person can write specs for your code.” The concept sounds interesting for writing browser tests with Selenium.

@Shashank gave a workshop on “scraping even the hardest websites.” Though I have used BeautifulSoup ² before, using Selenium with PhantomJS was a new learning. Another interesting note from the talk was using DeathByCaptcha API for solving captchas on websites.

So now my workflow for scraping websites is something like this:

Traditional GET requests

Requests.

Forms / POST requests

RoboBrowser (to go around CSRF and ASP websites).

Modern single-page-websites

It probably has REST APIs - use them.

For other Javascript heavy websites

Selenium + PhantomJS.

In another workshop, @Anand not only explained us how decorators work, but also made us do lots of exercises. One can go through the notebook here. By the end of the workshop we learned to write our own trace, memorize / cache and routing decorators.

Apart from decorators, I also learned about various magic commands in IPython Notebook:

%%file filename.py
# Python Code
# below it will save the code
# in the given filename

!python filename.py
# using ! we can run bash-commands

%magic
# To see all the available magic commands in IPython
# (found in an interaction with Eswar Vandanapu)

@Kushal gave a keynote on the importance of contributing upstream. By going through the source codes of our favorite programs, we can not only become a better programmer but also get an opportunity to contribute to them. I pledge to have a closer look at the source codes of two of my favorite Python packages - Rhythmbox and Django.

Guys from IIT Bhu talked about using SimpleCV to play flappy-birds using gestures. The library looks quite powerful to do some interesting stuff with cameras and images. I have a few toy-ideas in my mind now ³ - will see how they go.

@Kiran shared the problems he faced while creating LastUser service. I learned about database-backed sessions and about how they differ from cookie based sessions. Kiran also emphasized on using HTTPS by showing how easy it was to hijack sessions using FireSheep on an unsecured network.

Django uses database-backed sessions by default. While digging into the related documentation, I found the solution to some of my problems ⁴.

Second keynote speaker, @Michael Foord, emphasized on moving the projects from platform-as-a-service (such as Heroku other such hostings) to infrastructure-as-a-service (such as Amazon AWS). A hosting service should be considered as livestock or sheep instead of as pet. It should be easily migratable and redeployable whenever needed. Docker and JuJu are steps towards that.

@S Anand presented a talk on “faster data processing in python.” He began with basic pieces of codes and then optimized them incrementally - explaining the measurement techniques and conceptual approach towards optimization. The complete talk can be viewed online, while the notebook is available here.

Few important libraries mentioned in the talk were:

line_profiler

Shows the number of hits and time spent on each line of code.

Pandas and Numpy

For performing vector operations.

Numba @jit decorator

Converts a Python function to efficient machine instruction.

During the lightning talks, @Vineet showcased his library Pipdeptree, which displays all the pip dependencies in a tree format. It will surely come quite handy for generating sane requirements.txt.

@Ankur showcased his awesome new project ImportPython.com - a customized Python weekly newsletter. Apart from that, the website lists all the Python books (free as well as paid) categorized by reader’s levels. In future he plans to bring everything else, including videos, presentations etc. related to Python at one place.

Together, this is a long list of new libraries and concepts to try-out and practice. It should keep me busy well until the next PyCon.

These are few of the bash commands I often find useful:

z: Z allows jumping (cd) into frecently (i.e. frequently and recently) used directories. Instead of doing cd /home/pratyush/websites/project_name each time, I can now simply do z project_name. Z supports tab completion too. Link to Z library.

ps -u <username> -o pid,rss,command | awk '{print $0}{sum+=$2} END {print "Total", sum/1024, "MB"}': When accessing remote servers using shell, I often need to check the list of scripts running and the memory they are consuming. This command does exactly that. It is a sort of minimal task-manager. I found this command here.

php -S localhost:8888 -t .: Though I am not a PHP fan (anymore), this command serves the current directory using a live php server. This comes handy to tryout a local wordpress installation by just extracting the package.

python -m SimpleHTTPServer 8080 is a Python alternative for starting a live server from current directory to serve the files.

netstat -plunt: To see which ports are open and which application is on each one.

du file/path -chs: du is the disk-utility command. This command shows the total size of any directory. -c is for total, -h is for showing humanize file sizes and -s for showing only the summary. I often use this command on web-servers to find the size of file-system caches.

ssh-copy-id user@example.com: This appends the public key to the remote-host for password-less ssh logins.

mysqldump -u username -p --all-databases > alldbs.sql: For creating a backup of all the mysql databases.

mysql -u username -p < alldbs.sql: For restoring all the databases from the dump.

Both of the these mysql commands are high on performance. These come handy for creating a backup snapshots of databases.

I found these two commands here.

ssh -D 31500 user@example.com: This turns the SSH client into a SOCKS proxy server. It provides me a VPN on the fly. So if a website refuses to open, or is restricted to a particular country, I run this command and then update the proxy settings in Firefox as below:

1. Enable proxy in firefox.
2. Enter "127.0.0.1" for "SOCKS Host"
3. Enter "31500" (or whatever port we chose) for Port.

Full documentation for this trick is available here.

howdoi: HowDoI provides answers to programming questions from command line. Thus instead of opening a browser and getting distracted in web, I can now simply type in something like howdoi convert csv to namedtuple to get the leads.

I usually pass the -ac arguments: -a provides the full text of the answer, -c enables colorized output.

ab -n 100 -c 10 http://www.example.com: AB is a poor man’s website performance benchmarking tool. -c specifies the number of concurrent requests and -n specifies the total number of requests to be sent to a webpage. This comes handy while migrating websites to new servers or when making significant frontend changes.

wget "url" -c: WGET is for downloading files from command-line. I often find the download speeds significantly different in browser’s built-in download managers and wget. -c enables the resume support.

There are various other must-know bash commands such as awk, uniq, head which are super-useful in daily work. Akshay has covered them in a brief tutorial here.

These are the websites I use the most based on my browser’s frecency:

This post is inspired from S Anand’s blogpost.

• A is for amazon.in. Though I don’t shop too much, I usually use Amazon to browse though books and read their initial pages.
• B is bseindia.com - mostly for checking current stock prices and for keeping a track of latest corporate announcements.
• C shows one of my favorite tech blogs - codinghorror.com. Recently, have been visiting it more frequently to check the blog’s integration with Discourse. It is followed by cricket.yahoo.com, which I use to keep a track of latest scores.
• D is docs.google.com. I love to use the live sharing and commenting feature in Google Docs to collaborate on articles, letters and reports. For other writings, I prefer to use gVim.
• E is english.stackexchange.com. I recently read Strunk and White’s The Elements of Style; have been using this StackExchange to follow up with few of the rules mentioned in the book.
• F is (obviously) facebook.com. The runner-up is feedly.com - rss reader to keep a track of my favorite blogs. In future I would like to see Feedly climb above :).
• G is github.com - I love open-source projects.
• H is hootsuite.com - used to manage multiple Twitter accounts including @faltoo and @screener_in.
• I is irctc.co.in - for booking all the train tickets. It is followed by imdb.com for checking all the movie recommendations.
• J shows one of my favorite financial blogs janav.wordpress.com. It is followed by jpegmini.com, which I use to compress images in blog posts.
• K is kickass.to - for movies and music.
• L is localhost - a playground for all my development projects.
• M is (again obviously) mail.google.com.
• N is news.ycominator.com - I use it to stay updated with the latest programing developments. Lately, I have been using more of hckrnews.com for its better interface.
• O is openshift.redhat.com - for deploying new test projects for free.
• P is play.fully-faltoo.com - a custom domain used for testing different internal projects at different times. It is followed by pixlr.com/express/, used for quickly fixing the photographs.
• Q is Quora. I am not a huge fan of Quora but it is the only one with Q.
• R is Readiot.com - a side-project I have been working upon. It is like Evernote but saves everything in your DropBox.
• S is for screener.in - the project that takes most of my time :). It is followed by stackoverflow.com - my favorite resource to get and provide help.
• T is twitter.com - I like to keep track of latest updates through Twitter rather than watching news.
• U is ubuntu.com, an old and the only item in U. Now I like to use Arch.
• V is valuepickr.com, my favorite forum for in-depth discussions on stocks.
• W is webfaction.com - my webhost for all the production deployments.
• X is xiaomi.eu - an old one time item when I used MIUI Rom on my Nexus S.
• Y is youtube.com - I love to hear music and watch my favorite channels online.
• Z has no items.