I attended Pycon India during last couple of days. As always, it felt great to meet so many smart programmers creating amazing things. These are my notes from various workshops, talks and interactions 1.
In an interaction with @Gaurav, he mentioned about Behave library - “it is for behaviour-driven development. It is so intuitive that even a non techy person can write specs for your code.” The concept sounds interesting for writing browser tests with Selenium.
@Shashank gave a workshop on “scraping even the hardest websites.” Though I have used BeautifulSoup 2 before, using Selenium with PhantomJS was a new learning. Another interesting note from the talk was using DeathByCaptcha API for solving captchas on websites.
So now my workflow for scraping websites is something like this:
- Traditional GET requests
- Forms / POST requests
- RoboBrowser (to go around CSRF and ASP websites).
- Modern single-page-websites
- It probably has REST APIs - use them.
- Selenium + PhantomJS.
In another workshop, @Anand not only explained us how decorators work, but also made us do lots of exercises. One can go through the notebook here. By the end of the workshop we learned to write our own trace, memorize / cache and routing decorators.
Apart from decorators, I also learned about various magic commands in IPython Notebook:
%%file filename.py # Python Code # below it will save the code # in the given filename !python filename.py # using ! we can run bash-commands %magic # To see all the available magic commands in IPython # (found in an interaction with Eswar Vandanapu)
Guys from IIT Bhu talked about using SimpleCV to play flappy-birds using gestures. The library looks quite powerful to do some interesting stuff with cameras and images. I have a few toy-ideas in my mind now 3 - will see how they go.
@Kiran shared the problems he faced while creating LastUser service. I learned about database-backed sessions and about how they differ from cookie based sessions. Kiran also emphasized on using HTTPS by showing how easy it was to hijack sessions using FireSheep on an unsecured network.
Second keynote speaker, @Michael Foord, emphasized on moving the projects from platform-as-a-service (such as Heroku other such hostings) to infrastructure-as-a-service (such as Amazon AWS). The hosting services should be considered as livestock or sheeps instead of as pets. They should be easily migratable and redeployable whenever needed. Docker and JuJu are steps towards that.
@S Anand presented a talk on “faster data processing in python.” He began with basic pieces of codes and then optimized them incrementally - explaining the measurement techniques and conceptual approach towards optimization. The complete talk can be viewed online, while the notebook is available here.
Few important libraries mentioned in the talk were:
- Shows the number of hits and time spent on each line of code.
- Pandas and Numpy
- For performing vector operations.
- Numba @jit decorator
- Converts a Python function to efficient machine instruction.
@Ankur showcased his awesome new project ImportPython.com - a customized Python weekly newsletter. Apart from that, the website lists all the Python books (free as well as paid) categorized by reader’s levels. In future he plans to bring everything else, including videos, presentations etc. related to Python at one place.
Together, this is a long list of new libraries and concepts to try-out and practice. It should keep me busy well until the next PyCon.
Note: This is not a post about everything interesting that happened at the event. There were many other interesting talks and workshops. These are only the things I noted down during the event to implement them in my projects at sometime. ↩
Toy-ideas: 1. Write a script to take images each time I open the lid of laptop. 2. Write a script to automatically filter the best, find duplicates, and fix images from a large album. ↩
In a database-backed session, a session row is created in the database on each user login. The encrypted id of this row is stored in the user’s cookie. On each
is_authenticated()call, this session row is fetched. Thus this can be used to store session data of any size without incurring any additional overhead - example use cases: storing user’s votes, alerts, carts, trackers etc to show “already voted or item present in cart” flags. ↩