Yesterday, Screener.in stopped working in the evening. I logged in via remote shell, ran `top` and saw mysql using all the memory.
Next I did `tail -f /var/log/mysql/error.log` and saw this frightful error:
10:20:39 UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0xffed4075d9f0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
stack_bottom = ffff804c1748 thread_stack 0x100000
I upgraded Screener to Django 4.1 last week. I was surprised to see the support for conflicting updates in bulk_create method. This uses "insert...on duplicate key update" under the hood. [Django Docs].
INSERT...DUPLICATE UPDATE (MySQL docs) is one of my favourite SQL hacks. It allows us to create and update rows in bulk.
We pass the set of rows. The already existing rows are updated while the new ones are created. The "already existing" rows are detected on unique constraints. The good thing is that it is all done …
"Every line of code, every comment, every line-break should have a purpose."
We usually use Github's review feature for code reviews. It is hard, however, to jump between files when the changes are large. An ideal way would be to manually mark each and every line as checked during the review.
I use a combination of git reset and git add -p to do just that. Sharing the recipe below 🤫.
Algorithm - Switch to the branch we want to review. - Merge main into it. - Create a new review-branch from that branch. …
We have been of multitasking between tax returns, feature prototypes and code reviews for the last two weeks.
We discovered Section 54F this time. This section exempts capital gains (with no upper limit) if the proceeds are invested in buying or constructing a house. Very interesting. This is something that my Dad does. He creates a physical asset in every bull run.
Last year we discovered section 115BAA. It allows companies to opt for lower tax rates of 22%. Awesome!
Bulk and Block Deals on Screener
We added a lot of things in bulk and block deals over the last …
We are a very small tech team at Screener. Just 4-5 of us. Yet it becomes chaotic sometimes.
At the same time, there are companies which have 10s, 100s and 1000s of developers. They have fabulous products and keep rolling out new features. I admire them and try to learn a few tricks from them. About how they distribute work, how they keep the codebase sane and how they drive innovation.
These are few of the learnings and mental models from various books.
1. The E-Myth Revisited This book introduced me to the …
We had some trivial engineering challenges over last 2 weeks.
Reducing diff speed from 30+ seconds to 300ms We have a wiki feature at Screener. It allows anyone to modify and add key-insights for companies. In the backend, we use Google's diff-match-patch library. It generates diffs between multiple versions and merges them.
The Python version of the library had a long processing time for some edits. The moderation page was taking over 30 seconds to load.