We originally created Screener AI using WebSockets. While Django’s WebSockets worked pretty well, they had a few issues:

• The sessions were linked to the browser connection.
  
• If the user’s browser disconnected, the answer generation also stopped in the backend.
  
• This was especially problematic on mobile, where browsers are stopped after 5 seconds if the user switches apps or turns off the screen.
  

The above caused multiple cases where users didn’t receive the answer when they returned to the query they had asked.

That’s when we realised that we needed to decouple the user’s browser connection and the agent fetching the …

Last night, a simple delete query brought screener.in down. This is how the things went.

• One of the tables in the database had around 80 million rows.
  
• I ran a MySQL query to delete around 60 million rows.
  
• I went off to sleep and my remote shell got disconnected.
  
• 7 hours later, I woke to see the website performing very slowly.
  
• The disk utilisation showed writes of 80MB/s for the last 7+ hours.
  

After an hour of debugging, this is what solved it.

SET PERSIST innodb_redo_log_capacity = 4294967296; -- 4 GiB

Why did this happen?

I later learned that any …

Radhika and I had our second baby. He shares the birthday date with Radhika - 3rd September.

Enso, a writing tool, has a beautiful little feature called CoffeeMode. It enables writing privately in public places. Anything we write appears as a •.

I was sitting in a library recently, scribbling my random thoughts in my jrnl. This CoffeeMode sounds perfect for such moments (such as on airplanes). Interestingly, it was easy to implement this mode in Neovim using the conceal feature.

:syntax match CoffeeShop /[a-z]/ conceal cchar=• :set conceallevel=2 # might need to do this too active it in both normal and insert mode # this allows us to see the …

Yesterday I wrote a script to create backups for my server. Just when I ran the script, I realised it would take an hour to complete.

It was 7pm. I needed to leave for home.

I cursed myself for not doing bgrun bash backup.sh instead of bash backup.sh.

The thing is bgrun doesn’t exist.

The closest thing I found was:
nohup your_command > output.log 2>&1 & disown.

Let’s break it down:

• nohup = no hangup
• your_command = anything like bash backup.sh
• > output.log = put the output of the …

A bad carpenter blames his tools

I implemented metaWeblog APIs on this blog over the weekend. This allows me to use MarsEdit (or any other blogging tool) for writing posts.

I remember using Microsoft’s Live Writer a lot in the early days. The desktop interface reduces the friction of opening a webpage, logging in, and navigating various menus. I created my own blogging system to avoid the ever-growing complexity of WordPress. Still it is a multi-step work to write. MarsEdit removes this need for a browser and provides focus on writing.

What to write …

• First, our hard-disk got filled due to access logs
  
• Then our hard-disk got filled due to too many open files error logs
  
• Various services started hitting file descriptor limits
  
• The system started dropping packets and killing connections as the connections table got full
  

Is it a DDoS?

sudo netstat -nta …