Image: Roma, Caro, 1866-1937 · Public domain · Wikimedia Commons
Technology

Nobody Owns the Number That Kills a Third of Our Signups

Anonymous · the author chose not to be named
Sep 28, 2024 · 4 min read · Edited

Our signup funnel loses thirty-four percent of users between the phone-number screen and the verified screen. That number has been between thirty and thirty-eight percent for the entire time I have worked on consumer products in Bengaluru, across several companies.

It is the one-time password. The SMS arrives or it does not. When it does not, the user waits thirty seconds for the resend timer, taps resend, waits again, and somewhere in there most of them leave.

Everybody knows this. It is not a discovery. What I want to write about is why it has survived a decade of intense, well-funded, genuinely competent optimisation work, because the answer is not technical and I did not understand it until I sat in the wrong meeting.

Start with the shape of the failure, because it is not evenly distributed. The OTP arrives reliably on a good network in a metro. It arrives less reliably on a congested tower, on a roaming SIM, on certain operators at certain hours, on a phone whose SMS inbox is full of promotional messages, and on the handsets of people who have a second SIM they do not carry. Every one of those conditions correlates with being poorer, being further from a city, or being a woman using a shared handset. So the thirty-four percent is not a random third of India. It is a specific third, and it is close to the exact third every one of these companies says it exists to reach.

The technical fixes are all known and none of them is hard. You can use an authentication method that does not depend on a network round trip at all. You can fall back to a call. You can read the message automatically so the user never types it, which halves the abandonment on its own. You can let the account exist before verification and verify later, which is what every developer in the room will propose within four minutes.

So why is it still thirty-four percent?

Because of who owns what. The SMS gateway is procured by a platform or infrastructure team and its budget line is cost per message, negotiated with two or three vendors, and that team is measured on delivery rate and on price. Their delivery rate is excellent — ninety-six, ninety-seven percent, and they will show you the dashboard. The funnel is owned by a growth or product team and measured on conversion. Those are different teams with different managers and, at every company I have worked at, different quarterly goals.

Here is the part that took me years to see. The gateway team's ninety-seven percent and the product team's sixty-six percent are both true and they are not in contradiction, because the gateway measures delivery to the operator and the product measures a human completing a screen. Between those two numbers sits the operator queue, the handset, the promotional-inbox filter, and thirty seconds of a person's patience, and no team on the organisation chart has a metric that spans them.

I have watched this exact conversation four times. Growth says the OTP is killing us. Platform pulls up the delivery dashboard and says ninety-seven percent. Both sides leave believing the other has not looked at the data. Nobody is wrong and nobody is lying and the meeting produces nothing, and it produces nothing again the next quarter.

The objection worth taking seriously is that OTP persists because it is what regulators and banks effectively require, and in the parts of the Indian stack that touch money that is genuinely true and not something a product team gets to decide. Fair. But most of the thirty-four percent I am describing is lost before anything financial happens — it is a person creating an account to look at a catalogue. There is no regulation requiring an SMS to precede browsing. We do it because the signup flow was copied from a payments flow in about 2016 and nobody has revisited the reason.

What actually worked, the one time I saw it work, was not a technical change. A product manager got the gateway logs and the funnel events joined into one table, with the user as the key, and put a single number on a wall: percentage of people who asked for a code and were browsing four minutes later. One number, spanning both teams, owned by neither and therefore arguable by both.

It was fifty-eight percent. Everyone in the building found it appalling, which is the correct response to a number nobody had ever seen, and within a quarter it was seventy-one, by four changes that had all been proposed years earlier and had all previously died between the two dashboards.

I have tried to start that table at my current company twice. The obstacle both times was that joining the two data sources requires the gateway team to emit a user identifier they currently do not emit, which is a small piece of work on a roadmap that is fully allocated, justified by a number that does not yet exist because you cannot compute it without the change.

Postscript. The table got built, eventually, at a place I will not name, and the number was worse than the fifty-eight percent I quoted — it was in the forties on the two operators that matter most outside the metros. It moved. It took a quarter and four changes that had all been proposed before, and the person who did it was not on either of the two teams.

◉ 84 views

Comments

0