Why Kiwi6 shut down with 645,911 users

2022-12-31

Kiwi6 shut down on December 31st 2022, after 13 years online. I was the sole founder, and it was simultaneously one of the most fulfilling and stressful things I have ever done.

Kiwi6 was meant to be for all things audio. It evolved from a platform for indie musicians to include tools for podcasters. It's a tired cliché, but Kiwi6 evolved organically from a solution to a problem personally had: why can't I easily share files online?

It turns out lots of hobbyists, podcasters and musicians had the same problem. It also played into my obsession with cost and performance optimization: When I started Kiwi6 in 2009, storage and bandwidth were still enormously expensive, so a lot of my work over the years involved trying to reduce storage/bandwidth costs to make the numbers work. And for a short, glorious time, work they did! A combination of paid tiers and advertising sales convinced me to try working on the project full time for a few months in 2014, when revenue peaked at just shy of $10,000/month.

The business didn't work out and I think I know why; that's for a future update (edit: here it is). Today I want to detail why Kiwi6 can't continue to exist as "just" a side project. To give you a rough sense of scale, here are a few numbers:

Running a non-trivial scale service like Kiwi6 as a "hobby" is actually very expensive and time consuming, and numerous reasons contributed to this decision to shut down. Here are a few of them:

Server costs and maintenance: The primary web servers, database, and background workers were hosted on Linode and DigitalOcean (eventually the database was moved to AWS for reliability reasons). The content servers were physical servers I colocated in Chicago and New York City. There were a variety of hosting fees for auxiliary services like transactional emails, image uploads, etc. Even with penny-pinching optimizations, operational costs were easily thousands of dollars a month depending on load (and the hardware cost many additional thousands of dollars + many weekends to build, configure, and troubleshoot). Keeping these up and running was not super complicated or time consuming, but it did require constant availability to respond to incidents. I occasionally had to pay for remote hands to go into a datacenter and rack a new server or swap out a hard disk that died in a RAID array -- which was unpleasant for a number of reasons, one of which was I could tell the tech was very annoyed at the cheap bottom-tier hardware I bought.

Worse was when a box went down, as in the early days this meant an entire shard of files would be unavailable until the server could be repaired and brought back up. Then there was that time the alerting broke so when a server went down I didn't notice for days until I opened the support inbox. And sometimes files would just get corrupted on the disk due to hardware issues, and rsync would propagate that corruption so there were no good copies, and then a user would get really mad. I felt especially bad if it was a paid user and they didn't have a backup copy. This was all very stressful.

Service abuse: Power users made a lot of duplicate accounts to get around the limitations of the free service. I did a few things to discourage this (both to limit our costs but also to make it inconvenient enough that some people would hopefully upgrade to a paid tier for legitimate use cases). Eventually I had to build a system to ban certain domains from referring any traffic to us (regardless of the account) because these abusive users took up almost all available resources and either drove up costs or degraded the service for everyone. We never fully stopped "professional piracy" - but I think we made it hard enough that they couldn't reliably use Kiwi6 for it. Nevertheless this all added unwanted stress and excitement.

Software updates: The constantly changing best practices around build and deployment made it difficult to keep up with modernizing those systems while also building features to combat abuse and stay secure. Staying on top of security issues required regular updates to everything from the Linux kernel to the application code. Unfortunately, this was a constant struggle due to frameworks, libraries, languages, and platforms constantly reaching end-of-life and requiring migration just to keep doing what they were intended to do. Do you know how many times I had to rebuild the JavaScript deployment pipeline from 2009 to 2022? In order to deploy changes to backend code? Way too many times.

Copyright stuff: Copyright takedown notices ("DMCA" takedowns) happened often enough that I had to constantly monitor for them and remove infringing files. Like every other major service in existence, I eventually built a system to automatically respond to takedown notices that would remove the file and give a strike to the offending user. This required monitoring to ensure that it didn't break (it did, occasionally). Also, sometimes people would try to abuse the DMCA takedown system or try to dispute a takedown, and those were really annoying and time consuming exceptions to arbitrate.

Accounting, taxes and back office stuff: Back office work was a huge time sink and was basically a thread that never ended once you pulled on it. I incorporated for liability reasons, and to make it real and non-fake I opened bank accounts, credit cards, filed annual reports, and balanced the books so I could pay the correct amount of taxes and filing fees. And hiring people meant payroll taxes, unemployment taxes, and a bunch of other junk fees and quarterly reports. Also, did I mention sales taxes? I have so many stories about sales taxes. Anyways I can only debug "why my financial account syncing keeps breaking every 3-6 months, breaking my general ledger reconciliation workflow" for so long before my wife stops reminding me its 3am.

Privacy and legal stuff: As new laws and regulations like GDPR came online, work was required to stay in compliance. There was just never enough time to get it all done, and never enough advice. Do I need a friggin' cookie banner? Can I use Google Analytics in Europe? GDPR compliance work is both a massive one-time investment and ongoing friction; leaving this work a perpetual TODO was too risky, but also the committing to this much effort didn't make personal sense for me.

Random crime/law enforcement stuff: Occasionally, people would commit crimes and upload evidence of their crimes to Kiwi6, or use Kiwi6 in the commission of a crime. So then I'd get requests from police to share server logs several times a year. Once I got a subpoena from a state court to provide some evidence for a criminal trial. I eventually had a fax number set up for this exact purpose and then forgot to check it regularly, so they had to scan the fax and email it to me. Lol. Anyway, terrorism surveillance is a thing too; sometimes the feds ask for server logs, and it's not polite to say no. Also, people would try to DDOS the service or script kiddies would poke around doing shady stuff like automated pentests and then I'd have to get involved and ban them.

Content moderation: I was constantly dealing with issues like harassment, spam, and professional piracy. Reviewing profile pictures for dick pics is not the sort of thing I looked forward to on nights and weekends. And often people went out of their way to be unpleasant without doing anything strictly illegal; adjudicating what was considered harassment or spam was a huge time and energy drain. For example, I was shocked to discover that people (grown adults!) would record themselves making threats or bullying others, then posting the recordings online. These were not users who could be reasoned with, so the banhammer was often necessary. Many of the behaviors on the platform were so low quality that they were basically indistinguishable from spam, and eventually the comment system was used to send actual spam. So I integrated a spam checker API and shadow banned anyone caught spamming. However, the API sometimes made mistakes, so I built buttons to mark/unmark comments as spam, and just had to click those buttons forever, or the platform would be overrun with spam. Fun times.

Viruses/malware distribution: People kept trying to upload malware in increasingly creative ways. At first it was bitcoin_booster.exe so I just banned the file signatures; they eventually figured this out so I restricted EXE uploads to paid users only. Then they started subscribing with stolen credit cards so I had to turn off EXEs for everyone. So of course they switched to uploading ZIP files, and I had to severely restrict those. But then they just renamed them JPG or MP3 or whatever and...ultimately I ended up exploring a few integrations with antivirus systems but the costs were sort of prohibitive and I couldn't limit the scans to specific file types only as they kept figuring it out and renaming the files. Eventually this sort of died down because the best case for a malware distributor is their victim runs a freshly downloaded executable, and the "conversion rate" drops if they have to unzip something or change a file extension. I still got the occasional alert whenever Google Safe Browsing found something.
Kiwi6 Support Ticket
To this day I still don't know what this guy was trying to accomplish by sending me this email.

Customer support: I tried to provide good support, even to free users, but over time this wore me down. I regret every time I wasn't able to provide superior customer service…but people would write in with all sorts of problems; I tried my best over the years to be helpful. But with half a million users and hundreds of paid customers the support inbox sort of became a beast, and its name was Guilt:
Kiwi6 Support inbox
In hindsight, creating a paid tier of the service meant I had crossed a point of no return, and when life got in the way of answering support emails (or vice versa), I started to realize it was irresponsible to leave Kiwi6 in this kind of stasis.

Finally, the weirdest and by far the least pleasant part of running Kiwi6 was that multiple people stalked and harassed me for things like my content moderation. This included several instances of doxxing, and one bizarre incident trying to get me fired from my job over what was essentially a support ticket that spiraled out of control. I had to scrub any public association I had with the site for my own sanity.

Even years later, very persistent users would occasionally uncover my unlisted cell phone number and dial me for tech support (sorry buddy, never planning to return your voicemail), and this would remind me of the general unpleasantness of the worst of that harassment.

So I would like to not be harassed for taking down copyrighted music or recordings of bullying or whatever, and that was a not-small-part of my decision to shut everything down.

Alternatives

Selling it / giving it away: I briefly toyed with this idea, and even had some initial conversations in 2015, but quickly decided not to go this route. The number of suitable owners who could technically operate the site and would be motivated to take on such a responsibility was, in my estimation, approximately 0. Basically, Kiwi6 is not really economical for even myself to run, and any other owner would have even worse economics because the last few years I paid myself $0 to be on call 24/7 for the systems and support, and nobody can do it cheaper than me because I know all the systems.

The one possible exception is if a different owner did more unscrupulous things with the site/userbase to juice the financial returns. I didn't want to be responsible for that kind of outcome.

Automating it: This is actually just a subset of "selling it" since there's a tremendous amount of work that goes into preparing a service like Kiwi6 to be taken over by a new owner, and a decent amount of that work boils down to "make things more resilient" and "make things break in a more predictable and obvious way" which involves a lot of boring grunt work like documentation and rearchitecting for cloud and managed services. Again, once we start putting a price on keeping the service up and outsourcing that, it becomes a money sink. And there's always a huge amount of operating overhead -- like answering support emails, dealing with abuse, debugging a broken deployment pipeline, or preparing income reports -- that can't be automated away.

So anyway

I'm grateful to the handful of people who briefly helped build Kiwi6. We had some good moments.

Thank you to those who used Kiwi6 and maybe even paid for it. It was an honor to serve you.

And I'm sorry if I didn't return your email.