Another Example of How Not to Do Customer Relations

Mozy finally announced their 1.0 release for Mac OS X. I haven’t heard from these guys since I signed up for the notification, probably more than a year ago. Then today, out of the blue, I get an email that leads off with:

One of the nice things about being an ugly American is that I make all sorts of ignorant and culturally insensitive remarks about other countries. And as luck would have it, I had that opportunity last week when I visited Montreal. (That’s in Canada.)

The first thing I noticed about Montreal is that it’s a very clean city – like Toronto in that respect. Oh wait – actually, the first thing I noticed was the streets were on fire and full of police and rioting citizens because I guess they won an O-KAY (that’s hockey) game.

Admitting that you’re a jerk doesn’t give you a free pass to go ahead and act like a jerk. If anything, it’s worse because you’ve made it clear that you know it’s wrong.

And as a method of contacting a list of people who otherwise haven’t heard from your company before? This isn’t how I’d choose to establish a relationship with potential customers.

I’ll rest well knowing that Mozy won’t get any of my money, especially with so many other options available in their market.

Quality of Service

I clearly take my job a little too seriously.

I’ve been filling up some free hours these past couple of weeks putting some polish on ZingLists. Most of the changes involve tweaking a few things here and there, but the big one is that I’m adding a mobile interface for the most essential features, and that necessitated moving from Rails 1.2 to Rails 2.0.

Everything went smoothly during development, but late Friday, when I pushed the code to production, anything that touched the new code died with a scary-looking illegal instruction signal. My production server runs FreeBSD, and I had upgraded a few other packages to stay current with security issues and bug fixes, so this left me with the cold feeling of a completely dead site late on a Friday afternoon.

As it turns out, while I haven’t solved the problem yet, I got lucky through some over-engineering on my part when I first wrote my Capistrano deployment recipes. I do deployments in two stages. Stage 1: push the code, run migrations and background tasks (e.g generating a Google sitemap). Stage 2: move configuration files into place, switch the “current” symlink and restart processes. Since I hadn’t run stage 2 yet, everything was still working fine.

Twitter also pushed some new code on Friday, but they’re having problems big enough that the site may as well be down.

ZingLists is microscopic compared to Twitter in just about every way imaginable. I’m one guy with no VC, Twitter is a dozen-ish people with at least a few million in VC. Neither of us are making any money off our web sites. Shouldn’t I be the one that doesn’t care about my uptime?

I’m optimistic that I will solve my “illegal instruction” problem with Rails 2.0, so in the interest of priming this post for anyone else who runs into the same thing, here’s a summary of the issue. (I expect most of you will stop reading here.)

The “illegal instruction” seems to be a case of the Ruby interpreter not dealing well with a stack overflow. This could be from deep recursion, but I doubt it in my case since the same works fine in development. Even simple things like List.find(:all) from script/console causes the problem, and that’s a simple ActiveRecord query.

On FreeBSD 6.2, the Ruby port is always compiled with pthreads. Because of the way that this version of FreeBSD is written, if a dynamic shared object loaded into a running program needs a shared library (like pthreads), that library must be linked to by the executing program. In this case, that’s the Ruby interpreter. The default pthreads stack size on FreeBSD is relatively small compared to other operating systems, though I couldn’t track down an exact number. It’s in the ballpark of 64 KB or 128 KB. Linux usually uses something closer to 1 or 2 MB.

My theory at the moment is that Rails 2.0 nests method calls deeper than 1.2, just enough to exhaust the stack and trigger the illegal instruction signal. I’m guessing this is because the stack overflow results in some random bit of code being executed. I’m on the AMD64 platform, and I believe it marks non-code pages with the NX (no execute) bit, so a jump off to a random address is highly likely to be caught by the kernel.

I don’t really want to recompile or hack the Ruby interpreter to make pthreads stacks larger, since I’d have to make the same fix every time Ruby is updated. I’m going to try to replicate the problem in a 6.2 virtual machine, then upgrade it to 6.3 and possibly 7.0 to see if either of those solve it. Barring a response from the Ruby port maintainer, I may even look at switching to Ubuntu Linux, though I hate to do that since I’m otherwise very satisfied with FreeBSD.

Updated April 24: so as usual, admitting in public that I have a problem with some code virtually guarantees that the problem is in my code.  I had an infinite recursion bug due to an <tt>:include</tt> on an association.  FreeBSD simply died really fast, probably due its small thread stack, but I also was missing an adequate test case to catch the problem.

Introducing ZingLists

Last week, we finally launched our first product: ZingLists. It’s a community-oriented site for making and sharing lists. What kind of lists?

  • Simple, itemized lists for things like what to put in a home emergency/disaster kit
  • To-do lists where tasks have a due date and even a schedule for repeating the task
  • Fun, lighthearted lists of your favorite CDs, movies, places to go…

ZingLists is not the first time I’ve built a real web site, but it is my first serious project that uses Ruby on Rails and I have to say that it has been a very enjoyable experience. There is only one thing I can think of where I wanted to do a little more than Rails offered out of the box, and there wasn’t already a hook somewhere. (More about that later.)

The site is quite young at the moment and so public content is a little thin, but it will get better over time. Feature-wise, the site is very useful. I have had my personal to-do lists running from it for a couple of months, and have been using it for more generic list keeping as well.

There is almost always room for improvement, though, so if anyone has any feedback, feel free to leave a comment here or drop us a note at the support page.

Domain squatters are like car salesmen

Much to my surprise, one of the hardest things I’ve had to do while starting a new company is come up with a name for my product. Naming things isn’t one of my strengths, but the domain squatters have made this task much, much harder.

The biggest problem is that you can sit down and come up with a list of dozens of potential names, and then find that only a handful aren’t already registered by someone. The ones left are generally not very good. 37Signals says not to worry about getting the perfect domain, but I’m not yet convinced that you can ignore word-of-mouth (people literally speaking your domain name to their friends).

So if most “good” names are taken, that means buying one from a squatter. This brings me to my main point: negotiating with a domain squatter for a name is a lot like dealing with a car salesman. Usually, when you buy a car, the first offer or two from the salesman is outrageously high. He just hopes you haven’t done your research and that you’ll bite. Even armed with research, sometimes they won’t deal. The best thing to do is to get up and walk away. If they really want to sell a car, they’ll blink.

This is what happened to me. I made the lowest possible offer to start negotiations for a name I mostly liked, but wasn’t in love with. The reply was $800. Within two weeks, and mostly because I stopped responding to his offers, he came down to $150. I walked away, and he blinked.

The problem for squatters is that most domain names aren’t really worth anything at all on the open market. They’re only worth something to someone who has an idea or a partially finished application and needs a name. If the price is outrageous enough, we’ll just come up with something better. That gives us tremendous leverage: when the market for buyers is just a handful of people in the world, it’s essentially impossible to get them into a bidding war. So really what we’re trying to do is figure out the absolute lowest price the squatter will sell at and not hold on to that domain out of spite.

Unfortunately, even at $150, that means he can hold 14 or so other names that never sell and still break even, so this problem isn’t likely to go away any time soon.

The Value of Customer Service

A couple of months ago, there was a little bit of discussion on the web about customer service. Ryan Carson (of Carson Systems) related his experience from the business owner’s perspective that didn’t go very well.

Surprisingly, it doesn’t take an abusive customer to get the brush off from a company. My wife’s cell phone got wet one day. Not soaking wet, but wet enough that within a week or so it started turning itself off at random times.

She went shopping on eBay, found the same model (used) for $30 and bought it. It is locked to the Cingular network. We’re Cingular customers, though the merger with AT&T Wireless, but the phone doesn’t know about the merger. We hadn’t considered this, and her old SIM card didn’t work in the new phone.

Calling Cingular doesn’t help. They’re willing to sell us a new, Cingular SIM card for $30 or sign us up to a new contract and give her a new, free phone.

A quick Google search later and we’re reading instructions for how to unlock the phone ourselves, for free. Now her SIM card works fine in the phone. She’s moved over her contacts, wallpaper and wants her old ringtone back, too.

Next problem: the used phone from eBay isn’t set up quite the same and is having trouble getting to the ringtone she’s purchased. Another call to tech support is equally as fruitless. They won’t help us at all and outright lie about the phone not having the same features since it’s a Cingular phone and we’re AT&T customers. Or maybe it’s because it’s an unlocked phone. It’s so bad, they can’t keep their lies straight. She got it working herself by comparing the settings of the two phones side-by-side.

The moral of the story is this: we’re long-time customers, past the end of our contract, and nothing but momentum is keeping us with Cingular. They should realize this, and make an effort to keep us happy. Instead, they attempt to squeeze more money out of us and lie. All they have accomplished is to give us another reason to leave.


On a different note, how much fun is Ohio weather? Last week, it was 80. Today, 28 and:

Snow in April