This commentary on outages was interrupted ... by an outage.

Recently, Salesforce.com, the world's most popular hosted CRM provider, had system problems that prevented customers from accessing the service. I had planned on writing a story about the outage. I had also planned to get feedback from some other companies in the hosted CRM market regarding what they thought of the outage and perhaps how they are doing things differently to avoid such problems. As I was exchanging e-mails containing my questions with the various companies I thought would be good candidates for commenting on such an event, something interesting happened: TMC's in-house e-mail server crashed, taking my notes and responses down with it. We lost no data, just productivity.

[ILLUSTRATION OMITTED]

The irony, of course, is that I was able to witness first-hand what happens when systems that many people rely on don't work properly. In our case, it seemed like people were lost for the first few hours of downtime. If I had to compare it to something, I would say it was as if someone dimmed the lights to the romantic setting and everyone had a few drinks. There seemed to be a common need to stop working, come up for air and discuss the problem for hours, almost as if it were Happy Hour. It was amazing. Many people cannot function without e-mail, as they rely on it constantly. I, too, shared this paralysis to a degree. When you come to rely on a crucial piece of software, it's unbearable to be disconnected from it. In the case of e-mail, it's as if we have forgotten that we once used telephones to communicate, and that we have another place we can go to get contact information that will enable us to call customers.

People feel a physical connection with technology. Our applications become an extension of us and it seems from casual observation that when a critical application fails, it's as if our arms stop functioning correctly.

Now to put this in perspective: Let's look at what happened recently in the hosted CRM space. Salesforce.com applications were not available to the public--meaning the software was not usable. Companies, too, have the challenge of having their software running 24x7, and it's not unheard of to have outages when an application fails. Take our e-mail server as an example. After all, even with redundant hard disks in the form of RAID, you can still have failures. It happens sometimes. But hosted solutions are different--they are expected to be up all the time.

People say that Salesforce.com is a victim of its own success. Siebel even alluded to this fact a while back, indicating that Salesforce.com is incapable of scaling reliably. Others have said they feel that Salesforce.com has too many customers and has grown so quickly that it is impossible for them to support all of their customers effectively; thus the breakdowns in their solution.

A few competitors of the hosted CRM leader reached out to me to remind me that alternative solutions exist and Salesforce.com customers could look for alternatives. For example, the following e-mail hit my inbox about a week before I wrote this column:</p> <pre> By now, I am sure you have heard that Salesforce.com is experiencing yet another lengthy service outage. The most interesting part of this outage, however, is that they are not following through with their promise to their customers to keep them informed throughout the outage. If you cover this issue today, please feel free to contact Salesnet to discuss the issue--as a SaaS provider, they have a plan for keeping their customers informed in the rare case of any downtime incidents--a necessity in this market. In addition, please feel free to use this comment from Jonathan Tang, president and co-founder of Salesnet: "Last time Salesforce.com had this problem, Salesnet agreed that with a hosted system some downtime is, of course, inevitable; however, at some point it becomes unacceptable, especially on the last days of the month--a crunch time for every sales person. Even more unacceptable is their lack of follow-through with informing their customers. On January 6, Salesforce said they had learned from their mistakes with regard to informing customers, indicating that they planned to make available a beta version of their system status page in order to give customers real-time updates on system status and performance. The system has been down now for several hours, and no information is available for customers regarding the system status. Salesforce.com is becoming a victim of its own success, and they are failing their customers--at what point will their customers decide they have had enough? Please let me know if you would like any more information or would like to speak with Jonathan." </pre> <p>Naturally, I was intrigued and had to ask some questions. I quickly sent off a bunch of questions from my Blackberry, and a short time later I received the following responses:

Q: What is an acceptable amount of downtime for a hosted provider?

A: 99.6 percent of uptime is our SLA. We have had greater than 99.99 percent for the past three years.

Q: What sort of remedies should a customer seek as a result of excessive downtime?

A: Proactive communications should be top priority from the vendor. But we will offer credits back to customers for excessive unscheduled downtime (below 99.6 percent in any 24-hour period).

Q: At what point should customers consider switching hosted providers?

A: Regardless of downtime, at any point they feel that they are not being serviced adequately. The risk is minimal to switch. But experiencing all day and half-day outages several times a quarter (as was the case with SFDC) is certainly excessive unscheduled downtime.

Q: Can I have two sentences on how Salesnet differentiates itself?

A: Salesnet offers a process-centric CRM tool that allows a company to define, guide and track their sales methodology within our tool. Second, our configuration engine is more robust and flexible. One can map to the most comprehensive enterprise organizations.

I reached out to Siebel, as well, for answers, but with the Oracle merger completed, I am still looking to open up the channels of communications with the company, as the old ones are now closed. I also reached out to SAP, but they have a complex on-demand story and we had no time to fit in an interview by deadline. It is my hope to have a more in-depth wrap-up on this topic in the future.

Getting back to Salesforce.com, I asked Marc Benioff, the company's founder and CEO, some direct questions about the outage and was told he wasn't available for comment on such short notice. Here are some comments from the recent Merrill Lynch IT Services & Software CEO Conference, where Benioff gave a presentation discussing the outages:</p> <pre>

"We have been great at making our customers successful over the past few years. Customers are thirsty for success, and that's what we want to give them. But we cannot give that to them all the time, especially when we are making big changes to our hardware and our software. But more often than not, we've been able to bring more success to our users than other enterprise software company. As you know, since you've been following us for a long time, in the seven years of our company's history, what an excellent reliability rate we have--we've never fallen below a 99 percent reliability rate. Even in the fourth calendar quarter, we delivered about 99.7 percent uptime. That was during the month of December, when we experienced a brief outage on December 20th. In the month of December, our availability fell to a 99.5 percent uptime.

But that is still not where we want to be--we have very high expectations for ourselves. We have a brand new data center that just went in November 1st, and the two brief outages we had were around us shaking out our data center and some of the new hardware we have in our system. And as service providers yourself in banking, you know that no services provide 100 percent uptime in any industry. Kind of like eBay and Amazon, you have users who want 100 percent. And we are getting close to that--we want to be as close to 100 percent as possible. Now we have this brand new architecture to help us get there." </pre> <p>Certainly, the industry is gunning for Salesforce.com, and every company with a software-based solution has a hosted solution as well, or has one in the works. It is for this reason that Salesforce.com has to get it right. If they don't, they will certainly experience mass defections. In fact, I am surprised that Oracle/Siebel didn't make more noise about these outages. Perhaps the acquisition made them lose focus.

Speaking of focus, that may be another challenge for Salesforce.com. While they are focusing on uptime, they also have to keep an eye on numerous new initiatives they have recently rolled out. For example, they have a new Mirrorforce disaster recovery system that switches AppExchange live over the Internet. Hopefully this initiative will help keep the company's systems running more smoothly. Another initiative is Sandbox, which allows customers to replicate application environments for testing purposes.

AppExchange is designed to allow Salesforce.com to get into other areas such as finance/accounting and human resources. This platform was once known as CustomForce.

So while Salesforce.com expands its offerings and arguable is going broader, its competitors are focusing on hosted CRM and trying to find ways to catch up to the company. The service outages could not have come at a worse time and, in my opinion, the company will ensure these things never happen again to the best of their ability. But let's face it, outages happen. They are a sad fact of a life where everything runs on computers with moving parts and where 24x7 uptime is a journey, not a destination.

Still, the goal for the hosted CRM industry is 100 percent uptime, and until we reach that point, people like me will have lots of interesting articles to write and customers such as yourselves will have to weigh carefully all solutions before deciding to go hosted or the software route, and just as important, picking a hosted vendor you feel will have the best offering while maintaining extremely high levels of resiliency.

By: Rich Tehrani, Group Publisher, Group Editor-in-Chief, Technology Marketing Corporation