The NTT DATA Cloud Advisory Services team is frequently engaged by clients to evaluate the quality attributes of SaaS providers. While there's nothing we're going to tell clients about the infrastructure of Salesforce.com or Workday that most people don't already know, we're often surprised and confounded by some of the shortcuts taken by other SaaS providers (or application hosting providers, if you prefer) of all sizes. In this post, I will describe where some of these providers fall short, and questions you can ask to ensure you're making the right decision.
Resiliency impacts your business on a day-by-day basis. If users experience frequent disruptions, your decision to put the application "in the cloud" will not be looked upon favorably. This is an important area to drill deep into. If the vendor is unwilling to be transparent about how they've architected and designed their environment, then be wary. For example: does the hosting provider have application-level fault tolerance (e.g. resilient services-based infrastructure), platform-level fault tolerance (e.g. Microsoft Clustering or Oracle RAC), or are they merely relying on hypervisor-based fault tolerance (e.g. VMware)? Ideally you want to engage with a SaaS or application hosting provider that has designed fault tolerance and resiliency into the application itself with an active/active configuration, preferably across multiple geographic sites and regions.
You'll want to ask your hosting provider and their references specific questions about past performance. Great questions to ask a reference: When is the last time the provider had an outage? How long did it last? How was it fixed? Can they share a copy of the post-mortem/problem analysis report?
When cloudsourcing your application, you need to know and understand the kind of information that the application hosts and processes. Depending on the information hosted, you will put together a list of compliance objectives that are "must haves." It should be a "non-starter" if the application hosting provider fails to meet any of those objectives. If you store payment card information, or you transmit payment card data, even to an encrypted gateway, your provider needs to be PCI-DSS compliant. Typically this means regular scans from a QSV (Qualified Scanning Vendor). Your SaaS or hosting provider must have these scans done not only against the data center they are hosting in, but also the running application itself. This is an important distinction -- unfortunately, I have spoken to more than a handful of companies that claim that they have a PCI-DSS compliant hosted application environment because their data center is PCI-DSS compliant. Same applies for U.S. FINRA compliance measures if you're working in a heavily-regulated financial discipline, such as if you're a broker-dealer. If your systems host personal heathcare information, you'll want to have a BAA in place per the HIPAA Omnibus Rule.
3. Exit Strategy
I've written in the past about the importance of having an Exit Strategy when engaging a Cloud Service Provider. This remains critical. If your provider goes out of business, you need to have a plan for quickly securing and transporting your data to another system or location. Ideally, you should explore cloud-to-cloud backup solutions, or regularly-scheduled data dumps from the service provider. The application provider should give you enough control of the system such that you are able to download copies of your data in an ad hoc manner.
When SaaS/hosted applications stop consistently performing to user expectations, you can be assured that users will start to make noise and saturate your service desk with complaints that will introduce a substantial workload to process. You should understand how your the provider performs capacity planning. Are their seasonal factors for any of their client base that could increase system load? If so, how do they plan around that? The Internet introduces latency, and the further away your users are from the hosting site, the worse the latency will be. For a modern web-based application, latency may not be as crucial as with a thick-client app that requires Remote Desktop Services and/or Citrix sessions. For this type of architecture, a direct leased line solution may be required to reduce the number of hops. (This is nearly always the best way to go when consuming SaaS or hosted application solutions for reasons that extend beyond performance.) It is therefore important to ask about and understand the architecture of the application in addition to how it is delivered: Are Remote Desktop/Citrix services required? How is printing done? Can a leased line and/or MPLS solution be used?
These are just some of the questions you should ask any application vendor who is hosting an application for you. NTT DATA's Cloud Advisory Services performs and maintains benchmarks across a variety of different SaaS and application hosting providers, and we would be happy to work with your organization on validating if your future potential Cloud Service Providers meet your unique requirements.
- Jay Keyes, Vice President of Cloud Advisory Services, NTT DATA, Inc.