Thursday, May 30, 2013

Announcing Acme Air .. Performance Sample/Benchmark showcasing Mobile and Cloud at Web Scale

I am happy to announce a project I've been working on at IBM for some time today - The Acme Air Performance Sample and Benchmark.

Background and History

A year or so ago, we started to consider what the future of performance and benchmarking should be.  We specifically considered the following new architectural areas - Mobile, Cloud and Web Scale.  In looking at the standardized benchnmarking space and our own existing benchmarks many aspects of these technologies were missing.  Also, the existing benchmarks were created well before these industry trends existed and after some consideration, we decided it would be better to start from a clean slate than try to adjust existing workloads to these models.  In trying to come up with a story line for the new workloads, we looked for a compelling industry examples of a system of engagement (a term made popular by Geoffrey Moore).  After some consideration, we picked the airline industry.  Consider how the airline industry has already changed with in air connectivity, paperless boarding, and mobile applications that alert you of changing flights plans, etc.  We believe this industry (much like many others -- automotive, insurance, banking, energy, hospitality, health care -- to name a few), has dramatically changed its approach to business process in transformative ways already powered by the always-on mobile connection with their customers, employees, and partners, cloud hosted services, and analytics of the big data streaming from social, mobile and the internet of things.

Inside of IBM we have been looking at an implementation of this sample application within the context of performance and scalability.  We have talked with our customers on how to replicate similar implementations, best practices, and performance lessons learned.  Today we are starting to take that message external hoping to drive customer and partner collaboration around an open source contribution of the implementation at github.

Over the new few weeks now that this is public I plan to blog about the lessons already learned and the future contributions in this space by IBM and our partners.  Let me touch quickly on three core areas and hopefully this will spark interest in others to collaborate.  Feel free to leave comments if areas of this interest you.

Multi-Channel (Mobile)

We have designed the Acme Air services to be multi-channel.  What I mean by this is that we have the same core services offering up similar services to mobile, classic desktop browser and business partner Web API channels.  This means a Acme Air customer would get a consistent experience when interacting with Acme Air regardless of if they were using their laptop, the mobile Acme Air application, or through partners like Travelocity.  We accomplished this with well defined REST API's and leverage the IBM Worklight server technology (a technology component that can offer a secure, scalable native mobile facing environment to the traditional enterprise).

We also have implemented a hybrid mobile UI design across Android and iOS that leverages these services.  The hybrid approach means we can leverage web standards (HTML5, CSS, JavaScript) to provide a natively looking application to the Acme Air customers regardless of smartphone/tablet platform.  We also leverage technologies such as Apache Cordova to get access to mobile unique features (such as location, camera, etc) within the web standard programming model allowing the application to fully take advantage of the mobile decide aspects.


Cloud is a pretty vague word these days.  It can mean something different to each person who uses the term.  We have deployed Acme Air in about seven different cloud environments.  We have shown it using IaaS plus roll your own services, IaaS plus cloud provided services and PaaS (like CloudFoundry) based approaches.  We have even looked at advanced Web Scale platforms on top of IaaS such as Netflix OSS.  Acme Air has proven itself to be a versatile research tool for cloud given it has adapted well to each of these environments with elastic scalability.

Given how broad this term is, for now let me explain what each of these cloud environments gave us that was similar.  First, our development team around the application never wants to go back to asking for hardware, maintaining that hardware, and paying for the hardware for more than the time we use it.  We have gotten to a point where we expect the environment to be infinite in scope and readily available.  Also, with each level of abstraction provided by the cloud starting at IaaS and growing up through PaaS, we get more and more value from the underlying platform that provide us shorter time to value (less time focusing on the plumbing and more time focusing on our application and it's data).  Truly we have arrived at a model where the computing resources are a utility that can be turned on and off, are always there, and can scale up to our needs easily.

Web Scale

Web Scale is another term that has been thrown around pretty generally in the industry - so it's important to define what we mean.  We looked to the Programmable Web definition.  They have, for the past few years, documented which companies are handling more than a billion Web API calls per day calling this "club" the "Billionaire's Club".  While many of these companies are serving API's that are consumer facing and likely less transactionally critical than class enterprise applications, we believe the enterprises are moving into this space due to the systems of interaction, mobile, and internet of things.  We further believe that we need to refine this space to prove as we demonstrate web scale, that we do it with security, transactionality (when required) and reliability.

Looking at the performance world, lessons learned from web scaling applications are coming from born on the cloud companies such as Google, Twitter, Netflix and Facebook.  The lessons learned are being expressed at conferences between practitioners - not through known reference open source workloads.  In the standardized benchmarking space scale up has become more of a game of how much money can be spent by a vendor both on hardware and esoteric tuning and therefore less valuable to real world customers. By releasing Acme Air to open source along with performance papers over time we hope to fill this void of information helping the industry as a whole understand how to web scale most effectively.


I'm happy to be able to start to release this story.  I have so much to talk about based on what we have already learned.  I also know that we've only started to scratch the surface of this mobile, cloud performance and scalability story - which keeps me interested every day to show up at work and experiment further and learn.  On our open source github project page you'll find the start of this implementation.  We will have much more to add to this over time (our mobile application, our cloud devops enablement, our load testing scripts, our performance reporters, etc.).  If you are interested, I suggest you go ahead and download, build, deploy and play with what we have already released.  Currently you can try out an implementation written in an all Java stack including the WebSphere Liberty Profile and the WebSphere eXtreme Scale data grid or an all JavaScript version using the MongoDB NoSQL database.  Doing this evaluation now will give you the opportunity to learn before we start adding on to the story.