If you've been following along, you have seen a story building around of public cloud performance (and scale) and today I'm pleased to announce one of the most interesting chapters.
First, we introduced a new cloud and mobile sample application called Acme Air. Next, at IBM Impact 2013, we showed how IBM WebSphere middleware and the IBM cloud could run this sample workload as a benchmark at impressive web scale levels of load and scale.
More recently, we have shown the same application adjusted to use the NetflixOSS cloud platform on top of the Amazon cloud (and AMI's). We did this as we admired Netlix's leadership in the industry in open cloud-native architectures for high availability, dev ops, and web scaling (among other things).
While the NetflixOSS platform has been negatively commented on, we knew some of the comments pertained to perceived issues that were momentary. Specifically one of the comments was around portability and how being tied to Amazon's cloud might affect resulting adopters of Netflix's approach. In this latest work we question that perception showing not only portability of the core NetflixOSS architecture, but also starting to demonstrate the ability of the architecture across multiple public IaaS clouds. Specifically we showcase the technology running on IBM SoftLayer along with RightScale application cluster management.
Likely now you can see the method to our madness here. We first wanted to have a workload that was known to demonstrate cloud and mobile modern implementation and architecture to reset how we looked at performance. Next we wanted to demonstrate that the workload could run at interesting levels of scale. Next we added operational aspects to how we worked with the workload (high availability, devops, scaling, etc.) based on an open public cloud-native IaaS architecture. Finally, we showed how we could make this architecture open across clouds. At IBM, we are all about open cloud architecture as demonstrated in our work across OpenStack and CloudFoundry.
We are now looking at applying what we have learned across our entire cloud portfolio as well as our multiple Software as a Service offerings. IBM needs to continue to provide industry leadership in both IBM provided public services and cloud solutions that our customers can use to stand up their own public applications and services. NetflixOSS technology and the Netflix approach will certainly have impact on our approach to delivering these solutions going forward.
If you want to read about what we did we have extensively documented the work on this wiki and have contributed back our code changes under the "EmergingTechnologyInstitute" on github. We did this work completely in the open as encouraged by the Netflix Cloud Prize and following IBM's approach to open cloud architectures.
Check out the below video if you want to watch Acme Air / Netflix OSS running on the IBM SoftLayer cloud:
Direct Link (HD Version)
Thursday, September 12, 2013
Sunday, September 8, 2013
Acme Air NetflixOSS AMI's - For your consumption pleasure
TL;DR - Go here for AMI's for the core of the Netflix OSS platform using Acme Air as a sample.
Some time ago, we showed a working Acme Air recoded to use the Netflix OSS running on EC2 with Apache Tomcat and WebSphere eXtreme Scale at load. Later, we showed the same thing with the WebSphere Liberty Profile and WebSphere eXtreme Scale.
While these were great demonstrations, they weren't the easiest for someone to replicate themselves. To replicate the results, you have to build the Acme Air application from source and create two server images (back end auth service and front end web app), deploy and configure a WebSphere eXtreme Scale cluster, build Asgard from source and configure/deploy it as a server, build Eureka from source and configure/deploy it as a server, build Hystrix (for the console) and run it locally. While the standardized github and gradle based builds help make the code really easy, for the most part, easy to compile, the steps to configure and deploy the various servers are not trivial. It took us a few weeks of studying wiki's, reading code, and learning from the Netflix RSS sample to devise a "production" setup. It could take much longer than two weeks, especially if you are not a cloud platform engineer. The NetflixOSS platform has deep and broad value, but being able to bootstrap yourself on the platform is currently pretty tough.
We decided to make this easier for the masses by creating Amazing Machine Images (AMI's) of the six servers required to run a pretty basic setup of the platform. We're not the first to create AMI's of Netflix server components. Netflix themselves created an Asgard AMI and released it at the last meetup. Other Netflix Cloud Prize participants are creating Netflix server AMI's as part of their submissions. Some of those include Nicholas Whittier's work to create them with Puppet and Peter Sankauskas's work to create them with Ansible playbooks. While these AMI's are valuable, they lack two important aspects - a sample application and the information of how each of the servers should be tied together in a way that showcases the entire platform. Specifically, of the six AMI's required to run the end to end application of Acme Air, these AMI's would provide for two of the servers (Asgard and Eureka) but wouldn't help you immediately configure you application to access Eureka. This is not to say these other AMI's aren't valuable. They are likely built in a way that others can improve upon and learn from (something lacking in the AMI's we've built just to demonstrate the end to end configuration of the NetflixOSS platform). Eventually it would be nice to see Peter's work and/or Netflix themselves have not only Netflix server AMI's but AMI's for a sample application such as Acme Air and instructions of how to startup the entire platform end to end.
We decided to freeze dry our running images into AMI's and provide instructions of how to re-bootstrap the entire platform in a minimalist configuration. We figured that such an end to end view would be helpful to the community. While it will be important to enhance such an effort later to show further configuration for enhanced HA, security, and operational excellence, we decided it would be interesting to show the basic configuration first. You can get started using these AMI's using instructions we posted on the Acme Air Netflix wiki. All of the contained technology for the AMI's come from Apache 2.0 licensed github projects (Tomcat, NetflixOSS and Acme Air github projects) and for development use only WebSphere products (WebSphere Liberty Profile and WebSphere eXtreme Scale). I'd encourage you to review and agree to comply with the contained technologies' licenses before proceeding to work with the AMI's, but the main points are the use of the WebSphere technologies is for development use only and there is no support offered or implied by IBM or Apache or Netflix. These AMI's are just samples to be learned from.
If you find the AMI's useful or have questions, drop a comment on this blog or visit the Acme Air user group mailing list (please note that you're using the Netflix version of Acme Air).
Some time ago, we showed a working Acme Air recoded to use the Netflix OSS running on EC2 with Apache Tomcat and WebSphere eXtreme Scale at load. Later, we showed the same thing with the WebSphere Liberty Profile and WebSphere eXtreme Scale.
While these were great demonstrations, they weren't the easiest for someone to replicate themselves. To replicate the results, you have to build the Acme Air application from source and create two server images (back end auth service and front end web app), deploy and configure a WebSphere eXtreme Scale cluster, build Asgard from source and configure/deploy it as a server, build Eureka from source and configure/deploy it as a server, build Hystrix (for the console) and run it locally. While the standardized github and gradle based builds help make the code really easy, for the most part, easy to compile, the steps to configure and deploy the various servers are not trivial. It took us a few weeks of studying wiki's, reading code, and learning from the Netflix RSS sample to devise a "production" setup. It could take much longer than two weeks, especially if you are not a cloud platform engineer. The NetflixOSS platform has deep and broad value, but being able to bootstrap yourself on the platform is currently pretty tough.
We decided to make this easier for the masses by creating Amazing Machine Images (AMI's) of the six servers required to run a pretty basic setup of the platform. We're not the first to create AMI's of Netflix server components. Netflix themselves created an Asgard AMI and released it at the last meetup. Other Netflix Cloud Prize participants are creating Netflix server AMI's as part of their submissions. Some of those include Nicholas Whittier's work to create them with Puppet and Peter Sankauskas's work to create them with Ansible playbooks. While these AMI's are valuable, they lack two important aspects - a sample application and the information of how each of the servers should be tied together in a way that showcases the entire platform. Specifically, of the six AMI's required to run the end to end application of Acme Air, these AMI's would provide for two of the servers (Asgard and Eureka) but wouldn't help you immediately configure you application to access Eureka. This is not to say these other AMI's aren't valuable. They are likely built in a way that others can improve upon and learn from (something lacking in the AMI's we've built just to demonstrate the end to end configuration of the NetflixOSS platform). Eventually it would be nice to see Peter's work and/or Netflix themselves have not only Netflix server AMI's but AMI's for a sample application such as Acme Air and instructions of how to startup the entire platform end to end.
We decided to freeze dry our running images into AMI's and provide instructions of how to re-bootstrap the entire platform in a minimalist configuration. We figured that such an end to end view would be helpful to the community. While it will be important to enhance such an effort later to show further configuration for enhanced HA, security, and operational excellence, we decided it would be interesting to show the basic configuration first. You can get started using these AMI's using instructions we posted on the Acme Air Netflix wiki. All of the contained technology for the AMI's come from Apache 2.0 licensed github projects (Tomcat, NetflixOSS and Acme Air github projects) and for development use only WebSphere products (WebSphere Liberty Profile and WebSphere eXtreme Scale). I'd encourage you to review and agree to comply with the contained technologies' licenses before proceeding to work with the AMI's, but the main points are the use of the WebSphere technologies is for development use only and there is no support offered or implied by IBM or Apache or Netflix. These AMI's are just samples to be learned from.
If you find the AMI's useful or have questions, drop a comment on this blog or visit the Acme Air user group mailing list (please note that you're using the Netflix version of Acme Air).
Subscribe to:
Posts (Atom)