App Stacks with NetflixOSS Asgard, Eureka, and Ribbon

At a recent user meeting on NetflixOSS, I was asked how to share a single cloud environment (account) with various copies of an overall application stack.  I wondered the same thing when I was working at IBM using NetflixOSS.

This is a quick note to explain how to do this.

You may have noticed in Asgard, that it allows you when creating a cluster to specify a "stack".  At IBM we never specified the stack or if we did never used it at runtime.  Inside of Netflix, this "stack" ends up propagating down to instance as part of the environment, specifically as an environment variable called "NETFLIX_STACK".  You can do this by customizing the user data injected by Asgard or any other way that you pass down context.  You could also parse the auto scaling group name on the instance by querying the meta data url.  You can also make this any variable, say just "STACK".

So on an instance that was my acmeair webapp application, the application name was ACMEAIR_WEBAPP and the stacks are "aspyker" (for my development) and "smoke" for a CI smoke test I'm building.

If I go onto an instance in aspyker:

export NETFLIX_STACK="aspyker"

And on my smoke test instances:

export NETFLIX_STACK="smoke"

Now, how do we get the IPC layer to pay attention to this?  There is a cryptic note that tells you the key in the Ribbon Javadoc here.

"Ribbon supports a comma separated set of logical addresses for a Ribbon Client. Typical/default implementation uses the list of servers obtained from the first of the comma separated list and progresses down the list only when the prior vipAddress contains no servers. e.g. vipAddress settings ${foo}.bar:${port},${foobar}:80,localhost:8080 The above list will be resolved by this class as apple.bar:80,limebar:80,localhost:8080 provided that the Configuration library resolves the property foo=apple,port=80 and foobar=limebar"

Armed with this you can start to attack Eureka registration of your called service.  In your configuration, you need to set "eureka.vipAddress".  I typically set this to something like "ACMEAIR_AUTH_SERVICE".  That will mean that anyone looking up the "ACMEAIR_AUTH_SERVICE" vip address in Eureka will see your server.

If instead you set it to:

ACMEAIR_AUTH_SERVICE${NETFLIX_STACK}

it will register into Eureka as "ACMEAIR_AUTH_SERVICE/aspyker" or "ACMEAIR_AUTH_SERVICE/smoke" depending on the stack.  Both will end up under the same app in Eureka, but they are separately queriable via VIP address which is the default way Eureka gets queried from Ribbon.

Finally, you need to consume the service.  In Ribbon, you set the DeploymentContextBasedVipAddress.  Previously, I set this to "ACMEAIR_AUTH_SERVICE" which would mean it would only find services registered by that vip address.

If instead with my consumer being in a stack, I set the property as follows:

acmeair-auth-service-client.ribbon.DeploymentContextBasedVipAddresses=ACMEAIR_AUTH_SERVICE${NETFLIX_STACK}

A consumer in the "aspyker" stack will lookup

ACMEAIR_AUTH_SERVICEaspyker

and a consumer in the "smoke" stack will lookup

ACMEAIR_AUTH_SERVICEsmoke

You can take this a step further and do:

ACMEAIR_AUTH_SERVICE${NETFLIX_STACK},ACMEAIR_AUTH_SERVICE

This should according to the documentation, but I haven't personally tested, first use any instances from the stack and if none of those exist the instances in a "stackless" group.

Let me know if this works and helps you.  We'll find a way to get this back into official docs eventually.  I just wanted to get the info out here given the questions.