Let's set the stage with the definition of multitenancy and how this affects cloud computing.
Wikipedia definition of multitenancy:
"Multitenancy refers to a principle in software architecture where a single instance of the software runs on a server, serving multiple client-organizations (tenants). Multitenancy contrasts with multi-instance architectures where separate software instances (or hardware systems) operate on behalf of different client organizations. With a multitenant architecture, a software application is designed to virtually partition its data and configuration, and each client organization works with a customized virtual application."
Wikipedia further explains multitenancy in the context of cloud computing:
Multitenancy enables sharing of resources and costs across a large pool of users thus allowing for:
centralization of infrastructure in locations with lower costs (such as real estate, electricity, etc.),
peak-load capacity increases (users need not engineer for highest possible load-levels),
utilisation and efficiency improvements for systems that are often only 10–20% utilised.
It seems like everyone comes with their own definition of multitenancy and from what I can see they are all shades of the same definition. Specifically as you see below the differences are in the "designed to virtually partition its data and configuration" and to what extend that partitioning is possible to be affected by other users.
In an effort to have more meaningful conversations, I propose the following poor analogies based on humans inhabiting space.
The "Big Room"
Consider a really big room with one door. That door is where all people living in the room enter and exit. Also, there is a single toilet in the middle of the room. Also consider that we allow anyone to come in and one of the inhabitants is a bit crazy and loves to run around the room at full speed randomly bouncing off the walls. While this "big room" is multi-tenant (more than one human could live there) it doesn't well partition or protect inhabitants from each other. Also, the use of the toilet (a common resource) might be a bit more than embarrassing to say the least. I think most people I have talked to would consider this environment to lack even the weakest definitions of multi-tenancy.
The "Doorless Single Family Home"
Consider a typical North American single family home but remove all the internal doors. In this new analogy, we might still have crazy inhabitants (I have two and they are called kids). We can start to partition them from the rest of us by putting then in a door and they only once and a while escape into the areas affecting others. Now the toilet and other shared resources are easier to use safely, but still not as safe as you'd want. One other big change is the type of inhabitants and their ability to share. In a family, likely they all have semi-common goals and one won't destroy common resources (the toilet) and if they do the family works to ensure that it doesn't happen again. Finally, there is benefit of this family of living together as they likely share their services freely.
The improved "Single Family Home with Doors".
Consider the previous example, but add back in the typical doors - doors that can be opened and closed, but likely not locked. Now our private moments are improved. Also, the crazy kids won't bounce out of their rooms as frequently. The doors are there to help mistaken bad interactions, but the doors can be opened freely to help achieve family goals more quickly than with the doors closed.
The "Fraternity/Sorority house"
Continuing the poor analogy, what if we make the inhabitants have similar, but more divergent goals than a single family. Of course all of the inhabitants of a fraternity or sorority house want to graduate college and they might be working on similar subjects that they could share information and learning amongst themselves, but sometimes you really don't want your co-inhabitant to enter your part of the house. When that co-inhabitant is drunk (never happens in college, right?), you really would like a locked door between you and them. The co-inhabitant isn't really meaning to cause you harm, but they could cause you harm none-the-less, so you probably added a locked door just in case. Ok, I'll admit I now have totally lose the part of the analogy of the toilet, but likely there are still shared resources that the house works to protect and shared responsibly.
The "Apartment Building"
Now we finish the analogies with what I think most people I talk about multitenancy consider from the start. Consider an apartment where every tenant gets his or her own lockable front door. Also, all of their toilets or important resources are protected to just them. The inhabitants don't have any common shared goals. Therefore, the apartment living conditions make sense. However, these living conditions can be problematic in two ways. First, this is a more costly way to live and operate both for each inhabitant and their non-shared resources. Second, if any of the inhabitants have any shared goals, their lack of internal doors means a much slower communication channel and forward progress will be slower.
The Wrap-Up
Now going back to non-analogy aspect. Many of the NetflixOSS projects (Eureka/Asgard/etc) come from a model that I think best is described by "Doorless Single Family Home". There is nothing wrong with that for the type of organization Netflix is and likely when deployed inside of Netflix there are more doors added beyond the public OSS. At IBM, in our own usage I believe we need at least "Single Family Home with Doors" mostly to add some doors that protect us from new users of the cloud technology from accidentally impacting others. Some have argued that we need the "Fraternity/Sorority house" adding in locked doors until people are confident that people won't even with unlockable doors impact others. Adding locking of doors means things like adding owner writable only namespaces to Eureka, locking down which clusters can be changed in Asgard, providing segmented network VLAN's, etc. Finally, if we ever looked to run this fabric across multiple IBM customers (say Coke and Pepsi), it is hard to argue that we wouldn't need the full "Apartment Building" approach.
I hope this helps others in discussing multitenancy. I hope my own team won't get tired of these new analogies.