SVForum Meetup: Building a Cloud Platform for Real-Time Video: Learnings and Challenges

This meetup on Building a Cloud Platform for Real-Time Video: Learnings and Challenges was a very captivating presentation — except in the beginning, during introductory slides, when it was somewhat slow-moving¹ — on how BlueJeans have created a good recipe for videoconferencing service in the cloud.

SVForum Cloud Services SIG: "The Future of the Data Center"

By the title of the meetup, namely "The Future of the Data Center", you might conclude that the two scheduled talks would be about how to architect future data centers, racks, power, cooling, etc. But, the talks were a bit more focused on only two, but important, aspects of enterprise clouds:

Brad Stone, Nexenta: Cloud Storage.

Dave Roberts, ServiceMesh: The Big Win.

Cloud Fabrics and Virtual Networks

In the last couple of years, nearly every networking equipment vendor is touting their new data center fabric aimed at the creation of clouds. Take a look at the [partial] list below, culled from the respective web sites:

Company	Fabric Offering
Arista Networks	????
Brocade Communication Systems, Inc.	Data Center Fabric
Cisco Systems	Unified Data Center Fabric
Extreme Networks	Open Fabric Data Center
Force10 Networks	Open Cloud Networking
HP	FlexFabric
IBM	BladeCenter
Juniper Networks	QFabric
NEC	????

At the Silicon Valley Cloud Computing Group: Zynga presents a Real World Case Study

If you believe cloud computing is still in its infancy, you have not learnt about Zynga. While Zynga have a hybrid cloud that powers their game servers involving their data center, a private cloud, zCloud, and a public cloud, namely Amazon Web Services (AWS), the more interesting fact is that they expect to be profitable in 2011 with a net margin of 35%, or a net profit of $630m on a $1.8B revenue.

At the Cloud and Virtualization SIG on Virtual Networks

This SDForum Cloud and Virtualization SIG meeting saw a set of very exciting set of presentations by vCider, Big Switch and Cisco. There have been mature commercial availability of CPU and storage clouds, but there have been no rich commercial offerings of [virtual] networks¹.

Virtual Networking and Clouds.

Recently, in a blog post written by Chris Marino, titled Virtual Networking, It’s Not Just a VLAN…, I came across a beautiful definition of what a virtual network is:

"A virtual network is a network you control that runs on top of another network that you don’t control."

Cloud Computing according to Yahoo!

In a presentation yesterday at Cloud Computing at Yahoo! Lessons, Challenges and Futures, Yahoo! Cloud Wrangler Geoff Arnold presented a view of how the Yahoo network needs to evolve in the present day cloud computing environment.
Although the actual extent of the present day Yahoo network could not be divulged for obvious reasons, Geoff argued that public cloud providers like Amazon Web Services (AWS) could not support the functionality that is expected of the Yahoo network. Even if the public network were to be made use of for a 24x7x365 production network, Yahoo would have to rent bare metal CPUs networked and storaged¹ together, and I do not know if the economics for such a deployment is already there. (We know, though, that Netflix uses AWS for streaming movies to its subscribers). This is probably true of other network based services such as Facebook's, Google's, etc.

Yahoo! Cloud Architectural Vision

The Yahoo! cloud architectural vision that was laid out — slide 31 on Slideshare.net — somewhat explains why general purpose public cloud services may not be sufficient: There seems to be substantial functionality that is somewhat Yahoo-specific that is not necessarily readily available in the public cloud. And, Geoff talked about "planet scale" for most of the functionality Yahoo delivers. In addition to the familiar layers of SaaS, PaaS and IaaS, that are probably richer than their run-of-the-mill counterparts, he also sees a Knowledge as a Service (KaaS) layer sandwiched between SaaS and PaaS.

Overall, you come away with the awesome feeling that the Yahoo network is really very big.

The video of the presentation is at the Silicon Valley Cloud Computing Group channel at Ustream.

---

¹We can regard this kind of journalistic license [of creating new verbs on the fly] as an advantage of the computer science profession.

Proud Recipient of a Google Chrome OS Notebook Cr-48!

Happiness comes to you in different ways. Today, I was surprised, along with 199 others I believe, at the HTML5 Talk event conducted by Google at their campus in Mountain View. While HTML5 was definitely talked about, the key surprise was kept so until the last few minutes of the presentation by the Chrome OS product manager, when he announced that everyone attending the talk would get a Cr-48, as part of their pilot program!

Cloud computing is certainly taking hold, and you can read about Google CEO Eric Schmidt's thoughts at the announcement. There are several reviews already out on the Cr-48, including this one by IT World.

Here's a first impression of an engineering major, a sophomore — my son — on the Cr-48:

Microsoft and Apple have to rethink their computer strategy!

Nothing but the Web.

Netflix moving to Amazon Cloud by Hien Luu - A Review.

It is not often that you get to listen to a story that describes how a massively scalable application — as of October 2010, Netflix has nearly 17 million paid subscribers for their streaming solution — has been developed for the cloud, in this case the Amazon cloud, known as Amazon Web Services (AWS). Hien Luu described such a story. He was part of the implementation team at Netflix, and is presently with LinkedIn.
The event was conducted by Software and Architecture Modeling (SAM) Special Interest Group of the SDForum.

Netflix Moving To Cloud

View more presentations from hluu.

He made the presentation livelier by sprinkling interesting questions about Netflix throughout his presentation, and awarding a copy of the book titled Host Your Web Site in the Cloud to whoever in the audience answered a question with the greatest accuracy¹.

Hien's presentation involved four sections (Slide #2):

• Why Amazon cloud? The answer, as you would expect, is that "Netflix is not in the business of creating and operating data centers", even though they currently have 2 data centers for their private data, and the Amazon cloud proved to be an effective platform for hosting the Netflix streaming solution.
• Tour of AWS. This tour was a very useful introduction to AWS, particularly since the later portion of the talk makes use of several of the AWS components in creating the streaming solution.
• How Netflix leverages [the cloud]. Netflix uses ~4000 EC2² instances during encoding movies into digital format for streaming, typically a 15-day exercise, while ~1000 EC2 are normally deployed for servicing subscribers. The storage is of the order of petabytes. The storage for movies, built on AWS S3, has a rather large latency — 300 mS — and hence the S3 is front-ended by Memcached, and the resulting latency is brought down to about 5 mS. (Slide #30). Slide #32 highlights the storage usage: e.g., ~800M objects in SimpleDB, ~17M objects in S3. Slide #33 brings out the importance of some key features that would be desired in a cloud, but were missing from AWS, as Netflix prepared their implementation; Memcached is one of the missing services.
• Best practices. In this section, Hien highlighted the need for automation — Can you imagine setting up 4000 machines by hand? —, minimizing latency, ensuring storage persistence, instrumenting every step of the call, sharding, etc.

Overall, a lively evening, as you got to know how Netflix has prepared itself for paid subscriber growth for the streaming solution.

There were several announcements made at the beginning of the session:

• SAM SIG & Java SIG will be combined beginning January 2011
• Jian Zhang of GridX announced they have openings for software engineers — Hadoop, Java, Front-end UI programmers — in building a cloud based financial application for the Smart Grid.

---

¹One example question was "How many paid subscribers does Netflix currently have?". The answer is 17 million. There were more books than there were questions, and some lucky folks in the audience got the rest of them in the end.

²Elastic Compute Cloud.

The Action behind Clouds: Cisco's Acquisition of LineSider.

Clouds are good for cloud subscribers, but how can cloud providers offer on-the-fly provisioning such as is expected by the subscribers? Clearly, requests¹ for Infrastructure as a Service (IaaS) at a subscriber level — a certain combination of CPU, networking and storage — must be translated down to device-level, actionable, commands to both physical and virtual devices. And, these commands need to be effected as a single transaction in the cloud provider system. Cisco's intent last week, to acquire LineSider Technologies, brings this matter to the fore.

LineSider's white paper on Network Services Virtualization elaborates on the challenges by bringing out the limitations of point² products of the following types:

1. Configuration Management and Automation Engines
2. Inventory and Capacity Management
3. Network Alarm and Correlation Management

Their OverDrive product "... automatically initiates the creation of the required virtual machines (VMs). As the VMs are coming online the OverDrive virtualization engine defines and deploys the network access and security models across all required infrastructure devices (routers, switches, firewalls) as needed, to deliver the Cloud service to the defined end-users. The entire process is completed in seconds ...". Moreover, "Once the Business Policy is implemented and the Cloud service is active, the access and security models are bound to the end-point resources and persisted in the OverDrive engine."

What is the moral of the story? While it is important to create point technologies for the cloud, it is also important to think about their manageability in an 'end-to-end' scenario: The cloud subscriber is one of the ends, and the cloud provider is the other end, with all the devices in between — physical or virtual — associated with the cloud service in a persistent, auditable, manner.

---

¹Requests for cloud services are equally well applicable to both private enterprise cloud subscribers and public cloud subscribers.

²What were once 'end-to-end' solutions become 'point' solutions in an evolving system.

Do we need yet another blog on cloud computing?

Why do we need yet another blog on cloud computing? In this age of information overloading, responsible netizens need to clarify the reason for the creation of another blog.