Understanding True Cloud Computing: Could 10 year olds help?

Gartner's 2010 Hype Cycle report is out. Cloud Computing has now crossed the peak of hype curve.



Now, the question still is how many of us really know the true meaning of cloud computing? Wikipedia does its bit by explaining key concepts in cloud computing. Here is the closest definition I could see for Cloud Computing on wikipedia

Cloud computing is Internet-based computing, whereby shared servers provide resources, software, and data to computers and other devices on demand, as with the electricity grid.

It is a good starting point. But there are a number of things that are not well explained. To be fair with everybody, it is not easy to define cloud computing ! There is so much information out there that tries to explain the true meaning of cloud computing but it is hard to pinpoint the key parts.

Here is a very funny video of how 10 year olds explain what cloud computing is



I don't necessarily agree with everything in the video. When such kind of information is produced, the focus tends to shift on subset or related technologies based on who is sponsoring the content. The end result is that, you hear the things that sponsor wants you to listen. This might also mean you are missing the big picture similar to blind men and elephant tale.

The best information that I have found so far is from James Staten. The good thing about him is that he makes it easy for readers to help understand cloud computing. Here is how James defines cloud computing -

Cloud Computing is a standardized IT capability (services, software or infrastructure) in a pay-per-use, self-service way

The 3 main layers of cloud computing could be described as



Finally, it is important to distinguish some aspects of cloud computing from the traditional IT. Here is a great illustration:





In 2011, we are likely to see more and more customer transitions towards cloud. In other words, rather than "if", it is going to be a question of "when" the transition will take place! There have been several solutions that have been rebranded to look like cloud computing solutions. So, before somebody claims that they have cloud computing solution, it is important to make sure it truly meets basic characteristics for a cloud solution. Otherwise it would be just a case of "cloud-washing" !

I hope some of the things that I referred here would be useful for you to understand what "true" cloud computing is all about.

Google's attempt to bolster the security via crowdsourcing

Security remains on of top concerns in cloud computing and rightly so. But, how could a cloud service provider ensure that it is doing everything possible to address the security issues to an extent that customers feel comfortable?

Google is adopting a radical approach towards addressing these issues: crowdsourcing. Earlier this month, Google announced a cash reward offer for any interested individual (aka hacker) who could report vulnerabilities on its Web properties. The vulnerability reward program promises to pay anywhere between $500 to $3100 depending on severity of the issue. The company is hoping that the reward program will attract enough enthusiasts (hackers, students, researchers and so on) such that they will reveal certain issues that are worth paying for.

The program is now available to subset of Google properties like

• *.google.com
• *.youtube.com
• *.blogger.com
• *.orkut.com

Some of the types of issues that Google is most interested in finding about include

• XSS
• XSRF / CSRF
• XSSI (cross-site script inclusion)
• Bypassing authorization controls (e.g. User A can access User B's private data)
• Server side code execution or command injection

More details are available here

By the way... one more thing. Google will double the reward if the person decides to donate the amount to charity !

Any takers out there???

Boom Boom: Dell buys Boomi

For those of us who have spent years integrating heterogeneous systems know it very well that integrating the systems is a daunting task. Some of the challenges like interoperability, versioning, security, user experience issues, performance and scalability issues make it hard to pull off really robust integration .

In recent years though a number of cloud providers have been able to expose a lot of capabilities via APIs making it easy for integrators to integrate third party applications with them. Enterprises who are using these cloud providers are increasing depending on the integrations to implement their mission critical business process. For example.. your company might be using SAP/Oracle for CRM needs and then buys another company that is using cloud based CRM like salesforce.com/NetSuite. Considering the amount of data that resides inside these apps and business processes that are built around it, it is never easy to come up with a solution that could be readily agreed upon. Your company can decide to dump one of the systems in favor of other. However that is not an obvious choice in the short term. Better approach would to keep using both the systems with right bridge between them that would take care of combining the data in a meaningful fashion.

The Cloud Integration companies promise to address that very need to integrate the on-premises application and/or cloud apps. The promise with the integration connector comes with the claim that there is no coding, software or appliance is involved. Boomi touts itself as #1 Integration Cloud. It is not clear to me in what sense they are #1. But just going by the number of partners they have in this space, it is clear that they have very thriving echosystem of partners.

The biggest news in cloud computing last week is got to be Dell buying Boomi ! I must agree that somebody buying Boomi is of no surprise to me. But Dell buying Boomi comes as a surprise. I have not been following what Dell portfolio looks like for Cloud Computing. I presume they have some good assets in infrastructure space but not anything in pure software as prominent as Boomi. So Boomi acquisition is very interesting. I would guess that rather than augmenting they have, Boomi will prove as a catalyst to buy few more. We will have to wait and watch !

Do you know the value of mathematical constant π (pi) ?

We all know that π (pi) is a mathematical constant whose value is the ratio of circle's circumference to the diameter. For many of us, it simply means 3.14 which happens to be an approximate value of pi. I am sure few enthusiasts among us know more accurate value as 3.1415..... However that is still the approximate value.

Going by the history, numerous attempts were made to arrive at the exact value. But nobody could arrive at a number with nth bit as zero meaning the value was still not accurate. So, the question remains- what is the "real" value of pi and whether it is possible to get the exact value?

One Yahoo! cloud computing engineer has claimed that he is able to compute specific bits to an extent that nobody in the world has been able to do that. The result- The two quadrillionth bit of π is 0!

This is one heck of a remarkable achievement. So how he did it? He used hadoop technology that Yahoo has championed for years now. The computing needed amazing computing power. In fact, the computation took 23 full days and required 1000 different machines using Hadoop.

This should be celebrated as a great cloud computing milestone!

Read more at BBC website

Mobile Patents: Who is suing whom anyway?

Well...This is not exactly around cloud computing. It is all about mobile computing. But none the less... it is hard to ignore these news coming about the squabbles among the companies in mobile space.

As mobile computing gets pervasive, companies are vying to stay at the top of the top of increasingly competitive mobile market. Recently Microsoft sued Motorola over its line of Android-based smart phones. The popularity of Android software is also at the heart of a legal battle between Oracle and Google.

Guardian published an interesting chart around who is suing who in mobile business.



A long battle looms before it is clear which companies would be the winners. However one thing is clear ultimately it is the attorneys who would make quick bucks on this !!!

Using OAuth for Desktop Apps

There has been a lot of discussion lately around Twitter's use of OAuth. I had added a post around earlier around the countdown to OAuth. Now that the Twitter apps are forced to use OAuth, new issues and complains are popping up.

A post by Ryan Paul has attracted a lot of attention in his recent post around OAuth.

A lot of issues that he mentions could be attributed to Twitter's implementation of OAuth rather than generic OAuth itself. However one core issue that he brings up is around suitability of OAuth for desktop applications. The OAuth version 1.0a happens to be the latest version of the specification. I was under impression that OAuth 1.0a was meant to be used for web applications only. This is because the specification in general is pretty vague around use with desktop apps. However when I checked again, it does say clearly mentions this upfront: "OAuth is an open protocol to allow secure API authorization in a simple and standard method from desktop and web applications." desktop applications? Really?? I just think it is an overstatement to claim that OAuth is suitable for desktop oe mobile apps. Let's look at why it is not make it ideal to use for native app.

Credential Management Issues:
The API caller needs to identify itself by specifying who I am. That who I am is basically its OAuth credential (combination of consumer key and consumer secret). So in this case, this desktop and mobile app will need to have its own credentials that it needs to use. This credential management poses a couple of interesting challenges.

First, how does app gets its credentials? Is this something that app is shipped with? If so, would the same credentials be shared by every application that is installed? Once you start thinking about this process, you will realize that distribution and management of the credentials is a nasty problem.

Second aspect is around securing these credentials. This secret will be installed with the application. How can we ensure that these credentials would be secured inside the app? What kind of encryption/decryption would be used to keep the credentials. As Ryan Paul pointed out in his post, he could crack the OAuth credentials of the Twitter sample program itself :-(

The issues mentioned above pose less of an issue in server environment. The credentials are secured on the server typically at one place inside the server app. The users typically do not have any access to the installation.

Request Token Authorization Issue:
As part of the OAuth flow, user needs to authorize the access to his/her content. It is achieved by sending verifier code for request token. The OAuth 1.0a specification allows using out-of-band (oob) way where verifier code is handed to the user rather than as a part of browser redirect. However this is not ideal. This is because, the user is then supposed to key in the verifier back in the app. Various ways have been invented to work around this step. However none of them very user friendly. Earlier this year I read around half of the OAuth flows that are initiated on iPhone are never completed.


So, in summary, as I elaborated here any attempt to force OAuth for desktop apps is problematic. Google Buzz and Facebook have both tried to solve this issue in their own way outside OAuth 1.0a specification. Some of this is coming together in OAuth 2.0 draft in the form of different profiles. However it is still something we need to watch and wait for!

NoSQL ... No Kidding

Cloud computing puts a lot of demands over databases. While traditional databases are very good in what they do, they may not be the best fit for cloud computing.

Consider this scenario. Your cloud offering gains popularity and results in hundreds/thousands of new users overnight. You do want to make sure that your cloud offering can support those users and dramatic increase in demand. You might have started with single in house server. Now you must scale well by making use of multiple nodes. Adding new hardware and distributing the load is not trivial process. It also has cost implications. Research in this area suggests that it comes down to fulfilling the 3 important characteristics

* Consistency
This means that each client must always has the same view of the data.

* Availability
This means that all clients must be always read and write.

* Partition tolerance
This means that the system must work well across physical network partitions.

While it is desirable that DBMS should address all the three requirements, it is not possible. The underlying constrains and issues are well captured and described in what is popularly referred to as CAPS theorem.

So, depending on which characteristics are important, more desirable solutions are available. Nathan Hurst has done great job in classifying them.



The solutions are now collectively referred as NoSQL databases (sometimes distributed databases). The NoSQL databses are getting increasing popular to an extent that somebody who is starting out with new cloud offering should think about the value they bring in from the inception. I will try to discuss some of the popular distributed databases in subsequent post.

Deploying at the speed of light

In SaaS environment, it is very important to make sure the service is "always" available. The downtimes in the service impacts wide range of users especially if they are coming from a different timezone. Downtime between 3 AM to 6 AM in US ET may not seem disruptive to anybody in US. However for somebody who is working in Asia or Australia this might eat into major chunk of one's productive hours.

Anyway, the point I am at is in order to keep the service up for the most of the time, it is important to minimize the downtimes. Downtimes are usually needed for emergency patches or software refreshes. That is usually the only way to roll out new features and services. In SaaS environment, it might be daunting task to rollout new upgrade to a number of services. The downtime might then range from few minutes to hours depending on complexity of the environment.

The presentation by Larry Gadea demonstrates some of the challenges that are involved in deploying new builds to servers.

For starters, in case of Twitter,the deployment time was around 40 minutes for a server. With a number of innovative ideas the team was able to reduce it few seconds.

Here is the first general issue. If the farm is to be updated by copying the updates from one central server, usually the central server is the main bottleneck.

One way to solve the issue would be to use tree distribution. However it runs into a problem if one or more nodes in the tree fail resulting in servers that are not updated.


Twitter was able to use an approach that involved using techniques below



The new approach integrated with existing deployment app, Capistrano, resulted in getting the deployment time by about 99%. You can see it here

Twitter's Countdown to OAuth

OAuth 1.0 has been around for some time now. The adoption of OAuth has grown tremendously in last couple of years. The standard is evolving and there is significant interest in OAuth 2.0.

While companies are enthusiastically embracing OAuth, they are yet to dump their existing api authorization framework (example: salesforce.com). One can argue that drastic change like discontinuing support for non-OAuth based auth framework will impact hundreds (or thousands) of third party apps and might hurt the company. It is interesting though to understand what Twitter has announced. Twitter is doing courageous thing by publicaly announcing they are dumping BasicAuth in favor of OAuth. In other words, if you are to call Twitter APIs, you must use OAuth. The efforts needs to be appreciated. This is coming from a company which receives 75% of requests in the form APIs.

So, what really prompted Twitter to make that switch?

First of all, one can see that it is probably in Twitter's DNA to do the things right way. Twitter has a prime example of a company that is using Open Standards the way they are to be used. This has been obvious the way they did 'Sign in With Twitter' via OAuth rather than proprietary methods employed by Facebook in 'Facebook Connect'.

Now, let's dive into some technical aspects. In BasicAuth, user's credentials need to be sent with the API calls and it makes itself vulnerable to Man-in-the-Middle attacks. While this can be addressed by using SSL (secure transport layer), unless you do it right, it is problematic.

The next issue is around password management. Using BasicAuth means the API caller is forced to send the user's userid and password. That means users do need to share their passwords with API caller (external app). How many of us use the same password for multiple site? Would you be willing to share your password with third party app that you are not much familiar with? What happens if the api caller's website is breached? When you change your password, what happens to integration?....Once you start thinking on those lines, you will quickly realize that it is always problematic to share the passwords and trust different parties to manage the secrets. These slew of problems simply go away with OAuth.

Finally.. the service providers get enormous flexibility and traceability with OAuth. They can associate each API call to not only user who is making the call but also to each third party app that is making the call. It helps them to understand and analyze who is making those API calls.

Read more at http://www.countdowntooauth.com

There is an API for that !

Open APIs are exploding in numbers. Recently RWW published an article discussing the services that are already processing billions of APIs a day. The statistics is simply mind boggling. The top service providers are Google and Facebook. This should not be surprising. What is surprising is that they are getting 5 billion API calls a day ! That is right.. 5 billion calls a day !! Way to go !!!

Another interesting aspect is that for some of the services like Twitter or Salesforce more than half of the requests are submitted via APIs. That tells a lot around the usage model for these services.

For the pure techies.. if you are still wondering how much of this is SOAP vs REST, it turns out REST is popular 4:1 margin. This is hardly a race.

The days of whether we need APIs are gone. The picture below summarizes it the best.

Just like there is an app for that, it would be fair that there is an API for that!