5 Things I learned during the RedHat Summit

I attended Red Hat Summit earlier this month. I accidentally ran into a few of my old colleagues by accident and everyone was wondering where I have been for the last few years! Yes.. There is linked in that I occasionally visit and some local meet ups that I attend. But it is still pretty hard to keep up with. It has been a while ( 5-6 years actually) I haven't blogged about anything. So here I am blogging about some recent experiences ...

Red Hat Summit was a great community meetup with lots of interesting discussions and presentations around new (and also not-so-new) technologies. With 5000+ attendees in attendance and most of them coming from midsize and large companies, it was great to interact with a number of people. These guys were not your typical technology enthusiasts that you meet at a tech Meetup but technical leaders from established companies with ambitions to grow and adopt/make relevant technologies for growth . Here are my top 5 takeaways...

1. Enterprises are moving to the cloud at a much faster clip
It was great to see enterprises of all sizes moving to the cloud as fast as they can. These were multi-billion dollar financial institutions and government agencies talking about how they are overcoming traditional barriers like security, governance and regulatory compliance during the migration process. Keeping applications agnostic to underlying deployment infrastructure is very critical for a number of businesses. There were also presentations and discussions about using a middleware platforms like OpenStack and OpenShift container platform eliminates the direct dependency on cloud providers. Executives from cloud service providers like Amazon, Microsoft, Google, Rackspace and CenturyLink all were on the stage describing the support they are providing to make the transition successful. My favorite presentation was MS SQL Server running on Linux containers hosted on Azure and being accessed by dockerized apps running MS .NET and java apps alike.


2. Containers have already started to rule the app world
It was amazing to see a lot of momentum behind docker and containerization of the deployment platforms. Again these are not those Silicon Valley startups but intelligent banks like Deutsche bank moving to the containerization platforms. This is going to be an interesting area to follow for the next several years until it becomes the mainstream.

3. Kafka has become the defacto messaging solution
Move over MQ … companies of all sizes are looking at Kafka as the next generation alternative. With the growth in the data and messages, it is becoming more important to handle the data. Coupled with the microservices philosophy of 'dumb pipes', architects are loathe to push the orchestration and mediation to the middleware hub style solutions.

4. DevOps in NOT only about build and deployment automation
These days a number of companies are pitching software products that allow to create and manage the devops pipeline. However the pipeline is just one part of it. Ultimately it is a combination of tools, people and processes that yield true results. Educating and empowering the teams, setting up feedback loop, changing the mindset and making stakeholders comfortable in their new riles are the most challenging aspects. Often it turns out to be the real challenge where adoption fails. It was great to see several banks sharing their journey to DevOps and how they moved away from ‘create ticket’ mode to ‘self service’ as a measure of true agility.

5. Planning as-we-know-it is dead !
This comes from RedHat CEO.  It is a philosophical take and change in mindset around how companies tackle the problems and build solutions. The days of ‘traditional’ approach of 5 year strategic plans and long release cycles are over.  The companies that are successful today are the ones reinventing themselves with iterative cycle of Try -> Learn ->Improve. Actually this is practically the mode with which every startup works but it is a refreshing to see companies of all sizes buying into the same approach via thinking 'lean' and MVPs.

You can catch some of action here

LinkedIn: Congratulations ! You are 1 in 100 million !!!

Congratulations for being one in a 100 million users. In March this year LinkedIn reached milestone of 100 million registered users and I am sure you are one of those 100 million. The press release said -

We’re growing at roughly one million new LinkedIn members every week – that’s more than one new professional joining the network every second !

Here is some more info if you are interested -





While there is no doubt LinkedIn is hugely successful, the fact remains that it is only 1/5th or 1/6th size of that famous social network called Facebook. I know that LinkedIn is mainly targeted towards professionals but what strikes me is that amount of time a user spend on LinkedIn may not even compare with that on Facebook. In other words, LinkedIn is missing the stickiness that you would expect from these social sites that makes users visit them over and over again.

Curious to find out, what is missing I did a quick round of LinkedIn. I see that it has features like Groups, Company News and LinkedIn Answers. I would imagine they have right featureset for what they are meant for. Also it has basic app marketplace. But where are the apps that would be of interest to me? Take something like Hiring. Hiring is a very social activity since a lot of positions are filled based on networking and trust. Where is an app that for that? I do not see it. May be there is something like it available if you pay for a premium membership. But my point is that I believe apps like these would be of immense help to all the users.

One final point, now that LinkedIn had a very successful IPO. The stock doubled right after the opening. This buying frenzy ended up making the company valuation to be around $10B. If you do the calculation, that means every user is valued $100. Compare that to Microsoft's Skype acquisition which valued each user to be worth roughly around $13. I am not a stock expert nor I claim to be one. So rather than pondering over why is that the case, at least let's enjoy our moment of glory as one of the 100 million LinkedIn users !

Will IBM Watson turn into Dr. Watson?

IBM's Watson computer put on quite a show earlier this month in the game show Jeopardy ! Well, if you have not heard about Jeopardy, there is a good chance that you are not based in North America because it is a US based show. The unique thing about this show is that it not a trivial question and answers type of quiz program but it has questions with a lot of puns. That really makes it pretty hard for somebody (like me!) to get the answers right quickly ! Watson was pitted against 2 of the all time champions in Jeopardy. So by no means it was an ordinary match up. Watson came out on top after an interesting battle. You can catch glimpse here

As you would have guessed, Watson is an artificial intelligence computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project. For the technology enthusiasts, here is some information that will excite you. Watson is made up of a cluster of ninety IBM Power 750 servers with a total of 2880 POWER7 processor cores and 16 Terabytes of RAM. Watson's software runs thousands of algorithms on the questions it receives in parallel and then compares the answers to decide the best answer by attaching a percentage value. Watson's programming is written in both Java and C++ and uses Apache Hadoop distributed file system, Apache UIMA (Unstructured Information Management Architecture) framework.

We should herald Watson's feat as a new breakthrough in artificial intelligence. Watson demonstrates great ability to perform natural language processing. Watson breaks the questions into parts and tries to understand the exact question that it needs to answer. Then it uses a number of analytical techniques to come up with right answers.

While Jeopardy proved to be a great showcase for Watson to unleash its glory, it was not just built for the jeopardy showdown. After all it came with a huge price tag. Its unique ability to do natural language and analytical processing makes it a workhorse to solve array of problems. There are countless possibility in which it could be leveraged. There are some obvious areas where it could be of immense value.

IBM is reported working in healthcare with some hospitals, doctors, researchers and universities to build innovative solution in the medical field. We cannot expect Watson to diagnose the illness on its own (yet), however it can be an excellent tool that the doctors that would help them make better decisions.

For example, Watson might recognize that the kidney failure in our patient is linked to kidney failure in a patient in Buffalo and another in San Antonio; all three patients, he might inform me, were taking a “natural” weight loss supplement that contained a Chinese herb, aristolochia, that has been associated with more than 100 cases of kidney failure.

Other areas where it could be of immediate help would be customer call centers, employee help desks, equity portfolio analysis and so on. This link even claims that it would be great tool for match making !

It is too early to predict how IBM and others would be able to turn Watson into something more meaningful than a machine that vanquished its human opponents. But I sincerely think feel that it has potential to do that !

Understanding True Cloud Computing: Could 10 year olds help?

Gartner's 2010 Hype Cycle report is out. Cloud Computing has now crossed the peak of hype curve.



Now, the question still is how many of us really know the true meaning of cloud computing? Wikipedia does its bit by explaining key concepts in cloud computing. Here is the closest definition I could see for Cloud Computing on wikipedia

Cloud computing is Internet-based computing, whereby shared servers provide resources, software, and data to computers and other devices on demand, as with the electricity grid.

It is a good starting point. But there are a number of things that are not well explained. To be fair with everybody, it is not easy to define cloud computing ! There is so much information out there that tries to explain the true meaning of cloud computing but it is hard to pinpoint the key parts.

Here is a very funny video of how 10 year olds explain what cloud computing is



I don't necessarily agree with everything in the video. When such kind of information is produced, the focus tends to shift on subset or related technologies based on who is sponsoring the content. The end result is that, you hear the things that sponsor wants you to listen. This might also mean you are missing the big picture similar to blind men and elephant tale.

The best information that I have found so far is from James Staten. The good thing about him is that he makes it easy for readers to help understand cloud computing. Here is how James defines cloud computing -

Cloud Computing is a standardized IT capability (services, software or infrastructure) in a pay-per-use, self-service way

The 3 main layers of cloud computing could be described as



Finally, it is important to distinguish some aspects of cloud computing from the traditional IT. Here is a great illustration:





In 2011, we are likely to see more and more customer transitions towards cloud. In other words, rather than "if", it is going to be a question of "when" the transition will take place! There have been several solutions that have been rebranded to look like cloud computing solutions. So, before somebody claims that they have cloud computing solution, it is important to make sure it truly meets basic characteristics for a cloud solution. Otherwise it would be just a case of "cloud-washing" !

I hope some of the things that I referred here would be useful for you to understand what "true" cloud computing is all about.

Google's attempt to bolster the security via crowdsourcing

Security remains on of top concerns in cloud computing and rightly so. But, how could a cloud service provider ensure that it is doing everything possible to address the security issues to an extent that customers feel comfortable?

Google is adopting a radical approach towards addressing these issues: crowdsourcing. Earlier this month, Google announced a cash reward offer for any interested individual (aka hacker) who could report vulnerabilities on its Web properties. The vulnerability reward program promises to pay anywhere between $500 to $3100 depending on severity of the issue. The company is hoping that the reward program will attract enough enthusiasts (hackers, students, researchers and so on) such that they will reveal certain issues that are worth paying for.

The program is now available to subset of Google properties like

• *.google.com
• *.youtube.com
• *.blogger.com
• *.orkut.com

Some of the types of issues that Google is most interested in finding about include

• XSS
• XSRF / CSRF
• XSSI (cross-site script inclusion)
• Bypassing authorization controls (e.g. User A can access User B's private data)
• Server side code execution or command injection

More details are available here

By the way... one more thing. Google will double the reward if the person decides to donate the amount to charity !

Any takers out there???

Boom Boom: Dell buys Boomi

For those of us who have spent years integrating heterogeneous systems know it very well that integrating the systems is a daunting task. Some of the challenges like interoperability, versioning, security, user experience issues, performance and scalability issues make it hard to pull off really robust integration .

In recent years though a number of cloud providers have been able to expose a lot of capabilities via APIs making it easy for integrators to integrate third party applications with them. Enterprises who are using these cloud providers are increasing depending on the integrations to implement their mission critical business process. For example.. your company might be using SAP/Oracle for CRM needs and then buys another company that is using cloud based CRM like salesforce.com/NetSuite. Considering the amount of data that resides inside these apps and business processes that are built around it, it is never easy to come up with a solution that could be readily agreed upon. Your company can decide to dump one of the systems in favor of other. However that is not an obvious choice in the short term. Better approach would to keep using both the systems with right bridge between them that would take care of combining the data in a meaningful fashion.

The Cloud Integration companies promise to address that very need to integrate the on-premises application and/or cloud apps. The promise with the integration connector comes with the claim that there is no coding, software or appliance is involved. Boomi touts itself as #1 Integration Cloud. It is not clear to me in what sense they are #1. But just going by the number of partners they have in this space, it is clear that they have very thriving echosystem of partners.

The biggest news in cloud computing last week is got to be Dell buying Boomi ! I must agree that somebody buying Boomi is of no surprise to me. But Dell buying Boomi comes as a surprise. I have not been following what Dell portfolio looks like for Cloud Computing. I presume they have some good assets in infrastructure space but not anything in pure software as prominent as Boomi. So Boomi acquisition is very interesting. I would guess that rather than augmenting they have, Boomi will prove as a catalyst to buy few more. We will have to wait and watch !

Do you know the value of mathematical constant π (pi) ?

We all know that π (pi) is a mathematical constant whose value is the ratio of circle's circumference to the diameter. For many of us, it simply means 3.14 which happens to be an approximate value of pi. I am sure few enthusiasts among us know more accurate value as 3.1415..... However that is still the approximate value.

Going by the history, numerous attempts were made to arrive at the exact value. But nobody could arrive at a number with nth bit as zero meaning the value was still not accurate. So, the question remains- what is the "real" value of pi and whether it is possible to get the exact value?

One Yahoo! cloud computing engineer has claimed that he is able to compute specific bits to an extent that nobody in the world has been able to do that. The result- The two quadrillionth bit of π is 0!

This is one heck of a remarkable achievement. So how he did it? He used hadoop technology that Yahoo has championed for years now. The computing needed amazing computing power. In fact, the computation took 23 full days and required 1000 different machines using Hadoop.

This should be celebrated as a great cloud computing milestone!

Read more at BBC website

Mobile Patents: Who is suing whom anyway?

Well...This is not exactly around cloud computing. It is all about mobile computing. But none the less... it is hard to ignore these news coming about the squabbles among the companies in mobile space.

As mobile computing gets pervasive, companies are vying to stay at the top of the top of increasingly competitive mobile market. Recently Microsoft sued Motorola over its line of Android-based smart phones. The popularity of Android software is also at the heart of a legal battle between Oracle and Google.

Guardian published an interesting chart around who is suing who in mobile business.



A long battle looms before it is clear which companies would be the winners. However one thing is clear ultimately it is the attorneys who would make quick bucks on this !!!

Using OAuth for Desktop Apps

There has been a lot of discussion lately around Twitter's use of OAuth. I had added a post around earlier around the countdown to OAuth. Now that the Twitter apps are forced to use OAuth, new issues and complains are popping up.

A post by Ryan Paul has attracted a lot of attention in his recent post around OAuth.

A lot of issues that he mentions could be attributed to Twitter's implementation of OAuth rather than generic OAuth itself. However one core issue that he brings up is around suitability of OAuth for desktop applications. The OAuth version 1.0a happens to be the latest version of the specification. I was under impression that OAuth 1.0a was meant to be used for web applications only. This is because the specification in general is pretty vague around use with desktop apps. However when I checked again, it does say clearly mentions this upfront: "OAuth is an open protocol to allow secure API authorization in a simple and standard method from desktop and web applications." desktop applications? Really?? I just think it is an overstatement to claim that OAuth is suitable for desktop oe mobile apps. Let's look at why it is not make it ideal to use for native app.

Credential Management Issues:
The API caller needs to identify itself by specifying who I am. That who I am is basically its OAuth credential (combination of consumer key and consumer secret). So in this case, this desktop and mobile app will need to have its own credentials that it needs to use. This credential management poses a couple of interesting challenges.

First, how does app gets its credentials? Is this something that app is shipped with? If so, would the same credentials be shared by every application that is installed? Once you start thinking about this process, you will realize that distribution and management of the credentials is a nasty problem.

Second aspect is around securing these credentials. This secret will be installed with the application. How can we ensure that these credentials would be secured inside the app? What kind of encryption/decryption would be used to keep the credentials. As Ryan Paul pointed out in his post, he could crack the OAuth credentials of the Twitter sample program itself :-(

The issues mentioned above pose less of an issue in server environment. The credentials are secured on the server typically at one place inside the server app. The users typically do not have any access to the installation.

Request Token Authorization Issue:
As part of the OAuth flow, user needs to authorize the access to his/her content. It is achieved by sending verifier code for request token. The OAuth 1.0a specification allows using out-of-band (oob) way where verifier code is handed to the user rather than as a part of browser redirect. However this is not ideal. This is because, the user is then supposed to key in the verifier back in the app. Various ways have been invented to work around this step. However none of them very user friendly. Earlier this year I read around half of the OAuth flows that are initiated on iPhone are never completed.


So, in summary, as I elaborated here any attempt to force OAuth for desktop apps is problematic. Google Buzz and Facebook have both tried to solve this issue in their own way outside OAuth 1.0a specification. Some of this is coming together in OAuth 2.0 draft in the form of different profiles. However it is still something we need to watch and wait for!

NoSQL ... No Kidding

Cloud computing puts a lot of demands over databases. While traditional databases are very good in what they do, they may not be the best fit for cloud computing.

Consider this scenario. Your cloud offering gains popularity and results in hundreds/thousands of new users overnight. You do want to make sure that your cloud offering can support those users and dramatic increase in demand. You might have started with single in house server. Now you must scale well by making use of multiple nodes. Adding new hardware and distributing the load is not trivial process. It also has cost implications. Research in this area suggests that it comes down to fulfilling the 3 important characteristics

* Consistency
This means that each client must always has the same view of the data.

* Availability
This means that all clients must be always read and write.

* Partition tolerance
This means that the system must work well across physical network partitions.

While it is desirable that DBMS should address all the three requirements, it is not possible. The underlying constrains and issues are well captured and described in what is popularly referred to as CAPS theorem.

So, depending on which characteristics are important, more desirable solutions are available. Nathan Hurst has done great job in classifying them.



The solutions are now collectively referred as NoSQL databases (sometimes distributed databases). The NoSQL databses are getting increasing popular to an extent that somebody who is starting out with new cloud offering should think about the value they bring in from the inception. I will try to discuss some of the popular distributed databases in subsequent post.

Deploying at the speed of light

In SaaS environment, it is very important to make sure the service is "always" available. The downtimes in the service impacts wide range of users especially if they are coming from a different timezone. Downtime between 3 AM to 6 AM in US ET may not seem disruptive to anybody in US. However for somebody who is working in Asia or Australia this might eat into major chunk of one's productive hours.

Anyway, the point I am at is in order to keep the service up for the most of the time, it is important to minimize the downtimes. Downtimes are usually needed for emergency patches or software refreshes. That is usually the only way to roll out new features and services. In SaaS environment, it might be daunting task to rollout new upgrade to a number of services. The downtime might then range from few minutes to hours depending on complexity of the environment.

The presentation by Larry Gadea demonstrates some of the challenges that are involved in deploying new builds to servers.

For starters, in case of Twitter,the deployment time was around 40 minutes for a server. With a number of innovative ideas the team was able to reduce it few seconds.

Here is the first general issue. If the farm is to be updated by copying the updates from one central server, usually the central server is the main bottleneck.

One way to solve the issue would be to use tree distribution. However it runs into a problem if one or more nodes in the tree fail resulting in servers that are not updated.


Twitter was able to use an approach that involved using techniques below



The new approach integrated with existing deployment app, Capistrano, resulted in getting the deployment time by about 99%. You can see it here

Twitter's Countdown to OAuth

OAuth 1.0 has been around for some time now. The adoption of OAuth has grown tremendously in last couple of years. The standard is evolving and there is significant interest in OAuth 2.0.

While companies are enthusiastically embracing OAuth, they are yet to dump their existing api authorization framework (example: salesforce.com). One can argue that drastic change like discontinuing support for non-OAuth based auth framework will impact hundreds (or thousands) of third party apps and might hurt the company. It is interesting though to understand what Twitter has announced. Twitter is doing courageous thing by publicaly announcing they are dumping BasicAuth in favor of OAuth. In other words, if you are to call Twitter APIs, you must use OAuth. The efforts needs to be appreciated. This is coming from a company which receives 75% of requests in the form APIs.

So, what really prompted Twitter to make that switch?

First of all, one can see that it is probably in Twitter's DNA to do the things right way. Twitter has a prime example of a company that is using Open Standards the way they are to be used. This has been obvious the way they did 'Sign in With Twitter' via OAuth rather than proprietary methods employed by Facebook in 'Facebook Connect'.

Now, let's dive into some technical aspects. In BasicAuth, user's credentials need to be sent with the API calls and it makes itself vulnerable to Man-in-the-Middle attacks. While this can be addressed by using SSL (secure transport layer), unless you do it right, it is problematic.

The next issue is around password management. Using BasicAuth means the API caller is forced to send the user's userid and password. That means users do need to share their passwords with API caller (external app). How many of us use the same password for multiple site? Would you be willing to share your password with third party app that you are not much familiar with? What happens if the api caller's website is breached? When you change your password, what happens to integration?....Once you start thinking on those lines, you will quickly realize that it is always problematic to share the passwords and trust different parties to manage the secrets. These slew of problems simply go away with OAuth.

Finally.. the service providers get enormous flexibility and traceability with OAuth. They can associate each API call to not only user who is making the call but also to each third party app that is making the call. It helps them to understand and analyze who is making those API calls.

Read more at http://www.countdowntooauth.com

There is an API for that !

Open APIs are exploding in numbers. Recently RWW published an article discussing the services that are already processing billions of APIs a day. The statistics is simply mind boggling. The top service providers are Google and Facebook. This should not be surprising. What is surprising is that they are getting 5 billion API calls a day ! That is right.. 5 billion calls a day !! Way to go !!!

Another interesting aspect is that for some of the services like Twitter or Salesforce more than half of the requests are submitted via APIs. That tells a lot around the usage model for these services.

For the pure techies.. if you are still wondering how much of this is SOAP vs REST, it turns out REST is popular 4:1 margin. This is hardly a race.

The days of whether we need APIs are gone. The picture below summarizes it the best.

Just like there is an app for that, it would be fair that there is an API for that!