Linkedin - 99% Java Inside

For about 7 years that I have done active coding, it has been mostly using Java, a language I fell in love with when I wrote my first hello world program in it, way back in 1997. I liked the fact that it was natural for me to think, design and code in a language which was so close to OOP concepts. Everybody during that time in Bangalore seemed to be learning Java and java courses were selling like hot potatoes throughout ever mushrooming computer learning centers. Java was the language in demand and a passport to get a quick programming job. Some time later I came to know about EJBs and J2EE which were projected as some sort of Holy Grail of distributed systems. Everybody wanted to work on EJB projects. JSP/servlets were for lesser mortals. When I read EJB specification and J2EE book (I think it was wrox series) for first time, I felt like a student reading a high philosophy from the Zen Masters. It was beautiful. EJBs were the black art. Then sometime later came J2EE design patterns, pet shop and blueprints. You couldn’t; get through an interview if you didn’t know about Blueprints or if you didn’t know about session facade and value objects.

Well, those good old days…. lots of things happened after that within industry and with java language. I personally, have worked a bit with other languages like PHP which I got to learn because it seemed to be the lingua franca of internet. For me coding in PHP was difficult initially as I hated those nested arrays without which you couldn't do a lot in PHP and I missed beauty of strict typing of java. Exposure to internet made me realize that most large, high volume systems are written not in Java but in scripting languages. Possibly because such systems are easy to host and possibly because dynamic languages are natural fit for volatile business of internet. Whatever the reason, java has clearly lagged behind on web. Its focus has been behind firewall, within enterprise, backend processing. So for old time sake and love of a language, it is good to know when you come across a high volume internet site written entirely in java. Below is a presentation which explains how Linkedin runs on 99% java. Just to get an idea of scale, here are some numbers:

• 22 million members
• 4+ million unique visitors/month
• 40 million page views/day
• 2 million searches/day

Take a loot at architecture and how it has evolved. It is all about partitoning the data I guess and 1% is taken by C++ implmentation of custom JVM heap!!. Also read this post here.

View Upload your own

If embedded slide doesn't show up , use link to slideshare

URI design - Make it central to web application design

The beauty of HTTP is its simplicity. It is clear text protocol which essentially lists few methods (GET, POST etc) to operate on a resource. One of the cornerstones of HTTP protocol is concept of URIs. URI which is the address to reach a resource. A resource which could be abstract or physical. Simply put, URI is a string of characters which is reference to a resource. But it is amazing how this simple concept has been misused or underused. Architects and designers of enterprise software have hardly designed systems with URIs in mind (at least in pre-REST hype phase). So much so that sometime tasks a system is designed to do become hard or impossible to do.

Consider a system which is designed to act as web based employee directory or database. System records everything from employee's basic information, his photograph, his location, his contact details etc. So the primary function of such a system, one would assume is to, operate on set of resources which are employee records. And hence one would assume that every employee record would get represented by a URI which could be something like http://someempdir/employees/a/jhonSmith . This would mean that by passing the URI around, an employee can present his information to whosoever needs it. Plain and Simple.

But what happens if system is built ignoring basic tenets of URIs and HTTP? What if employee records are not URI reachable because system is perhaps using POST requests to do something which essentially is a GET request? So to access an employee record, one has to login to system and then use search functionality and then click on record to see the details but URI could be something like "http://someemdir/portals/wsd/somesbs/m orebs/search.asp?a=bs, making no sense to anybody. It would stay same for Jhon Smith and same for Ramesh kumar making It useless .And what if system does use Get request to access an employee record but generates URIs which can be mind numbing like http://someemdir/portals/wsd/som esbs/morebs/readuser.asp?resource=user&view=broad&application=e mpDirectory&dn=hN%3id=4%2COU%3Dusa%2COU%3DIN%2COU%3DEmployees%2CDC%3Dcom%2CDC%3Dabc%2CDC%3Dcom.

Now consider if I am handling a group of 100 employees and want to keep record of their details and share it with others, I can't use the employee database as I can't reach any employee record outside the system context. I can’t send a URI like http://someempdir/employees/a/jhonSmith to someone who needs Jhon's details. So I will have to create my own little database in an excel sheet and I will ask 100 guys to update it with their information. Employee information now is in two different places and I have to make sure that when guys leave my group or new guys join, I update my own little database and ask everybody else to make sure that their information in my database is in synch with what they have in their respective records in employee directory . And for all practical purpose the big database is all but redundant for me.

If system was designed with some attention to URIs and how information was to be represented, My task would have been easier. I would have just tracked a list of 100 URIs in a sheet. A list of pointers to central source of information. Now if there are 50 other people like me handling group of employees, one can imagine the time wasted in this inane activity of maintaining duplicated information. One can see many such examples in most of enterprise systems and that is why considerable amount of time is spent on unproductive tasks. Lots of times, manual tasks are done to gather information which is lying in all sort of system but is useless because of unnecessary barriers created in accessing it. A barrier could be as simple as not having readable, proper URI for a resource. From my own experience, the reason for this is overly focus on technologies, frameworks and implementation while designing a system, rather than key principles of information management.

While designing a web based system, it is important to consider URI design. In fact that should be beginning of design. Design how URI landscape would look like and how all resources in your system would be represented by URIs. If a product doesn’t support it, junk it.

Cloud Computing - A Design For Scalability?( Part1)

I have been hearing it for sometime in background, have noticed it being bandied about more frequently on my company's technical forums and have seen it appearing more regularly in my rss reader. The next buzzword , cloud computing. After spending few years in IT, I have become cicrumspect to buzzwords and hype that is generated around them and my own experience has been that doing IT is more simpler than it is actually made out to be. Anyway, yesterday I managed to pull myself to hear a talk on Cloud Computing and few slides later I got interested.

Just like in case of SOA, it will be futile to try to give an exact defintion of Cloud Computing. There are many depending upon what source you refer. Is Clound Coputing same as Grid and utlity computing or are there differences! Without going into it at this stage, the inference I could draw form my inital understanding of Cloud Computing is that - what SOA is to design of software, Cloud Computing is to deployment of software. SOA changes the way you look at application portfolio within an enterprise and Cloud Computing changes the way you look at utilising computing resources for deploying your applications. In fact SOA in hand with Cloud Computing takes Distributed computing to next evolutionaly stage. It is an evolution from distributed object paradigm (like CORBA, J2EE) to Distributed service paradigm. The abstraction is raised to a higher level. What clusters were for distributed object systems, Clouds could be for distributed service systems. It is a natural progression of good old distributed computing. In IT system architectures there always have been two main objectives:

1. Design For reuse
2. Design for Scalability

If design for reuse is more of software related concern, design for scalability is hardware related. Systems can only be truly scaled up by using more computing resource of CPU and memory. If SOA takes "design for reuse" to a higher level (even beyond enterprise boundaries at time), Cloud Computing takes "design for scalability" to next step. In a cloud you can achieve more scalability and more efficiently by transparently adding more computing capacity. And here when I say cloud computing I don't merely mean Amazon cloud or some other cloud implementations, but even clouds which enterprises can create behind thier own firewalls by using virtualisation. SOA makes us think outisde of siloed application and Cloud Computing does it for managing hardware resources. Instead of deploying various applications with differing usages in many different server in a datacenter, what if we could create , well ,...a cloud of these hardware resources and servers and have whole lot of applications deployed in cloud so that hardware is optimally utilised! And with Cloud Computing providers there would be another option of actually hiring out the hardware needs instead of buying your own capacity.

I think SOA with Cloud Computing and Virtualisation are what could be called disruptive technologies of future. Ok, you can ignore the "disruptive part". Just like SOA, Cloud Computing also has more business aspects to it and not mere technical aspects. In coming days, I will be more tuned to cloud computing and may be play with Amazon cloud and write about my experience with it.