Linkedin - 99% Java Inside
For about 7 years that I have done active coding, it has been mostly using Java, a language I fell in love with when I wrote my first hello world program in it, way back in 1997. I liked the fact that it was natural for me to think, design and code in a language which was so close to OOP concepts. Everybody during that time in Bangalore seemed to be learning Java and java courses were selling like hot potatoes throughout ever mushrooming computer learning centers. Java was the language in demand and a passport to get a quick programming job. Some time later I came to know about EJBs and J2EE which were projected as some sort of Holy Grail of distributed systems. Everybody wanted to work on EJB projects. JSP/servlets were for lesser mortals. When I read EJB specification and J2EE book (I think it was wrox series) for first time, I felt like a student reading a high philosophy from the Zen Masters. It was beautiful. EJBs were the black art. Then sometime later came J2EE design patterns, pet shop and blueprints. You couldn’t; get through an interview if you didn’t know about Blueprints or if you didn’t know about session facade and value objects.
Well, those good old days…. lots of things happened after that within industry and with java language. I personally, have worked a bit with other languages like PHP which I got to learn because it seemed to be the lingua franca of internet. For me coding in PHP was difficult initially as I hated those nested arrays without which you couldn't do a lot in PHP and I missed beauty of strict typing of java. Exposure to internet made me realize that most large, high volume systems are written not in Java but in scripting languages. Possibly because such systems are easy to host and possibly because dynamic languages are natural fit for volatile business of internet. Whatever the reason, java has clearly lagged behind on web. Its focus has been behind firewall, within enterprise, backend processing. So for old time sake and love of a language, it is good to know when you come across a high volume internet site written entirely in java. Below is a presentation which explains how Linkedin runs on 99% java. Just to get an idea of scale, here are some numbers:
- 22 million members
- 4+ million unique visitors/month
- 40 million page views/day
- 2 million searches/day
Take a loot at architecture and how it has evolved. It is all about partitoning the data I guess and 1% is taken by C++ implmentation of custom JVM heap!!. Also read this post here.
If embedded slide doesn't show up , use link to slideshare
Cult of Amateurs - Anti Web2.0 view
Most of enterprises these days are trying to get onto web2.0 bandwagon by launching AJAX sites, aggregators and blogs. No doubt, Web2.0 is about tools which have enabled individuals and communities to harness power of internet in newer ways, but web2.0 also has undeniable social ,cultural and political implications which are yet to be fully understood, especially by enterprises which are by nature hierarchical and control centric. Authors of Wikinomics seems to suggest , in web2.0 era, the emergence of a new kind of enterprise, more open, bottomed up, less hierarchal...an enterprise 2.0. All other institutions humanity has created would also get suffixed and versioned in newer world order of unprecedented democracy, freedom of speech and social participation. So we would have Government2.0, Education 2.0 and many others. "Wisdom of crowds" would rule and some sort of utopia of digital socialism would be achieved. The whole impact on our society would be huge and path altering.
The pace of technological changes and its impact on our society is so rapid that it is like churning of water. Residuals will only get settled once churning stabilizes. But for now, the churning is very engaging and exciting. We mostly read and hear people who have all good things to say about how web2.0, new digital media is leading us to a changed world order, kind of utopia, so it was refreshing in a provocative sense to read some contrarian views in “The Cult of Amateur" by Andrew Keen, who is some sort of contrarian of new digital media, internet and web2.0. Though I would disagree with many of his diatribes against web2.0 and internet, there are nonetheless some hard questions he raises through various arguments especially on mainstream journalism vs bloggers posing as reporters debate and on amateur versus professional content debates. He points put that "cut and paste" online culture would demolish all IP rights and rob many content owners from thier legitimate earnings form content they owned. Then he says that internet is becoming like an ocean of shrill opinions and truth is the vicitim. "In this era of exploding media, there is no truth except the truth you create for yourslef" . The information business he says " is being changed into sheer noise of hundred million bloggers all simultaneously talking about themselevs". Sample some of his "opinions" assembled in some kind of "anti-web2.0 manifesto"
The pace of technological changes and its impact on our society is so rapid that it is like churning of water. Residuals will only get settled once churning stabilizes. But for now, the churning is very engaging and exciting. We mostly read and hear people who have all good things to say about how web2.0, new digital media is leading us to a changed world order, kind of utopia, so it was refreshing in a provocative sense to read some contrarian views in “The Cult of Amateur" by Andrew Keen, who is some sort of contrarian of new digital media, internet and web2.0. Though I would disagree with many of his diatribes against web2.0 and internet, there are nonetheless some hard questions he raises through various arguments especially on mainstream journalism vs bloggers posing as reporters debate and on amateur versus professional content debates. He points put that "cut and paste" online culture would demolish all IP rights and rob many content owners from thier legitimate earnings form content they owned. Then he says that internet is becoming like an ocean of shrill opinions and truth is the vicitim. "In this era of exploding media, there is no truth except the truth you create for yourslef" . The information business he says " is being changed into sheer noise of hundred million bloggers all simultaneously talking about themselevs". Sample some of his "opinions" assembled in some kind of "anti-web2.0 manifesto"
Digital utopian economists Chris Anderson have invented a theoretically flattened market that they have christened the “Long Tail”. It is a Hayekian cottage market of small media producers industriously trading with one another. But Anderson’s “Long Tail” is really a long tale. The real economic future is something akin to Google a vertiginous media world in which content and advertising become so indistinguishable that they become one and the same (moregrist to that frankfurt-Prague-BuenosAires triangle).
The digital utopian much heralded “democratization” of media will have a destructive impact upon culture, particularly upon criticism. “Good taste” is, as Adorno never tired of telling us, undemocratic. Taste must reside with an elite (“truth makers”) of historically progressive cultural critics able to determine, on behalf of the public, the value of a work-of-art. The digital utopia seeks to flatten this elite into an ochlocracy. The danger, therefore, is that the future will be tasteless
(I particularly like this one ->) There is something of the philosophical assumptions of early Marx and Rousseau in the digital utopian movement, particularly in its holy trinity of online community,individual creativity and common intellectual property ownership. Most of all, it’s in the marriage of abstract theory and absolute faith in the virtue of human nature that lends the digital utopians their intellectual debt to intellectual Casanovas like young Marx and Rousseau
I can see where Andrew keen is coming from on most of his opinions. I agree that wisdom of crowds as espoused by many web2.0 evangelists would be contestable if we take a peek in human history. Crowds have hardly been wiser in thier choices than a lone individual could be. Traditional news media has become stale and parody, but most of citizen journalism has also been about lifting content from traditional sources and aggregating it. But largely, most of points which Andrew keen raises in his book are contestable and some are plainly elitist. Digital Utopians might sound similar to what Marx said about socialism but a fundamental difference between a collective farming and web2.0 digital communities is that later are by choice whereas former were by dictates. On the whole, it is a must read book to get different perspective on web2.0 revolution.
Sketchcast - Expressing through online sketching
Just stumbled upon this cool site where one can sketch and create online sketchcast. Looks like a useful idea. Quite possibly these guys are going to improve the site further and provide more sketching tools in future. Using tablet PC would be easier to sketch...using laptop was a pain , anyway here is my first sketchcast:
URI design - Make it central to web application design
The beauty of HTTP is its simplicity. It is clear text protocol which essentially lists few methods (GET, POST etc) to operate on a resource. One of the cornerstones of HTTP protocol is concept of URIs. URI which is the address to reach a resource. A resource which could be abstract or physical. Simply put, URI is a string of characters which is reference to a resource. But it is amazing how this simple concept has been misused or underused. Architects and designers of enterprise software have hardly designed systems with URIs in mind (at least in pre-REST hype phase). So much so that sometime tasks a system is designed to do become hard or impossible to do.
Consider a system which is designed to act as web based employee directory or database. System records everything from employee's basic information, his photograph, his location, his contact details etc. So the primary function of such a system, one would assume is to, operate on set of resources which are employee records. And hence one would assume that every employee record would get represented by a URI which could be something like http://someempdir/employees/a/jhonSmith . This would mean that by passing the URI around, an employee can present his information to whosoever needs it. Plain and Simple.
But what happens if system is built ignoring basic tenets of URIs and HTTP? What if employee records are not URI reachable because system is perhaps using POST requests to do something which essentially is a GET request? So to access an employee record, one has to login to system and then use search functionality and then click on record to see the details but URI could be something like "http://someemdir/portals/wsd/somesbs/m orebs/search.asp?a=bs, making no sense to anybody. It would stay same for Jhon Smith and same for Ramesh kumar making It useless .And what if system does use Get request to access an employee record but generates URIs which can be mind numbing like http://someemdir/portals/wsd/som esbs/morebs/readuser.asp?resource=user&view=broad&application=e mpDirectory&dn=hN%3id=4%2COU%3Dusa%2COU%3DIN%2COU%3DEmployees%2CDC%3Dcom%2CDC%3Dabc%2CDC%3Dcom.
Now consider if I am handling a group of 100 employees and want to keep record of their details and share it with others, I can't use the employee database as I can't reach any employee record outside the system context. I can’t send a URI like http://someempdir/employees/a/jhonSmith to someone who needs Jhon's details. So I will have to create my own little database in an excel sheet and I will ask 100 guys to update it with their information. Employee information now is in two different places and I have to make sure that when guys leave my group or new guys join, I update my own little database and ask everybody else to make sure that their information in my database is in synch with what they have in their respective records in employee directory . And for all practical purpose the big database is all but redundant for me.
If system was designed with some attention to URIs and how information was to be represented, My task would have been easier. I would have just tracked a list of 100 URIs in a sheet. A list of pointers to central source of information. Now if there are 50 other people like me handling group of employees, one can imagine the time wasted in this inane activity of maintaining duplicated information. One can see many such examples in most of enterprise systems and that is why considerable amount of time is spent on unproductive tasks. Lots of times, manual tasks are done to gather information which is lying in all sort of system but is useless because of unnecessary barriers created in accessing it. A barrier could be as simple as not having readable, proper URI for a resource. From my own experience, the reason for this is overly focus on technologies, frameworks and implementation while designing a system, rather than key principles of information management.
While designing a web based system, it is important to consider URI design. In fact that should be beginning of design. Design how URI landscape would look like and how all resources in your system would be represented by URIs. If a product doesn’t support it, junk it.
Google and Yahoo
How fast things change in Internet business. It was not many summers ago that Google was darling of everybody as someone who just wouldn't do anything wrong. And now suddenly, or it seems so, they are getting lots of bad press. In my opinion it was bound to happen. It is in nature of us to fear and hate Big. And when we have something small coming and challenging the Big, we like it. David and Goliath. But it is in nature of small to become Big and eventually become what it was fighting against. Google could challenge Microsoft because it was small, focussed, running on steroid and churning out innovations after innovations and even before Microsoft could understand what was happening the ground had shifted. Now Google dominates search market, has done spate of acquisitions and even its relationship with Mozilla is seen as its intentions of becoming a monopoly. So it will be interesting to see what happens next, now, that goole has acquired the bloat and its core is far away from its periphery.
Another poster boy of internet age, Yahoo is already going through tough times. After a Microsoft's failed attempt to acquire it, it is negotiating a deal with Google to stay afloat. (How fast tables turn. Not long back, Yahoo had almost but bought Google). As reported on techcrunch, Jerry Yang Chang has been on receiving end of criticism. Press seems to hint that Yang has to listen to his shareholders and safeguard their interests. Yes, but, didn’t Yahoo lose its way because it had hired Terry Semel, a Warner Brother executive, who thought that future for Yahoo was in becoming a media company and all the shareholder were excited! So unlike Google , which was cleverly controlled by Larry Page and , Sergey Brin, Yahoo lost touch from its technological roots and drifted even when smaller companies were creating social networking sites and You-Tubes!! I think, instead of shareholders , yahoo needs to re-focus on its customers and give them what they want to use. But it looks dismal. It’s hard for a big company to change course. Can it find its Blue Ocean to stay afloat is the question! It will be interesting to see what happens next.
Cloud Computing - A Design For Scalability?( Part1)
I have been hearing it for sometime in background, have noticed it being bandied about more frequently on my company's technical forums and have seen it appearing more regularly in my rss reader. The next buzzword , cloud computing. After spending few years in IT, I have become cicrumspect to buzzwords and hype that is generated around them and my own experience has been that doing IT is more simpler than it is actually made out to be. Anyway, yesterday I managed to pull myself to hear a talk on Cloud Computing and few slides later I got interested.
Just like in case of SOA, it will be futile to try to give an exact defintion of Cloud Computing. There are many depending upon what source you refer. Is Clound Coputing same as Grid and utlity computing or are there differences! Without going into it at this stage, the inference I could draw form my inital understanding of Cloud Computing is that - what SOA is to design of software, Cloud Computing is to deployment of software. SOA changes the way you look at application portfolio within an enterprise and Cloud Computing changes the way you look at utilising computing resources for deploying your applications. In fact SOA in hand with Cloud Computing takes Distributed computing to next evolutionaly stage. It is an evolution from distributed object paradigm (like CORBA, J2EE) to Distributed service paradigm. The abstraction is raised to a higher level. What clusters were for distributed object systems, Clouds could be for distributed service systems. It is a natural progression of good old distributed computing. In IT system architectures there always have been two main objectives:
- Design For reuse
- Design for Scalability
If design for reuse is more of software related concern, design for scalability is hardware related. Systems can only be truly scaled up by using more computing resource of CPU and memory. If SOA takes "design for reuse" to a higher level (even beyond enterprise boundaries at time), Cloud Computing takes "design for scalability" to next step. In a cloud you can achieve more scalability and more efficiently by transparently adding more computing capacity. And here when I say cloud computing I don't merely mean Amazon cloud or some other cloud implementations, but even clouds which enterprises can create behind thier own firewalls by using virtualisation. SOA makes us think outisde of siloed application and Cloud Computing does it for managing hardware resources. Instead of deploying various applications with differing usages in many different server in a datacenter, what if we could create , well ,...a cloud of these hardware resources and servers and have whole lot of applications deployed in cloud so that hardware is optimally utilised! And with Cloud Computing providers there would be another option of actually hiring out the hardware needs instead of buying your own capacity.
I think SOA with Cloud Computing and Virtualisation are what could be called disruptive technologies of future. Ok, you can ignore the "disruptive part". Just like SOA, Cloud Computing also has more business aspects to it and not mere technical aspects. In coming days, I will be more tuned to cloud computing and may be play with Amazon cloud and write about my experience with it.
Going Agile - Just another collection of good practices.
Agile practices for creating software within enterprises have been gaining ground in recent times. Even though shift is not tectonic but subtle, it is important nonetheless. One of main reasons for this has been that many big enterprises have realized that traditional methods have failed them at times. I have seen quite a few big enterprises spending millions of dollars on projects just to realize that projects very often overrun cost and don’t give them operational systems as they would have envisioned.
Do it Small and Detect early: Software projects fail for various reasons and going agile is not the panacea and it doesn't guarantee success either. But what Agile does is that, it reduces chances of a costly failure as progress of project is known to all stakeholders and an early decision about termination of project or change of course can be taken. A constant progress monitoring of project, continuous build and test, small iterations,sprints etc lead to early detection of problems if any. In this regard Agile is akin to RUP like iterative approach and breaking big into many smalls and doing continous health checks on them.
Focuss On People: But most importantly, What Agile also does is that it brings focus on people and project teams and put them in control. Process boundaries are lowered so there are no processes to cover for failures later. Fetish for processes is diluted so that team doesn’t drown beneath cumbersome and often stifling processes. People are the key ingredient for projects to either succeed or failure. Agile project teams are empowered to think and adapt as needed for success of project. Since teams are in control of project, they can not turn around and blame someone else or some process later for failure. Agile makes team members more visible and accountable. In principle, Agile just states a truth which many would want to ignore for sake of industrialization of software that People are most important element of code assembly line and heroes need not be discouraged.
One caution which enterprises adopting Agile has to follow is that they need to make sure that all pre-conditions for agile to work are met. Agile need some cultural changes and mindset adjustments. Agile teams are built around trust and openness to change. Most Agile teams are cross functional with Business and IT skills working in very close cooperation. Enterprises outsourcing their Software needs to vendors and using agile should keep in mind few things :
1.) Put key personnel in charge: Key Business and IT people should be put in charge of project (well, shouldn't that be true anyway ;)).
2.) Accountable Product Owners: Product owners should be people in your organization who can answer questions. Buck should stop with them. Very often cause of project failure is due to volatile requirements or difficulty in nailing them down properly and on time. Especially if organization is big and diverse, getting access to business people who can provide answers is difficult. This should be changed with empowered product owners being part of project team and not just as outsiders.
3.) Build Prescriptive Processes: Agile doesn’t mean no process. It doesn’t mean guerilla programming either. Agile means adapting quickly to changed scenario or context. Instead of a frozen, rigid, descriptive Process, a prescriptive process or Meta-process can be created so that agile teams are operating within a known process framework but with desired leeway to improvise and maneuver.
5.) Model for Offshore Agile: As is the norm these days, if agile team is distributed across locations, put some thought on how multi location latency can be managed. Decide if you would need proxy product owners in remote teams or how cross pollination would be done to maintain one team spirit of Agile. Remember that Agile can be adapted to your specific needs and constraints. Purists would say that distributed teams are not pure agile teams, but real spirit of Agile is adaptability and not maintaining purity.
6.) Agile doesn't mean cheap: Agile doesn’t mean less cost of doing software. In fact it can mean more build cost. Successful agile projects would create value in terms of reducing hidden costs which are usually accrued in traditional methods of tight control and change aversion.
7.) Fixed price versus TnM Get over fixed price mindset. Fixed cost projects don’t reduce your risks anyway. They only delay the costs by converting them into Hidden costs (Change requests, maintenance) .
8.) Not for all conditions: Agile might not be suited for all scenarios. In many cases, traditional methods of tighter control, elaborate requirements in beginning and managing change are required. There is never fit all solution.
Ultimately, Agile is just a collection of good development practices which puts value and emphasize on certain aspects of software engineering. Keep overzealous agile evangelists and those who think it is passing fad at equidistance.
1.) Put key personnel in charge: Key Business and IT people should be put in charge of project (well, shouldn't that be true anyway ;)).
2.) Accountable Product Owners: Product owners should be people in your organization who can answer questions. Buck should stop with them. Very often cause of project failure is due to volatile requirements or difficulty in nailing them down properly and on time. Especially if organization is big and diverse, getting access to business people who can provide answers is difficult. This should be changed with empowered product owners being part of project team and not just as outsiders.
3.) Build Prescriptive Processes: Agile doesn’t mean no process. It doesn’t mean guerilla programming either. Agile means adapting quickly to changed scenario or context. Instead of a frozen, rigid, descriptive Process, a prescriptive process or Meta-process can be created so that agile teams are operating within a known process framework but with desired leeway to improvise and maneuver.
5.) Model for Offshore Agile: As is the norm these days, if agile team is distributed across locations, put some thought on how multi location latency can be managed. Decide if you would need proxy product owners in remote teams or how cross pollination would be done to maintain one team spirit of Agile. Remember that Agile can be adapted to your specific needs and constraints. Purists would say that distributed teams are not pure agile teams, but real spirit of Agile is adaptability and not maintaining purity.
6.) Agile doesn't mean cheap: Agile doesn’t mean less cost of doing software. In fact it can mean more build cost. Successful agile projects would create value in terms of reducing hidden costs which are usually accrued in traditional methods of tight control and change aversion.
7.) Fixed price versus TnM Get over fixed price mindset. Fixed cost projects don’t reduce your risks anyway. They only delay the costs by converting them into Hidden costs (Change requests, maintenance) .
8.) Not for all conditions: Agile might not be suited for all scenarios. In many cases, traditional methods of tighter control, elaborate requirements in beginning and managing change are required. There is never fit all solution.
Ultimately, Agile is just a collection of good development practices which puts value and emphasize on certain aspects of software engineering. Keep overzealous agile evangelists and those who think it is passing fad at equidistance.
Subscribe to:
Posts (Atom)