Some information dynamics for Enterprise IT

While spending some time in an automobile factory recently, where I had chance of meeting many people from production and manufacturing background, I had to frequently explain what I do and since I was amidst many engineers, I found myself calling an information engineer. Not that software engineer is not well understood but It was easier to explain to logistics expert that my job was similar to his. He manages flow of goods and my job deals with flow of information. I know information engineering is just one of approaches we take in software engineering but using this perspective provides a different and interesting view of enterprise IT.

For effective information flow throughout the enterprise, info pipes need be laid down and taps and faucets installed at points where this information needs to be consumed, This can be understood as design of IT system landscape for enterprise and building applications (taps!). Together they can be called Info-structure. The master plan of information flow, the taps that need to be installed and how information will be consumed is what a typical enterprise information architecture would be. It is also required that on regular intervals it is checked that pipes are not clogged and that information is flowing smoothly. Clogged information flow is bad as it blocks and wastes precious human energy. To discover this ,the pertinent questions to ask is if employees have access to information which they need to know to do their jobs better and efficiently. The moment you find employees struggling with and groping for information, you would know that there are problems within organization’s info-structure. And the common symptoms of this are when you hear employees say : "I don’t have this information, I will call Tom", or, " Oh, I know that data is in excel sheet but it is not very recent", or "Let me call Mr. Gupta and check if he has the latest data on this".

In today's fast moving economy where business decision are to be made quickly, the information flow has to become more fluid. For example, in an automotive assembly line, use of SAP for running MRP and production planning is not enough. On demand information is needed, and not just by management but also at lower levels. The buyers department need to know historically which suppliers have problem delivering on time , the store manager need to know how parts age in inventory and if he needs to focus on logistics of certain parts more than the others. Business intelligence or BI is becoming important for this very reason and BI is akin to installing filters over the structured information flow. For information to become more liquid, the data trapped inside applications has to be unlocked and put on gradient. Enterprise also has to become porous as it has to absorb information from outside its boundaries and ,also , if I can put it that way , deliberately leak its own information outside its perimeter including to its customers. Reverse osmosis of information at enterprise boundaries!

Social media is also becoming an important element in enabling info-structure, as these tools are cheap and much effective ways of drawing employees into the information flow or let them plug into info-structure. They become social filters of information and can provide the ad-hoc plumbing over which unstructured information can flow overcoming departmental barriers. Their efficacy in knowledge based enterprises, where production of data or information is primary business is well understood but they are equally important in traditional businesses where information is result of business activities. To provide a simple example, a help manual on chassis number generation or a vehicle parts specification document is ok as a static document exchanged over emails. But, when kept on a Wiki, this information transforms from static to fluid and can flow more easily. And liquidity of information is only one advantage, distillation and refinement is another. Enterprise micro blogging platforms, wikis; they can enable social distillation of information. Increasingly, enterprise IT is going to be lot more about making information liquid and enabling its flow within enterprise and outside it as well. This diffusion of information will create a more "knowing" enterprise and that will be a key advantage in knowledge age.

(First posted on Capping IT Off)

Goodbye to Digital Decade

A day more and first decade of this century will be over. And what a decade it has been, marked with blistering pace of digital revolution which has left everything in sight singed. Such has been impact that decade is being called a digital decade. In span of 10 years we have seen massive changes thanks to advances in digital technology, explosion of internet and unleashing of waves of innovations. Come to think about it, how fast some of application and devices we use today and take for granted have sprung up e.g ; wikipedia: 2001, Myspace: 2003, Facebook: 2004,Youtube: 2005,Twitter: 2006,iPhone 2007,Kindle: 2007 and many more.

In span of 10 years, Google turned from a company to a verb, Facebook became a place to hang out, Youtube redefined blockbusters and Amazon survived dark ages of e-commerce to ride high on clouds. What started as a bug squashing exercise, turned into a dot com boom before crashing with a thud and reviving with everything 2.0 phenomenon. If we risked becoming verbose and blasé with blogger, twitter showed that 140 characters were more than enough to express anything. And this was much after a generation of kids had developed lingos not taught in any language academies, thanks to ubiquitous toy of decade called mobile phone. Oh well, the mobile itself, the question was not whether you had it but how smart it was. Wisdom of crowds, long tail, wikinomics, user generated content threatened some conventional business models and new paradigms emerged. Impact of social networking was not only felt in personal life but inside corporate walls as well, where some old ramparts could be seen crumbling. Low cost of communication and distribution ensured that the word which was with God was finally and truly appropriated by men.


In an old Indian philosophical thought, evolution, is seen as a progressive manifestation of nature ( to experience itself). Evolution doesn’t always result in what can be called a happy path scenario and in those circumstances self-correction is done. In similar vein, we can see human intelligence virtually exploding in many forms in this digital decade and moving towards some kind of distant Omega Point. The pace of this technology innovation has been so rapid that its impact on society's fabric and different aspects like political, legal and social is yet to be known fully. May be we are going to discover that not all innovations and technology made our life easier or fruitful after all. Now we have less time to do more things. Growth has become a monster which has to be constantly fed. "What Next!" drives us and there is almost sense of paranoia if we can’t see that Next. To some , digital technology and specially the Internet has ushered in a cult of the amateur that is "taking us back into the Dark Ages". Change always creates flux and that unsettles many. Regardless of different views, one thing is certain, that the mark has been made. Future generations will look back at this decade and perhaps wonder how it would have been to live through those years when epochal changes were taking place. Or possibly they are going to look back and wonder about a decade of digital stone age. Time will tell, as someone has said, future is not googlable.

(First appeared on Capping IT Off )

Does Agile mean no Processes and no Governance?

There are two generally held perceptions about agile. One, that agile is a kind of anarchist model with no processes, and second, that agile means no governance. Let’s look at the first perception that agile means no processes. We can understand how agile looks at need of processes by applying a well known principle called Occam's razor which states that ” one should not increase, beyond what is necessary, the number of entities required to explain anything”. The principle helps us to remove those concepts, variables or constructs that are not needed to explain a phenomenon. By doing this, it becomes easier to create a model which resembles reality much better and redundancy and inconsistency is removed. In essence, Occam's razor stipulates that when multiple theories are available to explain a problem, the simplest one is preferred. Nature likes simplicity. Ocaam's razor when applied to project management methods would imply that a Project should have only those processes which are required and not more. This is exactly what agile paradigm means with its use of low ceremony, just enough processes. Agile teams chose processes which are most relevant for them to meet their goals in their context. It means you do not pick set of pre-defined, rigid processes in a bunch and deploy them on a project. A process is good as long as it adds value to project; or it should be discarded or amended. So, why is to so hard to agree with this!

It is possibly due to the fact that traditional management methods have relied heavily on process standardization and execution efficiency. But it has not helped achieve predictability of results which is clear from number of IT project failures. Today's business scenario is vastly different with constant change and dynamic markets. Project teams like Organizations need to adapt quickly. Traditional methods, which are based on "execution as efficiency”, find this hard to achieve because they inherently oppose changes in system. In a Harvard business review paper published in 2008, Professor Amy Edmondson identifies a different approach to execution, called "execution as learning" which looks uncannily similar to agile thinking of software development. She calls execution as learning “a radically different organizational mind-set, one that focuses not so much on making sure a process is carried out as on helping it evolve". Agile practitioners would easily relate to her thoughts and “execution as learning” looks very similar to agile manifesto.

Second perception about agile is that there is no or very little Governance in agile methods. But the fact is that agile makes governance more effective as it puts onus on participation of all stakeholders in project. Traditional command and control governance models usually end up creating bureaucratic hurdles and illusion of control. Agile governance model on the other hand is about enabling the team and facilitating an environment where team can meet its objective without avoidable hindrances. Whole team principle is about shared vision and goals and that there are no finger pointing and putting the blame when and if things go wrong. Should project sponsor remain a distant illusive figure or rather become part of the team! Perception of agile being low on governance perhaps comes from principle of "self-organizing teams". Scott Ambler explains this principle when he says that "self organizing doesn’t mean that you are out of control and doing your own thing. It means that team members are themselves deciding how to meet the goals. But the goals they have to meet, the resources used and timeframe, are governed by organization". Agile governance he says is about keeping the baby and throwing the bathwater. And that might well summarize processes and governance perspective in agile.

(First appeared on Capgemini technology blog )

Capgemini's first ever Barcamp

It has been exciting time for past couple of days in preparations for Capgemini India's (and perhaps Capgemini's) first ever Barcamp which is happening tomorrow, 11th September in Mumbai. In fact we already have a Barcamp being run in Capgemini Bangalore today as part of this synchronised Capgemini India Barcamps. We had many people volunteering with tasks like creating logos, stickers, banners to coordinating with internal sponsors. The internal wiki was used for registration. We put up huge Barcamp banners in office with some fantastic posters created by Tarun. We weren't sure of how much participation we would get and were surprised to have more than 100 registrations. Not a Bad start.

Because of logistics and few other reasons the event is being attended by Capgemini employees only , so many people could comment that it is not a true, pure barcamp. But then purists are purists. They have a role to play. Our aim is to introduce unconferences and barcamps as an alternate method of learning within the company and also spread idea of barcamp among many people who are not exposed to it. I am sure we will soon have versions of Capgemini Barcamp running in thier pure form with wider and external participation.

For now, I hope this Barcamp would clear few mindblocks and give few people another perspective of learning and sharing in open, informal and collaborative enviornment....

Problem with Enterprise Search- new veriosn of GSA launched

Google has launched new version of GSA (Google Search Appliance) meant for deployment within the firewall to do enterprise searches. There are some improvements built inside yellow box which is how standard GSA is shipped. Google claims that GSA can now index many million more pages, can conduct searches faster than ever and show role based search results. So for so good. The problem is that enterprise search is not about searching millions of pages or spitting query results a bit faster. The problem is that enterprise search engines don't work. The real problem is that it is not easy to find anything relevant. And the problem is not with search engine like GSA. The real problem is in very nature of enterprise and its content management and intranet systems.

I have worked a bit with GSA sometime back. It is extremely easy to install and use. Mostly, you point it to sites and sources you want to index, trigger its crawler, let it build its index and there you start searching with familiar Google UI, all out of box. Google has spent a decade perfecting search algorithms. It uses complex rules to bubble up the most relevant results for a user query on top of result pages. Page rank or relevancy of a page is calculated based on how page is cited by other pages, their interlinking and metadata which crawlers extracts from page elements like title, header and other structural elements (typical SEO domain). And it works so well on web because web has huge scale. Web is huge mass of interlinking web pages where cross referencing through links is a norm. This Interlinking of pages is most important factor for determining relevancy of result pages for a query.

When I say that enterprise search doesn’t work, what I mean is that results which search engines throw are hardly relevant. Unlike on web search where first few results are almost always most relevant to a query, in enterprise search, it is hardly true. You can get results which are no where relevant to your query and most important and relevant documents might be buried somewhere in huge pile of results, where you would never reach. The reason is simple, that enterprise systems and intranets don’t use cross-linking of pages and sources. There no almost non existent cross references. For example how many times would you find a "Banking Industry opportunity PoV" document or page cross referencing to another document which could be "Banking Industry Trend Analysis"!! In fact, many enterprise systems are like document storage systems where all the documents in form of excels, ppts, pdfs are dumped as equal. Search engine has no way to find which most relevant document for a query is. On top of that, most enterprise systems and intranets are not optimized for search engines. How many times, have you seen web pages on intranet which have no title or use proper Meta data!! The content creators don’t use some basic practices to make their content “findable”. So another vital source for search engine to determine page relevancy is lost. And what we get in effect is a sputtering and struggling search engine trying hard to fish out that most important document for you.

In a way, it is not problem with technology but with very nature and realities of enterprise. IMHO, an effective intranet search engine has to provide more than "out-of- box" features; it should be "tweak-able". It has to work with understanding that:

1. Enterprise content is not cross linked and cross referenced, so relevancy logic successful on web wouldn't be much useful.
2. Scale of content is limited, unlike web where million of pages are cross-linking, enterprise content is not so vast.
3. Pages are not optimized for search engines. (But that should be fixed by a company's policy)
4. Some documents written by some "experts" or “communities” in enterprise become naturally more important or relevant! The relevancy logic has to account for that, but how! Engine administrators should be able to feed new relevancy rules into the engine.
5. User ratings of documents and META -TAGS on web pages should be given more weightage in calculating relevancy. On web, these are mostly ignored due to their misuse for search engine spamming, but this is not the case in enterprise where problem is opposite.
   

From another perspective, a human edited search engine could be more useful and effective within firewall. Automated search engine can still be used to find out what are users searching most of time (trends), and experts, knowledge managers or users can contribute to index and ranking of pages manually. New version of GSA also seems to have a similar feature for Do It Yourself (DIY) key-match and some features for administrators to influence search results.

Prayer for K2 Climbers

Ever since I have been on trek to ABC in Nepalese Himalayas, I have become fascinated with mountain climbing. It is perhaps toughest challenge any man can go through, both physically and mentally. Those tall irreverent himalyan mountains allow only the fittest and strongest of men to climb them and I have nothing but respect for men who indulge in this dangerous activity. Mountain climbing community is very small and plugging to it gives a different perspective on life. It is a different world altogether. Ever Since I read Maurice Herzog's classic book Annapurna detailing Frenchman's first and successful ascent of a 8000 meter peek, I have peeked into fascinating world of Louis Lachenal, Lionel Terray, Gaston Rebuffat and many such legendry climbers. The community has its own hierarchy with toughest, high altitude climbers on top. There were climbers like Anatoli Boukreev, who was daredevil to the extent of being mad, climbing without acclimatization or without oxygen. He was also climbing during 1996's Everest tragedy when 8 climbers were killed in one day. The expedition created lots of controversy with Jon kraukauer , a journalist and climber partly blaming Boukreev for not saving some of climbers trapped in snow storm. An Indian climber who died on same tragic day on Everest, was still lying dead just beneath the peak until recently. For almost 10 years his dead body was used by climbers as a milestone (called greenboots) on way to Summit. In recent times, many controversies and debates have arisen where some people within community have decried the attitude of climbers to reach the summit at any cost and not helping or rescuing other climbers who might be in distress. In 2006, David Sharp, a British Climber laid dying on the way to Everest peak, even while 40 other climbers passed him without offering any help or rescue. There is someting in us, men, which is tested only when faced with extreme adversity like in case of Joe Simpson and Simon Yates's disasterous climb on a Peruvian mountain so brilliantly captured in book and movie Touching the Void.


Even though men would often in their follies term their successful climbs and summits as conquests, these mountains invariably keep reminding them who the real lord is .Freak weather, falling stones, sudden avalanches, extreme weather, anything can kill even most experienced and trained climber as happened on K2 yesterday. In deadliest day in K2's history ,11 climbers reportedly died while scaling K2. K2 is said to be the most difficult of mountains to climb with its razor sharp ridges and unpredictable weather. The deadly mountain is also said to be cursed for women as only 5 women could reach the top, 3 of them died on decent and rest 2 died later on other mountains. Regardless of obvious danger, K2 keeps attracting climbers like bees to the flame. It is said to be a handsome and mighty mountain which is also equally unforgiving. Here is little silent prayer for those who died scaling it yesterday.

About Me

Some questions used to have simple answers......(roll mouse over picture)