Showing posts with label Data Analysis. Show all posts
Showing posts with label Data Analysis. Show all posts

February 25, 2012

The False Information G-d

The amount of data in the world is exploding and yet the belief in G-d is evaporating.  

A review in the Wall Street Journal (22 February 2012) of a book called "Abundance" points to the explosion of data with the prevalence of information technology. 

From the earliest civilization to 2003, all written information totaled 5 exabytes (an exabyte is a quintillion bytes or 1 followed by 18 zeros).

But this is nothing compared to the last number of years, "where the change is not just accelerating, but the rate of acceleration of change is itself accelerating."

Between 2003-2010, 5 exabytes of digital information was created every 2 days, and by next year, 5 exabytes will be produced every 10 minutes!

Similarly, Wired Magazine (March 2012) reports in an interview with George Dyson that the "Digital Universe" is growing organically and "cycling faster and faster and it's way, way, way more than doubling in scale every year. Even with the help of [tools like] Google and YouTube and Facebook, we can't consume it all."

According to ComputerWorld (13 February 2012) in Your Big Data To-Do List, with all this data being generated, there is a mistaken assumption that we have to consume it all like drinking from a firehose. The article references a McKinsey study that projects that by 2018, there will be a need for 140,000 to 190,000 additional data analysts and statistical experts to try and make sense of it all. The article suggests that instead of trying to grasp at all the data, we instead "data scoop" and "target projects that can showcase results as opposed to opting for the big-bang, big-data projects."

And tools are being developed and deployed to try to get our arms around the information rolling in around the world. For example, Bloomberg Businessweek (November 28 - December 4, 2011) describes the tool from Palantir being used by the military, Intel, and law enforcement agencies for data mining, link analysis, and even predictive analytics. 

These days, "It's like plugging into the Matrix"--in terms of the amount of data streaming in. One special forces member in Afghanistan describes it as follows: "The first time I saw it, I was like Holy crap. Holy crap. Holy crap." But the thinking is now-a-days that with tools like Palantir (and others), we "can turn data landfills into gold mines."

But while information is power, Harvard Business Review (September 2011) in Learning to Live With Complexity acknowledges "We are hampered by cognitive limits." And moreover, "Most executives think they can take in more information than research suggests they can." And harnessing data into information is constrained by the complexity involved--driven by the number, interconnections, and diversity of interacting elements.

The result is that while we are becoming in a sense data rich, in may ways, we are still information poor. And even with all the sensors, data, and tools available to search, access, and analyze it, we are becoming perhaps overconfident in our ability to get our arms around it all and in turn master the world we live in. 

The hubris in our abilities to use information technology is leading many to worship the proverbial information G-d, and in turn, they are forgetting the real one. According to the Pew Forum on Public and Religious Life (February 2010) as quoted in CNN, "young Americans are significantly less religious than their parents and grandparents were when they were young."  Moreover, a full one in four American millennials--those born after 1980--are not affiliated with any faith--they are agnostic or atheists.

Similarly, Mental Floss Magazine just a few months ago (November-December 2011) had various authored columns asking "Is G-d In Our Genes?" and another "Is G-d In a Pill?" questioning whether the age-old belief in G-d comes either from a genetic disposition in some to a drug-induced states in others.

While religion is a personal matter, and for a long time people have argued whether more people have died in wars over religion or money and power, as a person who believes in G-d, I find it most concerning that with the rise of information (technology) power in the last 30 years, and the exuberance and overconfidence generated from this, there is an associated decline in belief in G-d himself.  

While technology has the potential to raise our standard of living (in leaps and bounds even) and help solve many of our vexing problems, we cannot forget that technology is run by human beings who can choose to be good or evil and use information technology to either better mankind or the opposite, to destroy it. 

Ultimately, I believe that it is but G-d almighty who shapes the thoughts and destiny of mankind, so that one man sees just a string of bits and bytes--a matrix of zeros and ones--while another sees a beautiful new musical composition, the next terrorist attack, or even the amazing cure for cancer.  

(Source Photo: here)

Share/Save/Bookmark

January 27, 2008

Breakthrough Thinking and Enterprise Architecture

Breakthrough thinking is at the heart of great enterprise architecture. Incremental improvements in the organization are one thing, but dramatic breakthroughs that take an organization to a whole new level is what EA dreams are made of.

Harvard Business Review (HBR), December 2007, reports on how to achieve this breakthrough in thinking.

Firstly, HBR contends that brainstorming does not work for a few reasons:

  1. No structure—“most people are not very good at unstructured, abstract brainstorming.” Outside the box thinking is too vague for people to really get their arms around the problem and provide concrete solutions.
  2. Data analysis is constrained—“slicing the data in new ways—almost always produces only small to middling insights…the contents of every database are structured to correspond to insights that are already recognized, not the ones that aren’t.”
  3. Customer requirements can’t tell the whole story—customers “can rarely tell you whether they need or want a product that they have never seen or imagined."

So what do you do to get breakthrough thinking?

The approach “takes a middle path between the two extremes of boundless speculation and quantitative data analysis.

For example, “one question that can generate insights in any business is, “what is the biggest hassle about using or buying our product or service that people unnecessarily tolerate without knowing it?”

When you ask questions that create new boxes to think inside, you can prevent people from getting lost in the cosmos and give them a basis for making and comparing choices and for knowing whether they’re making progress.”

Here are some questions that drive breakthrough thinking:

  • “What is the biggest hassle about using or buying our product or service that people unnecessarily tolerate without knowing it?”
  • “How would our product change if it were tailored for every customer?”
  • “Which customers use or purchase our product in the most unusual way?”
  • “Who uses our product in ways we never expected or intended?”
  • “Who else is dealing with the same generic problem as we are but for an entirely different reason? How have they addressed it?”
  • “Which technologies embedded in our product have changed the most since the product was last redesigned?”
“The most fertile questions focus the mind on a subset of possibilities that differ markedly from those explored before, guiding people to valuable overlooked corners of the universe of possible improvements.”

From a participant’s standpoint, do the following to encourage breakthrough thinking.

  1. Selection—“select participants who can product original insights.”
  2. Engagement—“ensure that everyone is fully engaged;” provide incentives as appropriate. People are competitive by nature and a little competition can go a long way to idea generation.
  3. Group size—break the participants into groups of around four, since that group size encourages everyone to participate and not hide-out.
  4. Focus—set boundaries using preselected questions; “don’t worry about stifling creativity. It is precisely such boundaries…that will channel their creativity.”
  5. Results—At the end of the brainstorming, “narrow the list of ideas to the ones you will seriously investigate…nothing is more deflating to the participants of a brainstorming session than leaving at the end with no confidence that anything will happen as a result of their efforts.”

From a User-centric EA perspective, what better way to serve the users than by thinking how to serve them better with improved products and services? The chief enterprise architect should facilitate breakthrough thinking to create the target architecture and transition plan for the enterprise.


Share/Save/Bookmark