Showing posts with label Predictive Analytics. Show all posts
Showing posts with label Predictive Analytics. Show all posts

July 6, 2012

The Information Is On You

There was a fascinating article in the New York Times (17 June 2012) called: "A data giant is Mapping and Sharing the Consumer Genome."

It is about a company called Acxion--with revenues of $1.13 billion--that develops marketing solutions for other companies based on their enormous data collection of everything about you!
 
Acxion has more than 23,000 servers "collecting, collating, and analyzing consumer data...[and] they have amassed the world's largest commercial database on consumers."

Their "surveillance engine" and database on you is so large that they:

- "Process more than 50 trillion data 'transactions' a year."
- "Database contains information about 500 million active consumers."
- "About 1,500 data points per person."
- Have been collecting data for 40 years!

Acxion is the slayer of the consumer big data dragon--doing large-scale data mining and analytics using publicly available information and consumer surveys.

They collect data on demographics, socio-economics, lifestyle, and buying habits and they integrate all this data.

Acxion generates direct marketing solutions and predictive consumer behavior information.

They work with 47 of the Fortune 100 as well as the government after 9/11.

There are many concerns raised by both the size and scope of this activity.
 
Firstly, as to the information itself relative to its:

- Privacy
- Security

Secondly, regarding the consumer in terms of potential: 

- Profiling
- Espionage
- Stalking
- Manipulation 

Therefore, the challenge of big data is a double-edged sword: 

- On one hand we have the desire for data intelligence to make sense of all the data out there and use it to maximum affect.
- On the other hand, we have serious concerns about privacy, security, and the potential abuse of power that the information enables. 

How we harness the power of information to help society, but not hurt people is one of the biggest challenges of our time. 

This will be an ongoing tug of war between the opposing camps until hopefully, the pendulum settles in the healthy middle, that is our collective information sweet spot. 

(Source Photo: Andy Blumenthal)


Share/Save/Bookmark

February 25, 2012

The False Information G-d

The amount of data in the world is exploding and yet the belief in G-d is evaporating.  

A review in the Wall Street Journal (22 February 2012) of a book called "Abundance" points to the explosion of data with the prevalence of information technology. 

From the earliest civilization to 2003, all written information totaled 5 exabytes (an exabyte is a quintillion bytes or 1 followed by 18 zeros).

But this is nothing compared to the last number of years, "where the change is not just accelerating, but the rate of acceleration of change is itself accelerating."

Between 2003-2010, 5 exabytes of digital information was created every 2 days, and by next year, 5 exabytes will be produced every 10 minutes!

Similarly, Wired Magazine (March 2012) reports in an interview with George Dyson that the "Digital Universe" is growing organically and "cycling faster and faster and it's way, way, way more than doubling in scale every year. Even with the help of [tools like] Google and YouTube and Facebook, we can't consume it all."

According to ComputerWorld (13 February 2012) in Your Big Data To-Do List, with all this data being generated, there is a mistaken assumption that we have to consume it all like drinking from a firehose. The article references a McKinsey study that projects that by 2018, there will be a need for 140,000 to 190,000 additional data analysts and statistical experts to try and make sense of it all. The article suggests that instead of trying to grasp at all the data, we instead "data scoop" and "target projects that can showcase results as opposed to opting for the big-bang, big-data projects."

And tools are being developed and deployed to try to get our arms around the information rolling in around the world. For example, Bloomberg Businessweek (November 28 - December 4, 2011) describes the tool from Palantir being used by the military, Intel, and law enforcement agencies for data mining, link analysis, and even predictive analytics. 

These days, "It's like plugging into the Matrix"--in terms of the amount of data streaming in. One special forces member in Afghanistan describes it as follows: "The first time I saw it, I was like Holy crap. Holy crap. Holy crap." But the thinking is now-a-days that with tools like Palantir (and others), we "can turn data landfills into gold mines."

But while information is power, Harvard Business Review (September 2011) in Learning to Live With Complexity acknowledges "We are hampered by cognitive limits." And moreover, "Most executives think they can take in more information than research suggests they can." And harnessing data into information is constrained by the complexity involved--driven by the number, interconnections, and diversity of interacting elements.

The result is that while we are becoming in a sense data rich, in may ways, we are still information poor. And even with all the sensors, data, and tools available to search, access, and analyze it, we are becoming perhaps overconfident in our ability to get our arms around it all and in turn master the world we live in. 

The hubris in our abilities to use information technology is leading many to worship the proverbial information G-d, and in turn, they are forgetting the real one. According to the Pew Forum on Public and Religious Life (February 2010) as quoted in CNN, "young Americans are significantly less religious than their parents and grandparents were when they were young."  Moreover, a full one in four American millennials--those born after 1980--are not affiliated with any faith--they are agnostic or atheists.

Similarly, Mental Floss Magazine just a few months ago (November-December 2011) had various authored columns asking "Is G-d In Our Genes?" and another "Is G-d In a Pill?" questioning whether the age-old belief in G-d comes either from a genetic disposition in some to a drug-induced states in others.

While religion is a personal matter, and for a long time people have argued whether more people have died in wars over religion or money and power, as a person who believes in G-d, I find it most concerning that with the rise of information (technology) power in the last 30 years, and the exuberance and overconfidence generated from this, there is an associated decline in belief in G-d himself.  

While technology has the potential to raise our standard of living (in leaps and bounds even) and help solve many of our vexing problems, we cannot forget that technology is run by human beings who can choose to be good or evil and use information technology to either better mankind or the opposite, to destroy it. 

Ultimately, I believe that it is but G-d almighty who shapes the thoughts and destiny of mankind, so that one man sees just a string of bits and bytes--a matrix of zeros and ones--while another sees a beautiful new musical composition, the next terrorist attack, or even the amazing cure for cancer.  

(Source Photo: here)

Share/Save/Bookmark

November 26, 2011

Espionage, Social Media Style

You are being watched!
Good guys and bad guys are tracking your movements, rants and raves, photos, and more online.
For example, The Atlantic reported on 4 November 2011 in an article titled How the CIA Uses Social Media to Track How People Feel that "analysts are tracking millions of tweets, blog posts, and Facebook updates around the world."
Further, in January 2009, "DHS established a Social Networking Monitoring Center (SNMC) to monitor social networking sites for 'items of interest.'"
And even more recently in August 2011, DARPA invited proposals for "memetracking" to identify themes and sentiments online and potentially use this for predictive analysis.
The thinking is that if you can use online information to predict stock market movements as some have attempted, why not criminal and terrorist activity?
Similarly, The Guardian reported on 16 March 2010 FBI using Facebook in fight against crime and cautions that "criminals dumb enough to brag about their exploits on social networking sites have now been warned: the next Facebook 'friend' who contacts you may be an FBI agent."
This is reminescent of the work of private sector, Dateline NBC in using Internet chat rooms to catch sexual predators online by luring them to a house where the predators believed they were going to meet up with a underage girl for a tryst.
While these efforts are notable and even praiseworthy by the good guys--assuming you can get over the privacy implications in favor of the potential to have a safer society to live in--these activities should be carefully safeguarded, so as not to infringe on the rights and freedoms of those who behave legally and ethically.
But the good guys are not the only ones using the tools of the trade for monitoring and analyzing social networking activities--the bad guys too recognize the implicit information treasure trove available and have you in their crosshairs.
For example, in the last years Arab Spring, we have nation states tracking their citizens political activities and using their power over the Internet to shut off access and otherwise surpress democracy and human rights. Further, we have seen their use for cyberspying and testing offensive cyber attack capabilities--only the most recent of which was the alleged infiltration of a SCADA system for a Illinois water plant.
Moreover, this past week, Forbes (21 November 2011) reported in The Spy Who Liked Me that "your social network friends might not be all that friendly."
From corporate espionage to market intelligence, there are those online who "steadfastly follows competitors' executives and employees on Twitter and LinkedIn."
In fact, the notion of online monitoring is so strong now that the article openly states that "if you're not monitoring your competitors activity on social media, you may be missing out on delicious tidbits" and warns that "it's easy to forget that some may not have your best intersts at heart."
Additionally, while you may not think your posts online give that much away, when your information is aggregated with other peoples posts as well as public information, it's possible to put together a pretty good sketch of what organizations and individuals are doing.
Forbes lists the following sites as examples of the "Web Spy Manual" with lots of information to pull from: Slideshare, Glassdoor.com, Quora, iSpionage, Youtube as well as job postings and customer support forums.
When you are on your computer in what you believe to be the privacy of your own home, office, or wherever, do not be deceived, when you are logged on, you are basically as open book for all the world to see--good guys and bad guys alike.
(Source Photo: here)

Share/Save/Bookmark

October 3, 2010

Policing, Armed And Data-Rich

http://news.yahoo.com/video/us-15749625/predictive-policing-22251759

Watch how COMPSTAT (COMPuter STATistics) and technology is being used for predictive policing.

Data, geographic information systems (GIS), and business intelligence/analytics come together to predict and fight crime in major U.S. cities like LA, NY, and others.

As one officer said: "Information can predict the future. Information can lead you to make good decisions and it's shown in a business model everyday!"

Share/Save/Bookmark

August 1, 2010

A Peek at The Future of Information Search, Analysis, and Visualization

I am not endorsing any vendor or product, but here's an impressive video on what Recorded Future is doing with what they call a "Temporal Analytics Engine" to analyze open source information on the web as well as structured data sources.

This goes way beyond search as we do it today--this is a look into the future of iterative search, link analysis, information visualization (both temporally and spatially), and predictive analytics.

The video shows an example of how the technology can be used in counterterrorism efforts to "connect the dots" on the bad guys and protect our nation and its people.

Envision many other applications across government (including law enforcement), business, academia...check it out.


Share/Save/Bookmark