Showing posts with label Data Analytics. Show all posts
Showing posts with label Data Analytics. Show all posts

October 26, 2019

Don't Trust Your Gut Alone

Please see my new article in The Times of Israel called, "The Snake That Appeals To Your Gut."

The truth is, following one’s gut feelings alone is a way to avoid confronting or dealing with real data about what’s going on. While it’s true that information can be tough to get as well as to interpret, we certainly have to look not only at people’s words, but also at their deeds. We have to see them over an extended period of time, so we see whether there is consistency and if their integrity holds up under different situations and stressors.

We have hearts and minds and we need to make sure we are using both in making important decisions. Otherwise, see no evil, speak no evil, hear no evil—and what do you think you are precisely going to get?


(Credit Photo: Andy Blumenthal)
Share/Save/Bookmark

May 1, 2018

Information Is Power

Just wanted to share something I heard and liked about data and information:
"Everything is a record, record, record
in a table, table, table."
Can everything in life really be reduced to lines of records, with fields of data in tables of information?

This is the information age!

Analytics and Big Data rule!

Knowledge is power!

In any conflict, we seek information dominance and supremacy!

Artificial intelligence is the future!

Records are unique with their own sys.id.

Creativity and innovation are also records in the table--even if they are the one in a million. 

The more records and tables--the more dots and connections between them--the more intelligence we can glean. 

Yes, everything is a record, record, record in a table, table, table. ;-)

(Source Photo: Andy Blumenthal)
Share/Save/Bookmark

September 27, 2017

A Mountain Of Data

So I heard this interesting perspective on information and data analytics...

Basically, it comes down to this: 
"Most organizations are data rich, but information/insight poor."

Or put another way:
"Data is collected, but not used."

Hence we don't know what we don't know and we end up making bad decisions based on poor information. 

Just imagine if we could actually make sense of all the data points, connect them, visualize them, and get good information from them.

How much better than a pile of rocks is that? 

(Source Photo: Andy Blumenthal)
Share/Save/Bookmark

February 22, 2016

Make Up Your Mind

It all started with the Staples "Easy" button that says robotically when pressed, "That was easy!" 

Then came the "B.S." button that yells out, "That was bullsh*t!"

Now we have the decision and indecisional buttons for "Sorry," "Yes," "Maybe," and "No."

Very much like organizational decision-making and politics where either we can't make up our minds, hedge our bets because we simply don't know, or make decisions on imperfect knowledge or with plenty of biases.

It's funny-sad how instead of decisions and progress, some people lie and pretend that what they are saying has any reality or basis to it despite proof to the compelte opposite. 

For example, over and over again, we hear some politicians say there is no military solution in Syria, yet Russia has proved that completely false turning the tides of the war in Assad's favor and driving back the U.S.-backed rebels and recapturing dozens of towns and cities.

You can probably think of plenty more examples as this is the germy spin that we all must swim and navigate in. 

If only, we could just press a "truth" button to get past all the garbage thrown at us then maybe we could get down to business and really get something done. ;-)

(Source Photo: Dannielle Blumenthal)

Share/Save/Bookmark

July 5, 2015

Big Data, Small Moments

There is a definite rhythm to our lives. 

And by analyzing the peak times of Google search terms, we can get a good picture of what it is (as Seth Stephens-Davidowitz notes in the New York Times Sunday Review).

- From starting a new day to taking care of bathroom business, looking for healing, and even goofing off. 

- Midday is some personal time for shopping, travel plans, and a news update. 

- The evening is a nice dinner and maybe some sexual intimacy.

- The night time is scariest with anxiety about health, leading to panic and thoughts of suicide, and easing off with drugs and pornography. 

- As we roll towards the early hours of the next day, we have a philosophical reawakening with contemplation about the meaning of life and our place in it. 

If we can get all this just from some data analytics of Google search terms, can you imagine what else we can learn about the masses and YOU, the individuals that make it up. ;-)

(Source Graphic: Andy Blumenthal)

Share/Save/Bookmark

January 25, 2015

Size And Smell

So apparently data mining can be used for all sorts of research...

In the New York Times today, Seth Stephens-Davidowitz tries his hand with google search results to better understand people's feelings about sex. 

Though Stephens-Davidowitz doesn't explain how he gets these google statistics...here are some standouts:

As you might have guessed, the biggest complaint from men--and women--is that they don't get/have enough sex. 

For both (as you might imagine in a primarily--95%--heterosexual world), traditional surveys show that it's about once a week.

However, the author says this is exaggerated (yeah, is it surprising that people exaggerate about this?) and it's actually only about 30 times a year--or once every 12 days.

So there are a lot of search on "sexless" or "won't have sex with me."

Observing that "sex can be quite fun," he questions, "why do we have so little of it?"

And he concludes that it's because we have "enormous anxiety" and insecurity about our bodies and sexuality.

Again, you probably wouldn't need data mining to guess the results, but men's biggest worry is about their penis size, and one of women's most toxic worries--a "strikingly common concern"--is about the smell of their vagina.

For men, they actually google questions about genital size more often than they have questions about any other body part; in fact, more than "about their lungs, liver, feet, ears, nose, throat, and brain combined."

So much for health consciousness versus machismo pride. 

The funny thing is apparently women don't seem to care so much about this with only about 1 search on this topic for every 170 searches that men do on this. 

Surprising to most men, about 40% of the searches women do conduct on this topic is "complaints" that it is too big!

Not that size doesn't matter to women, but for them it's about the size of their breasts and butts--and again, bigger being generally considered better.

In this case, most men seem to agree. 

Another issue men are concerned about is premature ejaculation and how to make the experience last longer.

However, here women seem to be looking for information about half and half on how to make men climax more quickly on one hand, and more slowly on the other. 

Overall, men are from Mars and women from Venus, with lot's of misunderstanding between the sexes.

The conclusion from this big data study...everyone calm down and just try to enjoy each other more.

Amazing the insights we can get from data mining! ;-)

(Source Photo: here with attribution to Daniel)
Share/Save/Bookmark

October 12, 2013

Parole By Analytics

Interesting article in the Wall Street Journal about parole boards using software to predict repeat offenders before letting someone go free. 

What used to be a decision based on good behavior during time served, showing remorse to the parole board, and intuition is being augmented with "automated assessments" that include inmate interviews, age of first arrest, type of crime, and so forth.

At least 15 states have adopted "modern risk assessment methods" to determine the potential for recidivism. 

Individuals are marked as higher risk if they are:

- Young--age 18-23 (and impulsive)
- Offense was drug-related
- Suspended or expelled from school
- Quit a job prior to having another one 
- Single or separated
- Diagnosed with a mental disorder
- Believes that it's not possible to overcome their past. 

Surprisingly, violent criminals (rapists and murders) are actually considered lower risk those guilty of nonviolent property crimes--the thinking being the someone convicted of robbery is more likely to repeat the criminal behavior because the crime is one that "reflects planning and intent."

Honestly, I think it is more than ridiculous that we should rank violent criminals less risky than thieves and release them because they had what is considered an "emotional outburst."

Would you rather have some thieves back on the street or murders and rapists--rhetorical question!

But it just shows that even the best of systems that are supposed to help make better decisions--can instead be misused or abused.

This happens when there is either bad data (such as from data-entry mistakes, deceptive responses, and missing relevant information) or from poorly designed decision rules/algorithms are applied.

The Compas system is one of the main correctional software suites being used, and the company Northpointe (a unit of Volaris) themselves advise that officials should "override the system's decisions at rates of 8% to 15%."

While even a 1/7 error rate may be an improvement over intuition, we need to still do better, especially if that 1 person commits a violent hideous crime that hurts someone else in society, and this could've been prevented. 

It's certainly not easy to expect a parole board to make a decision of whether to let someone out/free in 20 minutes, but think about the impact to someone hurt or killed or to their family, if the wrong decision is made. 

This is a critical governance process that needs:

- Sufficient time to make important decisions
- More investment in tools to aid the decision process
- Refinement of the rules that support release or imprisonment
- Collection of a broad base of interviews, history, and relevant data points tied to repeat behavior
- Validation of information to limit deception or error.

Aside from predicting whether someone is likely to be repeat offenders, parole boards also need to consider whether the person has been both punished in accordance with the severity of the crime and rehabilitated to lead a productive life going forward. 

We need to decide people's fates fairly for them, justly for the victims, and safely for society--systems can help, but it's not enough to just "have faith in the computer." ;-)

(Source Photo: Andy Blumenthal)
Share/Save/Bookmark

August 31, 2012

Can a Computer Run the Economy?

I am not talking about socialism or totalitarianism, but about computers and artificial intelligence.

For a long time, we have seen political infighting and finger-pointing stall progress on creating jobs, balancing trade, taming the deficits, and sparking innovation. 

But what if we somehow took out the quest for power and influence from navigating our prosperity?

In politics, unfortunately no one seems to want to give the other side the upper hand--a political win with voters or a leg-up on with their platform.

But through the disciplines of economics, finance, organizational behavior, industrial psychology, sociology, geopolitics, and more--can we program a computer to steer the economy using facts rather than fighting and fear?

Every day, we need to make decisions, big and small, on everything from interests rates, tax rates, borrowing, defense spending, entitlements, pricing strategies, regulating critical industries, trade pacts, and more.

Left in the hands of politicians, we inject personal biases and even hatreds, powerplays, band-standing, bickering, and "pork-barrel" decision making, rather than rational acting based on analysis of alternatives, cost-benefits, risk management, and underlying ethics. 

We thumb our noises (rightfully) at global actors on the political stages, saying who is rational and who is perhaps just plain crazy enough to hit "the button."

But back here at home, we can argue about whether or not the button of economic destructionism has already been hit with the clock ticking down as the national deficit spirals upward, education scores plummet, and jobs are lost overseas?

Bloomberg BusinessWeek (30 August 2012) suggests using gaming as a way to get past the political infighting and instead focus on small (diverse) groups to make unambiguous trade-off decisions to guide the economy rather than "get reelected"--the results pleasantly were cooperation and collaboration.

Yes, a game is just a game, but there is lesson that we can learn from this--economic decision-making can be made (more) rationally by rewarding teamwork and compromise, rather than by an all or nothing, fall on your sword, party against party, winner takes no prisoner-politics. 

I would suggest that gaming is a good example for how we can improve our economy, but I can see a time coming where "bid data," analytics, artificial intelligence, modeling and simulation, and high-performance computing, takes this a whole lot further--where computers, guided and inspired by people, help us make rational economic choices, thereby trumping decisions by gut, intuition, politics, and subjective whims .

True, computers are programmed by human beings--so won't we just introduce our biases and conflict into the systems we develop and deploy?

The idea here is to filter out those biases using diverse teams of rational decision-makers, working together applying subject matter expertise and best practices and then have the computers learn over time in order to improve performance--this, separate from the desire and process to get votes and get elected.

Running the economy should not be about catering to constituencies, getting and keeping power for power sakes, but rather about rational decision-making for society--where the greatest good is provided to the greatest numbers, where the future takes center stage, where individuals preferences and rights are respected and upheld, and where ethics and morality underpin every decision we make.  

The final question is whether we will be ready to course-correct with collaboration and advances in technology to get out of this economic mess before this economic mess gets even more seriously at us?

(Source Photo: here with attribution to Erik Charlton)

Share/Save/Bookmark

August 3, 2012

FOIA Making Us Stronger

To commemorate 46 years since the Freedom of Information Act (FOIA) was passed on July 4, 1966, the Project on Government Oversight (POGO) came out with a infographic showing the significant progress that has been made in government transparency and areas they still see for possible improvement. 

Similarly, Government Executive Magazine ran an feature article in June 2012 called "The Truth Behind Transparency," calling progress with open government as "tough to gauge."

The basic idea of FOIA as the website for Sunshine Week put it is: "the public's right to know about its government."

Obviously, as GovExec points out, one of the main questions over the years with FOIA is "how quickly and fully do agencies respond to FOIA requests?"

To much and too soon, and do you perhaps put at risk various sensitive information, jeopardizing elements of the functioning of government itself?

Too little and too late, and then is the opportunity for mismanagement, waste, fraud, and abuse simply an after fact?

As Beth Novek, former deputy chief technology officer for open government, described it, open government is a "shorthand for open innovation or the idea that working in a transparent, participatory, and collaborative fashion helps improve performance, inform decision-making, encourage entrepreneurship and solve problems more effectively."

Transparency can aid in accountability by shedding a light on leadership and its performance management. It can also be a great opportunity to bring new ideas and opinions to the fold, perhaps leading to better decisions and results, at the end of the day, for all. 

The challenge for government is to guard against any information risks to the safety and security of our nation.  

An informed nation, is a stronger nation--to me, it is a foundation of a government "of the people, by the people, for the people."

Government and the people working together, duly informed, to confront our toughest challenges and solve our greatest problems.


Share/Save/Bookmark

July 6, 2012

The Information Is On You

There was a fascinating article in the New York Times (17 June 2012) called: "A data giant is Mapping and Sharing the Consumer Genome."

It is about a company called Acxion--with revenues of $1.13 billion--that develops marketing solutions for other companies based on their enormous data collection of everything about you!
 
Acxion has more than 23,000 servers "collecting, collating, and analyzing consumer data...[and] they have amassed the world's largest commercial database on consumers."

Their "surveillance engine" and database on you is so large that they:

- "Process more than 50 trillion data 'transactions' a year."
- "Database contains information about 500 million active consumers."
- "About 1,500 data points per person."
- Have been collecting data for 40 years!

Acxion is the slayer of the consumer big data dragon--doing large-scale data mining and analytics using publicly available information and consumer surveys.

They collect data on demographics, socio-economics, lifestyle, and buying habits and they integrate all this data.

Acxion generates direct marketing solutions and predictive consumer behavior information.

They work with 47 of the Fortune 100 as well as the government after 9/11.

There are many concerns raised by both the size and scope of this activity.
 
Firstly, as to the information itself relative to its:

- Privacy
- Security

Secondly, regarding the consumer in terms of potential: 

- Profiling
- Espionage
- Stalking
- Manipulation 

Therefore, the challenge of big data is a double-edged sword: 

- On one hand we have the desire for data intelligence to make sense of all the data out there and use it to maximum affect.
- On the other hand, we have serious concerns about privacy, security, and the potential abuse of power that the information enables. 

How we harness the power of information to help society, but not hurt people is one of the biggest challenges of our time. 

This will be an ongoing tug of war between the opposing camps until hopefully, the pendulum settles in the healthy middle, that is our collective information sweet spot. 

(Source Photo: Andy Blumenthal)


Share/Save/Bookmark