Showing posts with label Data Management. Show all posts
Showing posts with label Data Management. Show all posts

September 27, 2017

A Mountain Of Data

So I heard this interesting perspective on information and data analytics...

Basically, it comes down to this: 
"Most organizations are data rich, but information/insight poor."

Or put another way:
"Data is collected, but not used."

Hence we don't know what we don't know and we end up making bad decisions based on poor information. 

Just imagine if we could actually make sense of all the data points, connect them, visualize them, and get good information from them.

How much better than a pile of rocks is that? 

(Source Photo: Andy Blumenthal)

March 1, 2012

Dashboarding The Information Waves

I had an opportunity to view a demo of a dashboarding product from Edge called AppBoard, and while this is not a vendor or product endorsement, I think it is a good example to briefly talk about these types of capabilities. 

Dashboard products enable us to pull from multiple data sources, make associations, see trends, identify exceptions, and get alerts when there are problems.

Some of the things that I look for in dashboard tools are the following:

- Ease of use of connecting to data 

- Ability to integrate multiple stovepiped databases

- A variety of graphs, charts, tables, and diagrams to visualize the information

- Use of widgets to automatically manipulate the data and create standardized displays

- Drag and drop ability to organize the dashboard in any way you like to see it

- Drill down to get more information on the fly 

While there are many tools to consider that provide dashboards, information visualization, and business intelligence, I think one of the most important aspects of these is that they be user-centric and easy to implement and customize for the organization and its mission.

When making critical decisions (especially those involving life and death) and when time is of the essence--we need tools that can be can be easily navigated and manipulated to get the right information and make a good decision, quickly. 
As a fan of information visualization tools, I appreciate tools like this that can help us get our arms around the "information overload" out there, and I hope you do too.

(All Opinions my own)


February 29, 2012

Progressing From Data to Wisdom

I liked this explanation (not verbatim) by Dr. Jim Chen of data, information, knowledge, and wisdom.

- Data: This is an alphanumeric entity and/or symbol (ABC, 123, !@#...)

- Information: This is when entities are related/associated to each other and thereby derive meaning. (Information = Data + Meaning)

- Knowledge: This is information applied to context. (Knowledge = Data + Meaning + Context)

- Wisdom: This is knowledge applied to multiple contexts. (Wisdom = Data + Meaning + (Context x N cases)).

I'd like to end this blog with a short quote that I thought sort of sums it up:

"A man may be born to wealth, but wisdom comes only with length of days." - Anonymous
(Source Photo: here)


August 23, 2009

E-memory and Meat Memory

As we move towards a “paperless society” and migrate our data to the computer and the Internet, we can find personal profiles, resumes, photos, videos, emails, documents, presentations, news items, scanned copies of diplomas and awards, contact lists, and even financial, tax, and property records.

People have so much information on the web (and their hard drive) these days that they fear one of two things happening:

  1. Their hard drive will crash and they will lose all their valuable information.
  2. Someone will steal their data and their identity (identity theft)

For each of these, people are taking various precautions to protect themselves such as backing up their data and regularly and carefully checking financial and credit reports.

Despite some risks of putting “too much information” out there, the ease of putting it there, and the convenience of having it there—readily available—is driving us to make the Internet our personal storage device.

One man is taking this to an extreme. According to Wired Magazine (September 2009), Gordon Bell is chronicling his life—warts and all—online. He is documenting his online memory project—MyLifeBits—in a book, called Total Recall.

“Since 2001, Bell has been compulsively scanning, capturing and logging each and every bit of personal data he generates in his daily life. The trove includes Web Sites he’s visited (22,173), photos taken (56,282), docs written and read (18,883), phone conversations had (2,000), photos snapped by SenseCam hanging around his neck (66,000), songs listened to (7,139) and videos taken by (2,164). To collect all this information, he uses a staggering assortment of hardware: desktop scanner, digicam, heart rate monitor, voice recorder, GPS logger, pedometer, Smartphone, e-reader.”

Mr. Bell’s thesis is that “by using e-memory as a surrogate for meat-based memory, we free our minds to engage in more creativity, learning, and innovation.”

Honestly, with all the time that Bell spends capturing and storing his memories, I don’t know how he has any time left over for anything creative or otherwise.

Some may say that Gordon Bell has sort of an obsessive-compulsive disorder (OCD)—you think? Others that he is some sort of genius that is teaching the world to be free and open to remembering—everything!

Personally, I don’t think that I want to remember “everything”. I can dimly remember some embarrassing moments in elementary school and high school that I most sure as heck want to forget. And then there are some nasty people that would be better off buried in the sands of time. Also, some painful times of challenge and loss—that while may be considered growth experiences—are not something that I really want on the tip of my memory or in a file folder on my hard drive or a record in a database.

It’s good to remember. It’s also sometimes good to forget. In my opinion, what we put online should be things that we want or need to remember or access down the road. I for one like to go online every now and then and do some data cleanup (and in fact there are now some programs that will do this automatically). What I thought was worthwhile, meaningful, or important 6 months or a year ago, may not evoke the same feelings today. Sometimes, like with purchases I made way back when, I think to myself, what was I thinking when I did that? And I quickly hit the delete key (wishing I could do the same with those dumb impulse purchases!). Most of the time, I am not sorry that I did delete something old and I am actually happy it is gone. Occasionally, when I delete something by accident, then I start to pull my hair out and run for the backup—hoping that it really worked and the files are still there.

In the end, managing the hard drive takes more work then managing one’s memories, which we have little conscious control over. Between the e-memory and the meat memory, perhaps we can have more of what we need and want to remember and can let go and delete the old and undesired one—and let bygones be bygones.

June 21, 2009

Making More Out of Less

One thing we all really like to hear about is how we can do more with less. This is especially the case when we have valuable assets that are underutilized or potentially even idle. This is “low hanging fruit” for executives to repurpose and achieve efficiencies for the organization.

In this regard, there was a nifty little article in Federal Computer Week, 15 Jun 2009, called “Double-duty COOP” about how we can take continuity of operations (COOP) failover facilities and use them for much more than just backup and business recovery purposes in the case of emergencies. 

“The time-tested approach is to support an active production facility with a back-up failover site dedicated to COOP and activated only during an emergency. Now organizations can vary that theme”—here are some examples:

Load balancing—“distribute everyday workloads between the two sites.”

Reduced downtime—“avoid scheduled outages” for maintenance, upgrades, patches and so forth.

Cost effective systems development—“one facility runs the main production environment while the other acts as the primary development and testing resource.”

Reduced risk data migration—when moving facilities, rather than physically transporting data and risk some sort of data loss, you can instead mirror the data to the COOP facility and upload the data from there once “the new site is 100 percent operational.”

It’s not that any of these ideas are so innovatively earth shattering, but rather it is their sheer simplicity and intuitiveness that I really like.

COOP is almost the perfect example of resources that can be dual purposed, since they are there “just in case.” While the COOP site must ready for the looming contingency, it can also be used prudently for assisting day-to-day operational needs.

As IT leaders, we must always look for improvements in the effectiveness and efficiency of what we do. There is no resting on our laurels. Whether we can do more with less, or more with more, either way we are going to advance the organization and keep driving it to the next level of optimization. 


March 9, 2008

Better Data Management and Enterprise Architecture

What is Information Technology? Well in simple terms its technology that enables information processing, storage, sharing, and accessibility. The business needs information to carry out its functions, processes, activities, and tasks. The systems and their underlying technologies process the underlying data to get it to the people who need it in our organizations.

Government Computer News, 21 January 2008 has an article by Mike Daconta (previously from the Department of Homeland Security) that offers tip on better data management.

  1. Data privacy audit—“given that identity theft and government data loss are of public concern, you should conduct an audit of the privacy vulnerability of your data assets.”
  2. Data dictionary—the article calls for a business glossary to communicate across organizational boundaries; to ensure that terms mean the same thing to everyone. I would call this an enterprise data dictionary.
  3. Data mashups—use web applications to combine data and/or functionality from more than one source.
  4. Data elements—“expose each major data entity in your business glossary [I would say in your data inventory]…with a standard set of create, read, update, and delete services. You then build higher-level services on top of these foundational services” for SOA.

The article has a simile for describing data as follows: “if money is an organization’s lifeblood, and people its muscles, data is the nervous system.”

But data is not really the nervous system, instead the network is the nervous system, since it is the network that relays messages back and forth from one body part to another.

So what body part is data like?

Data is the electrical impulses carried by the nervous system that tells the various body parts what they need to do.

Interestingly, when is a person declared dead? When they have no brain function anymore. Not when they cannot eat or breathe (machines can perform this artificially to keep a person “alive”.) But if the brain that processes the data is not functioning, then we declare a person dead. Without the ability to process data, neither an organization nor a person can survive.


February 3, 2008

SOA, Data Management and Enterprise Architecture

Often I hear business and IT people say that Service Oriented Architecture (SOA) is the way ahead to achieve greater interoperability of systems, process integration, and business efficiency in the enterprise. Usually, this is quickly followed by discussion about rolling out the Enterprise Service Bus (ESB). However, very infrequently do I hear discussion about the necessary data architecture to achieve the vision of SOA.

Stephen Lahanas has an interesting article, “Enable SOA Transformation and Cross-Domain Data Fusion” in DM Review Magazine, January 2008 that addresses the importance of Data Management to SOA. (

“While SOA has long considered universal description, discovery and integration (UDDI) as its primary discoverability mechanism, the reality is that nearly all integration with an SOA environment is based upon data exchange and will ultimately be demonstrated through data exploitation interfaces (agile BI). Once SOA architects fully realize the implications of this revelation, then agile data architecture will become the facilitating mechanism for cross-domain data fusion and enterprise integration.

Mr. Lahanas provides four fundamental elements for building an agile data architecture, as follows:

  1. Actionable enterprise architecture—“a tangible way to connect architecture layers (EA, segment and implementation) and perspectives (application, process, data). One of the main reasons that large integration projects fail is due to the inability to successfully map the various architectures within a meaningful combined picture. Every agile data architecture begins here.”
  2. Federated data orchestration—“allow data owners to collaboratively manage resources across domains based upon a shared set of rules rather than a shared single data model. This is an excellent example of a user-centric approach. One of the major shortcomings of massive data warehouse projects has been the lost connections between users and developers and resulting data integrity issues.
  3. Enterprise Master Data Management—“Metadata is not just a technical consideration; it can define productivity in our knowledge economy.”
  4. Agile Business Intelligence—“A new generation of BI capabilities is bringing user control to more sophisticated report generation tools with much more accurate results… Based upon user queries and activities, we gather metadata, optimize caches and determine cross-domain mapping strategies.”
Two ideas that I particularly like in Stephen’s article are:

  1. Linkage of SOA to Data Architecture—“Agile data architecture is a parallel and complementary design philosophy and methodology to SOA and agile application development. Both can be mapped together within the larger actionable enterprise architecture.”
  2. Focus on User-centricity—“User-centricity is the primary motivating force behind the development of all agile solutions. The user provides:
  • Immediacy – the desire for near-term real-world capability.
  • Relevance and context.
  • Performance expectations.
  • Direction, domain knowledge and the logic behind every solution.”

In developing SOA, we cannot forget the necessity of building a meaningful data architecture that facilitates the discovery and exchange of data via SOA and ESB. Further, all our architecture and IT solutions must be user-centric if they are to be relevant and effective to the enterprise’s end-users.