We are in an information economy and now more than ever business needs information to conduct their functions, processes, activities, and tasks.
To effectively conduct our business, the information needs to be relevant and reliable. The information should be current, accurate, complete, understandable, and available.
Information integrity is essential for enabling better decision-making, improving effectiveness, and reducing risk and uncertainty.
However, according to DMReview, 8 February 2008, “information within the [corporate] data warehouse continues to be inaccurate, incomplete, and often inconsistent with its sources. As a result, data warehouses experience low confidence and acceptance by users and consumers of downstream reports.”
“The Data Warehousing Institute estimates that companies lose more than $600 million every year due to bad information.”
What are some of the challenges to information integrity?
- “Complex environments, [in which organizations] constantly generate, use, store, and exchange information and materials with customers, partners, and suppliers.”
- “Accelerating change in the business environment [and] changing needs of business users”
- “Increasing complexity of source systems and technology”
- “Expanding array of regulations and compliance requirements”
“Change and complexity introduce information integrity risk. Accelerating change accelerates information integrity risk. Compliance makes information integrity an imperative rather than an option.”
What are the particular challenges with data warehouses?
- Questionable input information—“Several source systems feed a data warehouse. Data may come from internal and external systems, in multiple formats, from multiple platforms.”
- Lack of downstream reconciliation—“As information traverses through the source systems to a data warehouse, various intermediate processes such as transformations may degrade the integrity of the data. The problem becomes more acute when the data warehouse feeds other downstream applications.”
- Inadequate internal controls—these include controls over data input, processing, and output, as well as policies and procedures for change management, separation of duties, security, and continuity of operations planning.
From an enterprise architecture perspective, information integrity is the linchpin between the businesses information requirements and the technology solutions that serves up the information to the business. If the information is no good, then what good are the technology solutions that provide the information to the business? In other words, garbage in, garbage out (GIGO)!
As enterprise architects, we need to work with the business and IT staffs to ensure that data captured is current, accurate, and complete, that it is entered into the system correctly, processed accurately, and that outputs are distributed on a need to know basis or as required for information sharing purposes, and is protected from unauthorized changes.
Using business, data, and systems models to decompose the processes, the information required for those, and the systems that serve them up helps to identity possible information integrity issues and aids in designing processes that enable quality information throughput.
Additionally, security needs to be architected into the systems from the beginning of their lifecycle and not as an afterthought. Information confidentiality, integrity, availability, and privacy are essential for an information secure enterprise and for information quality for mission/business performance.