Showing posts with label data warehouse. Show all posts
Showing posts with label data warehouse. Show all posts

February 12, 2008

Information Integrity and Enterprise Architecture

We are in an information economy and now more than ever business needs information to conduct their functions, processes, activities, and tasks.

To effectively conduct our business, the information needs to be relevant and reliable. The information should be current, accurate, complete, understandable, and available.

Information integrity is essential for enabling better decision-making, improving effectiveness, and reducing risk and uncertainty.

However, according to DMReview, 8 February 2008, “information within the [corporate] data warehouse continues to be inaccurate, incomplete, and often inconsistent with its sources. As a result, data warehouses experience low confidence and acceptance by users and consumers of downstream reports.”

“The Data Warehousing Institute estimates that companies lose more than $600 million every year due to bad information.”

What are some of the challenges to information integrity?

  1. Complex environments, [in which organizations] constantly generate, use, store, and exchange information and materials with customers, partners, and suppliers.”
  2. Accelerating change in the business environment [and] changing needs of business users”
  3. “Increasing complexity of source systems and technology
  4. Expanding array of regulations and compliance requirements

“Change and complexity introduce information integrity risk. Accelerating change accelerates information integrity risk. Compliance makes information integrity an imperative rather than an option.”

What are the particular challenges with data warehouses?

  1. Questionable input information—“Several source systems feed a data warehouse. Data may come from internal and external systems, in multiple formats, from multiple platforms.”
  2. Lack of downstream reconciliation—“As information traverses through the source systems to a data warehouse, various intermediate processes such as transformations may degrade the integrity of the data. The problem becomes more acute when the data warehouse feeds other downstream applications.”
  3. Inadequate internal controls—these include controls over data input, processing, and output, as well as policies and procedures for change management, separation of duties, security, and continuity of operations planning.

From an enterprise architecture perspective, information integrity is the linchpin between the businesses information requirements and the technology solutions that serves up the information to the business. If the information is no good, then what good are the technology solutions that provide the information to the business? In other words, garbage in, garbage out (GIGO)!

As enterprise architects, we need to work with the business and IT staffs to ensure that data captured is current, accurate, and complete, that it is entered into the system correctly, processed accurately, and that outputs are distributed on a need to know basis or as required for information sharing purposes, and is protected from unauthorized changes.

Using business, data, and systems models to decompose the processes, the information required for those, and the systems that serve them up helps to identity possible information integrity issues and aids in designing processes that enable quality information throughput.

Additionally, security needs to be architected into the systems from the beginning of their lifecycle and not as an afterthought. Information confidentiality, integrity, availability, and privacy are essential for an information secure enterprise and for information quality for mission/business performance.


September 3, 2007

Business Intelligence and Enterprise Architecture

“Business intelligence (BI) refers to applications and technologies that are used to gather, provide access to, and analyze data and information about company operations. Business intelligence systems can help companies have a more comprehensive knowledge of the factors affecting their business, such as metrics on sales, production, internal operations, and they can help companies to make better business decisions.” (Wikipedia)

Business intelligence includes warehousing data and mining data (sorting large amounts of data to find relevant information). Metadata (data about data) aids in the mining of useful nuggets of information. The warehousing and mining of data for business intelligence is often referred to as a decision support system.

User-centric EA is business (and technology) intelligence!

  • EA is a knowledge base and warehouse of information: BI warehouses date for decision support applications in the organization. Similarly, EA synthesizes and stores business and technical information across the enterprise to enable better decision making. EA uses applications like Systems Architect, DOORS, Metis, Rationale, and others to capture information in a relational database repository and model business, data, and systems. The intent is to develop a knowledge base for the capture, mining and analysis of data to enhance IT planning and governance.
  • EA provides for mining, querying, and reporting: BI tools use online analytical processing (OLAP) tools like Cognos, BusinessObjects, Hyperion, and SAS that utilize multi-dimensional database cubes for manipulating data into different views, and provides for analysis and reporting. Similarly, User-centric EA provides for analysis and reporting of performance measures, business functions, information requirements, applications systems, technology products and standards, and security measures. While EA tools are more limited than general BI tools in terms of OLAP capabilities like online queries, I believe that these tools will move in this direction in the future.
  • EA uses information visualization to communicate effectively: BI tools provide executive dashboard capabilities for displaying executive information in a user-friendly GUI format. Like an executive dashboard, EA often displays business and technology information in profiles and models that make extensive use of information visualization to communicate effectively and at strategic, high-level views to decision makers.

In is the role of the chief enterprise architect to sponsor, communicate, and educate on the use of EA for business and technology intelligence in the organization.