Thursday, December 8, 2022
HomeBusiness Intelligence7 Frequent Knowledge High quality Issues

7 Frequent Knowledge High quality Issues

data quality problems

Having Knowledge High quality issues is a standard – and expensive – challenge. In keeping with Gartner, poor-quality information prices organizations a mean of $12.9 million yearly. Knowledge High quality makes use of elements akin to accuracy, consistency, and completeness in figuring out the worth of the info. Excessive-quality information will be trusted, whereas low-quality information is inaccurate, inconsistent, or incomplete. Along with important quantities of misplaced income, utilizing low-quality information can lead to poor enterprise choices and lowered operational effectivity. 

Poor-quality information will weaken and harm vital enterprise actions, akin to operating e-mail campaigns and figuring out repeat prospects. 


Begin your coaching in Knowledge Governance, Knowledge High quality, Knowledge Structure, Knowledge Modeling, and extra with our course subscriptions.

Clear, correct, high-quality information permits a corporation to make clever choices and attain targets. The higher high quality the info, the extra possible it’s that gross sales and advertising efforts will likely be profitable. The affect of poor Knowledge High quality on gross sales and advertising can embrace things like unreliable buyer concentrating on or disagreeable buyer experiences. 

Moreover, poor Knowledge High quality can forestall automation from working correctly. 

There are a number of the way gross sales and advertising promoting will be automated. However, as a result of automated promoting campaigns depend on excessive Knowledge High quality (or accuracy), they will alienate potential prospects if that information is as a substitute poor high quality.

Sadly, fixing Knowledge High quality issues isn’t a once-and-done exercise. It’s a course of requiring steady consideration.

Knowledge Governance: Accountability and Expertise 

Typically talking, Knowledge Governance applications, that are a mixture of expertise and human habits, are liable for Knowledge High quality – in addition to complying with varied laws. Software program is often used to offer automated companies for processing the info, whereas people should be skilled in the perfect methods to advertise high-quality information.

Having a single individual, the information steward, be liable for the training of employees and the upkeep of this system total is an environment friendly manner of selling high-quality information.

The information steward is liable for educating the employees on how you can help good Knowledge Governance, and assuring the software program is working appropriately. (In lots of organizations, the info steward studies to the chief information officer, who in flip studies to the Knowledge Governance committee.)

A well-designed Knowledge Governance program, which incorporates human intervention, will right poor Knowledge High quality points.

Frequent Knowledge High quality Issues, and Cope with Them 

Poor Knowledge High quality promotes dangerous decision-making. Having high-quality information promotes good decision-making. It is very important resolve Knowledge High quality issues as shortly as doable. Some Knowledge High quality points are extra frequent than others, and are listed under:

Knowledge inconsistencies: This downside happens when a number of methods are storing info with out utilizing an agreed upon, standardized methodology of recording and storing info. Inconsistency is typically compounded by information redundancy. For instance, a buyer’s final title being recorded earlier than their first title in a single division, and vice versa in several departments. One more downside is when one shops information in a PDF format, whereas one other makes use of Microsoft Docs. 

Fixing this downside requires the info be homogenized (or standardized) earlier than or because it is available in from varied sources, probably by means of the usage of an ETL information pipeline.

Incomplete information: That is typically thought of the commonest challenge impacting Knowledge High quality. Key information columns will likely be lacking info, usually inflicting analytics issues downstream. 

A superb methodology for fixing that is to put in a reconciliation framework management. This management would ship out alerts (theoretically to the info steward) when information is lacking.

Orphaned information: It is a type of incomplete information. It happens when some information is saved in a single system, however not the opposite. If a buyer’s title will be listed in desk A, however their account will not be listed in desk B, this is able to be an “orphan buyer.” And if an account is listed in desk B, however is lacking an related buyer, this is able to be an “orphan account.” 

An automated service that checks for consistency when information is downloaded into tables A and B is a possible answer. Discovering the supply of the issue (usually a human) is another choice.

Irrelevant information: Irrelevant information is all over the place. Screening it out upfront, earlier than storage, will be time-consuming, and should eradicate information that “may very well be” helpful. Sadly, storing nice chunks of information is costlier and fewer sustainable than making the trouble to display screen out the ineffective information upfront. Screening out the ineffective information is extra environment friendly and cost-effective from a big-picture perspective. 

To resolve this downside, setting limits (typically often known as information capturing rules) ought to develop into a analysis requirement. Broadly talking, if the info can be utilized to perform an finish objective, it’s truthful sport. If not, the info shouldn’t be collected.

Outdated information: Outdated information, like outdated info, loses worth, and over time will not characterize actuality. Issues change. Storing outdated information is an pointless expense. It could confuse employees, and it has a unfavourable affect on performing information analytics. Storing information after a sure period of time gives no worth and promotes information decay. 

The Knowledge Governance software program ought to have a “GDPR precept on retention” choice, which will be set to reserve it for “not than essential.”

Redundant information: Occasionally, a number of individuals inside a corporation will seize the identical information, repeatedly. Not solely is that this a waste of employees time (six individuals accumulating the identical information, when just one is required), however there’s the expense of storing the redundant information.

A grasp information administration program can be utilized to resolve this challenge.

Duplicate information: When information is duplicated, it’s saved in two or extra areas. Usually, this isn’t a lot of a difficulty, except the duplicated information is “outdated,” of poor high quality, or being duplicated a number of instances. Whereas pretty straightforward to detect, it may be a little bit troublesome to repair. 

For relational (SQL) databases, there’s a characteristic referred to as “normalization” that can be utilized to cope with duplications. Moreover, grasp information administration controls will be carried out to help a “uniqueness examine.” This management checks for actual duplicates of saved information and purges one (or extra) duplicates. 

Greatest Practices for Knowledge High quality

Utilizing greatest practices can act as a type of preventative upkeep and assist to keep away from Knowledge High quality issues. 

  • Automation: Cloud computing makes it straightforward to entry information from a number of totally different sources, but in addition comes with the problem of integrating information from totally different sources and in several codecs. Coping with this problem requires the info be cleansed and de-duplicated. (Usually, an information preparation device is used to scale back the quantity of human labor.)
  • The need of normal consensus: If solely 75% of a enterprise’s employees are dedicated to making sure good Knowledge High quality, then it’s cheap to count on “some” of the info will likely be of low high quality. All of administration, and all of the employees coping with information, should perceive the significance of Knowledge High quality and take duty for sustaining it. That is the place the info steward is available in – first, as an educator and, when wanted, as the info police, to implement Knowledge Governance insurance policies.
  • Measuring Knowledge High quality: A formulation has been developed that permits for tough measurements of a corporation’s Knowledge High quality. By creating a measurement system to find out the standard of the info, and utilizing it, downside areas will be recognized and corrected, leading to higher-quality information. This may be scheduled as a month-to-month Knowledge High quality audit. Measuring Knowledge High quality will not be the identical as correcting the errors. It merely clarifies which areas are having issues.
  • Growing a Knowledge Governance program: If the enterprise doesn’t have already got a Knowledge Governance program, it’s most likely time to develop one. A Knowledge Governance program will be described as a set of insurance policies, roles, processes, and requirements that promote the environment friendly use of information for reaching the enterprise’s targets. 
  • Educating employees and administration: This must be organized by the info steward, with the assistance of the chief information officer. Since homework typically isn’t an choice, time should be scheduled throughout work hours. This may very well be achieved for a number of hours, with virtually everybody attending, or it may very well be achieved with small teams of employees, or some mixture of the 2. 
  • A single supply of fact (SSoT): This idea helps to guarantee all employees making choices are utilizing the identical predetermined, extremely reliable supply. Many important enterprise choices depend on correct, high-quality information, and utilizing a trusted supply will decrease errors. An SSoT is often one centralized storage space for all of the enterprise info. (Some analysis information should come from outdoors sources, however data in regards to the enterprise ought to come from the SSoT.) 


Poor Knowledge High quality can have an incredible affect on vital analysis tasks, akin to enterprise intelligence and growing the client expertise. Fixing Knowledge High quality issues must be one of many group’s high priorities, and clever investing in it’ll enhance effectivity and improve earnings.

Picture used beneath license from



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments