16 February 2016

AI for Data Validation and Verification

It is predicted that robots will replace many jobs in next 30 years time. However, one of the first critical roles they need to replace is the verification and validation of data input from human error. This is one of most common problems that occurs in business where human error can cause fraud not to mention decline someone from an application that they may have made for a mortgage, loan, security clearance, or recruitment. Furthermore, a business requires data entry in supply chain, ledger accounting, and more. As one can tell the role of data entry is critical across multiple business sectors. Often times the manual task of replacing a form entry into a system by a human needs to be replaced through automation. Furthermore, critical verification and validation checks need to be in place to ensure the data is correct as well as to meet compliance and mitigate risk. Data Validation is usually the aspect of checking that the data entered is sensible and reasonable. However, it does not check the accuracy of such data. Types of validation incorporate checking: digits, format, length, acceptable value lookup, presence of field entry, range, and spelling. Data Verification is usually to check that the data entered matches the source. This can be checked in two ways: double entry and proofreading data. Most of these, if not all can be automated as part of an intelligent agent role designation that can semantically understand the context of the data for validation while at the same time being able to check for the verification of data entry. These days forms are scanned or copied rather than manually entered. However, even such processes require being able to read the handwriting. The intelligent agent needs to be able to understand the different forms of handwriting to deduce characters of a language and semantically understand the meaning without diluting the context of the form nor the data. In process, an intelligent agent needs to be able to process vast quantities at speed greater than that possible for a human i.e. batch processing. Big Data Pipelines have made significant in roads towards automation in the data mining and retrieval with options for stream processing of information. Forms on the web are another aspect of data entry that is often used and entered into a backend database which surely need more intelligent means of validation and verification. Even the role of call center agent can be replaced. Additionally, the knowledgeable intelligent agent will need speech recognition, ability for text-speech analysis, as well as affective understanding of human emotions as part of customer service. At same time, the intelligent agent will need to both facilitate knowledgeable understanding of domain context while processing new information as part of the data entry step. Multitasking is something that computers have been better at than most humans while avoiding error. But, for specialized agents and robots it becomes more complex in learning as tasks get diversified. As we look forward into the future, we are likely to increase trust in artificial intelligence for everyday things while making our lives more complex in other areas of life especially human relationships. In process, data drifts everywhere around us and we adapt to ubiquitous technology as part of a new lifestyle.