28 May 2021

TypeDB

TypeDB

Drawbacks of SHACL

SHACL is a constraint language for validating RDF graphs against a set of conditions. It works in a closed-world assumption when the end goal is an open-world assumption. The validation is only really defined against those set conditions of constraint. There will be cases when validation cases are missed at point of inference. The entire search space can not be tested against constraints defined over a set of conditions. Approach can be seen as rather opposite to the intended goal. What follows from a SHACL validation can lead to a form of reverse engineering leading to a partially closed-world assumption criteria. It may be better to introduce SHACL earlier in the process rather than later in the process so as to avoid conflict of outcomes. SHACL validation tests can quickly go out of hand if acceptance tests against requirements that are defined in a question/answer form become a form of integration validation tests where constraints have overlapping dependencies. One can notice how quickly SHACL tests form unmaintainable targets and impervious constraints that are derived from set of conditions. In all fairness one may want to validate the graph at point of when it is in a state of a closed-world assumption and not after it has been generalized to an open-world assumption.

23 May 2021

Who is regarded as an expert?

Is an expert, someone with years of practical experience or someone with a Phd that doesn't know how to apply any of the theory in to practice? In, most cases, an expert is someone that has a balance of both and the practical experience to match, but not necessarily someone with a Phd. In fact, invariably, an expert does not have a Phd and usually finds the whole approach pointless. However, the way things move in the world, one is an expert one minute, and pretty much an outdated expert the next. It takes at least 10 years of practically applied experience to become an expert at something. By which time, either the approach or the technology becomes outdated. So, is it even worth becoming an expert? In some areas there is no other way. However, in most cases an expert is a pointless notion as it is all fairly subjective. But, conservatives will still focus more weight on academics than on experience. Ultimately, people learn the hard way when their financial pockets run dry and they need someone to deliver on a solution in the shortest amount of time rather than take any further risks with useless Phds who have no clue whatsoever and have provided quantifiably and qualitatively zero results in terms of return on investment to an organization. The safest bet for organizations is to develop a team of astute generalists that can adapt to change who have the ability to self-learn, self-explore, and self-grow in their practical endeavors, which is the whole point of what a degree is supposed to prepare and enable an individual at point of their graduation.

22 May 2021

Key Things Missing In Academics

Academics cannot replace practical skills. Many academically inclined people, that tend to have a Phd, have theoretical backgrounds but lack sufficient skills to convert it into practice for it to be of any use to business. Many academic courses at universities also discount certain key areas that almost always occur in practice or in the practical world would be defined as common sense. The following highlight four core areas that almost always are necessary in solving business problems in practice that are never sufficiently taught in academics: 

  • Rationalization of complexity:
    • Understanding complexity of the world we live in and the fact something that can be done in academic theory may not be possible in practice, given the resources at the time. Understanding such aspects of computational cost, architectural constraints, input/output dependencies, third-party dependencies, scalability requirements, resilience, latency, load, bandwidth, performance, time constraints, funding limitations, management buy-in, cloud resources, access to data, licensing requirements, state management, copyright restrictions, regulatory requirements, skill availability, and other such complexity constraints.
  • Rationalization of uncertainty:
    • Understanding that in practice there are many uncertainty variables, that need to be taken into account, that often may get ignored in academic theory. Often this involves a certain degree of risk. Such risks could take on many forms of economic, geopolitical, social, environmental, government regulation, criminal, accidents, errors, event state outcomes, customer behavior, market demand/trends, failure to deliver, health and safety, loss of team resource, loss of funding, sudden eventual changes, and any such factors outside of immediate control.
  • Rationalization of noisy data:
    • Understanding that in practice data is almost always noisy and that someone will not provide you clean data on a silver spooned platter.
  • Rationalization of context:
    • Understanding that in practice everything has context, and it is in this context that things can be constructively applied within the bounds of rational pragmatic thinking. There is no silver bullet that can magically solve all problems of the world. Often context understanding comes with practice as it involves all the above key areas of complexity, uncertainty, and being able to handle noisy data. In most cases, a formula is not provided and the problem is not defined. One has to formulate the problem present in the data, and discover a formula to solve such a problem.
As a side note, it seems as the more academic one gets, the more inclined one becomes, at not using common sense. And, even the most simple tasks become difficult to accomplish or require additional help from others. So it seems, academics is a trajectory that promises a lot initially but quantifiably or qualitatively delivers little in return to the individual in society over the long-term.

Matterhorn

Matterhorn

16 May 2021

Six Elements of Web Intelligence

  • Look 
  • Listen 
  • Learn 
  • Connect 
  • Predict 
  • Correct

What is a Knowledge Graph

A knowledge graph has a few key characteristics:

  • Is a type of knowledge base containing semantically integrated and interconnected graph-structured data
  • The representation is in a machine-readable standard that follows an open-world assumption
  • The representation forms an abstraction of a connected graph of relationships 
  • An ability to reason over the connected representation, defined in an open-world assumption, in order to build inference on new relationships
  • The queryable representation is iteratively evolvable through machine-inference, therefore dynamic and not static
  • The ability to semantically infer over machine-readable data allows the abstract representation of data to extend from information into knowledge
  • The semantic context is prevalent and embedded in the knowledge representation, namespaced, and tends to be defined as interlinked URI resources
  • The representation can be stored in any noSQL database, that is able to support the serialized data format, but for performance reasons it tends to be stored in a native graph-like triplestore or a quadstore
  • Property graph without the machine-readable representation and inference layer is not a knowledge graph 
  • Without machine-readable representation and inference layer the representation is static, merely data, likely to be defined in a closed-world assumption that can be queried and managed in a database leaving much of the form of inference to the person running the analytics
  • With additional inference layer and rich machine-readable representation, the database can take a performance hit in both reads and writes, in most cases writes are sub-optimal to reads, and therefore a write operation tends to be done in the background as a periodic bulk function and only a read operation is made available to the end user (an example of this can be seen between Wikipedia, which is available for read/writes, and DBpedia, as knowledge graph which is bulk loaded every so often and made available for read-only)
  • Many semantic graphs still work on the client-server basis with vertical scaling option of the pre-2000s era, which may pose an issue for heavy read/writes where reliance is on one core server, to avoid downtime make bulk writes offline on a hot swappable as a replicated instance, once resolved then swap the instances 
  • In a knowledge graph, only metadata is stored (data about data which is in a connected and machine-readable form) and not the changing data itself (keep the data separate from the metadata)
  • A knowledge graph is generally intended for searchability, queryability, discoverability, and findability cases and not for heavy transactional cases
  • If one wants heavy read/writes then opt for a transactional option, in most cases where such databases may not support inference nor provide semantic compliance, but may provide sharding, horizontal scaling, static property graphs in their serialized data representations which may be compliant with the tinkerpop stack, alternatively, an in-memory analytical graph may also be an option to avoid heavy I/O performance hits (bare in mind, without inference + machine-readable representation it is no longer a knowledge graph but merely a static connected graph-structured data representation stored in a database)
  • Knowledge Graph is a special case of a Knowledge Base defined as per the graphical representation that it holds in the data abstraction in form of subject-predicate-object triples, however, machine-readable knowledge does not necessarily have to be stored in form of a triple, the W3C have defined a specific set of standards for working with semantic data, but is not necessarily the only way