2102 03406 Symbolic Behaviour in Artificial Intelligence

Symbolic AI’s strength lies in its knowledge representation and reasoning through logic, making it more akin to Kahneman’s “System 2” mode of thinking, which is slow, takes work and demands attention. That is because it is based on relatively simple underlying logic that relies on things being true, and on rules providing a means of inferring new things from things already known to be true. At the height of the AI boom, companies such as Symbolics, LMI, and Texas Instruments were selling LISP machines specifically targeted to accelerate the development of AI applications and research. In addition, several artificial intelligence companies, such as Teknowledge and Inference Corporation, were selling expert system shells, training, and consulting to corporations. It wasn’t until the 1980’s, when the chain rule for differentiation of nested functions was introduced as the backpropagation method to calculate gradients in such neural networks which, in turn, could be trained by gradient descent methods. For that, however, researchers had to replace the originally used binary threshold units with differentiable activation functions, such as the sigmoids, which started digging a gap between the neural networks and their crisp logical interpretations.

He is worried that the approach may not scale up to handle problems bigger than those being tackled in research projects. “But as we expand and exercise the symbolic part and address more challenging reasoning tasks, things might become more challenging.” For example, among the biggest successes of symbolic AI are systems used in medicine, such as those that diagnose a patient based on their symptoms. First, a neural network learns to break up the video clip into a frame-by-frame representation of the objects.

The second AI summer: knowledge is power, 1978–1987

To build AI that can do this, some researchers are hybridizing deep nets with what the research community calls “good old-fashioned artificial intelligence,” otherwise known as symbolic AI. The offspring, which they call neurosymbolic AI, are showing duckling-like abilities and then some. “It’s one of the most exciting areas in today’s machine learning,” says Brenden Lake, a computer and cognitive scientist at New York University. Building on the foundations of deep learning and symbolic AI, we have developed technology that can answer complex questions with minimal domain-specific training. Initial results are very encouraging – the system outperforms current state-of-the-art techniques on two prominent datasets with no need for specialized end-to-end training. Their Sum-Product Probabilistic Language (SPPL) is a probabilistic programming system.

• A separate inference engine processes rules and adds, deletes, or modifies a knowledge store.
• Thus contrary to pre-existing cartesian philosophy he maintained that we are born without innate ideas and knowledge is instead determined only by experience derived by a sensed perception.
• Our researchers are working to usher in a new era of AI where machines can learn more like the way humans do, by connecting words with images and mastering abstract concepts.
• They use this to constrain the actions of the deep net — preventing it, say, from crashing into an object.

DOLCE is an example of an upper ontology that can be used for any domain while WordNet is a lexical resource that can also be viewed as an ontology. YAGO incorporates WordNet as part of its ontology, to align facts extracted from Wikipedia with WordNet synsets. The Disease Ontology is an example of a medical ontology currently being used.

IBM’s new AI outperforms competition in table entry search with question-answering

This only escalated with the arrival of the deep learning (DL) era, with which the field got completely dominated by the sub-symbolic, continuous, distributed representations, seemingly ending the story of symbolic AI. And while the current success and adoption of deep learning largely overshadowed the preceding techniques, these still have some interesting capabilities to offer. In this article, we will look into some of the original symbolic AI principles and how they can be combined with deep learning to leverage the benefits of both of these, seemingly unrelated (or even contradictory), approaches to learning and AI. This page includes some recent, notable research that attempts to combine deep learning with symbolic learning to answer those questions. Another benefit of combining the techniques lies in making the AI model easier to understand. Humans reason about the world in symbols, whereas neural networks encode their models using pattern activations.

However, contemporary DRL systems inherit a number of shortcomings from the current generation of deep learning techniques. For example, they require very large datasets to work effectively, entailing that they are slow to learn even when such datasets are available. Moreover, they lack the ability to reason on an abstract level, which makes it difficult to implement high-level cognitive functions such as transfer learning, analogical reasoning, and hypothesis-based reasoning. Finally, their operation is largely opaque to humans, rendering them unsuitable for domains in which verifiability is important. In this paper, we propose an end-to-end reinforcement learning architecture comprising a neural back end and a symbolic front end with the potential to overcome each of these shortcomings. As proof-of-concept, we present a preliminary implementation of the architecture and apply it to several variants of a simple video game.

Statistical Mechanics of Deep Learning

Parsing, tokenizing, spelling correction, part-of-speech tagging, noun and verb phrase chunking are all aspects of natural language processing long handled by symbolic ai, but since improved by deep learning approaches. In symbolic AI, discourse representation theory and first-order logic have been used to represent sentence meanings. Latent semantic analysis (LSA) and explicit semantic analysis also provided vector representations of documents. In the latter case, vector components are interpretable as concepts named by Wikipedia articles. For other AI programming languages see this list of programming languages for artificial intelligence.