Ontotext Case: Automated Detection of Anomalous Industry Classification in Linked Data. @ Global Datathon 2018

Andrey Tagarev and Nikola Tulechki - Ontotext

2018-09-20

1 FactForge: Data Integration

  • DBpedia (the English version) 496M
  • Geonames (all geographic features on Earth) 150M
  • owl:sameAs links between DBpedia and Geonames 471K
  • Company registry data (GLEI) 3M
  • Panama Papers DB (#LinkedLeaks) 20M
  • Other datasets and ontologies: WordNet, WorldFacts, FIBO
  • News metadata (2000 articles/day enriched by NOW) 473M
  • Total size (1611M explicit + 328M inferred statements) 1939М

2 FactForge: News Metadata

  • Metadata from Ontotext’s Dynamic Semantic Publishing platform
    • News stream from Google
    • Automatically generated as part of the NOW.ontotext.com semantic news showcase
  • News stream from Google since Feb 2015, about 50k news/month
    • Over 1M news articles at present
    • ~70 tags (annotations) per news article
  • Tags link text mentions of concepts to the knowledge graph
    • Technically these are URIs for entities (people, organizations, locations, etc.) and key phrases

3 FactForge: Classes

4 FactForge: Organisations

5 Bottom-up Industry Classification

  • Leverage wikipedia manual classification
  • Normalize and organize
    • Merge and order to produce a hierarchy
    • 17361 industry tags with no clear connection
    • 9560 literals and 7801 IRIs
  • The result
    • 32 top-level industries
    • 240 industries total organized in multiple levels
    • tag clusters give richness of expression

6 The Problem

… as always- messy data.

  • Companies are sometimes incorrectly classified (Type I error)
  • Some companies are not classified in all appropriate industries (Type II error)

7 The Task

Describe and evaluate a technique for identifying both types of errors in the data.

8 The Data

9 The Data II

  • Simple csv dump
    • Contains all organisations
    • Includes several useful direct features
      • name
      • description
      • wikipedia categories
      • location
      • industries

10 The Data III

  • Deep graph features
    • No dump as there are too many possibilities
    • Provided are sample SPARQL queries to extract some promising features for each organisation
      • direct links to wikipedia articles
      • direct links from wikipedia articles
      • indirect links through news mentions
      • mapping from industries to lists of keywords
      • rich geographical information (through geonames)
      • parent/subsidiary relations
    • Mentors can provide assistance for modifying the queries or creating new ones for novel features

11 The Data IV

http://factforge.net/sparql