Thesis 2: An Entire Framework: OpenSherlock
In which we describe a migration being studied which substitutes ElasticSearch for Solr


  • Separate instances of ElasticSearch
    • Topic Map
      • Serves as a symbol library for all topics
    • IDocument Collection
      • IWordGram Collection
        • Serves as a neural-like network of all harvested text resources
    • Conceptual Graph
      • Serves as a knowledge model to tie together all topics
    • TSC
      • Qualitative process models derived from harvesting
  • Anticipatory Behavior

Big Picture Diagram


Topic Map

  • Well-merged Graph
  • Created through
    • Ontology import
    • Machine reading
    • Social interaction
      • Knowledge Gardening

IDocument Collection

  • An IDocument is a container which shepherds all harvesting activities related to
    • Any document harvested through machine reading
    • Any Ontology Class which has descriptions
    • Any Topic Node which has descriptions

IWordGram Collection

  • Their place in the hierarchy of concepts
    • IDocument representation of harvested document
      • IParagraph collections
        • ISentence Collections
          • SentenceScanner
            • Uniform text processor
          • IWordGram collections
            • All IWordGram collections are pooled across all harvested documents
              • No duplicate objects: one serves all instances
            • ITuple Collections
              • The final workhorses used by
                • Topic Map
                • Conceptual Graphs
                • TSC


  • Qualitative Process Theory [Ken Forbus]
    • Actors
    • Relations
    • States
    • Process Rules
    • Episodes
      • Process Models
  • Created while harvesting
  • Created and updated in background tasks
  • Used in a case-based-reasoning way while harvesting
    • Provides anticipatory clues

Conceptual Graph

  • Components [John Sowa]
    • Concept Lattice
    • Predicate Lattice
    • Graphs
      • Knowledge Graphs
        • Harvested
      • Canonical Graphs
        • Mapped
          • Category Theory?

Anticipatory Behavior

  • Means by which harvesting occurs
    • What is already known helps form expectations of what is being harvested next
      • Qualitative models form expectations
      • Canonical graphs are anticipatory schemas for knowledge modeling
      • Topic Map helps keep named entities and relations well organized

First-Order Workflow

  • Pretrain
    • Import Ontologies, etc
  • Read text
    • New IDocument per document read
    • Scan for paragraphs (IParagraph)
    • For each paragraph
      • Scan for sentences (ISentence)
        • For each sentence
          • Scan for WordGrams (IWordGram)
          • Scan for Tuples (ITuple)
        • Sentences without Tuples are un-resolved sentences
  • Second-Order Workflow

Second-Order Workflow

  • Model Building
    • Conceptual Graphs
    • Process Models
    • Probability Models
  • Use Models to
    • Help solve un-resolved sentences
    • Help answer questions
    • Support Third-Order Workflow
  • Generate research tasks to further solve un-resolved sentences
    • Web search
    • User interaction
  • Third-Order Workflow

Third-Order Workflow

  • Generalized background tasks
    • Wide-ranging ​topic map merge ​studies
    • Wide-ranging ​contradiction​ studies
    • Wide-ranging ​un-connected dot​ studies
    • Wide-ranging ​link validation​ studies
OpenSherlock Project »OpenSherlock Project
Architecture Ideas »Architecture Ideas
Thesis 2: An Entire Framework: OpenSherlock
+Comments (0)
+Citations (0)