Managed Mode - REST API

SemSpect Managed Mode #

RDF SemSpect can be started without any predefined databases. We provide a REST API to create/delete databases and replace their content while the server is running (hotswap). The UI not only monitors hotswaps but is also aware of the creation and deletion of new databases.

REST API Routes #

Create Database #

POST /api/database/{DATABASE_ID}
{
  "metadata": {
    "databaseCaption": "DB caption",
    "contentDescription": "",
    "exampleKey1": "test",
    "exampleKey2": 42
  },
  "config": {
    "indexing": {
      "disableDomainRangeEntailment": true,
      "rdfDataSources": [
          "file:///..." 
      ]
    },
    "exploration": {
      "showTopClassInTree": false,
      "showClassesAndPropertiesAsResources": false
    },
    "presentation": {
      "locales": ["en", "de", "fr"]
    }
  }
}

Notes #

  • Metadata

    • databaseCaption and contentDescription are special fields with dedicated meaning:
      • databaseCaption: a human-readable label for the database
      • contentDescription: a free-form description of the database contents
    • All other fields in metadata are arbitrary key–value pairs. They are stored as-is and can be used for custom metadata (not displayed in UI)

  • Config

    • The sections indexing, exploration, and presentation are optional
    • They use the same fields and semantics as in the standard YAML configuration
    • If omitted, the system will apply default values

  • Request body

    • The body itself is optional. If omitted, SemSpect will create the database with default configuration and no metadata.

Returns #

200 OK
{
   "id": "{DATABASE_ID}",
   "dataVersion": "0",
   "index": {
      "status": "ready",
      "timestamp": "{TIMESTAMP_FROM_CREATE_OPERATION}",
      "contentDescription": ""
   },
   "operations": [ 
     {
       "operationId": "{OPERATION_ID}",
       "type": "CREATE_DATABASE",
       "status": "done",
       "timestamp": "{TIMESTAMP_FROM_SUBMISSION}",
       "description": ""
     },
     ...
   ]
}

400 BAD REQUEST
{
  "errors": {
    "error": "bad_request",
    "message": "The database name {DATABASE_ID} contains illegal characters: \"#?\". The only non alphanumeric characters allowed are \"-\", \".\", \"-\", \"~\""
  }
}

409 CONFLICT
{
  "error": "conflict",
  "databaseId": {DATABASE_ID}
}

423 Locked
{
  "errors": {
    "error": "database_operation_in_progress",
    "operationId": "{OPERATION_ID}"
  }
}

Database Status #

All databases #

GET /api/database

Returns:

200 OK
{
  "databases": [
     ...
     (list of single database stati, see below)
     ...
   ]
}

Single database #

GET /api/database/{DATABASE_ID}

Returns:

200 OK
{
   "id": "{DATABASE_ID}",
   "dataVersion": "{LAST_TRANSACTION_ID}",
   "databaseCaption": "{HUMAN_READABLE_LABEL}",
   "metadata": {
      "exampleKey1": "test",
      "exampleKey2": 42
   },
   "index": {
      "status": "ready|offline",
      "timestamp": "{TIMESTAMP_FROM_CREATE_OR_LAST_SUCCESSFUL_PUT_OPERATION}",
      "contentDescription": "{DESCRIPTION_FROM_LAST_SUCCESSFUL_PUT_OPERATION}"
   },
   "operations": [ # last operations
     {
       "operationId": "{OPERATION_ID}",
       "type": "CREATE_DATABASE|DELETE_DATABASE|IMPORT_CONTENT",
       "status": "starting|ongoing|done|failed",
       "timestamp": "{TIMESTAMP_FROM_SUBMISSION}",
       "description": "{DESCRIPTION_FROM_LAST_SUCCESSFUL_PUT_OPERATION}",
       "error": "{EXCEPTION_MESSAGE}" # only on error status
     },
     ...
   ]
}

Delete Database #

DELETE /api/database/{DATABASE_ID}

Returns: #

204 NO CONTENT

404 NOT FOUND
{
  "error": "not_found",
  "databaseId": "{DATABASE_ID}"
}

Upload Data #

Remark: Sources can be paths of files on the SemSpect server or (most of the time) URLs.

PUT /api/database/{DATABASE_ID}/content

Payload: 
{
  "dataSources": ["{SOURCE_1_PATH_OR_URL}", "{SOURCE_2_PATH_OR_URL}"...],
  "contentDescription": "{DESCRIPTION_TEXT}"
}

Returns: #

200 OK
{
  "operationId": "{OPERATION_ID}",
  "type": "IMPORT_CONTENT",  
  "status": "starting",
  "timestamp": "{TIMESTAMP_FROM_SUBMISSION}",
  "description": "{DESCRIPTION_TEXT}"
}

404 NOT FOUND
{
  "error": "not_found",
  "databaseId": "{DATABASE_ID}"
}

423 Locked
{
  "errors": {
    "error": "database_operation_in_progress",
    "operationId": "{OPERATION_ID}"
  }
}

Operations #

The operations run asynchronously and are only accessible over the databases status routes. They have the following states:

  • starting: the operation is currently being initialized (this status should only be very briefly visible)
  • ongoing: the operation is currently in progress
  • done: the operation completed successfully
  • failed: processing was aborted due to an error. See error field.

Operation Persistence #

Operations are persisted to disk across database sessions in server-indices/operations.yaml by a dedicated background thread.

  • Completion requirement: Only operations that reach a terminal state (done or failed) are persisted. If the server is killed during an ongoing operation (e.g., during a hotswap), the operation is discarded to prevent confusing ongoing states upon restart.
  • Retention limit: A maximum of 10 operations are kept per database (both in-memory and on disk). Older operations are automatically trimmed.
  • Deletion: When a database is deleted, all of its associated operations are also permanently removed.

HOWTO #

Start the SemSpect server with the semspect-server script:

% ./semspect-server.sh

Check that the status route is reachable:

% curl http://localhost:8080/api/database | jq

{
  "databases": []
}

You can already open the UI under http://localhost:8080/

Create a database (note the single quotes):

% curl -X POST 'http://localhost:8080/api/database/my_database' | jq 

{
  "id": "my_database",
  "dataVersion": "0",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:37:25.294+02:00",
    "contentDescription": ""
  },
  "operations": [
    {
      "operationId": "my_database-e2ccb4f4-cd97-4feb-a57b-c80941fa07b4",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:37:25.294+02:00",
      "description": ""
    }
  ]
}

Upload data from a local file or URL:

% curl -X PUT 'http://localhost:8080/api/database/my_database/content' -H 'Content-Type: application/json'  --data-binary '{"dataSources": ["/path/to/local/file/test.ttl"], "contentDescription" : "test dataset"}'

{
  "operationId": "my_database-c5c2c094-a0f0-443a-8011-4f2f1dca16ab",
  "type": "IMPORT_CONTENT",
  "status": "starting",
  "timestamp": "2024-06-06T08:41:23.369+02:00",
  "description": "test dataset"
}

The UI checks every 30 seconds for hotswaps. You can monitor the progress in the server console or call the database status route:

  • Indexing ongoing example:
% curl http://localhost:8080/api/database/my_database

{
  "id": "my_database",
  "dataVersion": "0",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:42:56.539+02:00",
    "contentDescription": ""
  },
  "operations": [
    {
      "operationId": "my_database-3cb03946-6e71-4c96-8068-d9d712fc6450",
      "type": "IMPORT_CONTENT",
      "status": "ongoing",
      "timestamp": "2024-06-06T08:44:20.193+02:00",
      "description": "test dataset"
    },
    {
      "operationId": "my_database-149f8e51-a2ef-452d-8ff1-2d9abc967878",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:42:56.539+02:00",
      "description": ""
    }
  ]
}
  • Indexing done example (note change of index directory after to hotswap):
% curl http://localhost:8080/api/database/my_database

{
  "id": "my_database",
  "dataVersion": "4294967296",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:44:20.193+02:00",
    "contentDescription": "test dataset"
  },
  "operations": [
    {
      "operationId": "my_database-3cb03946-6e71-4c96-8068-d9d712fc6450",
      "type": "IMPORT_CONTENT",
      "status": "done",
      "timestamp": "2024-06-06T08:44:20.193+02:00",
      "description": "test dataset"
    },
    {
      "operationId": "my_database-149f8e51-a2ef-452d-8ff1-2d9abc967878",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:42:56.539+02:00",
      "description": ""
    }
  ]
}

Environment Variables #

The Spring server settings can be set through environment variables. Example:

SERVER_PORT=8081 
SERVER_SERVLET_CONTEXT_PATH="/semspect" 
SERVER_FRAME_ANCESTORS="https://www.example.org"  # allows multiple comma separated values