SemSpect Managed Mode #

RDF SemSpect can be started without any predefined databases. We provide a REST API to create/delete databases and replace their content while the server is running (hotswap). The UI not only monitors hotswaps but is also aware of the creation and deletion of new databases.

REST API Routes #

Database Status #

All databases #

GET /api/database

Returns:

200 OK
{
  databases: [
     ...
     (list of single database stati, see below)
     ...
   ]
}

Single database #

GET /api/database/{DATABASE_ID}

Returns:

200 OK
{
   id: "{DATABASE_ID}",
   dataVersion: "{LAST_TRANSACTION_ID}",
   index: {
      status: "ready|offline",
      timestamp: "{TIMESTAMP_FROM_CREATE_OR_LAST_SUCCESSFUL_PUT_OPERATION}",
      description: "{DESCRIPTION_FROM_LAST_SUCCESSFUL_PUT_OPERATION}",
      indexDirectory: "{INDEX_PATH_ON_SERVER}"
   },
   operations: [ # last operations (max 2)
     {
       operationId: "{OPERATION_ID}",
       type: "CREATE_DATABASE|DELETE_DATABASE|IMPORT_CONTENT",
       status: "starting|ongoing|done|failed",
       timestamp: "{TIMESTAMP_FROM_SUBMISSION}",
       description: "{DESCRIPTION_FROM_LAST_SUCCESSFUL_PUT_OPERATION}",
       error: "{EXCEPTION_MESSAGE}" # only on error status
     },
     ...
   ]
}

Create Database #

POST /api/database/{DATABASE_ID}

Returns:

200 OK
{
   id: "{DATABASE_ID}",
   dataVersion: "0",
   index: {
      status: "ready",
      timestamp: "{TIMESTAMP_FROM_CREATE_OPERATION}",
      description: "",
      indexDirectory: "{INDEX_PATH_ON_SERVER}"
   },
   operations: [ 
     {
       operationId: "{OPERATION_ID}",
       type: "CREATE_DATABASE",
       status: "done",
       timestamp: "{TIMESTAMP_FROM_SUBMISSION}",
       description: "",
     },
     ...
   ]
}

400 BAD REQUEST
{
  errors: {
    error: "bad_request",
    message: "The database name {DATABASE_ID} contains illegal characters: \"#?\". The only non alphanumeric characters allowed are \"-\", \".\", \"-\", \"~\""
  }
}

409 CONFLICT
{
  error: "conflict",
  databaseId: {DATABASE_ID}
}

423 Locked
{
  errors: {
    error: "database_operation_in_progress",
    operationId: "{OPERATION_ID}"
  }
}

Delete Database #

DELETE /api/database/{DATABASE_ID}

Returns:

204 NO CONTENT

404 NOT FOUND
{
  error: "not_found",
  databaseId: "{DATABASE_ID}"
}

Upload Data #

Remark: Sources can be paths of files on the SemSpect server or (most of the time) URLs.

PUT /api/database/{DATABASE_ID}/content

Payload: 
{
  dataSources: ["{SOURCE_1_PATH_OR_URL}", "{SOURCE_2_PATH_OR_URL}"...],
  description: "{DESCRIPTION_TEXT}"
}

Returns:

200 OK
{
  operationId:"{OPERATION_ID}",
  type: "IMPORT_CONTENT",  
  status: "starting",
  timestamp: "{TIMESTAMP_FROM_SUBMISSION}",
  description: "{DESCRIPTION_TEXT}"
}

404 NOT FOUND
{
  error: "not_found",
  databaseId: "{DATABASE_ID}"
}

423 Locked
{
  errors: {
    error: "database_operation_in_progress",
    operationId: "{OPERATION_ID}"
  }
}

Operations #

The operations run asynchronously and are only accessible over the databases status routes. They have the following states:

• starting: the operation is currently being initialized (this status should only be very briefly visible)
• ongoing: the operation is currently in progress
• done: the operation completed successfully
• failed: processing was aborted due to an error. See error field.

HOWTO #

Start the SemSpect server with the semspect-spring script:

./semspect-spring.sh

Check that the status route is reachable:

% curl http://localhost:8080/api/database | jq

{
  "databases": []
}

You can already open the UI under http://localhost:8080/

Create a database (note the single quotes):

curl -X POST 'http://localhost:8080/api/database/my_database' | jq 

{
  "id": "my_database",
  "dataVersion": "0",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:37:25.294+02:00",
    "description": "",
    "indexDirectory": "/path/to/semspect/directory/managed-indices/my_database/index-0"
  },
  "operations": [
    {
      "operationId": "my_database-e2ccb4f4-cd97-4feb-a57b-c80941fa07b4",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:37:25.294+02:00",
      "description": ""
    }
  ]
}

Upload data from a local file or URL:

% curl -X PUT 'http://localhost:8080/api/database/my_database/content' -H 'Content-Type: application/json'  --data-binary '{"dataSources": ["/path/to/local/file/test.ttl"], "description" : "test dataset"}'

{
  "operationId": "my_database-c5c2c094-a0f0-443a-8011-4f2f1dca16ab",
  "type": "IMPORT_CONTENT",
  "status": "starting",
  "timestamp": "2024-06-06T08:41:23.369+02:00",
  "description": "test dataset"
}

The UI checks every 30 seconds for hotswaps. You can monitor the progress in the server console or call the database status route:

• Indexing ongoing example:

% curl http://localhost:8080/api/database/my_database

{
  "id": "my_database",
  "dataVersion": "0",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:42:56.539+02:00",
    "description": "",
    "indexDirectory": "/path/to/semspect/directory/managed-indices/my_database/index-0"
  },
  "operations": [
    {
      "operationId": "my_database-3cb03946-6e71-4c96-8068-d9d712fc6450",
      "type": "IMPORT_CONTENT",
      "status": "ongoing",
      "timestamp": "2024-06-06T08:44:20.193+02:00",
      "description": "test dataset"
    },
    {
      "operationId": "my_database-149f8e51-a2ef-452d-8ff1-2d9abc967878",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:42:56.539+02:00",
      "description": ""
    }
  ]
}

• Indexing done example (note change of index directory after to hotswap):

% curl http://localhost:8080/api/database/my_database

{
  "id": "my_database",
  "dataVersion": "4294967296",
  "index": {
    "status": "ready",
    "timestamp": "2024-06-06T08:44:20.193+02:00",
    "description": "test dataset",
    "indexDirectory": "/path/to/semspect/directory/managed-indices/my_database/index-1"
  },
  "operations": [
    {
      "operationId": "my_database-3cb03946-6e71-4c96-8068-d9d712fc6450",
      "type": "IMPORT_CONTENT",
      "status": "done",
      "timestamp": "2024-06-06T08:44:20.193+02:00",
      "description": "test dataset"
    },
    {
      "operationId": "my_database-149f8e51-a2ef-452d-8ff1-2d9abc967878",
      "type": "CREATE_DATABASE",
      "status": "done",
      "timestamp": "2024-06-06T08:42:56.539+02:00",
      "description": ""
    }
  ]
}

Environment Variables #

The Spring server settings can be set through environment variables. Example:

SERVER_PORT=8081 
SERVER_SERVLET_CONTEXT_PATH="/semspect" 
SERVER_FRAME_ANCESTORS="https://www.example.org"  # allows multiple comma separated values