Settings API Reference¶

The Heritage Data Processor Settings API provides endpoints for managing application configuration, including retrieving, updating, and persisting configuration changes to YAML files without server restart.

Base URL¶

All endpoints are prefixed with /config.

Configuration Management¶

Get Application Configuration¶

Retrieves the currently loaded application configuration and the absolute path to the configuration directory.

Endpoint: GET /config/get

Response:

{
  "configData": {
    "paths": {
      "hdpc_schema": "hdpc_schema.yaml",
      "prompts_file": "prompts.yaml",
      "metadata_mapping_file": "metadata_mapping.yaml"
    },
    "zenodo": {
      "sandbox_base_url": "https://sandbox.zenodo.org/api",
      "production_base_url": "https://zenodo.org/api"
    },
    "server": {
      "port": 5001,
      "debug": false
    },
    "file_validation": {
      "max_file_size_mb": 500,
      "allowed_extensions": [".xml", ".json", ".obj", ".mtl"]
    }
  },
  "configDirAbsPath": "/home/user/hdp/config"
}

Response Fields:

configData (object): Complete application configuration loaded from config.yaml
configDirAbsPath (string): Absolute path to the directory containing the configuration file

Configuration Structure:

The configuration data structure depends on the application's config.yaml file and typically includes:

paths: File paths for schemas, mappings, and other configuration files
zenodo: Zenodo API endpoints and settings
server: Application server settings
file_validation: File validation rules
Additional application-specific settings

Status Codes:

200 OK: Configuration retrieved successfully
500 Internal Server Error: Configuration not loaded or empty, or error processing configuration

Error Response:

{
  "error": "Configuration not loaded or is empty. Check server logs."
}

Save Application Configuration¶

Saves updated configuration to the config.yaml file with automatic backup creation and live reload.

Endpoint: POST /config/save

Request Body:

{
  "paths": {
    "hdpc_schema": "hdpc_schema.yaml",
    "prompts_file": "prompts.yaml",
    "metadata_mapping_file": "metadata_mapping.yaml"
  },
  "zenodo": {
    "sandbox_base_url": "https://sandbox.zenodo.org/api",
    "production_base_url": "https://zenodo.org/api"
  },
  "server": {
    "port": 5001,
    "debug": true
  }
}

Request Parameters:

Complete configuration object (any valid YAML-serializable structure)

Response:

{
  "message": "Configuration saved successfully."
}

Status Codes:

200 OK: Configuration saved and reloaded successfully
400 Bad Request: No configuration data received
500 Internal Server Error: Configuration file path not set or save operation failed

Save Process:

The endpoint performs the following operations:

Validation: Verifies configuration data is present in request
Backup Creation: Renames existing config.yaml to config.yaml.bak
File Write: Writes new configuration to config.yaml using YAML format with 2-space indentation
Live Reload: Calls load_configuration() to reload settings into the running application without restart
Logging: Logs successful save and reload

Automatic Rollback:

If any error occurs during the save process, the endpoint automatically restores the backup file by renaming config.yaml.bak back to config.yaml.

YAML Formatting:

The configuration is saved with the following YAML options:

default_flow_style=False: Uses block style (not inline)
sort_keys=False: Preserves original key order
indent=2: Uses 2-space indentation

Prompts Management¶

Get Prompt Keys¶

Retrieves available prompt keys from the prompts.yaml file, supporting both top-level and nested key retrieval.

Endpoint: GET /config/get_prompt_keys

Query Parameters:

prompt_id (string, optional): Specific prompt identifier to retrieve sub-keys for

Response (No prompt_id):

[
  "metadata_extraction",
  "description_generation",
  "keyword_suggestion",
  "title_refinement"
]

Response (With prompt_id):

GET /config/get_prompt_keys?prompt_id=metadata_extraction

[
  "system_prompt",
  "user_template",
  "temperature",
  "max_tokens"
]

Behavior:

Without prompt_id: Returns all top-level keys from prompts.yaml, excluding the special settings key

With prompt_id: Returns sub-keys for the specified prompt ID if it exists and is a dictionary. Returns empty array if the prompt ID is not found or is not a dictionary

Status Codes:

200 OK: Prompt keys retrieved successfully
404 Not Found: Prompts file not found at specified path
500 Internal Server Error: Prompts file path not configured, file is not valid YAML dictionary, or unexpected error

File Location:

The prompts file path is read from config.yaml at paths.prompts_file. The path is resolved relative to the configuration directory.

Path Resolution¶

Configuration Directory¶

The API uses Flask's application context to store the configuration file path:

config_file_path = current_app.config.get("CONFIG_FILE_PATH")

All relative paths in the configuration (such as paths.prompts_file) are resolved relative to the parent directory of config.yaml.

Example:

If config.yaml is at /home/user/hdp/config/config.yaml and paths.prompts_file is set to prompts.yaml, the resolved path will be /home/user/hdp/config/prompts.yaml.

Error Handling¶

Common Error Scenarios¶

Configuration Not Loaded:

GET /config/get

Response: 500 Internal Server Error

{
  "error": "Configuration not loaded or is empty. Check server logs."
}

No Configuration Data Provided:

POST /config/save
Content-Type: application/json

{}

Response: 400 Bad Request

{
  "error": "No configuration data received."
}

Configuration File Path Not Set:

If the CONFIG_FILE_PATH is not set in the Flask application context:

Response: 500 Internal Server Error

{
  "error": "Configuration file path is not set."
}

Prompts File Not Found:

GET /config/get_prompt_keys

Response: 404 Not Found

{
  "error": "Prompts file not found at /path/to/prompts.yaml"
}

Prompts File Path Not Configured:

If paths.prompts_file is missing from config.yaml:

Response: 500 Internal Server Error

{
  "error": "Path to prompts.yaml ('paths.prompts_file') not defined in config."
}

Invalid YAML Structure:

If the prompts file is not a valid YAML dictionary:

Response: 500 Internal Server Error

{
  "error": "Prompts file is not a valid YAML dictionary."
}

Backup Management¶

Automatic Backup Creation¶

When saving configuration, the endpoint automatically creates a backup with the .bak extension:

Backup Path: {config_file_path}.bak

Example: If config.yaml is at /config/config.yaml, the backup will be /config/config.yaml.bak

Backup Behavior:

Only the most recent backup is retained (overwrites previous backups)
Backup is created before writing the new configuration
If save fails, the backup is automatically restored

Manual Backup Restoration¶

Administrators can manually restore the backup by renaming config.yaml.bak to config.yaml and restarting the application.

Live Reload¶

Configuration Reload¶

After successfully saving the configuration file, the endpoint reloads it into the running application:

load_configuration(current_app, config_file_path)

Benefits:

No server restart required for configuration changes
Immediate effect of configuration updates
Seamless user experience during configuration management

Affected Components:

The reload affects all application components that read from current_app.config.get("LOADED_CONFIG").

Usage Examples¶

Example 1: Retrieve Current Configuration¶

GET /config/get

Response:

{
  "configData": {
    "paths": {
      "hdpc_schema": "hdpc_schema.yaml"
    },
    "server": {
      "port": 5001
    }
  },
  "configDirAbsPath": "/home/user/hdp/config"
}

Example 2: Update Server Port¶

POST /config/save
Content-Type: application/json

{
  "paths": {
    "hdpc_schema": "hdpc_schema.yaml",
    "prompts_file": "prompts.yaml"
  },
  "server": {
    "port": 8080,
    "debug": true
  },
  "zenodo": {
    "sandbox_base_url": "https://sandbox.zenodo.org/api"
  }
}

Response:

{
  "message": "Configuration saved successfully."
}

Example 3: List All Available Prompts¶

GET /config/get_prompt_keys

Response:

[
  "metadata_extraction",
  "description_generation",
  "keyword_suggestion"
]

Example 4: Get Prompt Sub-Keys¶

GET /config/get_prompt_keys?prompt_id=metadata_extraction

Response:

[
  "system_prompt",
  "user_template",
  "temperature"
]

Example 5: Query Non-Existent Prompt¶

GET /config/get_prompt_keys?prompt_id=nonexistent_prompt

Response:

[]

Logging¶

Application Logger¶

The module uses Python's standard logging with logger name settings_bp:

Info Level:

"Configuration saved and reloaded successfully."

Error Level:

"Error in /api/config/get: {exception}"
"Error in /api/config/save: {exception}"
"Error fetching prompt keys: {exception}"

All error logs include full stack traces with exc_info=True.

File Operations¶

YAML Processing¶

The API uses PyYAML for reading and writing YAML files:

Reading: yaml.safe_load() for security (prevents arbitrary code execution)

Writing: yaml.dump() with custom formatting options

Encoding: UTF-8 encoding for all file operations

File System Operations¶

Backup Creation: Uses os.rename() for atomic file operations

Path Resolution: Uses pathlib.Path for cross-platform path handling

Existence Check: Uses path.exists() before file operations

Security Considerations¶

Safe YAML Loading¶

The prompts endpoint uses yaml.safe_load() instead of yaml.load() to prevent arbitrary code execution from malicious YAML files.

Configuration Validation¶

The save endpoint does not perform schema validation on the configuration structure. Administrators should ensure configuration changes are valid before saving.

Backup Protection¶

Only one backup level is maintained. Consider implementing version control or external backup mechanisms for critical configuration files.

Integration with Application¶

Flask Application Context¶

All endpoints rely on Flask's application context for accessing configuration:

current_app.config.get("LOADED_CONFIG"): Current configuration data
current_app.config.get("CONFIG_FILE_PATH"): Path to config file

Configuration Loader¶

The save endpoint calls the load_configuration() function from the config module to reload settings:

from ..config import load_configuration

load_configuration(current_app, config_file_path)

This ensures all application components receive the updated configuration.

Data Types & Formats¶

Configuration Data¶

Configuration data can be any valid YAML structure, typically nested dictionaries with strings, numbers, booleans, arrays, and nested objects.

Prompt Keys¶

Prompt keys are returned as JSON arrays of strings:

["key1", "key2", "key3"]

Empty array is returned when no keys are found or the prompt ID does not exist.