WASDI Configuration Documentation
This document describes the structure and configuration options for the
wasdiConfig.json file, which contains all WASDI system configuration
settings.
Overview
The main WASDI configuration is stored in a single JSON file that
represents the WasdiConfig Java object and all its child
configuration objects. This configuration is used by all WASDI
components including:
Launcher
Trigger
Scheduler
Web Server
Database Utils
Main
These are the configuration properties at root level of the JSON file.
### Root Level Properties
{
"nodeCode": "wasdi",
"printServerAddress": "",
"mainNodeCloud": "CREODIAS",
"usersDefaultNode": "wasdi",
"defaultSkin": "wasdi",
"activateSubscriptionChecks": true,
"systemUserName": "appwasdi",
"systemUserId": 2042,
"systemGroupName": "appwasdi",
"systemGroupId": 2042,
"baseUrl": "https://www.wasdi.net/wasdiwebserver/rest/",
"connectionTimeout": 10000,
"readTimeout": 10000,
"msWaitAfterChmod": 1000,
"shellExecLocally": true,
"useLog4J": true,
"logHttpCalls": true,
"logLevel": "INFO",
"logLevelServer": "",
"logLevelLauncher": "",
"logLevelScheduler": "",
"logLevelTrigger": "",
"addDateTimeToLogs": false,
"filterInternalHttpCalls": true,
"useNotebooksDockerAddress": false,
"nvidiaGPUAvailable": false
}
Root Properties Description
Property |
Type |
Description |
|---|---|---|
|
String |
Unique identifier for this WASDI node (default: “wasdi”) |
|
String |
URL of the WASDI print server |
|
String |
Cloud provider of the main node (e.g., “CREODIAS”). Cloud Providers are just strings. They are used here in the main section, and in the dataProviders Config. This cloud name is used to understand if the node is in the same cloud of the dataProvider or not, and this information is useful to select the best data fetching option. |
|
String |
Default node assigned to new users |
|
String |
Default UI skin to apply (default: “wasdi”) |
|
Boolean |
Flag to activate or not the check and management of subscriptions |
|
String |
System user name for WASDI processes (default: “appwasdi”) |
|
Integer |
System user ID (default: 2042) |
|
String |
System group name (default: “appwasdi”) |
|
Integer |
System group ID (default: 2042) |
|
String |
Base URL of WASDI API |
|
Integer |
Connection timeout for third-party API calls (ms) |
|
Integer |
Read timeout for third-party API calls (ms) |
|
Integer |
Wait time after chmod commands (ms) |
|
Boolean |
Execute external components
locally vs using Docker
commands. WASDI uses
different command line tools
(e.g., gdalinfo, gdal_warp)
and Python scripts for
utilities and data providers.
When |
|
Boolean |
Use Log4J configuration vs standard output |
|
Boolean |
Enable HTTP call logging |
|
String |
General log level |
|
String |
Web server specific log level |
|
String |
Launcher specific log level |
|
String |
Scheduler specific log level |
|
String |
Trigger specific log level |
|
Boolean |
Add timestamp to log lines |
|
Boolean |
Filter internal HTTP calls from HTTP logs |
|
Boolean |
Use Docker internal names for Jupyter notebooks |
|
Boolean |
Enable NVIDIA GPU support for containers: set it true if this node have a GPU |
Database Configuration
MongoDB Configurations
WASDI supports multiple MongoDB instances for different purposes:
{
"mongoMain": {
"address": "localhost",
"port": 27017,
"dbName": "wasdi",
"user": "username",
"password": "password",
"replicaName": ""
},
"mongoLocal": {
"address": "localhost",
"port": 27017,
"dbName": "wasdilocal",
"user": "username",
"password": "password",
"replicaName": ""
}
}
The most important is the mongoMain section: this must target the main Mongo Db of the main node. All the computational nodes MUST be able to reach this database: usually this is done using an SSH tunnel.
The main node MUST have only mongoMain.
Any computational node MUST have mongoLocal: this is a small subset of the main database with only the collections that are used locally by the computational node.
MongoDB Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
MongoDB server address |
|
Integer |
MongoDB server port |
|
String |
Database name |
|
String |
Database username |
|
String |
Database password |
|
String |
Replica set name (if applicable) |
Authentication Configuration
Settings about the authentication system.
Keycloak Configuration
Keycloak is used as authentication server for WASDI.
{
"keycloack": {
"address": "https://auth.wasdi.net",
"cliSecret": "cli-secret",
"authTokenAddress": "https://auth.wasdi.net/auth/realms/wasdi/protocol/openid-connect/token",
"introspectAddress": "https://auth.wasdi.net/auth/realms/wasdi/protocol/openid-connect/token/introspect",
"confidentialClient": "wasdi-confidential",
"client": "wasdi",
"clientSecret": "client-secret",
"realm": "wasdi",
"sessionExpireHours": 24
}
}
Keycloak Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Keycloak server address |
|
String |
CLI client secret |
|
String |
Token endpoint URL |
|
String |
Token introspection endpoint URL |
|
String |
Confidential client name |
|
String |
Public client name |
|
String |
Client secret |
|
String |
Keycloak realm name |
|
Integer |
Session expiration time in hours |
File System Paths Configuration
In this section all the main paths needed by WASDI are configured.
{
"paths": {
"downloadRootPath": "/data/wasdi/",
"serializationPath": "/data/wasdi/params/",
"metadataPath": "/data/wasdi/metadata/",
"dockerTemplatePath": "/data/wasdi/dockertemplate/",
"sftpRootPath": "/data/wasdi/sftp/",
"geoserverDataDir": "/data/wasdi/geoserver/",
"sen2CorePath": "/usr/local/sen2cor/",
"userHomePath": "/home/appwasdi/",
"missionsConfigFilePath": "/opt/wasdi/config/missions.json",
"gdalPath": "/usr/bin/",
"wasdiTempFolder": "/tmp/wasdi/",
"pythonExecPath": "/usr/bin/python3",
"traefikMountedVolume": "/data/wasdi/traefik/",
"s3VolumesBasePath": "/mnt/wasdi/users-volumes/",
"wasdiConfigFilePath": "/opt/wasdi/config/wasdiConfig.json"
}
}
Paths Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Base root path containing subfolders (workspaces, metadata, styles, workflows, processors, images) |
|
String |
Path where parameters are serialized |
|
String |
Metadata files path |
|
String |
Docker templates path |
|
String |
Root of the local SFTP server |
|
String |
Geoserver data directory |
|
String |
Sen2Core binary path |
|
String |
User home path |
|
String |
Missions configuration file path |
|
String |
GDAL binary path |
|
String |
Temporary files folder |
|
String |
Full path to Python executable |
|
String |
Traefik mounted volume path |
|
String |
S3 volumes mount folder path |
|
String |
Path to this configuration file |
Docker Configuration
{
"dockers": {
"extraHosts": [],
"pipInstallWasdiAddress": "https://pypi.org/simple/",
"internalDockersBaseAddress": "localhost",
"numberOfAttemptsToPingTheServer": 4,
"millisBetweenAttmpts": 5000,
"millisBetweenStatusPolling": 1000,
"numberOfPollStatusPollingCycleForLog": 30,
"millisWaitAfterDelete": 15000,
"millisWaitAfterDeployScriptCreated": 2000,
"millisWaitForLogin": 4000,
"dockerComposeCommand": "docker-compose",
"internalDockerAPIAddress": "http://127.0.0.1:2375/",
"dockersDeployLogFilePath": "/var/log/wasdi/dockers.log",
"logDockerAPICallsPayload": false,
"removeDockersAfterShellExec": true,
"removeParameterFilesForPythonsShellExec": true,
"dockerNetworkMode": "net-wasdi",
"processorsInternalPort": 5000,
"groupAdd": [],
"registers": [],
"shellExecCommands": {},
"processorTypes": [],
"eoepca": {}
}
}
Docker Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
Array[String] |
Extra hosts to add to containers (may be needed in some clouds for network reasons) |
|
String |
Address to use to access PyPI to install waspy |
|
String |
Address to use to reach internal dockers (default: “localhost”) |
|
Integer |
Number of attempts to ping server before deciding it’s down (default: 4) |
|
Integer |
Time (ms) to wait between attempts to check if docker is started (default: 5000) |
|
Integer |
Time (ms) to wait between calls to docker engine API for status (default: 1000) |
|
Integer |
Number of cycles before waitContainer function logs activity (default: 30) |
|
Integer |
Time (ms) to wait for docker to complete delete operation (default: 15000) |
|
Integer |
Time (ms) to wait after deploy.sh file creation (default: 2000) |
|
Integer |
Time (ms) to wait for docker login operation (default: 4000) |
|
String |
Command to use to start docker compose (default: “docker-compose”) |
|
String |
Address of the local Docker instance API (default: “http://127.0.0.1:2375/”) |
|
String |
Path of file with docker build logs (default: “/var/log/wasdi/dockers.log”) |
|
Boolean |
Enable logging of payload for Docker Engine API calls (default: false) |
|
Boolean |
Remove containers after shell execute (true for production, false for debug) |
|
Boolean |
Remove input/output files after Python shell exec (true for production) |
|
String |
Docker network mode (default: “net-wasdi”, can be overridden by shell exec items) |
|
Integer |
Standard processors internal port (default: 5000) |
|
Array[String] |
List of group IDs to add to docker create command |
|
Array [DockerRegi- stryConfig] |
List of supported docker registries |
|
Map[String, ShellExec ItemConfig] |
Map local shell exec commands
to equivalent docker
commands. When
|
|
Array[Proces- sorTypeConfig] |
Configuration of processor types |
|
EOEPCAConfig |
Configuration of EoEpca related docker parameters |
ShellExecItemConfig Properties
Each ShellExecItemConfig object in the shellExecCommands map
contains the following properties:
Property |
Type |
Description |
|---|---|---|
|
String |
Name of the docker image to use instead of the command |
|
String |
Version of the docker image to use |
|
Boolean |
If true (default), all command line parts are passed as docker args. If false, the first element is not passed to docker command line |
|
Boolean |
If true, the command is executed locally even if WASDI is configured to be dockerized (default: false) |
|
Boolean |
If true, WASDI removes the path from the first command in the arg list |
|
String |
Prefix that will be added as arg[0] of the shell execute |
|
Array[String] |
List of additional mount points for this specific docker |
|
Boolean |
Enable override of system/docker config (user, group, network) with specific values for this docker |
|
String |
System user name for this docker (default: “appwasdi”) |
|
Integer |
System user ID for this docker (default: 2042) |
|
String |
System group name for this docker (default: “appwasdi”) |
|
Integer |
System group ID for this docker (default: 2042) |
|
String |
Docker network mode for this container (default: “net-wasdi”) |
ProcessorTypeConfig Properties
Each ProcessorTypeConfig object in the processorTypes array
contains the following properties:
Property |
Type |
Description |
|---|---|---|
|
String |
Name of the processor type |
|
Array[String] |
List of additional mount points for this specific docker |
|
Array[String] |
List of additional commands for this specific docker |
|
Array[Enviro- nmentVariable Config] |
List of environment variables to pass when creating the container |
|
String |
Name of the base image to use (if needed) |
|
String |
Version of the base image to use (if needed) |
|
Array[String] |
Personalized extra hosts for this processor type |
|
Boolean |
If true, mount only
|
|
Array[String] |
List of file names that must not be downloaded when zipping the processor |
Docker Registry Configuration
{
"registers": [
{
"address": "nexus.wasdi.net",
"user": "username",
"password": "password",
"priority": 1,
"isDefault": true
}
]
}
Docker Registry Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Unique identifier of the registry |
|
String |
Username for registry authentication |
|
String |
Password for the above user |
|
String |
HTTP address of the registry |
|
Integer |
Priority of the registry (lower numbers = higher priority) |
|
String |
HTTP address of the registry API |
|
String |
Name of the repository inside the registry (default: “wasdi-docker”) |
Messaging Configuration
Configurations related to messages.
RabbitMQ Configuration
{
"rabbit": {
"host": "localhost",
"user": "guest",
"password": "guest",
"port": 5672,
"queueName": "wasdi"
}
}
RabbitMQ Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
RabbitMQ username for authentication |
|
String |
RabbitMQ password for the above user |
|
String |
RabbitMQ server hostname or IP address |
|
Integer |
RabbitMQ server port (default: 5672) |
|
String |
RabbitMQ exchange name for message routing |
Notifications Configuration
{
"notifications": {
"mercuriusAPIAddress": "https://mercurius.cimafoundation.org/",
"pwRecoveryMailTitle": "WASDI Password Recovery",
"pwRecoveryMailSender": "noreply@wasdi.net",
"pwRecoveryMailText": "Password recovery email text",
"sftpMailTitle": "WASDI SFTP Account",
"sftpManagementMailSender": "noreply@wasdi.net",
"sftpMailText": "SFTP account notification text",
"wasdiAdminMail": "admin@wasdi.net"
}
}
Notifications Configuration Properties
Property |
Type |
Description |
|---|---|---|
`` mercuriusAPIAddress`` |
String |
Address of the Mercurius service (CIMA service API to send e-mails) |
|
String |
Title of the password recovery email |
|
String |
Sender address for password recovery emails |
|
String |
Text content of the password recovery email |
|
String |
Title of the SFTP account notification email |
|
String |
Sender address for SFTP account emails |
|
String |
Text content of the SFTP account notification email |
|
String |
Declared WASDI administrator email address |
Data Provider Configuration
Each Data Provider is a component able to query and fecth data from a generic external service that can offer one or more data collection.
The main business entities involved in this operation are:
Platform: this is the type of data. Usually identified as a Satellite Mission. Platorms are for example Sentinel1, Sentinel2, ENVISat etc. Each Platform, in general, can be found in one or more data providers.
Query Executors / catalogue: the Query executor is the WASDI hierarchy used to query the Data Provider Catalogue
Provider Adapters: Objects used by the launcher to download/import files from an external service.
Each Data Provider can support one or more platforms (and each platform can be supported by one or more Data Providers.)
All Data Providers have: .name .description .link .user .password
Different data providers uses also different parameters. This config has been re-written after that many data providers were already avaiable and is done to support also legacy objects.
Is due to control for Each Data Provider witch params are really needed.
At the moment of fetching some data, WASDI will try to use the “best” Data Provider: this is an algorithm based on the availability of the data providers and on the Cloud Environment declared by the Node and the data provider itself.
For example, a node in Creodias, will try to select the Creodias Data Provider since it assumes it will be faster beeing in the same cloud.
A node in LSA Data Center, will probably choose LSA Data Center.
In any case, in case of problems, WASDI will try to back-up using all the Data Providers available for that specific Platform.
Each Data Provider can also have two additional config files: these are not mandatory. Are meant to be generic JSON files, that may be needed by any specific Data Provider for its own purpose.
The two files are: - parserConfig: config file dedicated to the Query Executor - adapterConfig: config file dedicated to the Data Adapter
{
"dataProviders": [
{
"name": "COPERNICUS",
"queryExecutorClasspath": "wasdi.dataproviders.CopernicusQueryExecutor",
"providerAdapterClasspath": "wasdi.dataproviders.CopernicusProviderAdapter",
"pythonScript": "",
"description": "Copernicus Data Provider",
"link": "https://scihub.copernicus.eu/",
"searchListPageSize": "25",
"defaultProtocol": "https://",
"user": "username",
"password": "password",
"apiKey": "",
"localFilesBasePath": "",
"urlDomain": "https://scihub.copernicus.eu/",
"connectionTimeout": "10000",
"readTimeout": "30000",
"adapterConfig": "",
"parserConfig": "",
"cloudProvider": "COPERNICUS",
"supportedPlatforms": ["S1", "S2", "S3"]
}
]
}
Data Provider Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Name/Code of the data provider (required to get QueryExecutors and ProviderAdapters) |
|
String |
Full class name and path of the QueryExecutor implementation (used to create the Java class with reflection) |
|
String |
Full class name and path of the ProviderAdapter implementation (used to create the Java class with reflection) |
|
String |
Python script path for External Python Provider |
|
String |
Description of the data provider |
|
String |
Link to the data provider’s website |
|
String |
Size for paginated requests when making WASDI searchList operations |
|
String |
|
|
String |
Path to parser config JSON file for query conversion |
|
String |
Username for the data provider |
|
String |
Password for the data provider |
|
String |
API key for the data provider |
` localFilesBasePath` |
String |
Local base folder for direct file access (when defaultProtocol is “file://”) |
|
String |
API address of the data provider |
|
String |
Specific connection timeout for this data provider |
|
String |
Specific read timeout for this data provider |
|
String |
Path to file with specific ProviderAdapter configurations |
|
String |
Code of the cloud provider where this data provider is hosted |
|
Array [String] |
List of platforms supported by this data provider |
Catalogue Configuration
Catalogues are strongly connected with Data Providers. A Catalogue is a service we can use to query a data collection. As each data provider can support more platforms (data collections), each platform can in general be queried in different catalogues.
In this catalogues section, we are intructing WASDI, for each platform, what are the avaiable catalogues. WASDI will try to query the first one first: if there is any problem, WASDI can switch on the other ones in the list.
Each name of a catalogue, must match a Data Provider.
There is a kind of double link: in the Data Provider, we list all the platforms supported. In this catalogues section, we list for each platform the list of Catalogues (Data Providers) that we can query.
The two list in general should match, but is possible for the administrator, for example, to drop from the catalogues list a Data Provider that we do not want to query but, eventually, only to fecth data.
{
"catalogues": [
{
"platform": "S1",
"catalogues": ["COPERNICUS", "ONDA", "CREODIAS"]
},
{
"platform": "S2",
"catalogues": ["COPERNICUS", "ONDA", "CREODIAS"]
}
]
}
Catalogue Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Code of the Platform Type as defined by the Platform class (e.g., “S1”, “S2”, “S3”) |
|
Array [String] |
List of catalogues supporting the Platform Type. Each string is the code of a QueryExecutor/ProviderAdapter. The first element has the highest priority. |
Additional Service Configurations
GeoServer Configuration
{
"geoserver": {
"address": "http://localhost:8080/geoserver/",
"user": "admin",
"password": "geoserver",
"workspace": "wasdi"
}
}
GeoServer Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
GeoServer server address (e.g., “http: //localhost:8080/geoserver/”) |
|
String |
GeoServer username for authentication |
|
String |
GeoServer password for the above user |
|
String |
Maximum dimension in MB to publish single images. Over this limit, WASDI will create a pyramid of the image |
|
String |
GDAL retile command for pyramid creation (default: “gdal_retile.py -r bilinear -levels 4 -ps 2048 2048 -co TILED=YES”) |
|
Boolean |
Special debug mode for PublishBand operation. Forces input DownloadedFile to be gathered from database using /data/wasdi/ instead of real local folder (default: false) |
|
String |
Default layer used to get style images (default: “wasdi:ESA_CCI_LAND_COVER_2015”) |
SFTP Configuration Structure
SFTP Configuration
{
"sftp": {
"address": "sftp.wasdi.net",
"user": "sftpuser",
"password": "sftppassword",
"port": 22,
"uploadRootPath": "/upload/"
}
}
SFTP Configuration Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Sender address for SFTP account emails |
Note: The JSON example above shows additional properties (``address``, ``user``, ``password``, ``port``, ``uploadRootPath``) that may be used in specific configurations but are not currently defined in the SftpConfig.java class.
S3 Bucket Configuration
{
"s3Bucket": {
"accessKey": "access-key",
"secretAccessKey": "secret-key",
"region": "eu-west-1",
"bucketName": "wasdi-bucket"
}
}
S3BucketConfig Properties
This configuration is dedicated to the specific access to Ecostress Data, may be not of interest in a personal installation.
Property |
Type |
Description |
|---|---|---|
|
String |
Access Key for S3 bucket authentication |
|
String |
Secret Key for S3 bucket authentication |
|
String |
S3 service endpoint URL |
|
String |
Name of the S3 bucket |
|
String |
Folder structure or path within the S3 bucket |
Stripe Configuration
{
"stripe": {
"secretKey": "sk_test_...",
"publicKey": "pk_test_...",
"webhookSecret": "whsec_..."
}
}
StripeConfig Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Stripe API key for authentication and payment processing |
|
List<StripeProductConfig> |
List of product-related configuration entries |
StripeProductConfig Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Product ID in Stripe |
|
String |
Product URL for accessing product details |
Storage Usage Control
{
"storageUsageControl": {
"enabled": true,
"maxDaysBeforeWarning": 7,
"maxDaysBeforeDeletion": 30,
"adminEmail": "admin@wasdi.net"
}
}
StorageUsageControl Properties
Property |
Type |
Description |
|---|---|---|
|
Long |
Maximum storage space (in bytes) for FREE subscription plans (default: 20GB) |
|
Long |
Maximum storage space (in bytes) for STANDARD subscription plans (default: 100GB) |
|
int |
Number of days to wait before proceeding to workspace deletion after warning email is sent (default: 10) |
|
boolean |
If true, workspaces with invalid subscriptions will not be actually deleted, but admin notification emails will be sent instead |
|
Warning EmailConfig |
Configuration for warning emails sent to users before workspace deletion |
WarningEmailConfig Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Subject line of the warning email (default: “Storage exceeded in WASDI”) |
|
String |
Body content of the warning email with placeholders for user info, storage size, limit, and warning delay |
Skin Configuration
{
"skins": [
{
"name": "wasdi",
"description": "Default WASDI Skin",
"logoPath": "/assets/logos/wasdi.png",
"primaryColor": "#1976d2"
}
]
}
SkinConfig Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Name identifier for the skin (default: “wasdi”) |
|
String |
Path to the logo image file (default: “ /assets/icons/logo-only.svg”) |
|
String |
Path to the logo text/name image file (default: “ /assets/icons/logo-name.svg”) |
|
String |
URL link to help documentation (default: “https://wasd i.readthedocs.io/en/latest/”) |
|
String |
URL link to support channels (default: “ht tps://discord.gg/FkRu2GypSg”) |
|
String |
Primary brand color in hex format (default: “#43526B”) |
|
String |
Secondary brand color in hex format (default: “#009036”) |
|
|
List of default categories for the skin |
Node Score Configuration
{
"nodeScore": {
"minTotalMemoryGBytes": 30
}
}
NodeScoreConfig Properties
Property |
Type |
Description |
|---|---|---|
|
int |
Minimum total memory in GB required for node scoring (default: 30) |
SNAP Configuration
{
"snap": {
"auxPropertiesFile": "/usr/lib/wasdi/launcher/snap.auxdata.properties",
"launcherLogActive": true,
"webLogActive": true,
"launcherLogFile": "/var/log/wasdi/snap-launcher.log",
"webLogFile": "/var/log/wasdi/snap-web.log",
"launcherLogLevel": "SEVERE",
"webLogLevel": "SEVERE"
}
}
SnapConfig Properties
Property |
Type |
Description |
|---|---|---|
|
String |
Full path to SNAP auxiliary properties file (default: “/usr/lib/wasdi/launc her/snap.auxdata.properties”) |
|
boolean |
Flag to activate SNAP logs in the launcher (default: true) |
|
boolean |
Flag to activate SNAP logs in the web server (default: true) |
|
String |
Full path to SNAP launcher log file |
|
String |
Full path to SNAP web server log file |
|
String |
SNAP launcher log level (default: “SEVERE”) |
|
String |
SNAP web server log level (default: “SEVERE”) |
Scheduler Configuration
This section configures the WASDI Scheduler. For each operation type, it can add a queue. Is up to the admin to refine the configuration given the capabilities of the node where it is installed.
All the operations that are not explicitly assigned, will go in the default queue.
The admin can then add all the needed queues.
{
"scheduler": {
"processingThreadWaitStartMS": "2000",
"processingThreadSleepingTimeMS": "2000",
"launcherPath": "/opt/wasdi/launchers/launcher.jar",
"javaExe": "java",
"killCommand": "kill -9",
"maxQueue": "50",
"timeoutMs": "300000",
"lastStateChangeDateOrderBy": -1,
"sometimesCheckCounter": 30,
"watchDogCounter": 30,
"activateWatchDog": true,
"redirectLauncherOutputs": false,
"launcherOutputPath": "",
"useVirtualNodeCode": false,
"defaultFairRoundRobin": false,
"defaultFairRoundRobinMaxProcessesCount": 3,
"defaultFairRoundRobinParentId": false,
"defaultFairRoundRobinParentIdMaxProcessesCount": 3,
"defaultSpecialWaitCondition": false,
"defaultMaxWaitingQueue": 100,
"emergencyFallbackEveryNCycles": 5,
"schedulers": [
{
"name": "default",
"maxQueue": "50",
"timeoutMs": "300000",
"opTypes": "RUNPROCESSOR,RUNIDL,RUNMATLAB",
"opSubType": "",
"enabled": "true",
"fairRoundRobin": false,
"fairRoundRobinMaxProcessesCount": 3,
"fairRoundRobinParentId": false,
"fairRoundRobinParentIdMaxProcessesCount": 3,
"specialWaitCondition": false,
"maxWaitingQueue": 100
},
{
"name": "download",
"maxQueue": "10",
"timeoutMs": "600000",
"opTypes": "DOWNLOAD",
"opSubType": "",
"enabled": "true",
"specialWaitCondition": true,
"maxWaitingQueue": 50
}
]
}
}
SchedulerConfig Properties
These are the configurations of the default scheduler
Property |
Type |
Description |
|---|---|---|
|
String |
Number of milliseconds to wait after a process is started |
|
String |
Number of milliseconds to sleep between scheduler cycles |
|
String |
Full path of Launcher jar file |
|
String |
Local Java command line executable |
|
String |
OS kill command for terminating processes |
|
String |
Default maximum queue size |
|
String |
Default queue timeout in milliseconds |
|
int |
Direction to order process workspaces in scheduler queue (default: -1) |
|
int |
Number of cycles before starting periodic checks (default: 30) |
|
int |
Counter for deadlock detection when only waiting processes exist (default: 30) |
|
boolean |
Flag to activate or deactivate the watch dog functionality (default: true) |
|
boolean |
If true, asks launcher to redirect its output to file |
|
String |
Destination path used when launcher output redirection is enabled (default: “”) |
|
boolean |
Enables virtual node code filtering for scheduler execution (default: false) |
|
boolean |
Enables fair round robin per user in CREATED list (default: false) |
|
int |
Max consecutive processes per user when fair round robin is active (default: 3) |
|
boolean |
Enables second round robin by parentId inside selected user queue (default: false) |
|
int |
Max consecutive processes per parentId group when parentId round robin is active (default: 3) |
|
boolean |
Enables special wait condition to consider waiting queue before triggering processes (default: false) |
|
int |
Max waiting processes admitted before breaking FIFO rules (default: 100) |
|
int |
In emergency mode, fallback to default CREATED candidate every N cycles when no unblock candidate is found (default: 5) |
|
ArrayList |
List of configured scheduler queues |
SchedulerQueueConfig Properties
Each of these, is a specific queue.
Property |
Type |
Description |
|---|---|---|
|
String |
Name identifier of the queue |
|
String |
Maximum number of elements allowed in this queue |
|
String |
Queue timeout in milliseconds for this specific queue |
|
String |
Comma-separated operation types supported by this queue |
|
String |
Operation subtype (requires opTypes to contain only one operation) |
|
String |
Flag to enable or disable this queue |
|
boolean |
Flag to apply special wait condition considering waiting queue (default: false) |
|
int |
Maximum waiting processes before breaking FIFO rules (default: 100) |
|
boolean |
Enables fair round robin per user in CREATED list (default: false) |
|
int |
Max consecutive processes per user when fair round robin is active (default: 3) |
|
boolean |
Enables second round robin by parentId inside selected user queue (default: false) |
|
int |
Max consecutive processes per parentId group when parentId round robin is active (default: 3) |
Scheduler Defaults and Queue Overrides
The scheduler-level properties prefixed with default are global defaults
applied to queue behavior.
If a queue in schedulers defines the corresponding queue-level property,
that queue value takes precedence for that queue. Otherwise, the scheduler
default is used.
Main mappings:
defaultFairRoundRobin->fairRoundRobindefaultFairRoundRobinMaxProcessesCount->fairRoundRobinMaxProcessesCountdefaultFairRoundRobinParentId->fairRoundRobinParentIddefaultFairRoundRobinParentIdMaxProcessesCount->fairRoundRobinParentIdMaxProcessesCountdefaultSpecialWaitCondition->specialWaitConditiondefaultMaxWaitingQueue->maxWaitingQueue
Load Balancer Configuration
{
"loadBalancer": {
"includeMainClusterAsNode": false,
"diskOccupiedSpaceMaxPercentage": 90,
"metricsMaxAgeSeconds": 600,
"minTotalMemoryGBytes": 30,
"activateMetrics": true
}
}
LoadBalancerConfig Properties
Property |
Type |
Description |
|---|---|---|
|
boolean |
Flag to include main node in computational node evaluation (default: false) |
|
int |
Maximum percentage of disk space occupied to consider a node available (default: 90) |
|
int |
Maximum age in seconds for node metrics before considering node down (default: 600) |
|
int |
Minimum RAM in GB required for a node to avoid low performance penalty (default: 30) |
|
boolean |
Flag to enable/disable metrics-based node selection (default: true) |
Configuration File Location
The configuration file is typically located at: -
/opt/wasdi/config/wasdiConfig.json (production) -
/data/wasdi/config/wasdiConfig.json (development)
Loading the Configuration
The configuration is loaded automatically when WASDI components start up
using the WasdiConfig.readConfig(String configFilePath) method. The
loaded configuration is available statically through
WasdiConfig.Current.
Environment-Specific Configurations
Different environments (development, staging, production) should have separate configuration files with appropriate values for:
Database connections
External service URLs
Authentication settings
File system paths
Docker registry settings
API keys and secrets
Security Considerations
Store sensitive information (passwords, API keys) securely
Use environment variables or secure vaults for production secrets
Regularly rotate authentication credentials
Restrict file system permissions on the configuration file
Use HTTPS for all external service communications
Validation
The configuration is validated when loaded. Missing required properties or invalid values will cause WASDI components to fail during startup with appropriate error messages.