Wasdi Libraries Concepts

This tutorial has to goal to introduce the main WASDI App development concepts, that can be used in all the supported programming language to get the best from the WASDI libs.

Introduction

To create a WASDI account, just click on the New User? Register here! link

The main goal of the WASDI libraries is to let you develop in your onw enviroment. Do you prefer Windows, Apple or Linux? Do you use Python, IDL, Java,C#, Javascprit or Matlab? Do you use an integrated IDE, or you code using VIM? For WASDI is the same.

The world is full of different possibilities and environments and we do not want to introduce a new one. We think that every one should concentrate to obtain the result not to learn a new tool. So WASDI libraries are designed to be intalled and used in every environment.

In python you can use:

pip install wasdi

In IDL or Matlab/Octave, just donwload the lib files and save on your computer.

For Java, just download and link our jar to your project.

For C#, install our nuget package.

For JavaScript, install our Node package, or link our cdn lib JavaScript Library.

All the WASDI libraries are Open Source.

Main Goals

The main goals on the libraries are:

  • Authenticate in WASDI: this is based on a config file that you must create on your computer, and keep for you on your computer: here you will declare your WASDI credentials that will be used by the lib to interoperate with the system.

  • Abstract the access to the files: this is the real main concept of WASDI, the only thing we require to our developers: ask the paths to WASDI!

The file access abstraction is very simple to use: make your code in a way that, when must access one file, it can ask the path to wasdi:

# Python Code
fileFullPath = wasdi.getPath("S2B_MSIL1C_20210124T141049_N0209_R110_T20KNV_20210124T160035.zip")

; IDL Code
fileFullPath = WASDIGETPATH("S2B_MSIL1C_20210124T141049_N0209_R110_T20KNV_20210124T160035.zip")

//Java: we created an object WasdiLib oWasdiLib = new WasdiLib();
String sFileFullPath = oWasdiLib.getPath("S2B_MSIL1C_20210124T141049_N0209_R110_T20KNV_20210124T160035.zip");

% We obtained a lib object calling Wasdi = startWasdi(config_path)
fileFullPath = wGetPath("S2B_MSIL1C_20210124T141049_N0209_R110_T20KNV_20210124T160035.zip")

All the lib functions, take in input just the file names and not the paths, and you should work in the same way.

Thank to this little rule, WASDI will be able to optimize the access to files: when you work locally, WASDI will keep the file to the cloud until is not really necessary to open it locally: in that moment, if not present yet, WASDI will download automatically the file for you. The same when you create your result and wants to add it to the cloud: WASDI will keep the local file until is possibile, and will automatically upload it if and when requested.

Files are exchanged only once, if and when needed: all is completely transparent to you, in all the supported languages.

Main Entities

To work with WASDI there are only few basic entities to consider:

  • Product: in general, any file in WASDI. Products are satellite images taken from a data provider, files imported by you, files generated by an application or a workflow.

  • Workspace: the workspace is a container of pruducts. Is where you can download images, run applications, run workflows. Each workspace has an owner and can be shared with users.

  • Workflow: SNAP graphs. SNAP is the ESA Open Source Tool to handle Sentinels and Other missions images. A SNAP graph can be uploaded in WASDI with a drag and drop and executed on the cloud

  • App: a WASDI app is a processor developed by some WASDI user, in some language, that can be used. An app can be your own code for instance. Each WASDI Application can be private, can be shared with selected users or can be public. Each Application can be shown in the marketplace or not. Can be free or not. The developer can decide any detail of his own application.

Applications Model

The typical WASDI Application can be represented by this schema:

../_images/functionmodel.jpg

EO Application is the real processor deployed with WASDI. When the End-User starts an application, he must set the input parameters. These parameters can be a date interval, a bounding box, a sensitivity index and any other specific option.

Usually EO Applications use these parameters to fetch EO Images that has to be elaborated and create the output added-value data. The End-User, once the processor is finished, can view the generated output usually in a web GIS Environment.

All the WASDI EO-Applications accept a key-value dictionary as Parameters.

In python and C#, it is a JSON File.

In Java, Matlab/Octave, IDL, it is a standard properties file in the format:

KEY=VALUE.

Parameters are your own inputs. Since WASDI is desinged to make your application running in the cloud, we ask you to make “pure code” that does not care how to get inputs from the user, but just USE the inputs given by the user. This is done in the parameters file: there you decide the inputs you need and there you can put and change your inputs. WASDI will let you read your inputs using a simple line of code:

#python
myParam = wasdi.getParamter("StartDate", new Date())

;IDL
myParam = WASDIGETPARAMETER("StartDate")

//Java: we created an object WasdiLib oWasdiLib = new WasdiLib();
String sMyParam = oWasdiLib.getParameter("StartDate");

% We obtained a lib object calling Wasdi = startWasdi(config_path)
myParam = wGetParameter("StartDate")

Configuration

All the libraries uses a config file to be initialized. For Java, Matlab/Octave, IDL config files are standard properties file in the format:

KEY=VALUE

For python and C# it is a JSON file.

The basic configuration, that can be used in almost all cases, is:

USER=your.email@domain.sample
PASSWORD=yourpassword
WORKSPACE=NameOfYourWorkspace
PARAMETERSFILEPATH=./params.txt

{
        "USER": "your.email@domain.sample",
        "PASSWORD": "yourpassword",
        "WORKSPACE": "NameOfYourWorkspace",
        "PARAMETERSFILEPATH": "./params.json"
}

Basic Parameters are:

  • User is your valid WASDI UserId, ie the mail you used to register.

  • Password is your valid WASDI password.

  • Workspace, is the workspace where you want to run the code you are writing.

  • ParametersFilePath File Path is the path where you have the parameters file for the code you are running.

Advanced configuration can be controlled adding these entries to the config file:

  • BASEPATH=c:/local/path/ - This is the local base path used by wasdi to read and save the data you are using in your application.

  • DOWNLOADACTIVE=1 - If 1, WASDI will automatically download the data you need in your code when requested

  • UPLOADACTIVE=1 - If 1, WASDI will automatically upload your data to the cloud when requested.

  • BASEURL=https://www.wasdi.net/wasdiwebserver/rest - base url of the WASDI APIs

  • WORKSPACEID=364c24ff-4891-4d0a-83bd-2772d292f918 - Id of the workspace, can be used as an alternative to the WORKSPACE name option

  • VERBOSE=1 1 to make local console verbose log of the lib, 0 to deactivate

  • REQUESTSTIMEOUT=5 - seconds of timeout for the lib http calls

Local File System

As it been stated before, libraries make an automatic optimized download and upload of files from your local PC to the cloud when and only when is needed. This functionality is smart and is activated only when you work on your computer; when you will deploy your app to the cloud WASDI will directly access the files.

This means that you will have your files on your computer, and this can be useful to double check your results, open the files with other tools like ENVI or QGis, copy the files to other locations and whatever you may need.

By default, for all the languages, WASDI use as base folder the home folder of your computer user and adds a .wasdi folder. Can be:

  • Linux: /home/[your user]/.wasdi

  • Windows: C:\Users\[your user]\.wasdi

In the .wasdi folder WASDI will create a subfolder for each user and for each workspace. Is very important to remember: this will be done only if and when is needed!! if there is no need to access the file locally, all we be delegated to the cloud and no folder will be created. If a file is accessed locally, the folder will be created and the file downlaoded.

Each workspace folder will be named as the workspaceId. The Workspace Id is a guid. You can find the workspace Id from the web application in two ways:

../_images/workspaceid.jpg

It is in the address bar, when you are in the Editor section. You can click on the info button and read from the property window the Workspace Id.

The folder structure will be something similar:

../_images/folders.jpg

  • Linux: /home/[your user]/.wasdi/[WASDI_User]/[WorkspaceId]/

  • Windows: C:\Users\[your user]\.wasdi\[WASDI_User]\[WorkspaceId]\

These are your folders, you can do what you want of that folders. Again: only and when needed, WASDI will search there for the needed files and, if not available, will download it.

Basic Functionalities

The basic functionalities are:

  • Access users’ workspaces and files

  • Search EO Images

  • Import EO Images in the workspace

  • Execute SNAP Workflows

  • Execute other WASDI Applications

  • Execute basic GIS Operations (mosaic, multisubset)

  • Run Sen2Core

Advanced Functionalities

The advanced functionalities are:

  • Send log directly to the web user interface

  • Update the progress of the processing

  • Save a payload associated to each run of the app

  • Search and retrieve the execution of other processors and the relative payloads