Skip to main content

Data Sources

All applications interface with data sources to read and manipulate data. Data sources are of different types:

  • Databases
  • Web Services

Data has a common way to define data sources as micro executor configuration. These configurations are readable by all micro-executors and are compliant with multi-tenancy.

Data sources are defined in data_source_definiton.yaml file in DSL library. This file is a mapping between the data source ID as map key and its value. Data sources configuration are heterogeneous based on their types.

The typology of data sources are:

Relational Database Data Source​

relational_database_data_source_example:
  type: "RELATIONAL_DATABASE"
  description: "" # description
  uri: "" # db uri
  username: "" # db username
  password: "" # db password

Document Database Data Source​

document_database_data_source_example:
  type: "DOCUMENT_DATABASE"
  description: "" # description
  uri: "" # db uri
  database: "" # db name

Key-Value Database Data Source​

key_value_database_data_source_example:
  type: "KEY_VALUE_DATABASE"
  description: "" # description
  uri: "" # db uri
  database: "" # db name

Web Service Data Source​

web_service_data_source_example:
  type: "WEB_SERVICE"
  description: "" # description
  url: "" # ws url

AS400 libraries Data Source​

as400_libraries_data_source_example:
  type: "AS400_LIBRARIES"
  description: "" # description
  host: "" # as400 host
  username: "" # as400 username
  password: "" # as400 password
  libraries: [] # as400 library names
warning

Data sources configuration require micro executor reloading on change if you work in a non develop environment.

Variables and secrets​

In the data source configuration, you will need to set up secrets to avoid leaking sensitive information such as database credentials. To do this, you can set data source values as environment variables.

The syntax is as follows:

  • ${VARIABLE_NAME}: variable name with null as default
  • ${VARIABLE_NAME:=[default value]}: variable name with default value

i.e.

data_source_definition.yaml

relational_database_accounting:
  type: "RELATIONAL_DATABASE"
  description: "Relational database for Accounting"
  uri: "postresql://postgres-svc:5432/postgres"
  username: "${RELATIONAL_DATABASE_ACCOUNTING_USERNAME:=root}"
  password: "${RELATIONAL_DATABASE_ACCOUNTING_PASSWORD}"
warning

All components requiring data sources whose configuration is based on environment variables must contain the variables.

Use Data Source configuration in micro-executor​

The Data sources configuration can be accessible from ExecutionContext interface available in all SDK.