Database Configuration Step

Introduction

The database configuration step is used to configure the connection to the database.
Data Migrator provides multiple methods for configuring database connections:

Basic properties: Allows direct specification of connection details in the configuration file.
AWS Secret Manager: Enables secure retrieval of database credentials from AWS Secret Manager.
Database connection URL: Enables database access through a user-defined connection URL.

These options provide flexibility in how you configure and secure your database connections, allowing you to choose the method that best fits your infrastructure and security requirements.

Properties

Global properties

Property name	Description	Values	Default value	Mandatory
dataBaseName	The name of database This property can be stored in AWS secret manager, see dataBaseAWSSecretName property	String	None	Yes
dataBaseUser	The user used to connect to the database This property can be stored in AWS secret manager, see dataBaseAWSSecretName property	String	None	Yes
dataBasePassword	The password used to connect to the database This property can be stored in AWS secret manager, see dataBaseAWSSecretName property	String	None	Yes
dataBaseHost	DNS name or IP address of the database This property can be stored in AWS secret manager, see dataBaseAWSSecretName property [DOCKER] To use this property see DNS name or IP address in Docker	String	localhost	Yes
dataBasePort	The listener port This property can be stored in AWS secret manager, see dataBaseAWSSecretName property	Integer	5432	Yes
databaseType	The target database type	POSTGRE ORACLE MSSQL	POSTGRE	No
dataBaseAdditionalArgs	Enable to add connection properties to url. Example: add "?ssl=true" => jdbc:postgresql://localhost:5432/aeat?ssl=true	String	None	No
dataBaseDriverPath	Path of the database driver folder. It can be absolute or relative path. Only one driver need to be in this location [DOCKER] To use this property see path in Docker	String	See use default value.	No
dataBaseAwsSecretName	AWS Secret Manager stores the database connection details like host, port, user, password, etc., It's to avoid stealing sensitive informations. Data Migrator use this property to refer to Secret Name of the AWS Secret Manager, retrieve database information and establish DB connection. If this property is specified along with other properties (dataBaseHost, dataBasePort, dataBaseName, dataBaseUser, dataBasePassword, dataBaseServiceName) then databaseSecretName information will replace the data directly mentioned in the ini or property file. Refer FAQ to know how to add a customised field (ex: Oracle Service Name) for a database and use it in the Data Migrator	String	None	No
dataBaseAWSRegion	AWS Region where the database information like host, port, user, password, etc., are stored in the Secret Name	String	eu-west-3	No
dataBaseCustomConnectionUrl	This property can be used to provide the connection string for the database. If the dataBaseUser and dataBasePassword properties are defined in the configuration files with dataBaseCustomConnectionUrl, the values taken into account for username and password are those defined in the configuration file. If you wish to use only dataBaseCustomConnectionUrl for login, you must remove the dataBaseUser and dataBasePassword properties from the configuration files. [POSTGRES] In case we use the property stepDatabaseKillDropCreate=true, we must keep both properties databaseName and dataBaseUser in the configuration files or AWS Secret Manager. This property can be stored in AWS secret manager with the value dbcustomconnectionurl as key , see dataBaseAWSSecretName property	String	None	No

PostgreSQL properties

Property name	Description	Values	Default value	Mandatory
adminDataBaseName	The name of database to create	String	None	No
adminDataBaseUser	The super user owner	String	None	No
adminDataBasePassword	The password of super user owner	String	None	No
adminDataBaseCustomConnectionUrl	This property can be used to provide the connection string for the database. This property can be stored in the AWS Secrets Manager with the value admindbcustomconnectionurl as key, see the dataBaseAWSSecretName property	String	None	No
stepDatabaseKillDropCreate	This property can be used to drop old database and create a new one during migration.	true/false	true	No
stepDatabaseCreateScriptPath	This optional property helps to create a database with custom script. It's value can be relative or absolute path of the SQL file. More information is in this FAQ	String	None	No

Oracle properties

Property name	Description	Values	Default value	Mandatory
dataBaseServiceName	The service name of database to create This property can be stored in AWS secret manager, see dataBaseAWSSecretName property	String	None	No
customSqlldrUserId	This optional property can be used to provide the USERID for SQLLDR. This property can be stored in AWS secret manager with the value customSqlldrUserId as key, see the dataBaseAWSSecretName property This property becomes mandatory, if the dataBaseUser and dataBasePassword properties are not defined in the configuration files and the dataBaseCustomConnectionUrl is used to connect to a database.	String	None	No

Examples

PostgreSQL connection using properties

The following settings illustrate how to configure a PostgreSQL database connection using properties:

[Database - cnxPostgreSQL]
adminDataBaseName=postgres
adminDataBaseUser=postgres
adminDataBasePassword=postgres
dataBaseHost=localhost
dataBasePort=5432
dataBaseName=postgres
dataBaseUser=userTest
dataBasePassword=passwordTest
stepDatabaseKillDropCreate=true

This configuration section defines:

Administrative database name and superuser credentials for drop and create operations
Database server is located on localhost and uses port 1521
Database user is set to userTest
Database password is set to passwordTest
Option to drop and recreate the database

Oracle connection using properties

The following settings illustrate how to configure Oracle database connection using properties

[Database - cnxOracle]
databaseType=ORACLE
dataBaseHost=localhost
dataBasePort=1521
dataBaseName=DatabaseTest
dataBaseUser=userTest
dataBasePassword=passwordTest
dataBaseDriverPath=../drivers/ORACLE

This configuration section defines:

Database type is set to ORACLE
Database server is located on localhost and uses port 1521
Target database name is specified as DatabaseTest
Database user credentials have username set to userTest, and password set to passwordTest
Oracle database driver path is located at ../drivers/ORACLE

Oracle connection using secret manager

The following settings illustrate how to configure Oracle database connection using AWS Secret Manager

[Database - cnxOracle]
databaseType=ORACLE
dataBaseAWSSecretName=oracle-test
dataBaseDriverPath=../drivers/ORACLE

This configuration section defines:

Database type is set to ORACLE
AWS Secret Manager with the name oracle-test is utilized to retrieve database information and establish the database connection
Oracle database driver path is located at ../drivers/ORACLE

Oracle connection using database connection URL

The following settings illustrate how to configure Oracle database connection using connection url

[Database - cnxOracle]
databaseType=ORACLE
dataBaseName=DatabaseTest
dataBaseCustomConnectionUrl=jdbc:oracle:thin:Username/Password@localhost:1521:DatabaseTest
dataBaseDriverPath=../drivers/ORACLE

This configuration section defines:

Database type is set to ORACLE
Database name is set to DatabaseTest
Custom JDBC connection string, which includes:
- The database type
- Username and password (Username/Password)
- Host (localhost)
- Port (1521)
- Database name (DatabaseTest)
Oracle database driver path is located at ../drivers/ORACLE