AWS Blu Insights

FAQ

What is the format of the configuration file for an AWS Blu Age Data Migrator project?

The INI format

Which types of input data can be used by the AWS Blu Age Data Migrator?

CSV, EBCDIC and QDDS

Which database driver versions are compatible with the AWS Blu Age Data Migrator?

Compatible database drivers:

  • MSSQL: 12.6.0.jre11
  • ORACLE: 12.2.0.1
  • POSTGRESQL: 42.6.2

These jars are available through the links below. 

MSSQL - https://go.microsoft.com/fwlink/?linkid=2259203

Note: Unzip the downloaded sqljdbc_12.6.0.0_enu.zip and find mssql-jdbc-12.6.0.jre11.jar under the enu > jars folder.

ORACLE - https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/12.2.0.1/

POSTGRES - https://jdbc.postgresql.org/download/postgresql-42.6.2.jar

Where we need to put the driver files to configure the AWS Blu Age Data Migrator?

Follow these steps:

  1. Create a folder 'drivers' in your migration project root folder, i.e. migrationProjectPath.
  2. Create the folders (MSSQL, ORACLE, POSTGRE) in folder 'drivers'.
  3. Put in each folder the driver corresponding to each database (See driver versions).
drivers_path.PNG

How to use path in Docker?

To run the data migrator in the Docker by specifying the SQL Model JSON file or database driver path in the configuration, we need to add to the Docker command: -v pathInComputer:pathInDockerImage                 
And in the configuration file, we need to specify pathInDockerImage as path of sql model json file or database driver.

Docker command will be like this:              
docker run --rm -v [migrationProjectPath]:/home -v pathInComputer:pathInDockerImage 170644325583.dkr.ecr.eu-west-3.amazonaws.com/data-migrator:latest -root /home -configurationIni [configurationFilePath];[stepsConfigurationFilePath]

Parameters

  • -v: Volumes are the mechanism for persisting data generated and used by Docker containers
  • pathInComputer: Path of the SQL Model JSON file or database driver in the System/computer
  • pathInDockerImage: Path of the SQL Model JSON file or database driver in Docker image
  • migrationProjectPath: Absolute path to your migration project
  • configurationFilePath: Relative path to your migration configuration file
  • stepsConfigurationFilePath: Relative path to your migration steps configuration file

How to use database DNS name or IP address in Docker?

Use 'host.docker.internal' as the value for the dataBaseHost and server.host properties in your file if your database is installed on the server where you are running Docker.

During migration I am getting error with an empty line. What should I do?

In order to migrate the database which contains empty lines, use the property ignoreDeletedRecords with value as true. If still error is persisting with property, then check whether the latest version of the data-migrator tool is used.

How to add Oracle Service Name information to the secret in the AWS Secret Manager ?

By default, AWS Secret Manager doesn't provide a field to store the Oracle Service Name. Hence, with the help of edit option, add a row with key as 'dbservicename' and provides the corresponding value.

How can I create a custom PostgreSQL database using the AWS Blu Age Data Migrator?

Create a SQL file with custom database creation statement. Get the absolute or relative path of the file and give it as input for the parameter stepDatabaseCreateScriptPath. 

Note: 

  1. This parameter takes into consideration only if the value of the property stepDatabaseKillDropCreate as true.
  2. This script is not limited to database creation. Hence, it's responsibility of the user to provide the proper statements in a SQL file.

How can I execute queries after data insertion? 

Possible to execute queries after the data insertion by adding a section in the ini file. Data migrator reads and performs steps based on the section sequentially. Hence, create a new section after the data insertion section to run queries after data insertion.

[CSV2DB - Loading1]
csvQuote="
csvSeparator=;
csvOneLiner=true;
database=test
encoding=UTF-8
input.folder=data
multithread=5
temp.folder=dataTemp
[ExecuteSql - Execute extra queries]
database=test
input.folder=database/extra
multithread=1
plainExecution=true

How to install BCP Utility for MSSQL loading data?

The installation guide is available here.