AWS Blu Insights

Data Migrator - v2.0.4

Release v2.0.4

Improvements

We introduced new properties:

  • For Postgres, stepDatabaseCreateScriptPath allows the customer to specify a SQL script path to initialize the modernized database.
  • In case of blocker issue with bulk insertion, the disableFastDataInsertion disables it. It will decrease the performances, it is to use for POC project until a fix is done.

Bug fixes

  • If a SQL query fails, the Data Migrator will return a correct return code.
  • Now QDDS conversion step considers multithread property.
  • Column size is not limited to 128 characters for VARRAY type.

Release v2.0.3

Improvements

We introduced a new property:

  • For Oracle, customSqlldrUserId allows the customer to provide a full custom user identifier to Oracle SQL loader.

Bug fixes

  • The Unicode UFF0D was wrongly mapped in Custom930 charset.

Release v2.0.2

Bug fixes

  • Data Migrator handles data filenames with special characters.

Release v2.0.1

Improvements

We introduced new properties:

  • csvWithHeader shows to Data Migrator that input csv data files have a header line to skip.
  • oracle.ctlOptionClauseSqlldr allows the customer to provide the Oracle SQL loader's option clause.

Bug fixes

  • Fixed a SLF4j exception threw while performing QDDS conversion.
  • For VARCHAR column in EBCDIC data, when ebcdicNullByteIndicatorPositionMode is set to ALWAYS_LEFT, Data Migrator considers Null Byte Indicator to determine the actual column size.

Release v2.0.0

General

  • Data Migrator is now compatible with docker.
  • Data Migrator allows to use AWS Secrets Manager to retrieve connection details (see database identification).

Improvements

We introduced new properties:

  • sqlmodel.path to specify a custom location for SQLMODEL.json file.
  • ignoreDeletedRecords: in mainframe data files, deleted records are filled with blank characters, leading to an exception when Data Migrator reads a blank numeric field. It is now possible to configure Data Migrator, with this new property set to true, to ignore record starting with blanks.
  • dataBaseDriverPath to provide the location of the database drivers’s folder anywhere in the system/computer.
  • dataBaseServiceName to add the possibility of a connection to an Oracle database using a service name. This property is not compatible with dataBaseName property.
  • convertGraphicDataToFullWidth: By default, SQL graphic data is coded on two bytes. When it is not the case, this property needs to be set to true.
  • dataBaseCustomConnectionUrl allows the customer to provide a full custom connection URL to the database.


See https://bluinsights.aws/docs/data-migrator-migration-steps for more details.


Bug fixes

  • Data using charset Custom930 were fixed in this release.
    • all low-level and all high-level data bytes in a graphic type field were not correctly interpreted by Custom930 charset, resulting in unreadable character. 
    • The two bytes data FE FD was not mapped.
  • PostgreSQL database connection gets closed randomly when using a search path query.
  • Data Migrator handles BLOB columns for a specific case.
  • For Oracle, fixed an exception when loading a huge data file with the sqlldr command.
  • Fixed a path issue to support Windows and Unix systems.
  • Line feed character into string column data implied issue for Oracle. Generalize csvOneLiner property behavior for Oracle.
  • For PostgreSQL, when csvOneLiner property is used, decimal literal in the NUMERIC column was not formatted.