Handling Data Dependencies in EIM using IFB Files

In a typical data migration application using EIM, the IFB file contains a large number of sub-processes, which are called from the main process (SHELL). Often while running the EIM process, there are situations, when we would like to run only a sub set of these processes. Such situations normally arise in testing environments, when we prefer running only those processes for which testing needs to be performed.

For example consider a section of a typical IFB File, which includes the following processes. 

[Import Position]

TYPE  = IMPORT

BATCH = 11100000-11100005

TABLE = EIM_POSITION

ONLY BASE TABLES = S_PARTY, S_POSTN

[Import Position Organization] 

TYPE  = IMPORT

BATCH = 11200000-11200005

TABLE = EIM_POSITION

ONLY BASE TABLES = S_PARTY, S_ACCNT_POSTN

USE INDEX HINTS = TRUE

INSERT ROWS = S_PARTY, FALSE

UPDATE ROWS = S_PARTY, FALSE

;***************************************************************************

; IMPORT

;***************************************************************************

[Import Everything]

TYPE = SHELL

INCLUDE = "Import Organization"

INCLUDE = "Import Organization Address"

INCLUDE = "Import Org-Address Relationship"

INCLUDE = "Import Position"

INCLUDE = "Import Position Organization"

INCLUDE = "Employee Position Rel"

Suppose, the logic for migrating Position Organization relationship has undergone some change. Hence the “Import Position Organization” Process needs to be tested. So, in the IFB file we comment out all other processes and keep only INCLUDE = "Import Position Organization" uncommented. We delete the records in the S_ACCNT_POSTN table, which stores the Position Organization Relationship, and then run the EIM process. This works fine.

However, suppose the logic for migrating Position data has undergone some change, which is why the “Import Position” Process needs to be tested. But in this case, if we comment out all other processes and run the EIM only for “Import Position”, after deleting records from S_PARTY and S_POSTN, then the Position Organization relationship data records would become orphans. Similarly, records in all other tables, which have a foreign key relationship with S_POSTN, would become orphan records. This is because every time we delete a Base table (either manually or through EIM), and then run the EIM process to migrate records into it, SIEBEL generates new ROW_IDs for each of these records. Hence the foreign key reference that other tables have with this record would be lost.

Hence the thumb rule that should be followed is:

When testing data migration for a Master table (which has foreign key relationship with one or more child tables), run the EIM process not only to import the Master table records, but also to import records to all tables that have foreign key relationship with the Parent table.

In our example, therefore if we need to test the migration of Position Master records into S_POSTN, we would need to run the EIM process for   “Import Position”, "Import Position Organization" and"Employee Position Rel" to ensure that foreign key references are not lost.