How to FIX data inconsistency issues in TDS

How to FIX data inconsistency issues in TDS

By Siva Praturi

Replication is a technique used by directory servers to improve performance, availability, and reliability. The replication process keeps data in multiple directory servers synchronized.

Data inconsistency in directory servers arises mainly due to replication issues. Keeping the directory servers synchronized requires a diligent approach, including monitoring and maintenance. Any negligence in directory administration can result in significant differences in directory data across the cluster. 

In a clustered environment of TDS, data needs to be kept consistent among clustered members. Any changes done to master server needs to be propagated to peer or replica servers through replication technique. Due to various reasons, sometimes changes done to one server may not get replicated to other servers. This results in data inconsistency among them.

Data inconsistency among TDS cluster members can occur when:

1.       One server contains entries that do not exist on another TDS cluster member.

2.       Entry exists on both server but their attributes are different.

To synchronize TDS cluster members and bring them back in consistent state, the following approaches are adapted. 

1.       Importing the data from one TDS server using idsdb2ldif and exporting it to other server using idsldif2db or bulkload.

2.       Using idsldapdiff utility.

The following is the example LDAP structure for LDAP Master and Replica instances directory structure.  The example commands which are given underneath steps are used to synchronize “ou=TestOrg” organizationUnit from master to replica/master.

O=IBM

|-ou=itim  

|-ou=TestOrg

      |-ou=itim

   |-erglobalid=000000000000000000 

|-ibm-replicaGroup=default     

 1. Using idsdb2ldif and idslidf2db utility

 The following steps need to be followed to synchronize data between two or more LDAP instances.

 1.       Export LDIF file using idsdb2ldif/db2ldif server utility on master LDAP server

The idsdb2ldif/db2ldif is a server utility which is used to export entries from a directory stored in a relational database into a text file in LDAP Directory Interchange Format (LDIF). 

db2ldif [-o output_file -I instance_name [-f config_file]

        [-n filter_DN] [-c comments]

        [-k ?|key_seed -t key_salt] [-j] [-d debug_level]

        [[-s subtree_DN [-x]] | [-l] [-r]] [-W]] |    ?

This utility is available under “..\IBM\LDAP\V6.2\sbin” folder 

For syntax help, type “db2ldif -?” command 

Example: idsdb2ldif -I itimldap -s " ou=TestOrg, ou=itim, O=IBM " -j -o C:\LDAP_BACKUPs\ TestOrg.ldif     

2.       Delete LDAP subtree using idsldapdelete/ldapdelete server utility, in this scenario delete “ou=TestOrg” on LDAP replica server. Note: Take LDAP full backup before deleting this entry using idsdb2ldif utility.

The idsldapdelete/ldapdelete opens a connection to an LDAP server, binds, and deletes one or more entries. If one or more Distinguished Name (DN) arguments are provided, entries with those DNs are deleted. Each DN is a string-represented DN. If no DN arguments are provided, a list of DNs is read from standard input, or from file if the -i or -f flag is used. 

ldapdelete.exe [options] [DNs]

ldapdelete.exe [options] [-i file]

This utility is available under “..\IBM\LDAP\V6.2\bin” folder 

For syntax help, type “idsldapdelete -?” command 

Example: idsldapdelete -D cn=root -w password -s "ou=TestOrg, ou=itim, O=IBM"

3.       Skip all pending change entries and suspend replication on both master and replica LDAP servers using TDS Web Admin GUI or command line utilities.

4.       Import LDIF file which is generated in step 1 using ldif2db/idsldif2db utility 

The ldif2db/idsldif2db utility is used to import entries into LDAP server.  The database must already exist. The idsldif2db can be used to add entries to an empty directory database or to a database that already contains entries.

Note:  Before executing ldif2db/idsldif2db utility, TDS server must be stopped (both administration and instance).  Make sure that no applications are active and attached to the directory.  If applications are running using TDS Server then none of the import utilities will run. 

ldif2db [-i input_file -I instance_name [-f config_file]

        [-d debug_level] [-r yes | no] [-g] [-W output_file]] | -?

            This utility is available under “..\IBM\LDAP\V6.2\sbin” folder 

For syntax help, type “ldif2db -?” command  

Example: idsldif2db -i C:\LDAP_BACKUPs\ TestOrg.ldif 

5.       Resume replication on both master and replica TDS Servers.  To test replication create/update and delete entries and verify on both LDAP servers. 

2. Using Ldapdiff Utility

The idsldapdiff command line utility is designed to compare two directory subtrees on two different directory servers to determine if their contents match. It identifies differences in a replica server and its master and can be used to synchronize replicas.

Idsldapdiff performs two passes to make the servers are in sync. In the first pass, idsldapdiff traverses the Supplier server and does the following: Adds any extra entries on the supplier and to the consumer. Compares and fixes entries that exist on both the servers. In the second pass, idsldapdiff traverses the Consumer to check for any extra entries on the Consumer 

The tool traverses each entry in the directory subtree on the supplier server and compares its contents with the corresponding entry on the consumer server. Thus running the utility can take a long time and can generate lots of read requests to the supplier and consumer servers.  It is recommended to run the utility when no updates are being made to either of the directory servers.

This utility is a diagnostic and corrective tool it is not designed to run as routine maintenance. Depending on the replication-related errors observed in the log files, an administrator might decide to run the utility.

idsldapdiff -sh hostname -sp 389 -sD cn=root -sw password -ch consumerhostname -cp 389 -cD cn=root -cw password -b o=ibm,c=us  -a -F

This utility is available under “..\IBM\LDAP\V6.2\bin” folder 

For syntax help, type “idsldapdiff -?” command  

Example: ldapdiff -b " ou=TestOrg, ou=itim, O=IBM " -sh "mastertds.ibm.com" -ch "replicatds.ibm.com" -sD "cn=root" -sw password -cD "cn=root" -cw password –F -a 

Backup, Extract and Restore Tivoli Access Manager data

Backup, Extract and Restore Tivoli Access Manager data

By Siva Praturi

pdbackup

pdbackup utility is used to backup, extract and restore Tivoli Access Manager data

<TAM_Install_Dir>\etc\pdbackup.lst file contains Tivoli Access Manager data

<TAM_Install_Dir>\etc\pdinfo.lst file contains Tivoli Access Manager service information

It is good practice to backup using both options.

How to bakup Tivoli Access Manager data?

Use following commands to bakup Tivoli Access Manager data

pdbackup.exe -action backup -list "D:\APPS\IBM\Tivoli\Policy Director\etc\pdbackup.lst" -file pdbackup.lst_xxxx -path D:\temp

pdbackup.exe -action backup -list "D:\APPS\IBM\Tivoli\Policy Director\etc\pdinfo.lst" -file pdinfo.lst_xxxx -path D:\temp

If the command is run without any errors, it creates pdbackup.lst_xxxx.dar and pdinfo.lst_xxxx.dar files in D:\temp directory.

How to extract Tivoli Access Manager data?

Use following commands to extract Tivoli Access Manager data. This is to ensure the files are created properly before performing restore operation.

pdbackup.exe -action extract  -file D:\temp\pdinfo.lst_xxxx.dar -path D:\temp\pdinfo

pdbackup.exe -action extract  -file D:\temp\pdbackup.lst_xxxx.dar -path D:\temp\pdbackup

If the command is run without any errors, it extracts files to  D:\temp\pdinfo and  D:\temp\pdbackup  directory.

  

How to restore Tivoli Access Manager data?

Use following commands to restore Tivoli Access Manager data.

pdbackup.exe -action restore -file D:\temp\pdinfo.lst_xxxx.dar

pdbackup.exe -action restore -file D:\temp\pdbackup.lst_xxxx.dar

If the command is run without any errors, it restores files to <TAM_Install_Dir> from archive file.

Note:-

• Use current date / timestamp for xxxx. It will avoid overwriting of bakup files created.
• msg__pdbackup file will contain verbose output of ‘pdbakup’ commands. Review this file after executing the commands. Generally this file will be created in local\temp folder.
• ‘pdbakup’ with restore option will try to overwrite files in <TAM_Install_Dir> and it prompts for ‘yes/no/all’ options. In win 2008, there is a bug and this will not be visible in command prompt. So open msg__pdbackup file and enter your option.
• Stop TAM policy server service before restore operation.