Backup and Restore Tivoli Directory Server

Overview of backup and restore procedures for Tivoli Directory Server

By Siva Praturi

Tivoli Directory Server provides multiple methods for backing up and restoring directory server instance information. There are methods that back up the complete information for a directory server instance, and methods that back up only the data in the database. You can back up and restore Tivoli Directory Server using below options and they have their advantages and disadvantages.

• DB2 backup (db2 backup) and restore (db2 restore) commands
• Tivoli Directory Server backup (idsdbback) and restore (idsdbrestore) commands
• Tivoli Directory Server tools db2ldif and ldif2db.

Choosing an appropriate backup method is a very important decision in Tivoli Directory Server environments. In my view, it is always a safe side option to choose both automated LDIF export and database backups with LDIF export being stored on a DISK drive and database backup on TAPE device.

DB2 backup and restore commands

The db2 backup and db2 restore commands are provided by IBM DB2. The advantage to using these commands is performance and flexibility for specifying the location of the database files. The db2 restore command can be used to distribute the database across multiple disks or to simply move the database to another directory.

The disadvantage to the db2 backup and db2 restore commands is their complexity. Another disadvantage is the potential for incompatibility in backing up and restoring across platforms and across DB2 versions.

An important consideration when using db2 backup and db2 restore commands is the preservation of DB2 configuration parameters and system statistics optimizations in the backed-up database. The restored database has the same performance optimizations as the backed-up database.This is not the case with LDAP db2ldif, ldif2db, or bulkload.

idsdbback and idsdbrestore commands

The idsdbback and idsdbrestore commands are provided by Tivoli Directory Server to back up both the DB2 database and the directory server configuration. The advantage to using these commands is the backing up of the directory server configuration.

The other ways of backing up the directory server do not include the directory server configuration information. This information includes directory schema, configuration file and key stash file. Although it is possible to back these files up manually, the idsdbback andidsdbrestore commands make this task easier. However, it is important to note that idsdbback command can be used only when Tivoli Directory Server is not running.

The disadvantage to using the idsdbback and idsdbrestore commands is less flexibility in how the underlying DB2 restore is performed. For example, using the idsdbrestore command, the DB2 restore cannot be directed to distribute the database across multiple disks. Another disadvantage is the potential incompatibility in backing up the database on one platform (for example AIX) and restore on another platform (for example Windows).

db2ldif, ldif2db, and bulkload commands

The db2ldif and ldif2db commands are provided by Tivoli Directory Server to dump or restore the database to or from a file in Lightweight Directory Interchange Format (LDIF). The advantage to using these commands is portability, standardization and size. It is important to note that these tools do not preserve the DB2 configuration parameters and database optimizations parameters.

The output LDIF file can be used to restore a directory server on a different platform and possibly with some modification on a different directory server product, for example the Sun ONE Directory Server. The disk space requirement for the LDIF output from db2ldif is approximately 6.5 times less than the backed up database from db2 backup or idsdbback.

The disadvantage to using db2ldif and ldif2db is speed as compared to db2 backup and db2 restore. Because the db2 backup and restore commands are essentially a disk copy, they are the fastest alternative.

For restoring the database, the bulkload utility is many times faster than ldif2db command, yet is still much slower than db2 restore. Thebulkload utility takes approximately 12 times longer than a db2 restore. In order to restore a directory server using bulkload, the directory server must be empty. This is accomplished by restoring an empty database or by unconfiguring, then reconfiguring the directory server.

Choosing an appropriate backup method

What are the factors to consider in choosing a backup method?

The first choice in backup methods is between exporting data to LDIF and using a database backup. There are some advantages to exporting to LDIF:

• For directories with no more than a few hundred entries, it might be more time and space efficient to export the entries to LDIF as a backup, because there is extra overhead for a full database backup.
• The directory server can be running and accepting updates during the export to LDIF.
• LDIF is a portable format, so if you need to move data from an LDAP directory on Windows to one on AIX, using LDIF is the best approach.
• LDIF is also vendor neutral, so if you decide to move your data from some other vendor’s directory server to the Tivoli Directory Server, then exporting to LDIF is a good way to go. 

Now, if you are not moving data between directories and you have more than a few hundred entries, you would be better off doing a database backup. For large directories, it is much faster to do a database backup than to export all the data to LDIF. A database backup has these advantages:

• It is faster and more efficient for directories with thousands to millions of entries.
• It saves the database configuration settings, including those for the underlying tablespaces.
• DB2 provides the options to do either offline or online backups. It includes an option to do full backups or delta backups.
• Your saved backup images and the transaction logs allows for recovery right up to the last transaction completed prior to a disk crash. 

So, database backups are usually the best approach, but you still need to decide if online or offline backups will work best for you. Doing offline backups is the simplest approach. It requires less administrative activity, and if you can afford to stop a directory server long enough to do a periodic backup, then it is probably the best solution for you.  On the other hand, if you cannot afford to stop a directory server to take a backup, then online backups are the way to go. The extra administrative overhead when a database is configured for online backups is related to the accumulation of transaction logs that necessitates some management of them.

How to FIX data inconsistency issues in TDS

How to FIX data inconsistency issues in TDS

By Siva Praturi

Replication is a technique used by directory servers to improve performance, availability, and reliability. The replication process keeps data in multiple directory servers synchronized.

Data inconsistency in directory servers arises mainly due to replication issues. Keeping the directory servers synchronized requires a diligent approach, including monitoring and maintenance. Any negligence in directory administration can result in significant differences in directory data across the cluster. 

In a clustered environment of TDS, data needs to be kept consistent among clustered members. Any changes done to master server needs to be propagated to peer or replica servers through replication technique. Due to various reasons, sometimes changes done to one server may not get replicated to other servers. This results in data inconsistency among them.

Data inconsistency among TDS cluster members can occur when:

1.       One server contains entries that do not exist on another TDS cluster member.

2.       Entry exists on both server but their attributes are different.

To synchronize TDS cluster members and bring them back in consistent state, the following approaches are adapted. 

1.       Importing the data from one TDS server using idsdb2ldif and exporting it to other server using idsldif2db or bulkload.

2.       Using idsldapdiff utility.

The following is the example LDAP structure for LDAP Master and Replica instances directory structure.  The example commands which are given underneath steps are used to synchronize “ou=TestOrg” organizationUnit from master to replica/master.

O=IBM

|-ou=itim  

|-ou=TestOrg

      |-ou=itim

   |-erglobalid=000000000000000000 

|-ibm-replicaGroup=default     

 1. Using idsdb2ldif and idslidf2db utility

 The following steps need to be followed to synchronize data between two or more LDAP instances.

 1.       Export LDIF file using idsdb2ldif/db2ldif server utility on master LDAP server

The idsdb2ldif/db2ldif is a server utility which is used to export entries from a directory stored in a relational database into a text file in LDAP Directory Interchange Format (LDIF). 

db2ldif [-o output_file -I instance_name [-f config_file]

        [-n filter_DN] [-c comments]

        [-k ?|key_seed -t key_salt] [-j] [-d debug_level]

        [[-s subtree_DN [-x]] | [-l] [-r]] [-W]] |    ?

This utility is available under “..\IBM\LDAP\V6.2\sbin” folder 

For syntax help, type “db2ldif -?” command 

Example: idsdb2ldif -I itimldap -s " ou=TestOrg, ou=itim, O=IBM " -j -o C:\LDAP_BACKUPs\ TestOrg.ldif     

2.       Delete LDAP subtree using idsldapdelete/ldapdelete server utility, in this scenario delete “ou=TestOrg” on LDAP replica server. Note: Take LDAP full backup before deleting this entry using idsdb2ldif utility.

The idsldapdelete/ldapdelete opens a connection to an LDAP server, binds, and deletes one or more entries. If one or more Distinguished Name (DN) arguments are provided, entries with those DNs are deleted. Each DN is a string-represented DN. If no DN arguments are provided, a list of DNs is read from standard input, or from file if the -i or -f flag is used. 

ldapdelete.exe [options] [DNs]

ldapdelete.exe [options] [-i file]

This utility is available under “..\IBM\LDAP\V6.2\bin” folder 

For syntax help, type “idsldapdelete -?” command 

Example: idsldapdelete -D cn=root -w password -s "ou=TestOrg, ou=itim, O=IBM"

3.       Skip all pending change entries and suspend replication on both master and replica LDAP servers using TDS Web Admin GUI or command line utilities.

4.       Import LDIF file which is generated in step 1 using ldif2db/idsldif2db utility 

The ldif2db/idsldif2db utility is used to import entries into LDAP server.  The database must already exist. The idsldif2db can be used to add entries to an empty directory database or to a database that already contains entries.

Note:  Before executing ldif2db/idsldif2db utility, TDS server must be stopped (both administration and instance).  Make sure that no applications are active and attached to the directory.  If applications are running using TDS Server then none of the import utilities will run. 

ldif2db [-i input_file -I instance_name [-f config_file]

        [-d debug_level] [-r yes | no] [-g] [-W output_file]] | -?

            This utility is available under “..\IBM\LDAP\V6.2\sbin” folder 

For syntax help, type “ldif2db -?” command  

Example: idsldif2db -i C:\LDAP_BACKUPs\ TestOrg.ldif 

5.       Resume replication on both master and replica TDS Servers.  To test replication create/update and delete entries and verify on both LDAP servers. 

2. Using Ldapdiff Utility

The idsldapdiff command line utility is designed to compare two directory subtrees on two different directory servers to determine if their contents match. It identifies differences in a replica server and its master and can be used to synchronize replicas.

Idsldapdiff performs two passes to make the servers are in sync. In the first pass, idsldapdiff traverses the Supplier server and does the following: Adds any extra entries on the supplier and to the consumer. Compares and fixes entries that exist on both the servers. In the second pass, idsldapdiff traverses the Consumer to check for any extra entries on the Consumer 

The tool traverses each entry in the directory subtree on the supplier server and compares its contents with the corresponding entry on the consumer server. Thus running the utility can take a long time and can generate lots of read requests to the supplier and consumer servers.  It is recommended to run the utility when no updates are being made to either of the directory servers.

This utility is a diagnostic and corrective tool it is not designed to run as routine maintenance. Depending on the replication-related errors observed in the log files, an administrator might decide to run the utility.

idsldapdiff -sh hostname -sp 389 -sD cn=root -sw password -ch consumerhostname -cp 389 -cD cn=root -cw password -b o=ibm,c=us  -a -F

This utility is available under “..\IBM\LDAP\V6.2\bin” folder 

For syntax help, type “idsldapdiff -?” command  

Example: ldapdiff -b " ou=TestOrg, ou=itim, O=IBM " -sh "mastertds.ibm.com" -ch "replicatds.ibm.com" -sD "cn=root" -sw password -cD "cn=root" -cw password –F -a