A few weeks ago, I had a task to add a new CS into already HA environments (DEV/TEST/PROD) to better support the load on these environments as well as adding a new repository on all Content Servers. These environments were installed a nearly two years ago already so it was really just adding something new into the picture. When doing so, the installation of a new repository on existing Content Servers (CS1 / CS2) was successful and without much trouble (installation in silent obviously so it’s fast & reliable for the CS and RCS) but then the new Remote Content Server (RCS/CFS – CS3) installation, using the same silent scripts, failed for the two existing/old repositories while it succeeded for the new one.
Well actually, the CFS installation didn’t completely fail. The silent installer returned the prompt properly, the repository start/stop scripts were present, the config folder was present, the dm_server_config object was there, aso… So it looked like the installation was successful but, as a best practice, it is really important to always check the log file for a silent installation because it doesn’t show anything on the prompt, even if there are errors. So while checking at the log file after the silent installer returned the prompt, I saw the following:
[dmadmin@content_server_03 ~]$ cd $DM_HOME/install/logs/
[dmadmin@content_server_03 logs]$ cat install.log
15:12:31,830 INFO [main] com.documentum.install.shared.installanywhere.actions.InitializeSharedLibrary - Done InitializeSharedLibrary ...
15:12:31,870 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCfsInitializeImportantServerVariables - The installer is gathering system configuration information.
15:12:31,883 INFO [main] com.documentum.install.server.installanywhere.actions.DiWASilentRemoteServerValidation - Start to verify the password
15:12:33,259 INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /tmp/655905.tmp/dfc.keystore
15:12:33,635 INFO [main] com.documentum.fc.client.security.internal.CreateIdentityCredential$MultiFormatPKIKeyPair - generated RSA (2,048-bit strength) mutiformat key pair in 352 ms
15:12:33,667 INFO [main] com.documentum.fc.client.security.internal.CreateIdentityCredential - certificate created for DFC <CN=dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa,O=EMC,OU=Documentum> valid from Fri Feb 01 15:07:33 UTC 2019 to Mon Jan 29 15:12:33 UTC 2029:
15:12:33,668 INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /tmp/655905.tmp/dfc.keystore
15:12:33,681 INFO [main] com.documentum.fc.client.security.impl.InitializeKeystoreForDfc - [DFC_SECURITY_IDENTITY_INITIALIZED] Initialized new identity in keystore, DFC alias=dfc, identity=dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa
15:12:33,682 INFO [main] com.documentum.fc.client.security.impl.AuthenticationMgrForDfc - identity for authentication is dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa
15:12:33,687 INFO [main] com.documentum.fc.impl.RuntimeContext - DFC Version is 7.3.0040.0025
15:12:33,939 INFO [Timer-2] com.documentum.fc.client.impl.bof.cache.ClassCacheManager$CacheCleanupTask - [DFC_BOF_RUNNING_CLEANUP] Running class cache cleanup task
15:12:34,717 INFO [main] com.documentum.fc.client.impl.connection.docbase.DocbaseConnection - Object protocol version 2
15:12:34,758 INFO [main] com.documentum.fc.client.security.internal.AuthenticationMgr - new identity bundle <dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa 1549033954 content_server_03.dbi-services.com hicAAvU7QX3VNvDft2PwmnW4SIFX+5Snx7PlA5hryuOpo2eWLcEANYAEwYBbU6F3hEBAMenRR/lXFrHFqlrxTZl54whGL+9VnH6CCEu4x8dxdQ+QLRE3EtLlO31SPNhqkzjyVwhktNuivhiZkxweDNynvk+pDleTPvzUvF0YSoggcoiEq+kGr6/c9vUPOMuuv1k7PR1AO05JHmu7vea9/UBaV+TFA6/cGRwVh5i5D2s1Ws7qiDlBl4R+Wp3+TbNLPjbn/SeOz5ZSjAmXThK0H0RXwbcwHo9bVm0Hzu/1n7silII4ZzjAW7dd5Jvbxb66mxC8NWaNabPksus2mTIBhg==>
15:12:35,002 INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /tmp/655905.tmp/dfc.keystore
15:12:35,119 INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: false
15:12:36,317 INFO [main] com.documentum.fc.client.privilege.impl.PublicKeyCertificate - stored certificate for CN
15:12:36,353 INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling in GR_DocBase a new record with this persistent certificate:
-----BEGIN CERTIFICATE-----
MIIDHzCCAgcCELGIh8FYcycggMmImLESjEYwDQYJKoZIhvcNAQELBQAwTjETMBEG
YXZxbFJuN1lRZFlUTXRQNnBWNnpRY3JBYTAeFw0xOTAyMDExNTA3MzNaFw0yOTAx
MjkxNTEyMzNaME4xEzARBgNVBAsMCkRvY3VtZW50dW0xDDAKBgNVBAoMA0VNQzEp
hKnQmaMo/wCv+QXZTCsitrBNvoomcT82mYzwIxV5/7cPCIHHMcJijsJCtunjiucV
MCcGA1UEAwwgZGZjX1VuSWF2cWxSbjdZUWRZVE10UDZwVjZ6UWNyQWEwggEiMA0G
HcL0KUImSV7owDqKzV3lEYCGdomX4gYTI5bMKAiTEuGyWRKw2YTQGhfp5y0mU0hV
ORTYyRoGjpRUuXWpdrsrbX8g8gD9l6ijWTSIWfTGO/7//mTHp2zwp/TiIEuAS+RA
eFw1pBLSCKneYgquMuiyFfuCfBVNY5Q0MzyPHYxrDAp4CtjasIrNT5h3AgMBAAEw
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC4Hli+niUAD0ksVVWocPnvzV10ZOj2
DQYJKoZIhvcNAQELBQADggEBAEAre45NEpqzGMMYX1zpjgib9wldSmiPVDZbhj17
KnUCgDy7FhFQ5U5w6wf2iO9UxGV42AYQe2TjED0EbYwpYB8DC970J2ZrjZRFMy/Y
A1UECwwKRG9jdW1lbnR1bTEMMAoGA1UECgwDRU1DMSkwJwYDVQQDDCBkZmNfVW5J
gwKynVf9O10GQP0a8Z6Fr3jrtCEzfLjOXN0VxEcgwOEKRWHM4auxjevqGCPegD+y
FVWwylyIsMRsC9hOxoNHZPrbhk3N9Syhqsbl+Z9WXG0Sp4uh1z5R1NwVhR7YjZkF
19cfN8uEHqedJo26lq7oFF2KLJ+/8sWrh2a6lrb4fNXYZIAaYKjAjsUzcejij8en
Rd8yvghCc4iwWvpiRg9CW0VF+dXg6KkQmaFjiGrVosskUjuACHncatiYC5lDNJy+
TDdnNWYlctfWcT8WL/hX6FRGedT9S30GShWJNobM9vECoNg=
-----END CERTIFICATE-----
15:12:36,355 INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: false
15:12:36,535 INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling a new registration record for dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa
15:12:36,563 INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - [DFC_SECURITY_GR_REGISTRATION_PUBLISH] this dfc instance is now published in the global registry GR_DocBase
15:12:37,513 INFO [main] com.documentum.fc.client.impl.connection.docbase.DocbaseConnection - Object protocol version 2
15:12:38,773 INFO [main] com.documentum.fc.client.impl.connection.docbase.DocbaseConnection - Object protocol version 2
15:12:39,314 INFO [main] com.documentum.install.shared.common.services.dfc.DiDfcProperties - Installer is adding it as primary connection broker and moves existing primary as backup.
15:12:41,643 INFO [main] - The installer updates dfc.properties file.
15:12:41,644 INFO [main] com.documentum.install.shared.common.services.dfc.DiDfcProperties - Installer is adding it as primary connection broker and moves existing primary as backup.
15:12:41,649 INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerEnableLockBoxValidation - The installer will validate AEK/Lockbox fileds.
15:12:41,656 INFO [main] com.documentum.install.shared.common.services.dfc.DiDfcProperties - Installer is changing primary as backup and backup as primary.
15:12:43,874 INFO [main] - The installer updates dfc.properties file.
15:12:43,874 INFO [main] com.documentum.install.shared.common.services.dfc.DiDfcProperties - Installer is changing primary as backup and backup as primary.
15:12:43,876 INFO [main] - The installer is creating folders for the selected repository.
15:12:43,876 INFO [main] - Checking if cfs is being installed on the primary server...
15:12:43,877 INFO [main] - CFS is not being installed on the primary server
15:12:43,877 INFO [main] - Installer creates necessary directory structure.
15:12:43,879 INFO [main] - Installer copies aek.key, server.ini, dbpasswd.txt and webcache.ini files from primary server.
15:12:43,881 INFO [main] - Installer executes dm_rcs_copyfiles.ebs to get files from primary server
15:12:56,295 INFO [main] - $DOCUMENTUM/dba/config/DocBase1/dbpasswd.txt has been created successfully
15:12:56,302 INFO [main] - $DOCUMENTUM/dba/config/DocBase1/webcache.ini has been created successfully
15:12:56,305 INFO [main] - Installer found exising file $DOCUMENTUM/dba/secure/lockbox.lb
15:12:56,305 INFO [main] - Installer renamed exising file $DOCUMENTUM/dba/secure/lockbox.lb to $DOCUMENTUM/dba/secure/lockbox.lb.bak.3
15:12:56,306 INFO [main] - $DOCUMENTUM/dba/secure/lockbox.lb has been created successfully
15:12:56,927 INFO [main] - $DOCUMENTUM/dba/config/DocBase1/server_content_server_03_DocBase1.ini has been created successfully
15:12:56,928 INFO [main] - Installer found exising file $DOCUMENTUM/dba/castore_license
15:12:56,928 INFO [main] - Installer renamed exising file $DOCUMENTUM/dba/castore_license to $DOCUMENTUM/dba/castore_license.bak.3
15:12:56,928 INFO [main] - $DOCUMENTUM/dba/castore_license has been created successfully
15:12:56,931 INFO [main] - $DOCUMENTUM/dba/config/DocBase1/ldap_080f123450006deb.cnt has been created successfully
15:12:56,934 INFO [main] - Installer updates server.ini
15:12:56,940 INFO [main] - The installer tests database connection.
15:12:57,675 INFO [main] - Database successfully opened.
Test table successfully created.
Test view successfully created.
Test index successfully created.
Insert into table successfully done.
Index successfully dropped.
View successfully dropped.
Database case sensitivity test successfully past.
Table successfully dropped.
15:13:00,675 INFO [main] - The installer creates server config object.
15:13:00,853 INFO [main] - The installer is starting a process for the repository.
15:13:01,993 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCreateContentFileServerPostSeq - logPath is $DOCUMENTUM/dba/log/content_server_03_DocBase1.log
15:13:03,079 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCreateContentFileServerPostSeq - logPath is $DOCUMENTUM/dba/log/content_server_03_DocBase1.log
15:13:04,149 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCreateContentFileServerPostSeq - logPath is $DOCUMENTUM/dba/log/content_server_03_DocBase1.log
15:13:05,187 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCreateContentFileServerPostSeq - logPath is $DOCUMENTUM/dba/log/content_server_03_DocBase1.log
15:13:06,256 INFO [main] com.documentum.install.multinode.cfs.installanywhere.actions.DiWAServerCreateContentFileServerPostSeq - logPath is $DOCUMENTUM/dba/log/content_server_03_DocBase1.log
15:14:06,352 INFO [main] - Waiting for repository DocBase1.content_server_03_DocBase1 to start up.
15:14:25,003 INFO [main] com.documentum.fc.client.impl.connection.docbase.DocbaseConnection - Object protocol version 2
15:14:25,495 INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /tmp/655905.tmp/dfc.keystore
15:14:25,498 INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /tmp/655905.tmp/dfc.keystore
15:14:25,513 INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: true
15:14:25,672 INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - assigning rights to all roles for this client on DocBase1
15:14:25,682 INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - found client rights: false
15:14:25,736 INFO [main] com.documentum.fc.client.privilege.impl.PublicKeyCertificate - stored certificate for CN
15:14:25,785 INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling in DocBase1 a new record with this persistent certificate:
-----BEGIN CERTIFICATE-----
MIIDHzCCAgcCELGIh8FYcycggMmImLESjEYwDQYJKoZIhvcNAQELBQAwTjETMBEG
YXZxbFJuN1lRZFlUTXRQNnBWNnpRY3JBYTAeFw0xOTAyMDExNTA3MzNaFw0yOTAx
MjkxNTEyMzNaME4xEzARBgNVBAsMCkRvY3VtZW50dW0xDDAKBgNVBAoMA0VNQzEp
hKnQmaMo/wCv+QXZTCsitrBNvoomcT82mYzwIxV5/7cPCIHHMcJijsJCtunjiucV
MCcGA1UEAwwgZGZjX1VuSWF2cWxSbjdZUWRZVE10UDZwVjZ6UWNyQWEwggEiMA0G
HcL0KUImSV7owDqKzV3lEYCGdomX4gYTI5bMKAiTEuGyWRKw2YTQGhfp5y0mU0hV
ORTYyRoGjpRUuXWpdrsrbX8g8gD9l6ijWTSIWfTGO/7//mTHp2zwp/TiIEuAS+RA
eFw1pBLSCKneYgquMuiyFfuCfBVNY5Q0MzyPHYxrDAp4CtjasIrNT5h3AgMBAAEw
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC4Hli+niUAD0ksVVWocPnvzV10ZOj2
DQYJKoZIhvcNAQELBQADggEBAEAre45NEpqzGMMYX1zpjgib9wldSmiPVDZbhj17
KnUCgDy7FhFQ5U5w6wf2iO9UxGV42AYQe2TjED0EbYwpYB8DC970J2ZrjZRFMy/Y
A1UECwwKRG9jdW1lbnR1bTEMMAoGA1UECgwDRU1DMSkwJwYDVQQDDCBkZmNfVW5J
gwKynVf9O10GQP0a8Z6Fr3jrtCEzfLjOXN0VxEcgwOEKRWHM4auxjevqGCPegD+y
FVWwylyIsMRsC9hOxoNHZPrbhk3N9Syhqsbl+Z9WXG0Sp4uh1z5R1NwVhR7YjZkF
19cfN8uEHqedJo26lq7oFF2KLJ+/8sWrh2a6lrb4fNXYZIAaYKjAjsUzcejij8en
Rd8yvghCc4iwWvpiRg9CW0VF+dXg6KkQmaFjiGrVosskUjuACHncatiYC5lDNJy+
TDdnNWYlctfWcT8WL/hX6FRGedT9S30GShWJNobM9vECoNg=
-----END CERTIFICATE-----
15:14:25,789 INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: true
15:14:25,802 INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - found client rights: false
15:14:25,981 INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling a new rights record for dfc_UnYQdYTP6pV6zRn7tQMIavqlcrAa
15:14:26,032 INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - [DFC_SECURITY_DOCBASE_RIGHTS_REGISTER] this dfc instance has now escalation rights registered with docbase DocBase1
15:14:26,052 INFO [main] com.documentum.install.appserver.jboss.JbossApplicationServer - setApplicationServer sharedDfcLibDir is:$DOCUMENTUM/shared/dfc
15:14:26,052 INFO [main] com.documentum.install.appserver.jboss.JbossApplicationServer - getFileFromResource for templates/appserver.properties
15:14:26,059 INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerAddDocbaseEntryToWebXML - BPM webapp does not exist.
15:14:26,191 INFO [main] com.documentum.install.server.installanywhere.actions.cfs.DiWAServerProcessingScripts2 - Executing the Docbase HeadStart script.
15:14:36,202 INFO [main] com.documentum.install.server.installanywhere.actions.cfs.DiWAServerProcessingScripts2 - Executing the Creates ACS config object script.
15:14:46,688 INFO [main] com.documentum.install.server.installanywhere.actions.cfs.DiWAServerProcessingScripts2 - Executing the This script does miscellaneous setup tasks for remote content servers script.
15:14:56,840 ERROR [main] com.documentum.install.server.installanywhere.actions.cfs.DiWAServerProcessingScripts2 - The installer failed to execute the This script does miscellaneous setup tasks for remote content servers script. For more information, please read output file: $DOCUMENTUM/dba/config/DocBase1/dm_rcs_setup.out.
com.documentum.install.shared.common.error.DiException: The installer failed to execute the This script does miscellaneous setup tasks for remote content servers script. For more information, please read output file: $DOCUMENTUM/dba/config/DocBase1/dm_rcs_setup.out.
at com.documentum.install.server.installanywhere.actions.cfs.DiWAServerProcessingScripts2.setup(DiWAServerProcessingScripts2.java:98)
at com.documentum.install.shared.installanywhere.actions.InstallWizardAction.install(InstallWizardAction.java:75)
at com.zerog.ia.installer.actions.CustomAction.installSelf(Unknown Source)
at com.zerog.ia.installer.AAMgrBase.an(Unknown Source)
at com.zerog.ia.installer.ConsoleBasedAAMgr.ac(Unknown Source)
at com.zerog.ia.installer.AAMgrBase.am(Unknown Source)
at com.zerog.ia.installer.AAMgrBase.runNextInstallPiece(Unknown Source)
...
at com.zerog.ia.installer.ConsoleBasedAAMgr.ac(Unknown Source)
at com.zerog.ia.installer.AAMgrBase.runPreInstall(Unknown Source)
at com.zerog.ia.installer.LifeCycleManager.consoleInstallMain(Unknown Source)
at com.zerog.ia.installer.LifeCycleManager.executeApplication(Unknown Source)
at com.zerog.ia.installer.Main.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.zerog.lax.LAX.launch(Unknown Source)
at com.zerog.lax.LAX.main(Unknown Source)
15:14:56,843 INFO [main] - The INSTALLER_UI value is SILENT
15:14:56,843 INFO [main] - The KEEP_TEMP_FILE value is true
15:14:56,843 INFO [main] - The common.installOwner.password value is ******
15:14:56,843 INFO [main] - The SERVER.SECURE.ROOT_PASSWORD value is ******
15:14:56,843 INFO [main] - The common.upgrade.aek.lockbox value is null
15:14:56,843 INFO [main] - The common.old.aek.passphrase.password value is null
15:14:56,843 INFO [main] - The common.aek.algorithm value is AES_256_CBC
15:14:56,843 INFO [main] - The common.aek.passphrase.password value is ******
15:14:56,843 INFO [main] - The common.aek.key.name value is CSaek
15:14:56,843 INFO [main] - The common.use.existing.aek.lockbox value is null
15:14:56,843 INFO [main] - The SERVER.ENABLE_LOCKBOX value is true
15:14:56,844 INFO [main] - The SERVER.LOCKBOX_FILE_NAME value is lockbox.lb
15:14:56,844 INFO [main] - The SERVER.LOCKBOX_PASSPHRASE.PASSWORD value is ******
15:14:56,844 INFO [main] - The SERVER.COMPONENT_ACTION value is CREATE
15:14:56,844 INFO [main] - The SERVER.DOCBROKER_ACTION value is null
15:14:56,844 INFO [main] - The SERVER.PRIMARY_CONNECTION_BROKER_HOST value is content_server_01.dbi-services.com
15:14:56,844 INFO [main] - The SERVER.PRIMARY_CONNECTION_BROKER_PORT value is 1489
15:14:56,844 INFO [main] - The SERVER.PROJECTED_CONNECTION_BROKER_HOST value is content_server_03.dbi-services.com
15:14:56,844 INFO [main] - The SERVER.PROJECTED_CONNECTION_BROKER_PORT value is 1489
15:14:56,844 INFO [main] - The SERVER.FQDN value is content_server_03.dbi-services.com
15:14:56,845 INFO [main] - The SERVER.DOCBASE_NAME value is DocBase1
15:14:56,845 INFO [main] - The SERVER.PRIMARY_SERVER_CONFIG_NAME value is DocBase1
15:14:56,845 INFO [main] - The SERVER.REPOSITORY_USERNAME value is dmadmin
15:14:56,845 INFO [main] - The SERVER.SECURE.REPOSITORY_PASSWORD value is ******
15:14:56,845 INFO [main] - The SERVER.REPOSITORY_USER_DOMAIN value is
15:14:56,845 INFO [main] - The SERVER.REPOSITORY_USERNAME_WITH_DOMAIN value is dmadmin
15:14:56,845 INFO [main] - The SERVER.REPOSITORY_HOSTNAME value is content_server_01.dbi-services.com
15:14:56,845 INFO [main] - The SERVER.CONNECTION_BROKER_NAME value is null
15:14:56,845 INFO [main] - The SERVER.CONNECTION_BROKER_PORT value is null
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_NAME value is
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_PORT value is
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_CONNECT_MODE value is null
15:14:56,846 INFO [main] - The SERVER.USE_CERTIFICATES value is false
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_KEYSTORE_FILE_NAME value is null
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_KEYSTORE_PASSWORD_FILE_NAME value is null
15:14:56,846 INFO [main] - The SERVER.DOCBROKER_CIPHER_LIST value is null
15:14:56,853 INFO [main] - The SERVER.DFC_SSL_TRUSTSTORE value is null
15:14:56,853 INFO [main] - The SERVER.DFC_SSL_TRUSTSTORE_PASSWORD value is ******
15:14:56,853 INFO [main] - The SERVER.DFC_SSL_USE_EXISTING_TRUSTSTORE value is null
15:14:56,853 INFO [main] - The SERVER.CONNECTION_BROKER_SERVICE_STARTUP_TYPE value is null
15:14:56,854 INFO [main] - The SERVER.DOCUMENTUM_DATA value is $DATA
15:14:56,854 INFO [main] - The SERVER.DOCUMENTUM_SHARE value is $DOCUMENTUM/share
15:14:56,854 INFO [main] - The CFS_SERVER_CONFIG_NAME value is content_server_03_DocBase1
15:14:56,854 INFO [main] - The SERVER.DOCBASE_SERVICE_NAME value is DocBase1
15:14:56,854 INFO [main] - The CLIENT_CERTIFICATE value is null
15:14:56,854 INFO [main] - The RKM_PASSWORD value is ******
15:14:56,854 INFO [main] - The SERVER.DFC_BOF_GLOBAL_REGISTRY_VALIDATE_OPTION_IS_SELECTED value is null
15:14:56,854 INFO [main] - The SERVER.PROJECTED_DOCBROKER_PORT_OTHER value is null
15:14:56,854 INFO [main] - The SERVER.PROJECTED_DOCBROKER_HOST_OTHER value is null
15:14:56,854 INFO [main] - The SERVER.GLOBAL_REGISTRY_REPOSITORY value is null
15:14:56,854 INFO [main] - The SERVER.BOF_REGISTRY_USER_LOGIN_NAME value is null
15:14:56,855 INFO [main] - The SERVER.SECURE.BOF_REGISTRY_USER_PASSWORD value is ******
15:14:56,855 INFO [main] - The SERVER.COMPONENT_ACTION value is CREATE
15:14:56,855 INFO [main] - The SERVER.COMPONENT_NAME value is null
15:14:56,855 INFO [main] - The SERVER.DOCBASE_NAME value is DocBase1
15:14:56,855 INFO [main] - The SERVER.CONNECTION_BROKER_NAME value is null
15:14:56,855 INFO [main] - The SERVER.CONNECTION_BROKER_PORT value is null
15:14:56,855 INFO [main] - The SERVER.PROJECTED_CONNECTION_BROKER_HOST value is content_server_03.dbi-services.com
15:14:56,855 INFO [main] - The SERVER.PROJECTED_CONNECTION_BROKER_PORT value is 1489
15:14:56,855 INFO [main] - The SERVER.PRIMARY_SERVER_CONFIG_NAME value is DocBase1
15:14:56,855 INFO [main] - The SERVER.DOCBROKER_NAME value is
15:14:56,856 INFO [main] - The SERVER.DOCBROKER_PORT value is
15:14:56,856 INFO [main] - The SERVER.CONNECTION_BROKER_SERVICE_STARTUP_TYPE value is null
15:14:56,856 INFO [main] - The SERVER.REPOSITORY_USERNAME value is dmadmin
15:14:56,856 INFO [main] - The SERVER.REPOSITORY_PASSWORD value is ******
15:14:56,856 INFO [main] - The SERVER.REPOSITORY_USER_DOMAIN value is
15:14:56,856 INFO [main] - The SERVER.REPOSITORY_USERNAME_WITH_DOMAIN value is dmadmin
15:14:56,856 INFO [main] - The SERVER.DFC_BOF_GLOBAL_REGISTRY_VALIDATE_OPTION_IS_SELECTED_KEY value is null
15:14:56,856 INFO [main] - The SERVER.PROJECTED_DOCBROKER_PORT_OTHER value is null
15:14:56,856 INFO [main] - The SERVER.PROJECTED_DOCBROKER_HOST_OTHER value is null
15:14:56,856 INFO [main] - The SERVER.GLOBAL_REGISTRY_REPOSITORY value is null
15:14:56,856 INFO [main] - The SERVER.BOF_REGISTRY_USER_LOGIN_NAME value is null
15:14:56,856 INFO [main] - The SERVER.SECURE.BOF_REGISTRY_USER_PASSWORD value is ******
15:14:56,856 INFO [main] - The SERVER.COMPONENT_ACTION value is CREATE
15:14:56,857 INFO [main] - The SERVER.COMPONENT_NAME value is null
15:14:56,857 INFO [main] - The SERVER.PRIMARY_SERVER_CONFIG_NAME value is DocBase1
15:14:56,857 INFO [main] - The SERVER.DOCBASE_NAME value is DocBase1
15:14:56,857 INFO [main] - The SERVER.REPOSITORY_USERNAME value is dmadmin
15:14:56,857 INFO [main] - The SERVER.REPOSITORY_PASSWORD value is ******
15:14:56,857 INFO [main] - The SERVER.REPOSITORY_USER_DOMAIN value is
15:14:56,857 INFO [main] - The SERVER.REPOSITORY_USERNAME_WITH_DOMAIN value is dmadmin
15:14:56,857 INFO [main] - The env PATH value is: /usr/xpg4/bin:$DOCUMENTUM/shared/java64/JAVA_LINK/bin:$DM_HOME/bin:$DOCUMENTUM/dba:$ORACLE_HOME/bin:$DOCUMENTUM/shared/java64/JAVA_LINK/bin:$DM_HOME/bin:$DOCUMENTUM/dba:$ORACLE_HOME/bin:$DM_HOME/bin:$ORACLE_HOME/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/dmadmin/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin
[dmadmin@content_server_03 logs]$
As you can see above, everything was going well until the script “This script does miscellaneous setup tasks for remote content servers” is executed. Yes that is a hell of a description, isn’t it? What this script is doing is actually running the “dm_rcs_setup.ebs” script (you can find it under $DM_HOME/install/admin/) on the repository to setup the remote jobs, project the RCS/CFS repository to the local docbroker, create the log folder and a few other things. Here was the content of the output file for the execution of this EBS:
[dmadmin@content_server_03 logs]$ cat $DOCUMENTUM/dba/config/DocBase1/dm_rcs_setup.out
Running dm_rcs_setup.ebs script on docbase DocBase1.content_server_03_DocBase1 to set up jobs for a remote content server.
docbaseNameOnly = DocBase1
Connected To DocBase1.content_server_03_DocBase1
$DOCUMENTUM/dba/log/000f1234/sysadmin was created.
Duplicating distributed jobs.
Creating job object for dm_ContentWarningcontent_server_03_DocBase1
Successfully created job object for dm_ContentWarningcontent_server_03_DocBase1
Creating job object for dm_LogPurgecontent_server_03_DocBase1
Successfully created job object for dm_LogPurgecontent_server_03_DocBase1
Creating job object for dm_ContentReplicationcontent_server_03_DocBase1
Successfully created job object for dm_ContentReplicationcontent_server_03_DocBase1
Creating job object for dm_DMCleancontent_server_03_DocBase1
The dm_DMClean job does not exist at the primary server so we will not create it at the remote site, either.
Failed to create job object for dm_DMCleancontent_server_03_DocBase1
[DM_API_E_BADID]error: "Bad ID given: 0000000000000000"
[DM_API_E_BADID]error: "Bad ID given: 0000000000000000"
[DM_API_E_BADID]error: "Bad ID given: 0000000000000000"
[DM_API_E_BADID]error: "Bad ID given: 0000000000000000"
[DM_API_E_NO_MATCH]error: "There was no match in the docbase for the qualification: dm_job where object_name = 'dm_DMClean' and lower(target_server) like lower('DocBase1.DocBase1@%')"
Exiting with return code (-1)
[dmadmin@content_server_03 logs]$
[dmadmin@content_server_03 logs]$
The RCS/CFS installation is failing because the creation of a remote job cannot complete successfully. It’s working properly for 3 out of the 5 remote jobs but not for the 2 remaining. Only one is shown in the log file because it didn’t even try to process the 2nd one since it failed already and therefore stopped the installation here. That’s why the start/stop scripts were there, the log folder was there, the dm_server_config was ok as well but there were some missing pieces actually.
The issue here is that the RCS/CFS installation isn’t able to find the r_object_id of the “dm_DMClean” job (it mention “Bad ID given: 0000000000000000”) and therefore it’s not able to create the remote job. The last message is actually more interesting: “There was no match in the docbase for the qualification: dm_job where object_name = ‘dm_DMClean’ and lower(target_server) like lower(‘DocBase1.DocBase1@%’)”.
The RCS/CFS installation is actually looking at the job with the name ‘dm_DMClean’, which is OK but it is also filtering only on the target_server which is equal to ‘docbase_name.server_config_name@…’ and here, it’s not finding any result.
So what happened? Like I was saying in the introduction, this environment was already installed several years ago in HA already. As a result of that, the jobs were already configured by us as we would expect them. Usually, we are configuring the jobs as follow (I’m only talking about the distributed jobs here):
| Job Name on CS1 | Job Status on CS1 | Job Name on RCS% | Job Status on RCS% |
| dm_ContentWarning | Active | dm_ContentWarning% | Inactive |
| dm_LogPurge | Active | dm_LogPurge% | Active |
| dm_DMClean | Active | dm_DMClean% | Inactive |
| dm_DMFilescan | Active | dm_DMFilescan% | Inactive |
| dm_ContentReplication | Inactive | dm_ContentReplication% | Inactive |
Based on this, we usually disable the dm_ContentReplication completely (if it’s not needed), we obviously leave the dm_LogPurge enabled (all of them) with the target_server set to the local CS it is supposed to run into (so 1 job per CS). Then for the 3 remaining jobs, it depends on the load of the environment. These jobs can be set to run on the CS1 by setting the target_server equal to ‘DocBase1.DocBase1@content_server_03.dbi-services.com’ or you can set them to run on ANY Content Server by setting an empty target_server (a single space: ‘ ‘). It doesn’t matter where they are running but it is important for these jobs to run and hence the setting to ANY available Content Server is better so it’s not bound to a single point of failure.
So the reason why the RCS/CFS installation failed is because we configured our jobs properly… Funny, right? As you could see in the logs, the dm_ContentWarning was created properly but that was because someone was doing some testing with this job and it was temporarily set to run on the CS1 only and therefore, when the installer checked it, it was a coincidence/luck that it could find it.
After the failure, there is normally not much done except creating the JMS config object, checking the ACS URLs and finally restarting the JMS but still, it is cleaner to just remove the RCS/CFS, clean the repository objects still remaining (the distributed jobs that were created) and then reinstalling the RCS/CFS after setting the jobs as the installer expects them to be…
Cet article Documentum – RCS/CFS installation failure est apparu en premier sur Blog dbi services.