Recently, I’ve been facing an issue more and more frequently where a Repository cannot register itself to a local/remote Docbroker. The root cause of the issue is simply that the hostname used for the projection to the Docbroker is too long which prevents the Repository to properly register to it. I’ve seen it recently because when working with Kubernetes, you can pretty quickly end-up with very very long hostnames… The issue isn’t linked to K8s in itself, it just helps to reproduce the issue. Alternatively, you can do the same thing by just tweeking the /etc/hosts file to lenghten a name that can then be used for the projection.
Alright so let’s demonstrate this issue. The simplest way to reproduce the issue is to have a FQDN that is strictly longer than 59 characters and a short hostname that is shorter:
[dmadmin@documentum-server-0 ~]$ hostname | wc -m 20 [dmadmin@documentum-server-0 ~]$ hostname documentum-server-0 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ hostname -f | wc -m 63 [dmadmin@documentum-server-0 ~]$ hostname -f documentum-server-0.documentum-server.dbi-ns.svc.cluster.local [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ grep "^host=" $DOCUMENTUM/dba/dm_launch_Docbroker host=documentum-server-0 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ dmqdocbroker -t `hostname` -p 1489 -c ping dmqdocbroker: A DocBroker Query Tool dmqdocbroker: Documentum Client Library Version: 16.4.0110.0058 Using specified port: 1489 Successful reply from docbroker at host (documentum-server-0) on port(1490) running software version (16.4.0110.0167 Linux64). [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ dmqdocbroker -t `hostname -f` -p 1489 -c ping dmqdocbroker: A DocBroker Query Tool dmqdocbroker: Documentum Client Library Version: 16.4.0110.0058 Using specified port: 1489 Successful reply from docbroker at host (documentum-server-0) on port(1490) running software version (16.4.0110.0167 Linux64). [dmadmin@documentum-server-0 ~]$
As you can see above, everything seems fine. Let’s see what is the current configuration and if it works properly:
[dmadmin@documentum-server-0 ~]$ grep "^host=" $DOCUMENTUM/dba/dm_launch_Docbroker host=documentum-server-0 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ $DOCUMENTUM/dba/dm_stop_Docbroker ./dmshutdown 16.4.0000.0248 Linux64 Copyright (c) 2018. OpenText Corporation. Shutdown request was processed by Docbroker on host documentum-server-0 (INET_ADDR: 01 2d3 01010164 documentum-server-0 1.1.1.100) Reply status indicates a success: OK [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ $DOCUMENTUM/dba/dm_launch_Docbroker starting connection broker on current host: [documentum-server-0] with connection broker log: [$DOCUMENTUM/dba/log/docbroker.documentum-server-0.1489.log] connection broker pid: 19073 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ cat $DOCUMENTUM/dba/log/docbroker.documentum-server-0.1489.log OpenText Documentum Connection Broker (version 16.4.0110.0167 Linux64) Copyright (c) 2018. OpenText Corporation 2019-11-16T11:30:44.737512 [DM_DOCBROKER_I_START]info: "Docbroker has started. Process id: 19073" 2019-11-16T11:30:44.747745 [DM_DOCBROKER_I_REGISTERED_PORT]info: "The Docbroker registered using port (1489)." 2019-11-16T11:30:44.747797 [DM_DOCBROKER_I_LISTENING]info: "The Docbroker is listening on network address: (INET_ADDR: family: 2, port: 1489, host: documentum-server-0 (1.1.1.100, e01234ac))" 2019-11-16T11:30:44.747812 [DM_DOCBROKER_I_REGISTERED_PORT]info: "The Docbroker registered using port (1490)." 2019-11-16T11:30:44.747828 [DM_DOCBROKER_I_LISTENING]info: "The Docbroker is listening on network address: (INET_ADDR: family: 2, port: 1490, host: documentum-server-0 (1.1.1.100, e01234ac))" [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ grep -A3 "TARGET\]" $DOCUMENTUM/dba/config/gr_repo/server.ini [DOCBROKER_PROJECTION_TARGET] host = documentum-server-0 port = 1489 proximity = 1 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ $DOCUMENTUM/dba/dm_start_gr_repo starting Documentum server for repository: [gr_repo] with server log: [$DOCUMENTUM/dba/log/gr_repo.log] server pid: 19245 [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ grep broker $DOCUMENTUM/dba/log/gr_repo.log 2019-11-16T11:31:48.749813 19245[19245] 0000000000000000 [DM_SERVER_I_START]info: "Sending Initial Docbroker check-point " 2019-11-16T11:31:49.008865 19318[19318] 0112d68780000003 [DM_DOCBROKER_I_PROJECTING]info: "Sending information to Docbroker located on host (documentum-server-0) with port (1490). Information: (Config(gr_repo), Proximity(1), Status(Open), Dormancy Status(Active))." [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ dmqdocbroker -t `hostname` -p 1489 -c getdocbasemap dmqdocbroker: A DocBroker Query Tool dmqdocbroker: Documentum Client Library Version: 16.4.0110.0058 Using specified port: 1489 ************************************************** ** D O C B R O K E R I N F O ** ************************************************** Docbroker host : documentum-server-0 Docbroker port : 1490 Docbroker network address : INET_ADDR: 01 2d3 01010164 documentum-server-0 1.1.1.100 Docbroker version : 16.4.0110.0167 Linux64 ************************************************** ** D O C B A S E I N F O ** ************************************************** -------------------------------------------- Docbase name : gr_repo Docbase id : 1234567 Docbase description : dbi dev k8s gr Govern docbase : Federation name : Server version : 16.4.0110.0167 Linux64.Oracle Docbase Roles : Global Registry Docbase Dormancy Status : -------------------------------------------- [dmadmin@documentum-server-0 ~]$
So with the above short name on both the Docbroker start script and the Repository projection (server.ini), you can see that it’s all working. Now let’s replicate the issue:
[dmadmin@documentum-server-0 ~]$ $DOCUMENTUM/dba/dm_shutdown_gr_repo
Stopping Documentum server for repository: [gr_repo]
OpenText Documentum iapi - Interactive API interface
Copyright (c) 2018. OpenText Corporation
All rights reserved.
Client Library Release 16.4.0110.0058
Connecting to Server using docbase gr_repo.gr_repo
[DM_SESSION_I_SESSION_START]info: "Session 0112d68780012505 started for user dmadmin."
Connected to OpenText Documentum Server running Release 16.4.0110.0167 Linux64.Oracle
Session id is s0
API> shutdown,c,T,T
...
OK
API> exit
Bye
Waiting for 90 seconds for server pid, 19245, to disappear.
Sat Nov 16 11:35:35 UTC 2019: Waiting for shutdown of repository: [gr_repo]
Sat Nov 16 11:35:35 UTC 2019: checking for pid: 19245
Sat Nov 16 11:35:45 UTC 2019: Waiting for shutdown of repository: [gr_repo]
Sat Nov 16 11:35:45 UTC 2019: checking for pid: 19245
repository: [gr_repo] has been shutdown
checking that all children (19251 19253 19254 19275 19297 19319 19362 19387 19570) have shutdown
[dmadmin@documentum-server-0 ~]$
[dmadmin@documentum-server-0 ~]$ grep -A3 "TARGET\]" $DOCUMENTUM/dba/config/gr_repo/server.ini
[DOCBROKER_PROJECTION_TARGET]
host = documentum-server-0
port = 1489
proximity = 1
[dmadmin@documentum-server-0 ~]$
[dmadmin@documentum-server-0 ~]$ sed -i "s,^host.*,host = `hostname -f`," $DOCUMENTUM/dba/config/gr_repo/server.ini
[dmadmin@documentum-server-0 ~]$
[dmadmin@documentum-server-0 ~]$ grep -A3 "TARGET\]" $DOCUMENTUM/dba/config/gr_repo/server.ini
[DOCBROKER_PROJECTION_TARGET]
host = documentum-server-0.documentum-server.dbi-ns.svc.cluster.local
port = 1489
proximity = 1
[dmadmin@documentum-server-0 ~]$
[dmadmin@documentum-server-0 ~]$ grep broker $DOCUMENTUM/dba/log/gr_repo.log
2019-11-16T11:38:34.593108 19715[19715] 0000000000000000 [DM_SERVER_I_START]info: "Sending Initial Docbroker check-point "
2019-11-16T11:38:34.655402 19787[19787] 0112d68780000003 [DM_DOCBROKER_I_PROJECTING]info: "Sending information to Docbroker located on host (documentum-server-0.documentum-server.dbi-ns.svc.cluster.lo) with port (1489). Information: (Config(gr_repo), Proximity(1), Status(Open), Dormancy Status(Active))."
[dmadmin@documentum-server-0 ~]$
The message seems correct, right? You might be fooled by the message but if you look carefully at it, you can actually see that the host in the parenthesis is cut. The value is “documentum-server-0.documentum-server.dbi-ns.svc.cluster.lo” instead of “documentum-server-0.documentum-server.dbi-ns.svc.cluster.local“. That’s 59 characters exactly and nothing more. Since our FQDN is longer, then the Repository cut the hostname and keep only the first 59 characters. If you look at the log file directly of if you include the error messages as well in the grep (with the -i option for example for the uppercase BROKER errors), then you can see the failure:
[dmadmin@documentum-server-0 ~]$ grep -i broker $DOCUMENTUM/dba/log/gr_repo.log 2019-11-16T11:38:34.593108 19715[19715] 0000000000000000 [DM_SERVER_I_START]info: "Sending Initial Docbroker check-point " 2019-11-16T11:38:34.655141 19787[19787] 0112d68780000003 [DM_DOCBROKER_E_NETWORK_ERROR]error: "An error occured performing a network operation: ((112) Error opening connection). Network specific error: ( errno: 0, message: Success)." 2019-11-16T11:38:34.655364 19787[19787] 0112d68780000003 [DM_DOCBROKER_E_CONNECT_FAILED_EX]error: "Unable to connect to DocBroker. Clients please check your dfc.properties file for a correct host. Server please check your server.ini or target attributes in server config. Network address: (INET_ADDR: family: 2, port: 1489, host: documentum-server-0.documentum-server.dbi-ns.svc.cluster.lo (0.0.0.0, 00000000))." 2019-11-16T11:38:34.655402 19787[19787] 0112d68780000003 [DM_DOCBROKER_I_PROJECTING]info: "Sending information to Docbroker located on host (documentum-server-0.documentum-server.dbi-ns.svc.cluster.lo) with port (1489). Information: (Config(gr_repo), Proximity(1), Status(Open), Dormancy Status(Active))." [dmadmin@documentum-server-0 ~]$ [dmadmin@documentum-server-0 ~]$ dmqdocbroker -t `hostname` -p 1489 -c getdocbasemap dmqdocbroker: A DocBroker Query Tool dmqdocbroker: Documentum Client Library Version: 16.4.0110.0058 Using specified port: 1489 ************************************************** ** D O C B R O K E R I N F O ** ************************************************** Docbroker host : Docbroker port : Docbroker network address : Docbroker version : ************************************************** ** D O C B A S E I N F O ** ************************************************** -------------------------------------------- [dmadmin@documentum-server-0 ~]$
So as you can see above, just putting a hostname too long in the server.ini (or inside the Repository) will prevent it to project itself to the Docbroker and therefore you will never be able to work or even stop the Repository (kill is the only option). Therefore, take care that the hostname you are using for the projection isn’t too long.
Cet article Documentum – Beware of long hostnames for the Docbroker est apparu en premier sur Blog dbi services.