Rotate weblogic and managed server log file to a folder in a different mount point/file system.

Rotate weblogic and managed server log file to a folder in a different mount point/file system.

Log Rotation Does Not Happen When Rotated Log Directory And Current Log Are On Two Different File Systems / Mountpoints (Doc ID 1562494.1)

To Bottom

---

In this Document Symptoms Cause Solution References APPLIES TO: Oracle WebLogic Server - Version 10.0 and later Generic Linux Oracle Solaris on SPARC (32-bit) Oracle Solaris on SPARC (64-bit) Oracle Solaris on x86 (32-bit) Oracle Solaris on x86-64 (64-bit) SYMPTOMS One has a WebLogic Server (WLS) log rotation configuration in place such that current log is written to one directory and rotation is done to other directory. However it has been observed that rotation directory does not have the rotated files. Consider following WLS server log configuration for WLS instance "AdminServer" (a snippet from DOMAIN_HOME/config/config.xml): <server>     <name>AdminServer</name>     <log>       <file-name>/u01/current/logs/AdminServer/AdminServer.log</file-name>       <file-min-size>5</file-min-size>       <rotate-log-on-startup>true</rotate-log-on-startup>       <log-file-rotation-dir>/u02/archive/logs/AdminServer</log-file-rotation-dir>       <memory-buffer-severity>Debug</memory-buffer-severity>     </log> In this configuration, one might see that once AdminServer.log reaches 5KB, WLS seems to create a new log in"/u01/current/logs/AdminServer" however the rotated log file which is supposed to be saved into /u02/archive/logs/AdminServer is not there. You might look into standard out file with an intention to see what happened, however there might be no trace of any error there. CAUSE One might be hitting a known limitation of JDK and OS (click here) if current log file exists on one filesystem and the log rotation directory on another filesystem. To verify this, there are two ways: Check whether the file-name and log-file-rotation-dir locations are on two different mount points. Following commands (generic linux) can be used: [apir@mac5.oracle.vm:/home/apir]$ df Filesystem           1K-blocks      Used Available Use% Mounted on /dev/sda1              2030768    320972   1604972  17% / /dev/sda2              4061540     89832   3762064   3% /u01 /dev/sda3              7936256    292584   7234020   4% /u02 #For NFS based mounts: [apir@mac5.oracle.vm:/home/apir]$ /usr/sbin/nfsstat -m  /home from mac4.oracle.vm:/export/home  Flags: rw,vers=3,rsize=1048576,wsize=1048576,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=mac4.oracle.vm ... ... From the "df" output, we can see that /u01 (current log is being written to a subdirectory of this directory) and /u02 (our rotation directory is located in this tree) are two different mount points. Hence the specified JDK limitation is applicable here. Additionally, one can use the "strace" utility (or its equivalent on applicable OS) to trace the system calls and locate the issue. How to use strace: ============== Start the WLS instance which faces the issue. Retrieve the process ID of the WLS instance (for example using "ps -ef | grep java"). Run the strace command as:          strace -f -o /tmp/strace.out -p <pid> Stop the strace once server comes in RUNNING mode Note:"2" and "3" need to be executed before you see the following in the standard out of the server: <xxxxxxxxxxxxxxxxxx> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool> Reviewing the strace output, if we are hitting the limitation we should be able to see following entry: 27199 rename("/u01/current/logs/AdminServer/AdminServer.log", "/u02/archive/logs/AdminServer/AdminServer.log00001") = -1 EXDEV (Invalid cross-device link) SOLUTION Place the current log and the rotation directory on the same file system.