Classification : Public

UNIX Knowledge
for
WLA
Jawad Ahmed
2008

Classification : Public
Todays topics
File Related Commands
Process Related Commands
Resource Utilization/statistics
Other useful commands
Real-time Usage Scenarios
Script Executions foreground,
background, nohup, cron
Classification : Public
File related commands
cat , more , tail ,touch, mkdir , rm , ln ,
ls , chmod , cp , tar , jar , find , grep
ln s <target> <symblink>
find <dir> name <filename>
tar cvf <tarfile> <dir>/*
tar xvf <tarfile>

Classification : Public
Redirecting stdout , stderr
0 filedescriptor for standard input
1 - filedescriptor for standard output
2 filedescriptor for standard error
./startWeblogic > start.out
./startWeblogic > start.out 2>&1
Classification : Public
append and pipe
./monitorprocess.sh >> proc_mon.out
ps ef | grep <user>
Classification : Public
Process related commands
ps , kill -15/9/3 , pfiles , ptree , pstack
ps fu <user>
ps ef
kill -15 <pid> - soft kill
kill -9 <pid> - hard kill
Kill -3 <pid> - thread dump
pfiles <pid> - file descriptors
ptree <pid> - process tree showing ppids

Classification : Public
Taking Thread dump
nohup ./startWeblogic.sh > start.out 2>&1 &
ps ef|grep startWeblogic.sh (Find processid)
ptree <pid> (Find the child java processid)
kill -3 <pid> (send SIGQUIT to java pid)
Look PROCESS console logfile for thread
dump

Classification : Public
Resource Utilization/Statistics
top , prstat , netstat , iostat , vmstat ,
df , du , swap

Classification : Public
top command
Top command will display the high CPU utilizing processes list

last pid: 16532; load averages: 1.30, 1.15, 0.79
08:09:05
116 processes: 113 sleeping, 1 zombie, 1 stopped, 1 on cpu
CPU states: 92.0% idle, 5.8% user, 2.2% kernel, 0.0% iowait, 0.0%
swap
Memory: 16G real, 6307M free, 11G swap in use, 36G swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
3515 gspace 133 29 10 337M 279M sleep 66.0H 2.09% java
14169 admin 121 0 10 1330M 825M sleep 8:46 1.74% java
10150 admin 121 0 10 1330M 771M sleep 8:14 1.55% java
6076 admin 124 0 10 1351M 701M sleep 5:37 1.32% java
1925 admin 139 0 10 1347M 801M sleep 8:00 1.31% java

Classification : Public
prstat command
prstat -a
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
14169 admin 1330M 825M sleep 0 10 0:00.42 2.1% java/121
14757 admin 1266M 865M cpu546 29 10 0:00.50 1.8% java/135
1925 admin 1348M 801M sleep 0 10 0:00.40 1.7% java/139
10150 admin 1331M 772M sleep 0 10 0:00.42 1.7% java/121
6076 admin 1351M 710M sleep 0 10 0:00.41 1.6% java/124
21947 admin 1266M 789M sleep 0 10 0:00.45 0.8% java/120
27533 admin 1269M 873M sleep 0 10 0:00.46 0.6% java/117
23089 admin 1282M 896M sleep 0 10 0:00.47 0.6% java/120
3515 gspace 337M 279M sleep 29 10 0:02.13 0.1% java/135
NPROC USERNAME SIZE RSS MEMORY TIME CPU
22 admin 11G 7098M 45% 0:06.25 11%
87 root 623M 211M 1.2% 126:21.35 0.2%
2 gspace 458M 343M 2.2% 0:02.17 0.2%
2 logread 3200K 2424K 0.0% 0:00.00 0.0%
1 daemon 2752K 1240K 0.0% 0:00.00 0.0%
Total: 114 processes, 1927 lwps, load averages: 1.27, 1.22, 0.88

http://developers.sun.com/solaris/articles/prstat.html
Classification : Public
Load Average
Load Average gives the load on the
system for the past 1 , 5 and 15 min.
Load Average is a measure of active
processes and Ready Queue (Processes
waiting to execute).
Load Average below 3 is considered
good.
Classification : Public
vmstat command
$ vmstat
procs memory page disk faults cpu
r b w swap free re mf pi po fr de sr s2 s3 s4 s1 in sy cs us sy id
0 0 66 1855624 192024 144 545 307 130 162 0 66 6 6 0 0 53 391 910 36 10 53
Classification : Public
iostat
$ iostat -xt
extended disk statistics tty cpu
disk r/s w/s Kr/s Kw/s wait actv svc_t %w %b tin tout us sy wt id
sd0 2.6 10.0 20.7 22.7 0.1 0.2 59.2 6 19 0 84 3 85 11 0
sd1 4.2 1.0 310.5 8.0 0.0 0.2 47.2 2 23
sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd3 10.2 1.6 51.4 12.8 0.1 0.3 31.2 3 31
Classification : Public
netstat command
Network statistics
$ netstat -na| grep 8110
*.8110 *.* 0 0 24576 0 LISTEN
10.171.1.3.35287 10.171.16.3.8110 32768 0 32768 0 ESTABLISHED
10.171.1.3.8110 10.171.16.3.35287 32768 0 32768 0 ESTABLISHED
10.171.1.3.35291 10.171.16.3.8110 32768 0 32768 0 ESTABLISHED
10.171.1.3.8110 10.171.16.3.35291 32768 0 32768 0 ESTABLISHED

Status could be one of the following:
CLOSED - Closed. The socket is not being used.
LISTEN - Listening for incoming connections.
SYN_SENT - Actively trying to establish connection
SYN_RECEIVED - Initial synchronization of the connection under way
ESTABLISHED - Connection has been established
CLOSE_WAIT - Remote shut down; waiting for the socket to close
FIN_WAIT_1 - Socket closed; shutting down connection
FIN_WAIT_2 - Socket closed; waiting for shutdown from remote.
TIME_WAIT - Wait after close for remote shutdown retransmission.

Classification : Public
df and du commands
$ df -k
Filesystem kbytes used avail capacity Mounted on
/dev/vx/dsk/rootvol 1987399 108466 1819312 6% /
/dev/vx/dsk/usr 1987399 1417772 510006 74% /usr
/proc 0 0 0 0% /proc
fd 0 0 0 0% /dev/fd
mnttab 0 0 0 0% /etc/mnttab
/dev/vx/dsk/var 1987399 1522783 404995 79% /var
swap 938280 16 938264 1% /var/run
swap 939000 736 938264 1% /tmp
/dev/vx/dsk/wkstdg/vol01
3932128 2780812 758104 79% /app01
/dev/vx/dsk/opt 9340301 7525409 1721489 82% /opt
/dev/vx/dsk/wkstdg/vol02
12575932 11043215 275124 98% /export/home

$ du -sk .
498368 .



Classification : Public
swap command

$ swap -s
total: 6799480k bytes allocated + 1737544k reserved = 8537024k used,
940392k available


$ swap -l
swapfile dev swaplo blocks free
/dev/vx/dsk/swapvol 227,8 16 4094704 16
/dev/vx/dsk/wkstdg/extraswap 227,104002 16 8601584 720896


Classification : Public
Other useful commads
[admin@swsweb016 ~]$ whoami
admin

[admin@swsweb016 ~]$ who am i
jawahd pts/3 Jun 3 02:13 (10.209.217.245)

[admin@swsweb016 ~]$ w
08:32:52 up 214 days, 22:20, 1 user, load average: 1.28, 1.33, 1.09
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
jawahd pts/3 10.209.217.245 02:13 0.00s 0.07s 0.01s
sshd: jawahd [priv]

[admin@swsweb016 ~]$ finger
Login Name Tty Idle Login Time Office Office
Phone
jawahd Jawad Ahmed pts/3 Jun 3 02:13
(10.209.217.245)
[admin@swsweb016 ~]$
Classification : Public

Classification : Public
Scenario 1: High Filesystem Utilization
df -k .

du dak . |sort n|tail -50
Classification : Public
Scenario 2: Port conflict
netstat na| grep <port>
getservbyport(int port_number, const
char *protocol_name)
pfiles <pid>|grep sockname |grep port
Classification : Public
Scenario 3: Searching log files
grep i error <logfile>
grep v pattern <logfile>
grep i error <logfile>|wc l
-i option used for ignore case
-v option for all lines except the pattern
Classification : Public
Scenario 4: wrong entry in config file
find <dir> type f exec grep pattern {} \;
-print

To replace the pattern
:1,$ s/oldstring/newstring/g
Classification : Public
Scenario 5: High CPU Utilization
10.1 Run the prstat command on the Java process.
$ prstat -L -p 9499 1 1
PID USERNAME SIZE RSS STATE PRI
NICE TIME CPU PROCESS/LWPID
9499 usera 153M 100M sleep 58 0 0:00.22 0.6% java/8
9499 usera 153M 100M sleep 58 0 0:00.10 0.2% java/10
9499 usera 153M 100M sleep 58 0 0:00.11 0.1% java/9
9499 usera 153M 100M sleep 58 0 0:00.03 0.0% java/5
9499 usera 153M 100M sleep 58 0 0:01.01 0.0% java/1
9499 usera 153M 100M sleep 58 0 0:00.00 0.0% java/12
9499 usera 153M 100M sleep 58 0 0:00.00 0.0% java/11
9499 usera 153M 100M sleep 58 0 0:00.00 0.0% java/14
9499 usera 153M 100M sleep 58 0 0:00.00 0.0% java/13
9499 usera 153M 100M sleep 59 0 0:00.07 0.0% java/7
9499 usera 153M 100M sleep 59 0 0:00.00 0.0% java/6
9499 usera 153M 100M sleep 59 0 0:00.00 0.0% java/4
9499 usera 153M 100M sleep 58 0 0:00.11 0.0% java/3
9499 usera 153M 100M sleep 58 0 0:00.00 0.0% java/2
Classification : Public






10.2 Run the pstack command.
pstack 9499 > 9499_pstack.out

Classification : Public




10.3 Examine the pstack output
for "lwp# 8"
-------- lwp# 8 / thread# 76 --------------------
ff29d190 poll (e2e81548, 0, bb8)
ff24d154 select (0, 0, 0, e2e81548, ff2bf1b4, e2e81548) + 348
ff36b134 select (0, bb8, 7fffffff, fe4c8000, 0, bb8) + 34
fe0f62e4 __1cCosFsleep6FpnGThread_xl_i_ (0, bb8, fe4c8000, 1, 0, 1e2fd8) + 234
fe23f050 JVM_Sleep (2, 0, bb8, fe4de978, fe4c8000, 1e2fd8) + 22c
0008f7ac ???????? (e2e818d4, bb8, 1e2fd8, 984a4, 0, 109a0)
0008c914 ???????? (e2e8194c, 1, fe4d6a80, 98564, 8, e2e81868)
fe5324e8 __1cMStubRoutinesG_code1_ (e2e819d8, e2e81c10, a, f6cb5000, 4, e2e818f0) +
3ec
fe0cbe94
__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGT
hread__v_ (e2e81c08,fe4c8000, e2e81b54, 1e2fd8, 8e764, e2e81c10) +308
fe1f6dbc
__1cJJavaCallsMcall_virtual6FpnJJavaValue_nLKlassHandle_nMsymbolHandlee81c08,
e2e81b54) + 150pnGThread__v_(f6cb64b8, e2e81b40, e2e81b44, fe4c8000, e2d8) +
60e_5pnGThread__v_ (e2e81c08, e2e81c04, e2e81c00,e2e81bf4, e2e81bec, 1e2f8000,
e2e81d10, 1e, e) + 120FpnKJavaThread_pnGThread__v_ (f6817ff8, 1e2fd8, fe4c
7fd70) + 3d8cKJavaThreadDrun6M_v_ (e2e02000, fe4d3e34, fe4c8000, 7fd70, 1e2fd8,
fe213ec8 _start (fe4c8000, fe625d10, 0, 5, 1, fe401000) + 20
ff36b728 _thread_start (1e2fd8, 0, 0, 0, 0, 0) + 40

Classification : Public
10.4 Get a thread dump of the
server
kill -3 9499
Classification : Public
10.5 Map lwp# to Thread#
Since lwp# 8 mapped to thread #76, you can
calculate the hex value of 76 to be 4c.
This maps to nid=0x4c in the JVM thread dump:
"Thread-6" prio=5 tid=0x1e2fd8 nid=0x4c waiting
on monitor [0xe2e81000..0xe2e819d8]
at java.lang.Thread.sleep(Native Method)
at
weblogic.management.deploy.GenericAppPoller
.run(GenericAppPoller.java:139)
Classification : Public
Batch Job Executions
Foreground
Background
nohup
cron jobs
Classification : Public
Foreground Jobs
Shell prompt does not return untill the
execution of job done
Ctrl+c will terminate the job (process)
./startWeblogic
Classification : Public
Background Jobs
Job can be started in background by
giving & at the end
Shell prompt will come back, you can
execute other jobs
Job will terminate once the terminal is
closed
./startWeblogic &
Classification : Public
No hangup
Jobs started with nohup will not
terminate even after closing the
terminal window
nohup writes the status into a file
nohup.out
nohup ./startWeblogic &
Classification : Public
Cron jobs
Cron is a unix utility to schedule jobs.
Cron has a format to specify the job timing
* * * * * Job_Name
minute (0-59), hour (0-23),
day of the month (1-31), month of the year
(1-12),
day of the week (0-6 with 0=Sunday).
crontab l listing cron jobs
crontab e for editing