How to know witch process is running in a particular port on AIX using KDB

Fellas, here is the gig, please, don’t cry! 😀

Suppose your application is not working well (or crashed, or stops and cannot goes online again) because someone install a new software that uses the same port as the application and leave the server, simple like this – believe (it happens). What a cool thing right?

Well, In this example, i will use the port 1334, but, in this question itself we will discuss when you need to remove a socket from a determined port but the rmsock command itself cannot tell you – WITCH PROCESS is being held the process, something like this:

# netstat -Aan |grep *.1334
f1000e0000180bb8 tcp4       0      0  *.1334                *.*                   LISTEN

Normal case:

# rmsock f1000e0000180bb8 tcpcb
The socket 0xf1000e0000180808 is being held by proccess 3211434 (writesrv).

Note that, sometimes AIX can give you all the job listing the PID 3211434, the process writesrv but, sometimes…

Worst case (and that’s what we gonna discuss it now):

# rmsock f1000e0000180bb8 tcpcb
The socket 0xf1000e0000180808 is being held by Kernel/Kernel Extension.

And now i ask you: WTF?

Now, let’s use KDB to starting playing around. Note that the output is too detailed, so you need to take your time to check it out what you need!

PS: If the protocol used by the socket is UDP, you need to change to “inpcb” instead of “tcbcp”.

Back to Life!

# kdb
           START              END 
0000000000001000 00000000058A0000 start+000FD8
F00000002FF47600 F00000002FFDF9C8 __ublock+000000
000000002FF22FF4 000000002FF22FF8 environ+000000
000000002FF22FF8 000000002FF22FFC errno+000000
F1000F0A00000000 F1000F0A10000000 pvproc+000000
F1000F0A10000000 F1000F0A18000000 pvthread+000000
read vscsi_scsi_ptrs OK, ptr = 0x5AB0380
(0)>

Then, you type: sockinfo f1000e0000180bb8 tcpcb

(0)> sockinfo f1000e0000180bb8 tcpcb
---- TCPCB ----(@ F1000E0000180BB8)----
    seg_next......@F1000E0000180BB8  seg_prev......@F1000E0000180BB8  
    t_softerror... 00000000 t_state....... 00000001 (LISTEN)
    t_timer....... 00000000 (TCPT_REXMT)
    t_timer....... 00000000 (TCPT_PERSIST)
    t_timer....... 00000000 (TCPT_KEEP)
    t_timer....... 00000000 (TCPT_2MSL)
    t_rxtshift.... 00000000 t_rxtcur...... 00000006 t_dupacks..... 00000000 
    t_maxseg...... 000005B4 t_force....... 00000000 
    t_flags....... 00000000 ()
    t_oobflags.... 00000000 ()
    t_template....@0000000000000000  t_inpcb.......@F1000E0000180AA0  
    t_iobc........ 00000000 t_timestamp... 22B9A601 snd_una....... 00000000 
    snd_nxt....... 00000000 snd_up........ 00000000 snd_wl1....... 00000000 
    snd_wl2....... 00000000 iss........... 00000000 
    snd_wnd....... 0000000000000000 rcv_wnd....... 0000000000000000 
    rcv_nxt....... 00000000 rcv_up........ 00000000 irs........... 00000000 
    snd_wnd_scale. 00000000 rcv_wnd_scale. 00000000 req_scale_sent 00000000 
    req_scale_rcvd 00000000 last_ack_sent. 00000000 timestamp_rec. 00000000 
    timestamp_age. 00000006 rcv_adv....... 00000000 snd_max....... 00000000 
    snd_cwnd...... 000000003FFFC000        snd_ssthresh.. 000000003FFFC000 
    t_idle........ 00000006 t_rtt......... 00000000 t_rtseq....... 00000000 
    t_srtt........ 00000000 t_rttvar...... 00000006 t_rttmin...... 00000002 
    max_rcvd...... 0000000000000000        max_sndwnd.... 0000000000000000 
    t_peermaxseg.. 000005B4 snd_in_pipe... 00000000 
    sack_data.....@0000000000000000         snd_recover... 00000000 
    snd_high...... 00000000 snd_ecn_max... 00000000 snd_ecn_clear. 00000000 
    t_splice_with.@0000000000000000         t_splice_flags 00000000 


-------- TCB --------- INPCB  INFO ----(@ F1000E0000180AA0)----
    next........@0000000000000000  prev........@0000000000000000  
    head........@00000000061DDC00  faddr_6.....@F1000E0000180AC0  
    iflowinfo... 00000000 fport....... 00000000 fatype...... 00000000 
    oflowinfo... 00000000 lport....... 00000536 latype...... 00000000 
    laddr_6.....@F1000E0000180AD8  socket......@F1000E0000180808  
    ppcb........@F1000E0000180BB8  route_6.....@F1000E0000180AF8  
    ifa.........@0000000000000000  flags....... 00000400 
    proto....... 00000000 tos......... 00000000 ttl......... 0000003C 
(0)> more (^C to quit) ?

Let’s hit ENTER

rcvttl...... 00000000 rcvif.......@0000000000000000  
    options.....@0000000000000000  refcnt...... 00000000 
    lock........ 0000000000000000  rc_lock..... 0000000000000000 
    moptions....@0000000000000000  hash.next...@F1000A002C047D10  
    hash.prev...@F1000A002C047D10  timewait.nxt@0000000000000000  
    timewait.prv@0000000000000000  inp_v6opts  @0000000000000000  
    inp_pmtu....@0000000000000000  inp_fastlo..@0000000000000000  

---- SOCKET INFO ----(@ F1000E0000180808)----
    type........ 0001 (STREAM)
    opts........ 0002 (ACCEPTCONN)
    linger...... 0000 state....... FFFF8080 (PRIV)
    pcb.....@F1000E0000180AA0  proto...@00000000061B5688  
    lock....@F1000E0000166A80  head....@0000000000000000  
    q0......@0000000000000000  q.......@0000000000000000  
    q0len....... 0000 qlen........ 0000 qlimit...... 0005 
    timeo....... 0000 error....... 0000 special..... 0E08 
    pgid.... 0000000000000000  oobmark. 0000000000000000 

snd:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000001000  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF 

MBUF LIST

rcv:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000000001  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000

Hit ENTER again

lockwtg. FFFFFFFFFFFFFFFF  

MBUF LIST

    tpcb....@0000000000000000  fdev_ch.@0000000000000000  
    sec_info@0000000000000000  qos.....@0000000000000000  
    gidlist.@0000000000000000  private.@0000000000000000  
    uid..... 00000000 bufsize. 00000000 threadcnt00000000 
    nextfree@0000000000000000  
    siguid.. 00000000 sigeuid. 00000000 sigpriv. 00000000 
    sndtime. 0000000000000000  sec  0000000000000000  usec 
    rcvtime. 0000000000000000  sec  0000000000000000  usec 
    saioq...@0000000000000000  saioqhd.@0000000000000000  
    accept.. 00000000008F001F  frcatime 00000000 
    isnoflgs 00000000 ()
    rcvlen.. 0000000000000000  frcaback@0000000000000000  
    frcassoc@0000000000000000  frcabckt 0000000000000000 
    iodone.. 00000000          iodonefl 00000000 ()
    ioarg...@0000000000000000  refcnt.. 0000000000000001 
    trclev........... 0001 

proc/fd:  49/3
proc/fd: fd: 3
              SLOT NAME     STATE      PID    PPID          ADSPACE  CL #THS

pvproc+00C400   49*writesrv ACTIVE 03100AA 02E0078 000000081C327400   0 0001



(0)> 

The complete command show this (for this particular process):

PS: Interesting information that you gonna use are in bold.

---- TCPCB ----(@ F1000E0000180BB8)----
    seg_next......@F1000E0000180BB8  seg_prev......@F1000E0000180BB8  
    t_softerror... 00000000 t_state....... 00000001 (LISTEN)
    t_timer....... 00000000 (TCPT_REXMT)
    t_timer....... 00000000 (TCPT_PERSIST)
    t_timer....... 00000000 (TCPT_KEEP)
    t_timer....... 00000000 (TCPT_2MSL)
    t_rxtshift.... 00000000 t_rxtcur...... 00000006 t_dupacks..... 00000000 
    t_maxseg...... 000005B4 t_force....... 00000000 
    t_flags....... 00000000 ()
    t_oobflags.... 00000000 ()
    t_template....@0000000000000000  t_inpcb.......@F1000E0000180AA0  
    t_iobc........ 00000000 t_timestamp... 22B9A601 snd_una....... 00000000 
    snd_nxt....... 00000000 snd_up........ 00000000 snd_wl1....... 00000000 
    snd_wl2....... 00000000 iss........... 00000000 
    snd_wnd....... 0000000000000000 rcv_wnd....... 0000000000000000 
    rcv_nxt....... 00000000 rcv_up........ 00000000 irs........... 00000000 
    snd_wnd_scale. 00000000 rcv_wnd_scale. 00000000 req_scale_sent 00000000 
    req_scale_rcvd 00000000 last_ack_sent. 00000000 timestamp_rec. 00000000 
    timestamp_age. 00000006 rcv_adv....... 00000000 snd_max....... 00000000 
    snd_cwnd...... 000000003FFFC000        snd_ssthresh.. 000000003FFFC000 
    t_idle........ 00000006 t_rtt......... 00000000 t_rtseq....... 00000000 
    t_srtt........ 00000000 t_rttvar...... 00000006 t_rttmin...... 00000002 
    max_rcvd...... 0000000000000000        max_sndwnd.... 0000000000000000 
    t_peermaxseg.. 000005B4 snd_in_pipe... 00000000 
    sack_data.....@0000000000000000         snd_recover... 00000000 
    snd_high...... 00000000 snd_ecn_max... 00000000 snd_ecn_clear. 00000000 
    t_splice_with.@0000000000000000         t_splice_flags 00000000 


-------- TCB --------- INPCB  INFO ----(@ F1000E0000180AA0)----
    next........@0000000000000000  prev........@0000000000000000  
    head........@00000000061DDC00  faddr_6.....@F1000E0000180AC0  
    iflowinfo... 00000000 fport....... 00000000 fatype...... 00000000 
    oflowinfo... 00000000 lport....... 00000536 latype...... 00000000 
    laddr_6.....@F1000E0000180AD8  socket......@F1000E0000180808  
    ppcb........@F1000E0000180BB8  route_6.....@F1000E0000180AF8  
    ifa.........@0000000000000000  flags....... 00000400 
    proto....... 00000000 tos......... 00000000 ttl......... 0000003C 
    rcvttl...... 00000000 rcvif.......@0000000000000000  
    options.....@0000000000000000  refcnt...... 00000000 
    lock........ 0000000000000000  rc_lock..... 0000000000000000 
    moptions....@0000000000000000  hash.next...@F1000A002C047D10  
    hash.prev...@F1000A002C047D10  timewait.nxt@0000000000000000  
    timewait.prv@0000000000000000  inp_v6opts  @0000000000000000  
    inp_pmtu....@0000000000000000  inp_fastlo..@0000000000000000  

---- SOCKET INFO ----(@ F1000E0000180808)----
    type........ 0001 (STREAM)
    opts........ 0002 (ACCEPTCONN)
    linger...... 0000 state....... FFFF8080 (PRIV)
    pcb.....@F1000E0000180AA0  proto...@00000000061B5688  
    lock....@F1000E0000166A80  head....@0000000000000000  
    q0......@0000000000000000  q.......@0000000000000000  
    q0len....... 0000 qlen........ 0000 qlimit...... 0005 
    timeo....... 0000 error....... 0000 special..... 0E08 
    pgid.... 0000000000000000  oobmark. 0000000000000000 

snd:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000001000  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF 

MBUF LIST

rcv:cc...... 0000000000000000  hiwat... 0000000000004000 
    mbcnt... 0000000000000000  mbmax... 0000000000010000 
    lowat... 0000000000000001  mb......@0000000000000000  
    sel.....@0000000000000000  events...... 0000 
    iodone.. 00000000          ioargs..@0000000000000000  
    lastpkt.@0000000000000000  wakeone. FFFFFFFFFFFFFFFF 
    timer...@0000000000000000  timeo... 00000000 
    flags....... 0000 ()
    wakeup.. 00000000          wakearg.@0000000000000000  
    lockwtg. FFFFFFFFFFFFFFFF  

MBUF LIST

    tpcb....@0000000000000000  fdev_ch.@0000000000000000  
    sec_info@0000000000000000  qos.....@0000000000000000  
    gidlist.@0000000000000000  private.@0000000000000000  
    uid..... 00000000 bufsize. 00000000 threadcnt00000000 
    nextfree@0000000000000000  
    siguid.. 00000000 sigeuid. 00000000 sigpriv. 00000000 
    sndtime. 0000000000000000  sec  0000000000000000  usec 
    rcvtime. 0000000000000000  sec  0000000000000000  usec 
    saioq...@0000000000000000  saioqhd.@0000000000000000  
    accept.. 00000000008F001F  frcatime 00000000 
    isnoflgs 00000000 ()
    rcvlen.. 0000000000000000  frcaback@0000000000000000  
    frcassoc@0000000000000000  frcabckt 0000000000000000 
    iodone.. 00000000          iodonefl 00000000 ()
    ioarg...@0000000000000000  refcnt.. 0000000000000001 
    trclev........... 0001 

proc/fd:  49/3
proc/fd: fd: 3
              SLOT NAME     STATE      PID    PPID          ADSPACE  CL #THS

pvproc+00C400   49*writesrv ACTIVE 03100AA 02E0078 000000081C327400   0 0001



(0)>

Where:

– writesrv is the process itself;
– ACTIVE is the state (of course);

And, the MOST IMPORTANT ONE:

– 03100AA, that is the PID in Hex.

If you stills on KDB, you can convert it using kdb function hcal, like this:

(0)> hcal 03100AA
Value hexa: 003100AA          Value decimal: 3211434

(0)> 

Or, You can also use perl to convert it to decimal:

# perl -le 'print hex("03100AA");'
3211434

Tell me if you have a smart way to find the process and pid using a different way, i want to know, and of course, love to share it! Joao Bosco Cortez Filho

How to check VLAN ID number on AIX?

To check VLAN ID on AIX you can issue the command entstat filtering the result as the follow:

# entstat -d en0 | grep 'Port VLAN ID'
Port VLAN ID:   800

Script to show Total, Free and Used Memory on AIX

Just Copy and Paste:

(memory=`prtconf -m | awk 'BEGIN {FS=" "} {print $3/1024}'`
usedmem=`svmon -G | grep memory | awk 'BEGIN {FS=" "} {print $3/256/1024}'`
freemem=`echo $memory-$usedmem | bc -l`
clear
echo
echo "Memory Results:"
echo "----------------------"
echo
echo "Avai Mem: $memory GB"
echo "Free Mem: $freemem GB"
echo "Used Mem: $usedmem GB"
echo
echo)

Result:

Memory Results:
----------------------

Avai Mem: 2 GB
Free Mem: 0.69649 GB
Used Mem: 1.30351 GB

Have something different to Share? Let me know!! Joao Bosco Cortez Filho

Awesome Command to show top 15 processes using memory on AIX

Need to know who is on the top processes using memory on AIX?, here is:

Command:

# svmon -Pt15 | perl -e 'while(<>){print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)}'

Output:

# svmon -Pt15 | perl -e 'while(<>){print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)}'
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
 5636288 rmcd             75559    69009      384    80108      N     Y     N
 6094908 java             43698    14846     4153    51997      N     Y     N
 3801272 java             41497    14821     5718    51391      N     Y     N
 3866846 cimserver        24446    14811     4472    33101      N     Y     N
 5112018 cimprovagt       24249    14775      572    29118      N     Y     N
 6160420 cimlistener      22622    14775     1396    28326      N     Y     N
 6553824 rpc.mountd       21649    14774      401    26376      N     Y     N
 7471188 sshd             21581    14772      384    26269      N     N     N
 3473552 tier1slp         21564    14772     1881    27692      N     N     N
 4456702 IBM.MgmtDomai    21550    14781      440    26383      N     Y     N
 3342532 rpc.statd        21541    14775      457    26329      N     Y     N
 3408040 clcomd           21510    14775      509    26356      N     Y     N
 4915204 IBM.DRMd         21507    14793      672    26583      N     Y     N
 5374150 topasrec         21478    14772      396    26186      N     N     N
 6750398 ksh              21472    14772      384    26096      N     N     N

Any Questions? Please Write Me! Joao Bosco Cortez Filho