Re: [Collectl-interest] Suggestion: Additional NFS data from /proc/self/mountstats (same as nfsiost

I was wondering it the tool might run something and measure times, clearly
something collectl can't do given the way it currently works - you wouldn't
want it to stall waiting for something.

If you want to get an idea what it takes to write a collectl plugin, have a
look at /usr/share/collectl/misc.ph which is a farily simply plugin that
does a lot of things one could even exclude depending on your needs.  To
use it you simply do something like this:

$ collectl --import misc
waiting for 1 second sample...
#<------Misc------>
# UTim  MHz MT Log
     5 1272  0   5
     5 1800  0   5
     5 1236  0   5

and what you see are the cpu speed, nfs mounts (I actually forgot it did
this ;)) and how many users are logged in.  Naturally being fully
integrated, you can also combine its output with other things collectl
knows about such as like this, noting collectl also has it's own version of
hello world':

$ collectl --import hello:misc -sc
waiting for 1 second sample...
#<----CPU[HYPER]-----><-Hello-><------Misc------>
#cpu sys inter  ctxsw   Total   UTim  MHz MT Log
   2   0  7877  32674     140       5 1218  0   5
   1   0  5161  16135     230       5 1416  0   5


and you can also get the output included in the tab file so you can plot it.

-mark



On Mon, Mar 14, 2016 at 11:45 AM, Thomas Oliw <tho...@er...>
wrote:

> Hi Mark,
>
>
>
> Thanks!
>
>
>
> Yes, another switch.. How I have waited! J
>
>
>
> I looked at /proc/self/mountstats and could not see any timingdata
> either.. So where nfsiostat gets the RTT values is a bit of a mystery.
>
> I did find some references of a nfs-iostat.py script that might give a
> clue.
>
>
> http://git.linux-nfs.org/?p=steved/nfs-utils.git;a=blob;f=tools/nfs-iostat/nfs-iostat.py;h=9626d42609b9485c7fda0c9ef69d698f9fa929fd;hb=HEAD
> )
>
> I think it runs several times and calculates delta?!
>
>
>
> If it helps, this is output from nfsiostat on one of our RedHat 6.5
> servers:
>
> (As you can see the RTT for write operations are very high).
>
>
>
> [root@myserver collectl]# nfsiostat 10 2 /proj/eiffel002_config_fem001
>
>
>
> nfsserv.somedomain.se:/vol/volp01234/data_config mounted on
> /proj/data_config_server001:
>
>
>
>    op/s         rpc bklog
>
> 218.81            0.00
>
> read:             ops/s            kB/s           kB/op
> retrans         avg RTT (ms)    avg exe (ms)
>
>                  12.293         697.798          56.763        0
> (0.0%)           7.972          15.149
>
> write:            ops/s            kB/s           kB/op
> retrans         avg RTT (ms)    avg exe (ms)
>
>                  38.325         1322.385         34.505      392
> (0.0%)          35.313         1237.445
>
>
>
> nfsserv.somedomain.se:/vol/volp01234/data_config mounted on
> /proj/data_config_server001:
>
>
>
>    op/s         rpc bklog
>
>   68.20            0.00
>
> read:             ops/s            kB/s           kB/op
> retrans         avg RTT (ms)    avg exe (ms)
>
>                   0.000           0.000           0.000        0
> (0.0%)           0.000           0.000
>
> write:            ops/s            kB/s           kB/op
> retrans         avg RTT (ms)    avg exe (ms)
>
>                  69.700         4481.492         64.297        0
> (0.0%)         658.139         71380.063
>
>
>
>
>
> I understand that  this is hard to build and test without NFS systems!
>
> I just wanted to throw out the suggestion and see what happens.
>
>
>
> Learning to write a plugin for collectl is tempting. I am not a
> programmer, but have fiddled around with some simple perl scripts in the
> past.
>
> I’ll do some reading on the webpage and try to get an idea of how the
> plugin stuff works.
>
> You’ll never get rid of me if I start.. J
>
>
>
> Kind Regards,
>
>
>
> Thomas
>
>
>
>
>
>
>
> *From:* Mark Seger [mailto:mj...@gm...]
> *Sent:* den 14 mars 2016 14:56
> *To:* Thomas Oliw
> *Cc:* col...@li...
> *Subject:* Re: [Collectl-interest] Suggestion: Additional NFS data from
> /proc/self/mountstats (same as nfsiostat-command)
>
>
>
> Always happy to hear from happy users.
>
>
>
> I just looked at /proc/xx/mountstats, which actually applies to all pids,
> self is just a shortcut to yourself.  The problem with pid-based stats is
> it can be a lot of overhead to read any more stats than collectl already
> reads, but my thought was I might be able to add something optionally.  Oh
> boy, another switch!  ;)
>
>
>
> But when I looks at these stats I did't see anything about timing and only
> saw info on what is mounted.  That said, I'd think since nfs is a shared
> resource, there might be timing data for nfs in generat, but my systems
> currently don't use nfs and I might need to do some experiments to see what
> happens if/when I do configure it.
>
>
>
> Worse case, especially if you're a collectl fan, you might be able to
> write your own plugin if you're a perl user.  The benefit there is once you
> see how easy it is to write a plugin you then might be able to add even
> more metrics, possibly at the application level if you find that useful.
> If so, I'm always ready to help...
>
>
>
> I'm out of town this week but I'll try to revisit next week when I return.
>
>
>
> -mark
>
>
>
> On Mon, Mar 14, 2016 at 8:12 AM, Thomas Oliw <tho...@er...>
> wrote:
>
> Hi,
>
>
>
> I love collectl and use it extensively for many performance related
> troubleshooting/monitoring  tasks in our server park.
>
> The possibility to run live and/or record to file is a fantastic mix of
> features and very useful!
>
>
>
> However, one thing that I miss, is NFS Response time data…
>
> We use lots of NFS shares in our environment, and that particular metric
> is one of the most useful ones in my opinion.
>
>
>
> As a complement to collectl, I use “nfsiostat” when NFS is suspected to be
> a performance bottleneck.
>
> It shows me a number of good metrics and has a “RTT” (Round Trip Time)
> field, that at least gives me a hint of the NFS server responsetime.
>
> If I read the documentation correct, it gets its data from  /proc/self/mountstats.
>
>
>
> I think it would be very useful if those metrics could be collected in collectl as well.
>
> The nfsiostat tool itself is a bit crude, at least in our a bit aged
> RedHat environment and for us it would be convenient to have these metrics
> managed with collectl instead.
>
>
>
> Just a suggestion…
>
> Thanks for the collectl tool!!
>
>
>
> Kind Regards,
>
>
>
> Thomas Oliw
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
> _______________________________________________
> Collectl-interest mailing list
> Col...@li...
> https://lists.sourceforge.net/lists/listinfo/collectl-interest
>
>
>