[op5-users] Réf. : Re: Réf. : Re: Nagios start time delay with Merlin

[Andreas Ericsson]

nicolas.raspail at bnpparibas.com wrote:
> 
> I will try the new beta. Is a tar file available or this version must be 
> retreive from the git repository ?
> 

You'll need to get it from git, I think. We still haven't gotten around
to providing tar-balls automagically every time we tag. That should
happen in the next few days though, as we're in release-frenzy at the
moment and usually face a slightly calmer period the first week or so
after a major release.

>>>         * nagios has been restarted at timestamp 1251281934 and since 
>>> that, no check have been made until timestamp 1251282809 ! 15 mins of 
>>> update, that is a lot of time.
>> And far, far more than we're experiencing here. Merlin is designed in
>> such a way that it rather drops messages than interferes with the
>> running Nagios daemon, so what you're seeing is almost certainly not
>> a result of Merlin doing something weird.
>>
>> This should be alleviated by upgrading to the latest Merlin version
>> though, since it ignores all events Nagios throws at it until Nagios
>> has entered the main event execution loop. Under such circumstances,
>> the startup time can't be affected at all by Merlin.
>>
> 
> Ok, but without Merlin, my Nagios starts immediately some checks. With 
> NDO,
> there is also a delay (7/8 mins), and the mysql server is very busy during
> this period. Maybe there is a problem else, but I can't find where.
> 

With the latest Merlin, checks should start immediately with merlin too.

> 
>>> Also, During these tests, I have see my check and hosts latencies grow 
> up 
>>> and now, with Merlin enabled, I have a large number of orphaned 
> checks.
>>> With Merlin I have the following latencies (and it is increasing as I 
>>> write my email) :
>>>
>>> Service Check Latency:  0.00 / 1299.86 / 160.156 sec
>>> Host Check Execution Time:      2.54 / 3.19 / 2.564 sec
>>> Host Check Latency:     0.00 / 723.49 / 304.415 sec
>>>
>>> Before Merlin, i don't have the exact values, but I remember that the 
>>> service latency was under 50s and the host latency under 1s
>>>
>> Was the latency slowly increasing before, or was it totally stable?
>> Merlin does add a small overhead to the processing of each check
>> result, status update and a plethora of other things. If your
>> latency was previously increasing slowly, Merlin will make it
>> increase faster. If it was stable before, it's possible that the
>> (very small) overhead that Merlin adds is pushing it over the
>> limit so that the latency starts converging on infinity.
> 
> Before Merlin, the latency was totally stable. I understand why Merlin and 
> NDO add
> a small overhead, but what I'm facing is a huge overhead.
> 
> I have disabled Merlin and enabled NDO to compare. Here is my actual 
> latency with NDO
> after 15/20 minutes :
> 
> Service Check Execution Time:   0.04 / 30.03 / 0.389 sec
> Service Check Latency:  0.00 / 1952.96 / 300.960 sec
> Host Check Execution Time:      2.54 / 2.69 / 2.566 sec
> Host Check Latency:     0.00 / 17.98 / 5.105 sec
> 
> And the latency is still decreasing as I write my email.
> 

Ah, I think I know what's happening now. Merlin schedules a reaper
event once every 5 seconds, where it basically just sits and waits
for input from the daemon. This reaper event can be dropped
completely if no pollers or peers are configured, so I'll make
sure to add something to that effect. This should make Merlin
perform better than NDOUtils, since we're transferring fewer
events and we're transferring them with a more efficient protocol.

-- 
Andreas Ericsson                   andreas.ericsson at op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.