v4.4.2 awefully slow

POP Peeper: Tech support, suggestions, discussion, etc.
Post Reply
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

v4.4.2 awefully slow

Post by henry66 »

jeff, just updated to the latest version, and all works, even the often problem-causing YAHOO accounts.

but but but: CHECK MAIL all accounts (on a really fast internet line) has become awefully slow, the counting down mssg at the bottom ''<account> PROCESSING DATA'' stays forever. what used to be seconds is now, same accounts, several minutes!

anything i could do/change/adjust/reset?

greetings - henry
User avatar
mjs
Moderator
Posts: 2216
Joined: Sun Jul 17, 2011 2:36 am

Re: v4.4.2 awefully slow

Post by mjs »

Hi Henry, just a few quick questions to cover until Jeff can get by the forum (a few general questions first):

1. Did you try rebooting your system after updating POP Peeper (this isn't normally necessary but is something that can sometimes help in this situation)?

2. What is the Operating System that you are using?

3. What ISP (Internet Provider) are you using?

4. What type of account(s) do you have in POP Peeper? (Outlook, Yahoo, Gmail etc)

5. What "Server type" are you using for these account(s)? (i.e. "Webmail", IMAP or POP3)

More specific to your question regarding your slow Mail Check:

6. What is your "Maximum simultaneous threads" setting at? (in main menu under "Tools" > "Options", select "Mail Check" - if you have a lot of accounts you could increase this number, a general rule of thumb for a minimum value here would be to take your number of accounts and divide by 3 or 4 for this setting)

7. Also in the same "Options", select "Message Retrieval" to see what your "Default retrieval options" are set at (i.e. Entire Message, Headers, etc etc, respective to the type of account(s) you have)?

8. Also in the same "Options", select "Connection" to check what you have set for your "Connection Type" setting?

Thanks for the information. :wink:
Good judgment comes from experience and a lot of that comes from bad judgment. - Will Rogers
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

lakrsrool, tks for jumping in. many of your questions make sense, but might now not really be needed: i have installed an older version (4.3) on my wifes latop, all the very same settings and accounts, and the difference is more than obvious - minutes, not seconds.

- 1. Did you try rebooting ...

hah, many times. i only write a mssg after i try everything possible on my side.

- 2. What is the Operating System

WIN7prof 64 on both

- 3. What ISP (Internet Provider) are you using?

several tried, no difference at all. i am in europe, i can even switch countries.

- 4. What type of account(s) do you have in POP Peeper?

mixture of all. but no outlook: YAHOO, GMAIL, POP3, IMAP - the accounts us such work fine on both systems - 14 accounts alltogether

- 6. What is your "Maximum simultaneous threads" setting at?

5 in both of them, seems to be the default.

- 7 - "Default retrieval options"

set to 'Entire Message' on all

- 8. "Connection Type" setting?

permanent in both.

not sure this info will help. the system was fast untill i updated two days ago, and the parallel test system is also fine.

greetings - henry
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

one addon: collecting mssgs initialy works equally fast on both sytems, and seem, specially with large attachments, only be governed by the actual internet speed.

it really only is that PROCESSING DATA phase that slows everything down. if i am honest, i dont even recall that one showing up on the older versions.

greets - henry
User avatar
mjs
Moderator
Posts: 2216
Joined: Sun Jul 17, 2011 2:36 am

Re: v4.4.2 awefully slow

Post by mjs »

Hmmm, that's very odd to hear that in your case just reverting from PPv4.4 back to PPv4.3 made a significant difference in Mail Check processing elapsed time.... this is something I'm sure Jeff would be interested in checking out further....

Not sure how you are differentiating or more specifically what your meaning is between the terms "collecting" messages as opposed to what you are referring to as the "processing" phase of a Mail Check.

Not that it matters at this time since you have installed a previous PP release but perhaps you could clarify in more detail specifically what you mean by this....

Now that you have switched back to the previous POP Peeper release I guess it doesn't matter, but just to let you know I think the default for "Maximum simultaneous threads" is actually 2, so you've probably changed this setting since you now have this setting at 5. That said, the value you have should not be a problem and considering the number of accounts I doubt there would be a need to increase that value.

One other point, if you had not switched back to PPv4.3, it would have been interesting to know whether switching the connection type setting from "Using a permanent connection..." to "Always assume I'm on line" might have made any difference. (I'm not sure in your case if this would have helped since your issue is Mail Check processing speed, but some ISP's work better using this alternate Connection type setting).
Good judgment comes from experience and a lot of that comes from bad judgment. - Will Rogers
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

. >> Not sure how you are differentiating or more specifically what your meaning is between
. >> the terms "collecting" messages as opposed to what you are referring to as the "processing"
. >> phase of a Mail Check

well, those mssgs are at the very bottom of PP, first is CONNECTING, then RETRIEVING, both are equally fast on 4.3 and 4.4.2. then on the newer one comes the count down of PROCESSING DATA, and that is the slow slow slow part.

. >> since you have installed a previous PP ...
. >> ... Now that you have switched back to the previous POP Peeper release

on a parallel laptop, but very similar to mine. so i can run them side-by-side.

. >> just to let you know I think the default for "Maximum simultaneous threads" is
. >> actually 2, so you've probably changed this setting since you now have this at 5

no, for sure i never touched this, it is at 5 in both cases.

. >> One other point, if you had not switched back to PPv4.3

i have NOT - these are two seperate machines!

. >> switching the connection type setting from "Using a permanent connection..." to
. >> "Always assume I'm on line" might have made any difference

the line speed during RETRIEVING is rather high, but during PROCESSING DATA its as good as dead, so i assume it has nothing to do with the internet as such, but is an internal PP thing - but to make sure, i will try your suggestion.

greetings - henry

PS: ok, just checked your last suggestion, after i emptied out and refilled two mailboxes, so that PP had something to do. no change at all - same result. 4.3 -> 12 seconds, 4.4.2 -> 68 seconds
User avatar
mjs
Moderator
Posts: 2216
Joined: Sun Jul 17, 2011 2:36 am

Re: v4.4.2 awefully slow

Post by mjs »

... then on the newer one comes the count down of PROCESSING DATA, and that is the slow slow slow part.
Ah, this clarifies what is occurring and yes this will significantly increase the elapsed time of a Mail Check.... This part of the message retrieval you are seeing in regards to displaying a "count down" when "processing data" as far as I'm aware only occurs if the account is getting repopulated with messages as a result of changing account protocols, i.e. in the case that the server protocol is changed for example from Webmail to IMAP etc. or possibly due to some issue with the messages requiring this to be done again and is in any case generally only a one-time occurrence so to speak for the first Mail Check for these type of situations. You should NOT be observing this on every Mail Check which of course when this occurs understandably does take a much longer time to process. In regards to this information you have provided Jeff would know more about this and explain better than I can, but again - this is certainly not what you should typically observe in the PP status bar for every Mail Check. This additional information should be helpful to Jeff when he stops by the forum of which he would know more about why this might be happening.
Good judgment comes from experience and a lot of that comes from bad judgment. - Will Rogers
User avatar
Jeff
Admin / Developer
Posts: 9252
Joined: Sat Sep 08, 2001 9:46 pm

Re: v4.4.2 awefully slow

Post by Jeff »

The "Processing data" is the AntiJunk checks.

A couple of questions:
1) What was the previous version you upgraded from?
Off the top of my head, I can't think of any significant difference in AntiJunk between v4.3 and v4.4. But if you were upgrading from v3.x then, yes, you'd probably notice a difference for the initial mail check. But the difference should be negligible when you're just retrieving the latest new mail.

2) AntiJunk does keep some statistics that should be helpful in this case. It only tracks the timing statistics for messages that you've retrieved in the current session (ie. it resets whenever PP has exited/restarted), so make sure that you've already seen the slowness, and then:
a) From PP's main menu, select Tools / AntiJunk -> Main interface
b) From AntiJunk's menu, select Tools / Display statistics
c) This will open a text file. I recommend saving this file somewhere so that we can reference back to it as needed. It can contain sensitive information (ie. email addresses) so I don't want you to post the entire thing here.
For now, the part we're interested in is a few lines from the top, and starts with "Filters" down to and including "Overall" (about 9 lines). Copy this and paste it here.

My guess is that either/both the URI or DNS Blacklists will be high. These rely on your DNS server and this could actually indicate a general issue with your system's network configuration that could be optimized.
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

jeff and laksrol, thanks.

just for this testing, i am running two systems in parallel on two different laptops (4.3 vs 4.4.2, all settings same, incl anti-junk, same accounts, same servers), but into the same router and service provider. i think that fact has been overlooked in some of your answers.

and certainly the slow PROCESSING DATA is there every time, not just once.

those junk settings were kept at default, never changed them. will post that TXT file later.

greetings - henry
Last edited by henry66 on Thu Jun 29, 2017 3:38 am, edited 1 time in total.
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

here we go. not sure what this will look like on the forum, you might have to copy/paste it and then stretch it.

the first one is the (slow) 4.4.2:

Jun 29, 2017, 09:44:08
Range: 2006-02-23 <=> 2017-06-29

Filter On Processed Eval-Spam Eval-Legt False-Pos False-Neg Effective Accuracy ms-Avg ms-Lo ms-Hi
------
White List ON 32856 0 15528 0 1 47.26% 99.99% 1.19 0.162 5.67
Black List ON 0 0 0 0 0 n/a n/a - - -
Rules ON 36473 452 2560 38 0 8.15% 98.74% 10.8 3.36 27.3
URI BlackList ON 9146 79 0 0 0 0.86% 100.00% 3.16 0.084 14.5
Bayesian ON 33 0 33 0 0 100.00% 100.00% 6.5 2.1 11
DNS BlackList ON 35152 78 0 0 0 0.22% 100.00% 3939 2086 5693
Overall ON 36472 397 18081 38 1 50.56% 99.79% 3960 2103 5706

and here the much faster 4.3:

Jun 29, 2017, 09:56:04
Range: 2006-02-23 <=> 2017-06-29

Filter On Processed Eval-Spam Eval-Legt False-Pos False-Neg Effective Accuracy ms-Avg ms-Lo ms-Hi
------
White List ON 30770 0 14400 0 1 46.80% 99.99% 0.793 0.11 5.27
Black List ON 0 0 0 0 0 n/a n/a - - -
Rules ON 34141 438 2460 38 0 8.38% 98.69% 14.9 1.97 51.5
URI BlackList ON 8609 78 0 0 0 0.91% 100.00% 85.2 0.05 1096
Bayesian ON 33 0 33 0 0 100.00% 100.00% 9.76 0.954 39.3
DNS BlackList ON 32829 76 0 0 0 0.23% 100.00% 271 0.518 1546
Overall ON 34141 383 16854 38 1 50.37% 99.77% 381 4 1663

hope this makes sense. to me they look VERY similar, but the obviously different ''ms-Avg ms-Lo ms-Hi''.

greetings - henry
User avatar
Jeff
Admin / Developer
Posts: 9252
Joined: Sat Sep 08, 2001 9:46 pm

Re: v4.4.2 awefully slow

Post by Jeff »

The "ms" values are actually the only ones we care about. "ms" is "milliseconds" (e.g. 1000ms = 1 second).

So here's where we see the significant difference in times:

Code: Select all

                  ms-Avg     ms-Lo       ms-Hi
DNS BlackList     3939       2086        5693
vs
DNS BlackList     271        0.518       1546
As this shows, every message on average is taking 3.5 seconds longer. So let's figure out why.

Step 1: make sure that POP Peeper is comparing apples to apples on both computers: using the same DNS blacklists.
1) From PP's main menu: Tools / AntiJunk -> Main interface
2) Select the "DNS Blacklist" page on the left
3) There are 2 entries here by default:
- bl.spamcop.net (enabled by default)
- sbl.spamhaus.org (disabled by default)

Do both computers have the same entry(s) enabled?


Step 2: Check that each computer is using the same DNS server:
1) Open a command prompt (type that into the Windows Start Menu to find and run it)
2) type:
ipconfig /all
and press <enter>

You may need to search for it, but you should find a value for "DNS Servers" which will look something like this:

Code: Select all

   DNS Servers . . . . . . . . . . . : 2001:558:feed::2
                                       2001:558:feed::1
                                       192.168.2.1
                                       0.0.0.0
This is my personal result. The first 2 are the DNS servers of my ISP; the 3rd (192.168.2.1) is my router. This is the order that my computer will try to resolve host names. So if the first DNS Server (feed::2) is down, then it will use feed::1 and then 192.168.2.1, if necessary.

Anyway, point is, you should check that the dns servers *and the order* are the same on both your computers.


Let me know what you find.
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow - getting there

Post by henry66 »

jeff, many thanks, we are getting closer, much closer. and you have to admit, with all this trying, re-installing versions, emptying out mailboxes and filling them again, and all this on two computers, i am trying to help as much as possible.

. >> the "ms" values are actually the only ones we care about

thats what i thought. also the OVERALL ms-values show just about the 1:10 difference i am experiencing.

i have gone thru most of your points, one by one: i am ending up with one issue you also pointed out, the DNS servers.

i have two available, and can switch between both of them: an IPv6 one (( fe80::1%11 )), and an IPv4 one, the router at (( 192.168.2.1 )). if i have both active, the computer tries the IPv6 one first, and often but not always succeeds - so not a good base for testing. simply would not be apples to apples.

eliminating one or the other, and going strictly IPv6 OR IPv4, i come up with the below results, tried several times, back and forth between both systems. before i forget, both antijunks are set to default.

and i have moved PP4.3 back and forth between both computers, same with PP4.4.2, so you can be sure it is not only apples to apples, but also also pears to pears.

- SYSTEM/servers set on IPv4 only: PP4.3 fast, PP4.4.2 fast

- SYSTEM/servers set on IPv6 only: PP4.3 fast, PP4.4.2 SLOW by about 10 times

so, there we are now. pretty limited to further trials - i wouldnt know what inside of PP4.4.2 could possibly slow it down on IPv6's - you tell me.

question: am i so far the only one having experienced this?

greetings - henry
User avatar
Jeff
Admin / Developer
Posts: 9252
Joined: Sat Sep 08, 2001 9:46 pm

Re: v4.4.2 awefully slow

Post by Jeff »

This:
> the computer tries the IPv6 one first, and often but not always succeeds
is not good. But what's your methodology for determining that? Also, if it fails when both are available, it's going to fail if it's the *only* one. Maybe I'm misinterpreting what you mean by "not always succeeds", but if it fails, I assume that means it times out which will obviously affect PP.

This may not be the best methodology either, but try this:
command prompt:
ping -t (ipv6)
where (ipv6) is the ip address and not literally "(ipv6)"

Let it run for about 5 minutes, then hit Ctrl+C and review the results. here's mine for comparison:

Code: Select all

    Packets: Sent = 298, Received = 298, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 22ms, Maximum = 117ms, Average = 25ms
The "0% loss" is the most important stat here. Followed by the "maximum" ms time, but when compared to how close the "average" is to the "minimum", it suggests I get a consistently low ping, with maybe an occasional spike (but still reasonable at 117ms).

--> let me know what your values are here.
(edit: you may also want to test your ipv4 dns server to see if it's better/worse/similar)

and i have moved PP4.3 back and forth between both computers, same with PP4.4.2, so you can be sure it is not only apples to apples, but also also pears to pears.
Well, there are still other contributing factors involved, like caching. Let's go back to your original stats, where you had a "low" value of "0.518" on v4.3 -- that's definitely the result of a cache hit. Your low on v4.4 was "2086" -- probably (hopefully) not the result of a cache hit. PP doesn't cache these itself, the caching is coming from Windows, your router, your modem, or whatever.

Just to assure you that there's nothing wrong with PPv4.4 or using an IPv6 DNS server, here are my current session's stats for DNSBL:
avg/lo/high: 241 / 0.002 / 608
And I should point out that I have both DNSBL's enabled (not something I would usually recommend), so that includes both of them. Further down in the stats, PP splits them out:
spamcop: 127 / 0.231 / 237
spamhaus: 167 / 0.148 / 383

There's no difference between v4.3 and v4.4.2 as far as how the DNSBL is checked. 99.9% of the time indicated by the stat consists of the time waiting for the DNS result.
henry66
Posts: 40
Joined: Sun May 24, 2015 3:26 am

Re: v4.4.2 awefully slow

Post by henry66 »

hi jeff

just tried to switch back to my IPv6 connenction, and it timed out. called my provider, they are having IPv6 problems i am told, and it will only be back monday/tuesday. they sounded miserable.

so real testing will continue then, meantime a few points. what better is there to do over the weekend (here starting in 2 hrs).

. >> IPv6 one first, and often but not always succeeds ... is not good

i know, thats why it turned either of them completely off for testing, back and forth.

. >>"IPv6 ... not always succeeds"

could have to to with their present switching it off completely.

. >> there are still other contributing factors involved, like caching

doubt it. i have tried too often with close to same results, so cached or not would show once or twice, but not dozends and dozends of times.

. >> the caching is coming from Windows, your router, your modem, or whatever

to avoid any such influences (router, modem, provider etc) i have now set up both 4.3 and 4.4.2 on the same laptop, as portable versions. this way we can eliminate a lot of possibilities and/or negative influences.

the DNS blacklist i will leave at default, BL enabled, SBL disabled. unless you tell me to turn both of them ON for testing purposes.

greets, and wish you a nice weekend - henry
User avatar
Jeff
Admin / Developer
Posts: 9252
Joined: Sat Sep 08, 2001 9:46 pm

Re: v4.4.2 awefully slow

Post by Jeff »

Ok, so barring any major surprises when your ISP fixes the problem, it sounds like this is basically an issue with your ISP? I mean, I can't explain why you were getting different results from different versions, but if they were on different computers at the time, maybe your other computer was defaulting to the ipv4 server (I don't think you mentioned anything about the dns servers on the "fast/normal" computer)?

In case it helps, you can have PP test any existing message and this should make it easier to do a comparison:
- Right-click on a message in PP and select AntiJunk -> Test message in sandbox

Just be sure to wait a little time between tests; cache hits will usually be <1ms, but I'd be suspicious of anything less than 20ms.


It's also worth testing the spamhaus BL, as it may be faster for you. Based on my stats, spamcop will detect more spam, but it also has a slightly higher false positive percentage (ie. suggesting a message is spam that is actually legit), although it's a fraction of a percentage point.
Post Reply