Weird characters; need help understand...

[Windance]

I've been using PP for a very long time, first by donation, then Pro. I'm looking for an explanation. Over the past couple of years, more an more email come with unreadable characters as follows. Most I've been able to have go to junk mail. Here are a few example.

1
Subject: ???????????Is? yo?ur? i?ns?ura?nc?e p?ol?ic?y w?o?r?ki?ng? ?as? ha??rd ??as? ?yo?u? ?????
From: "???????????Af?lac? "<store@news.exposeii.com>
The encoding for this one is utf-8. When I double click for images, the mail shows up ok.

2
Subject: ?????????????????Fo????cu???s?? ?o??n ??y???ou?r? r?ec?o?v??e??ry? ?– ?no?t? b?il??l?s? ?– ?w??it??h?? ??Af?la??c ?  
From: "?????????????????Af????la???c??  "<announcements@v.hover10.com>
Also utf-8.

3
Subject: þÿ
From: þÿ (no address so I can't blacklist it.)
(Also encoding show utf-8.)

I'm just trying to figure out what all the weird characters mean. I used to be able to fly through what I wanted to make as junk mail, or blacklist. Now, I'm not ture if some of this is legitimate, but unreadable header for some reason. I hope someone can help me understand. The þÿ one seems too suspicious without an address to blacklist. I'm not much for understanding irregularities. I just read and try to follow directions. But now, I need a better explanation than I can give myself. Thank you.

[mjs]

These messages presumably contain "unicode" of which PP does not support (unicode) at this time (thus, in this case, is currently "unreadable" just as you have suggested). To my knowledge, "unicode" support is currently underway (and I might add, so far, successfully tested) to be included in the PP version 6 release. Rest assured, "unicode" will be readable when it is supported. (I will occasionally get similar results that you are getting myself. Perhaps, in some cases, spammers may indeed exploit this situation to avoid SPAM filters.)

Jeff will likely have more details to add to this topic.

[mjs]

To provide more details (as I understand it) - let's just discuss the "?" as you see it (to keep it simple).

My understanding is that (Jeff will thankfully correct me if I'm off here ), as far what goes on (in the event "unicode" is used) - it is the following that occurs:

The question marks ("?") that PP renders are (in most cases) not intrinsically "question marks" at all, but are rather representative of (unicode) characters that PP is unable to display (hence the "?"). To elaborate, PP is not actually displaying the "?" - it is instead, Windows that is displaying the "?" in as much as the specific "unicode" character (represented by the "?") cannot be displayed correctly using the encoding method (to my knowledge, basically, 7-bit ascii) that PP uses at this time (all of which is expected to be addressed as of PPv6 ).

P.S. Stay tuned, Jeff may have some thoughts on how to use the Rule Wizard to generate Anti-Junk Rules to apply to these type messages in the meantime. (In general, I just delete them myself -- as, in my case, they're not that commonplace)

[Windance]

I appreciate this so much! I think I understand the gist of what you're saying. Using the Rule Wizard to generate Anti-Junk Rules would be helpful. Here's the thing, I have 18 email addresses although I no longer need them all. I am in my 70's, still a licensed RN but not working, only as a volunteer. Each box served a purpose, either professionally or for different aspects of my personal life. Anyway, until I consolidate and figure out which box get's mail from someone I still need, I will also keep getting about 100 or more emails every morning. Most of the weird ones go to junk mail from continuously having marked them as junk, and a few here and there get blacklisted. So, in the morning, I check one account at a time, legitimate ad junk, sort this way and that, deleting what needs to be deleted or marked as junk. 18 accounts, 100 + email takes me about 10 minutes to 15 minutes to deal with before actually looking at the new ones remaining. Your program is indispensable to me for those reasons. There is no comparison. But all said, with so many of this Unicode type coming in, I do my best marking them but I am sure I could do a lot better. Oh, and that "þÿ" email without any return address, I can't blacklist it but oddly, when I click the picture in the email, it opens to a legitimate, although advertising website with a URL. I get about 5 or 10 of them in the morning. Not one time have they been important.

So, I'm standing by and looking forward to more information! Again thanks!

Oh, well, while I have your attention, I've always wondered why when I get something from Amazon and click to open the website, it opens in the PP browser despite being set to open in the browser. My workaround is to 'open in new window'. I just wondered if there is a way to make it open in my external browser. I found some information with search but not a solution. That's it.

[mjs]

.... I just wondered if there is a way to make it open in my external browser....

Under the main menu settings: "Tools" > "PPtweak..." > "Misc 2" page - do you have the settings as circled in the screen-shot below (works for me, as far as what your wanting to do - should use what you have set as your Windows [default] browser):

Main Menu: Tools/PPtweaker/Misc 2

Oh and always keep in mind Windance -- do not hesitate to ask whenever you might have any questions.... (it sounds like this has perhaps been an issue for you, while using a "work-around", for some time now )

... Using the Rule Wizard to generate Anti-Junk Rules would be helpful....

Have you tried using the "Rule Wizard"? --- Jeff should be able to help you out as to the specific details on this (I'm not that up to speed on the intricacies of using it myself )...

.... I am in my 70's...

Welcome to the (elite ) club

[Jeff]

Yes, as mjs stated, these are unicode characters; and they appear as question marks because POP Peeper v5 doesn't support them (it will be supported in v6).

Unicode support may come as a mixed blessing in regards to this. Let's take this subject of yours for example:
???????????Is? yo?ur? i?ns?ura?nc?e p?ol?ic?y w?o?r?ki?ng? ?as? ha??rd ??as? ?yo?u? ?????

If you remove all the question marks, you get:
Is your insurance policy working as hard as you?
(that '?' at the end is probably an actual question mark)

What the spammers are doing is inserting unicode characters into the text, but the unicode characters are invisible to a human reading the text (there are lots of unicode characters that aren't intended to be visible). The purpose of doing this is to make it harder for spam filters to process the text; e.g. creating a simple filter to detect "insurance" won't work, because it's got so many other characters embedded in the word "insurance".

The reason I mentioned unicode support will be a mixed blessing -- right now, because PP does *not* support unicode, you'll see the ??? instead of unicode. Personally, that's all I need to know that the message is spam and I Junk it without a second thought. That will be a thing of the past in v6. However, I would like to create a custom filter that tries to detect this kind of spam, so we'll see how it goes.

[Windance]

Mjs, to answer you, I do have the 'Tools" > "PPtweak..." > "Misc 2" set to force to external system default browser. It's always email from Amazon and maybe one other. Just in case, I tried changing my system default to Chrome (from Chrome based Brave) and even tried putting the path to Chrome in. No luck, Amazon mail will open in the PP browser. Wondering, I tried 'open in another' window and it opened my system default browser, good enough. Just curious why Amazon.

Jeff, thanks for an incredible software, and mjs for your support, fast and easy to understand answers. Back when I actually used all of my accounts, some Yahoo, Gmail and some that I pay for on a private host. I could not have gone though 100 + emails as fast without PP. I am not one for learning more than I have to unless it's professional or for a hobby. But mjs has encouraged me to take another look at the Rule Wizard (already using it to a degree) and see if I can use it better. Although I have poked around in the available information, I think it's worth my time to to take an even closer look at how the program and settings function in general.

Jeff, I appreciate tha explanation of Unicode and PP. I saw another related post I didn't quite get. What you are saying makes sense now. As for all the ??? marks, I came to that conclusion as my way to mark as junk. 98% of these weird ones come directly to junk mail from my having been diligent at marking them as junk early on. But I still scan the list of new legitimate and junk emails to see if any should really be junk, blacklisted, or have other Rules applied. In the FAQ or somewhere, you said not to blacklist too much. I try to be careful when and how.

I can't thank you enough for the answers and again, an amazing program.

[Jeff]

Your reply did make me realize I forgot to mention a couple of other points:

Rule Wizard may not be too helpful in these types of email because the insertion of the invisible unicode characters could be random. For example, in the last email it could be "i?ns?ura?nc?e" but in the next email it could be "insuranc?e". This can be addressed using regex (see some of the pre-included "Pills" rules for examples [not the 'generic' one]) -- but regex can be extremely complicated and unless you're already a pro at regex, I would advise against creating such rules. So my advice is just to mark them as Junk and move on.

As for links not opening in an external browser -- some heavily scripted emails may do this. Eventually, POP Peeper will use a different HTML renderer and that may fix these types of problems. In the meantime, instead of right-click/open-in-new-window, you may try holding... ummm... shift? or ctrl? (brainfart), and that should have the same effect, so whatever's easier.

I think ctrl opens a new tab and shift opens a new window, so you want shift-click. probably.

[Windance]

Thanks Jeff. Yep, regex for me is not for this lifetime. Using the Rule Wizard is not necessarily for the emails I brought up in the OP, just getting to know it better and when it's worth using it. Right now, I'm just mostly learning how to work with the setting, colors, actions etc. You have written such a vast program, it's time for my relationship with it to grow. Out the box, it's not very complicated and the instructions are clear enough to keep it simple. I am about an advanced beginner. I've also found that reading the forum is teaching me a lot. This is the first time I've posted I think.. because it's valuable to see if the questions I may have were already answered in a way I understood.

Thanks for 'opening the browser tip'. Looking forward to version 6.