an "é" typed in an email i type with Bemail, ultimately show in Bemail as a "é"

[AlienSoldier]

An "é" typed in an email i type with Bemail, ultimately show in Bemail as a "é" if i check it back in my sent email directory. If i check it in my google mail browser interface, the sent email have the right "é" with no "é" to be seen.

[nephele]

Sounds like a UTF-8 file is beeing read as if it was ISO 8859-1 or 8859-15 or so, with what are you viewing the mail?

(Am asking since the mail i've send with Mail never got placed in the send folder, only in my home, but perhaps that is just a bug with imap..)

[AlienSoldier]

Viewing the mail with the native Mail app. The only difference is that is was sent. For exemple this notification when i opened it with Mail from my "In" directory showed the "é" fine.

Using SMTP.

[AlienSoldier]

Still happen.

Not sure anymore of what i was meaning by this btw "For exemple this notification when i opened it with Mail from my "In" directory showed the "é" fine" :)

Here is some reading of the mail as text:

first is sent from BMail. subject "more é" text "ké":

Subject: more =?iso-8859-15?q?=E9?=

X-Mailer: Mail/Haiku 3.0.3

Date: Sun, 19 Dec 2021 14:35:51 -0500

Message-Id: <5180961017-BeMail@shredder>

Mime-Version: 1.0

Content-Type: text/plain

Content-Transfer-Encoding: quoted-printable

k=C3=A9

second (the one i can read fine) is sent from gmail web interface, subject "more é" text "ké" (so exactly the same thing):

MIME-Version: 1.0

Date: Sun, 19 Dec 2021 13:46:43 -0500

Delivered-To: my email

Message-ID:

Subject: =?UTF-8?Q?more_=C3=A9?=

From: my email

To: my email

Content-Type: text/plain; charset="UTF-8"

Content-Transfer-Encoding: base64

a8OpDQo=

[nephele]

The first email is iso8859-15 encoded, Maybe the mailer is trying to open it with utf-8? Can you open the email and then select the encoding directly? Message->Encoding->Iso Latin 9 (ISO-8859-15)

If it then shows correctly the issue is that The mail application opens files as UTF-8 even though they are anotated as not beeing UTF-8.

A second issue is that the mailer apparently sends messages with iso8859-15 per default, even though it should really use UTF-8 (I mean, mailers /should/ deal with encodings... but then the end probably treats everything as utf-8 too depending on the system...)

[AlienSoldier]

Seem the body of text was encoded in utf-8 as it show correclty when i select "utf-8". Choosing encoding Iso Latin 9 (ISO-8859-15) make it appear badly again. So right now it open by default in so Latin 9 (ISO-8859-15) for text (subject was always displayed correctly).

One would expect that sending from bemail and receiving from bemail, the text would not change encoding along the trip.

[nephele]

Yes. it shouldn't.

But what you wrote confuses me more, the header of your first email sais it is iso8859-15 encoded, but if it is displayed correctly with utf-8 selected that would mean that it when sending originally put the wrong encoding info in the email, and then later uses this to select the wrong encoding.

Maybe this is a problem with sending the email then? Can you verify this? maybe send an email to yourself :D, it should arrive mostly identical except for more headers. See if the send one mentions iso8859-15 but then also displays correctly only with utf-8

[nephele]

It looks like the problem is with BeMail not including in the email which encoding it is using for the body, we should probably also include charset="UTF-8"

BeMail: Content-Type: text/plain

Gmail webmailer: Content-Type: text/plain; charset="UTF-8"

[nephele]

As an additional note: iOS Mail displays the email correctly even without the encoding specified, we should probably include it regardless.

[dsizzle]

I spent a bunch of time dealing with this stuff at a previous job. Here's how it works:

In the first email, this "Subject: more =?iso-8859-15?q?=E9?= " does NOT mean the email is encoded in iso-8859-15. That is only the encoding for that subject line. I can't remember the RFC number but that format is common. So you decode the part inbetween the last two question marks using the format in the first part. The "=E9" is the part that translates to the é character.

Then this is the encoding for the email: "Content-Transfer-Encoding: quoted-printable" "k=C3=A9" in "quoted-printable" format is "ké".

Similarly, for the second email, the *subject* line is UTF-8 encoded, much like the first email: "Subject: =?UTF-8?Q?more_=C3=A9?="

For the content of the second email, it's UTF-8 format, but it's encoded as base64: "a8OpDQo=" is base64 code for "ké"

You can even get these types of things nested inside each other, when mail is replied to and forwarded from different mail clients with different encodings.

So it looks like the mail itself is formatted correctly.

[dsizzle]

<deleted>

[nephele]

I can't really reproduce this anymore. I tried sending an email to myself with é as a content Here are the headers:

Subject: Re: test
X-Mailer: Mail/Haiku 3.0.3
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

é

It looks like the charset is send correctly, and in the code it also looks to cary this info around everywhere and then send it. maybe you are triggering an edge case? the normal case seems to work anymore

I do have utf-8 set in my mail preferences though, maybe this makes it behave nicely. what do you have set to reproduce it?

[AlienSoldier]

Humm the message encoding in the drop down menu (in the message area of the tool bar) was not kept after closing bemail. I then looked in preference and there is another encoding setting there. The one there was set to nothing, i mean not "nothing" as a choice but the whole rectangle was empty. I did set it to uft-8 now. Like you i now receive the correct character in the body of the text, but the problem is still there for the email sent before doing that setting manipulation. So it seem that haiku should need to be distributed with uft-8 encoding as default because mine seemed to be in undetermined state (my second computer was also empty setting). Once set to anything, it is not possible to have that option empty again so i guess the only way to trigger that bug is from a fresh install.

[dsizzle]

I suspect that's the issue - I think having the setting default to anything (maybe UTF-8) would fix it.