mail sends repeatedly forever, despite getting through

[Kev]

This was on haiku-r1a2-rc-hrev36423-x86gcc2hybrid-raw until I upgraded just now.

Using Gmail SMTP, I sent an e-mail today, just like normal. Well, somehow, the mail_daemon got confused, and didn't think it had sent it successfully. So it tried again, and kept trying, and kept succeeding. My friends got flooded with 80+ copies of this one e-mail before Gmail finally stopped accepting the messages altogether because I'd hit my daily cap.

Now, when I first caught on that it seemed to be sending something every time it checked the mail, I looked at the out folder, and the particular e-mail just said 'Sent' for its status.

No steps to reproduce it.

[stippi]

The mail_daemon will detect the use of certain language and phrases and switch to repeatedly sending a mail when it finds it is most likely intended as SPAM.

[Kev]

You're kidding, right?

[axeld]

Of course he is, or do you feel caught? :-)

Anyway, that's really a strange bug, and I've never seen it, despite me using the mail_daemon for years now. I don't have an idea what could have caused this; let's see if it happens again somewhere.

The mail_daemon is using queries to retrieve the mails to sent, maybe something went wrong there. You could use checkfs to see if your file system is okay. Other than that, some protocol problems could have been the culprit.

Did the daemon sent the mail out several times in a single session, or did it retry periodically?

[stippi]

BTW, I am a regular Beam user, so I don't have so much experience with mail_daemon. The other day I was debugging a problem in Mail which involved sending/saving a mail without the mail folders present. Of course it also asked me to start the mail_daemon. When I did, the mail_daemon tried to send all sorts of old mails, which may have had a pending attribute or not. Fortunately it wasn't properly configured so it didn't actually send anything. Perhaps the query approach is much too fragile. It could be ok to use queries to collect mails to be sent, but other properties should be taken into account, like where the mail is stored. I have no idea what other mails it could have tried to send on my machine, other than mails not in ~/mail/out.

[axeld]

That might also point to a compatibility problem between Beam and our mail_daemon. The latter uses the MAIL:flags field to locate the mails that haven't been sent yet.

[Kev]

I too have used the MDR for many years, both on BeOS and lately on Haiku.

It sent it out once every time it checked mail, which I had set to once a minute. It didn't even bring up the error log until it was filled with errors.

/> checkfs /boot checked 31010 nodes, 0 blocks not allocated, 0 blocks already set, 0 blocks could be freed

files 27761 directories 1992 attributes 705 attr. dirs 512 indices 40

[anevilyak]

Is it possible GMail's SMTP/IMAP implementation is doing something our protocols don't handle? This isn't the first time I've heard of weird issues trying to deal with GMail that way (and not just on Haiku either).

[axeld]

The mail_daemon checks whether or not the mail is really pending since hrev36820. Maybe your BFS index was invalid, and caused the problem - if that was the problem, it should not be able to happen anymore.

Without further information, one can only guess, though.

[antics]

I can confirm this bug, with steps to reproduce:

1. Create an email with an illegal attachment (​http://mail.google.com/support/bin/answer.py?answer=6590) and send it to a Gmail account.

2. Mail daemon will respond with an error message after sending has occured.

3. Email will end up in "Out" folder.

Mail will keep sending email everytime you check for new mail, even if you delete the email from Out folder. No mail is in Draft folder.

[humdinger]

BTW, I recently also had a problem with a mail that kept failing to send. I, too, deleted the "Error" mail from the out folder, but it kept failing to be sent. Solution: Remove the mail from Trash as well...
Apparently mail_daemon is going through my Trash... Dirty daemon! :)

[anevilyak]

Replying to humdinger:

BTW, I recently also had a problem with a mail that kept failing to send. I, too, deleted the "Error" mail from the out folder, but it kept failing to be sent. Solution: Remove the mail from Trash as well...

mail_daemon uses a BFS query to locate messages to send. The out folder is more or less purely cosmetic for the user's purposes, it will find them no matter where they are.

[humdinger]

I know, but IMO Trash should be filtered out "manually" in order to ignore deleted mails. As long as it isn't, looking into Trash is necessary.

[stippi]

Sending mails from the Trash is dangerous and definitely a bug. I thought mail_daemon already filtered the query against mails in the Trash? Apparently it doesn't work and deserves it's own ticket. ;-)

[phoudoin]

In src/kits/mail/MailProtocol.cpp, the TrashMonitor handler only watch the folder /boot/home/Desktop/Trash folder, which AFAICT don't exist anymore. Could be the issue regarding deleted mails still send?

I dunno if it's expected, but when other volume(s) are mounted in RW mode, entries moved to the desktop's Trash are not located in the source volume's trash folder but on the latest mounted volume one. Could be a side-bug BTW...

Anyway, TrashMonitor should be fixed to actually monitor the right folders, for every volume.

[scottmc]

Anyone willing to take ownership of this one? It seems bga is sleeping?

[phoudoin]

Replying to scottmc:

Anyone willing to take ownership of this one? It seems bga is sleeping?

I will. Dunno when, though, but I hope that I will be able to reproduce steps as in comment:9 in see what happends wrong then.

Please notice that pending mails located in some volume Trash folder should be ignored now since hrev39804-hrev39806, but it won't help when the rejected mail is located otherwere.

[phoudoin]

I hope you don't mind Clemens, but since your mail subsystem rewrite, you're most probably :-) more fluent with the new code than me to check this issue:

• is still there
• can be fixed and suggest how to do that, if you don't have time to do it yourself.

[czeidler]

What's the desired behaviour? Only send messages in out folder? Otherwise the ticket seems to be fixed. Or is there a general way to find out if a message is rejected?

[humdinger]

I'd say, if a mail couldn't be sent, pop up an alert with an understandable error message and offer to try resending or abort the effort, which will set its status to "Error" and won't be picked up for sending anymore.

[czeidler]

Is this ticket still valid? The main problem seams to be fixed by checking the pending flag. The trash problem is also fixed.

I think the invalid gmail mail problem is not a real problem. From ​http://mail.google.com/support/bin/answer.py?answer=6590 it says that the mail is not delivered in this case. If the mail_daemon prints an error in the log everything is fine. The user just have to delete the invalid mail. I think marking it with an error and not trying to send it any more is more confusing to the user. If you missed the error and don't check the status you don't know why daemon stopped sending this mail.

Will close the ticket. Feel free to reopen it if the problem still exists.