mail_daemon crash after retrieving ~20000 messages

[anevilyak]

I left mail_daemon synchronizing my GMail account overnight in IMAP mode. In the morning it had fetched around 20000 messages and was in a crashed state with the following backtrace:

stack trace, current PC 0x4ca022  CountItems__C5BList + 0x6:
  (0x700804bc)  0x93c9f3  CountItems__C12NestedString + 0x1f
  (0x700804ec)  0x93bd9a  FetchMessage__18IMAP4PartialReaderPCc + 0x19a
  (0x7008055c)  0x93ba94  Seek__18IMAP4PartialReaderxUl + 0x54
  (0x7008059c)  0xac0f7f  ProcessMailMessage__12FolderFilterPP11BPositionIOP6BEntryP8BMessageP5BPathPCc + 0x217
  (0x700808dc)  0x57f338  get_messages__16BMailChainRunnerP11BStringList + 0x370
  (0x70080a2c)  0x57e877  MessageReceived__16BMailChainRunnerP8BMessage + 0x34b
  (0x70080f0c)  0x3b4003  DispatchMessage__7BLooperP8BMessageP8BHandler + 0x5b
  (0x70080f3c)  0x3b5971  task_looper__7BLooper + 0x211
  (0x70080f7c)  0x3b555f  _task0___7BLooperPv + 0x3f
  (0x70080fac)  0x238c20  thread_entry + 0x20
debug_server: Killing team 598 (/boot/beos/system/servers/mail_daemon)

Not sure what else to report here other than that it's reliably reproducible ; if I delete all the messages and resync the account from scratch, it will crash every time, though I'm uncertain if the message count was always the same at time of crash.

[koki]

Subscribing.

[bga]

Did you try to reproduce this again? I am asking because the crash seems to be inside BList and, more specifically, in the CountItems method which does nothing but return a number. I wonder if this was not some kind of corruption.

[anevilyak]

I haven't tried recently, but I did reproduce this problem across quite a few different revisions already. If you want I'll try again, but it will take some time since it takes a few hours to retrieve that many for me.

[aldeck]

Might not be the case here, but i thought i should leave a note about #2760. Rene! remember? ;) cf hrev28871 and hrev28873. Corruption was happening when a BList was sorted by a non conforming function.

[anevilyak]

No longer reproducible, retrieved 35000 messages. I did notice a different problem though, which may warrant a new ticket: When I woke up in the morning, MDR was warning about having lost its connection to imap.gmail.com. However, picking send and receive did not re-establish it. After quit/restarting mail_daemon it worked and picked up 7 new messages. Should I open a ticket for that? Also, is it possible to improve the retrieval performance at all? It took several hours to get the messages for me.

[bga]

Yeah, it was probably fixed with your BList fix. Thanks aldeck for pointing this up. As soon as I fix the remaining issues, I will look into performance. I think we are doing one sync per message when receiving it and this could be the cause of the performance problem.