net.imap: add an IMAP4rev1 client - #28269
Conversation
…EXISTS Dovecot answers EXPUNGE and CLOSE with the removals alone and no new EXISTS behind them, so a client that waited for one went on reporting a count that included messages it had just watched being taken out. Each EXPUNGE now decrements the count, and an EXISTS the server did send still wins over that arithmetic. Found by running a session against a real Dovecot; the scripted server in the tests had been sending an EXISTS and hiding it.
Building a set of scattered numbers rescanned and re-sorted every range on each insertion, which is quadratic: twenty thousand numbers took ten seconds. The ranges are kept sorted and disjoint, so insertion now bisects for its place and merges in one pass. The decoder reached into the reader for a single octet at a time, with a one-element array allocated per byte. It reads a chunk at a time now, and a literal larger than a chunk goes straight into the caller's buffer. Four near-identical token loops become one, and a mailbox name that is already plain ASCII skips the UTF-7 encoder rather than walking it. seq_set, 20000 scattered 10074 ms -> 1.2 ms parse 200 messages, 462 KB 4.98 ms -> 2.50 ms utf7_encode, ascii name 309 ns -> 27 ns utf7_decode, ascii name 155 ns -> 19 ns
medvednikov
left a comment
There was a problem hiding this comment.
Thanks for the very thorough parser work and test coverage. I found several correctness issues that should be addressed before merge:
-
Greeting/authentication state is lost (
imap.v).read_greeting()acceptsPREAUTH, butnew_client()still callsLOGINwheneverusernameis set. APREAUTHgreeting already puts the connection in authenticated state, so bothLOGINandSTARTTLSare invalid there. The greeting'sCAPABILITYresponse code is also parsed and discarded, so a server can advertiseLOGINDISABLEDand the client will still sendLOGIN(including the password). Please retain the greeting state/capabilities and gate STARTTLS/authentication accordingly. Regression tests forPREAUTHwith configured credentials and for greeting-timeLOGINDISABLEDwould catch both cases. -
A failed
SELECT/EXAMINEleaves stale selected-mailbox state. IMAP returns to authenticated state after a failed selection.open_mailbox()only assignsc.selectedon success, so selecting a missing mailbox while another mailbox is selected leavesc.selectedpointing at the old mailbox. Clear the selected state (and any selected-mailbox counters) when a new selection is attempted/fails, and add a selected -> failed-select test. -
Several post-dial error paths can leak the transport. Failures during implicit TLS, STARTTLS, or the automatic login escape without
shutdown(). Separately, an untaggedBYEsetsis_open = false, so a laterclose()returns before closing the local socket. Please make every error after a successful dial, plus BYE/logout handling, tear the transport down exactly once. -
Malformed server
INTERNALDATEdata can panic the process.parse_internal_date()feeds unchecked.int()results intotime.new(), whose range validation panics. A response containing an out-of-range day/hour/etc. should return an IMAP parse error rather than terminate the client. The zone digits/ranges should be validated too. -
Unknown untagged responses containing literals still desynchronise the decoder. The fallback in
read_untagged()callsd.text(), which stops at the CRLF after a literal marker such as{3}and leaves the literal payload to be parsed as the next response. A transcript like* XDATA {3}\r\nabc\r\na1 OK done\r\ndemonstrates it. The unknown-response skip needs to be literal-aware to uphold the stated extension-safety guarantee. -
APPENDcan mislabel local time as UTC.format_internal_date()always emits+0000using the supplied struct's raw fields, whiletime.now()is local time. Convert withlocal_to_utc()before formatting (or emit the actual offset), otherwise the stored internal date is shifted by the caller's timezone.
Smaller edge case: seq_max = u32(0xffffffff) collides with the largest valid IMAP number. For example, parse_seq_set('4294967295')!.str() becomes *; using a wider internal bound would keep the numeric value distinct from the star sentinel.

net.smtpsends mail but nothing in vlib reads it, so a V program that wants to look at a mailbox has to shell out or reimplement IMAP. This addsnet.imap, a client for IMAP4rev1 (RFC 3501).I checked vpm and awesome-v first: there is no IMAP module in either.
Responses are tokenised, not searched
The module reads the grammar of RFC 3501 section 9 rather than looking for keywords in a response line. That is not tidiness, it is correctness: a message whose subject is
UID 999answers a search forUIDexactly as well as the real field does, and a body containing)ends a response that a bracket-counting parser would still be reading.wire.vis a decoder over the four shapes the protocol is built from: atoms, quoted strings, literals and parenthesised lists. It reads the connection directly with one octet of lookahead, so a literal is taken by count and its content is never offered back to the parser. An item this module does not model is stepped over whole byskip_value, so a server extension cannot desynchronise the rest of the response.Two grammar details worth naming, because both are easy to get wrong and both are covered by tests:
env-cc = "(" 1*address ")". The addresses in an envelope are concatenated with nothing between them, not separated by spaces like every other list in the protocol.[is a validATOM-CHAR; only]is excluded. Reading a fetch item name with a plain atom reader therefore swallows the opening bracket ofBODY[]and leaves the rest unreadable.Mailbox names
Names are ordinary UTF-8 in the API and travel as the modified UTF-7 of section 5.1.3, encoded on the way out and decoded on the way in. Without it,
list_mailboxeshands back&AMk-l&AOk-mentswhere the user's mailbox is calledÉléments. The encoding is checked against the two examples printed in the RFC and cross-checked against an independent implementation.Sequence sets
SeqSetkeeps message numbers as merged ranges, so a search matching fifty thousand messages is sent asFETCH 1:50000 .... A comma separated list of every number would be a command line no server is obliged to accept.What is implemented
CAPABILITY, NOOP, LOGOUT, STARTTLS, AUTHENTICATE PLAIN, LOGIN, SELECT, EXAMINE, CREATE, DELETE, RENAME, SUBSCRIBE, UNSUBSCRIBE, LIST, LSUB, STATUS, APPEND, CHECK, CLOSE, UNSELECT, EXPUNGE, SEARCH, FETCH, STORE, COPY, MOVE, and the UID forms of the last five.
FETCH parses UID, FLAGS, RFC822.SIZE, INTERNALDATE, ENVELOPE, BODYSTRUCTURE and any number of
BODY[...]sections, keyed by the specification the server echoed back.Some things that took deliberate handling:
+continuation, then the octets follow. Section 2.2.1 says the server may send untagged data or reject the command instead of continuing, so the wait handles both rather than assuming the next line is the+.Client.existsandClient.recentfollow them.ESEARCH(RFC 4731) andMOVE(RFC 6851) are supported;supports()is there to guard the optional ones.Tests
Four files, 513 assertions, no external server needed.
utf7_test.vcovers the RFC's own examples, a round trip over Japanese, Russian, Chinese, French and an emoji, the astral plane surrogate pair, and six malformed inputs that must be refused.seqset_test.vcovers merging, ordering,*, parsing and the fifty thousand message case.response_test.vruns the response reader over transcripts taken from the RFC, including its envelope and body structure examples, plus the two traps above: a body and an envelope subject that both read like protocol.imap_test.vdrives the client over a real socket against a scripted server, checking the exact bytes each command puts on the wire, the literal handshake for APPEND and for an eight bit password, and that an unsolicited EXISTS reaches the client.v fmt -verifyandv vet -Ware clean on all nine files, and none of them show up inv test-cleancode.