SMTP - Error Trapping and Authentication

[Tim Johnson]

Hi Folks:



Just tried out the smtp interface for newlisp and it is the easiest that

I've used. It would be helpful however to have information in parsing

the response string to trap errors. If anyone can point me in the right

direction, I would appreciate it. I.E. Should I always expect to find

the response code as the beginning of the response string?



Rebol and python throw errors if a successful transmission is not

 accomplished. I can observe that a successful transmission returns

221 at the head of the response string.



In addition - although I can sidestep using any SMTP server that requires

authentication - I think it would be very valuable to be able to send

authentication parameters. I've found one thread in this forum on this

matter, but couldn't find a conclusion. Again, I would welcome comments.



thanks

tim

[cormullion]

HI Tim. You might be referring to this thread - http://www.alh.net/newlisp/phpbb/viewtopic.php?p=7611">//http://www.alh.net/newlisp/phpbb/viewtopic.php?p=7611. With that code (oh there's a typo I think - send-str shouldn't be quoted in net-send-get-result) I can send and receive email using simple base64 authentication:

Code Select Expand

220 smtp.example.com ESMTP Service
AUTH PLAIN aGljeXJpbGhvcGV5b3VyZWhhcHB5bm93
sent: AUTH PLAIN aGljeXJpbGhvcGV5b3VyZWhhcHB5bm93
235 2.0.0 OK Authenticated
HELO example.com
sent: HELO example.com
250 example.com Hello dyn.example.com [91.122.83.67], pleased to meet you
MAIL FROM: <a>
 sent: MAIL FROM: <a>
250 2.1.0 <a>... Sender ok
RCPT TO: <a>
sent: RCPT TO: <a>
250 2.1.5 <a>... Recipient ok
DATA
sent: DATA
354 Enter mail, end with "." on a line by itself
TO: a.user@example.com
sent: TO: a.user@example.com
FROM: a.user@example.com
sent: FROM: a.user@example.com
SUBJECT: Greetings
sent: SUBJECT: Greetings
X-Mailer: newLISP v.9302 sent: X-Mailer: newLISP v.9302
sent:
How are you today? sent: How are you today?
.
sent: .
250 2.0.0 m2Q96FNw004236 Message accepted for delivery
QUIT sent: QUIT
221 2.0.0 example.com closing connection
true


Perhaps someone with the right knowledge can take up the story from here.

[Kirill]

Hello, world!



This is my first post, I've been lurking around here for a while. I'd like to make a note of a small bug in smtp.lsp, that could be squashed if anyone volunteers to add support for SMTP authentication.



Current smtp.lsp doubles single dots on a line by themselves. In SMTP, all dots at the beginning of a line should be doubled (cf. RFC2821 sec. 4.5.2), i.e. not only dots on a line by themselves:

-  Before sending a line of mail text, the SMTP client checks the first character of the line.  If it is a period, one additional period is inserted at the beginning of the line.

I have a wish to hack a bit on the SMTP module, but have a constant lack of time.



Best regards,

Kirill



P.S. Kirill is Кирилл in Russian, which also can be spelled as Cyril. So we're two Кириллs here. :-)

[Tim Johnson]

FYI

Regarding my question about error codes, I overlooked something:

send-mail returns nil if the transmission failed.

:-) So never-mind on the error codes.

[Lutz]

The following changed code in smtp.lsp would insert a period in each line starting with a period.

Code Select Expand

(define (mail-send-body )
    (net-send-get-result "")
    (dolist (lne (parse mail-body "rn"))
        (if (starts-with lne ".")
            (net-sent-get-result (append "." lne))
            (net-send-get-result lne)))
    (net-send-get-result "."))


The function to be replaced is at the end of the file. If somebody can test this, I can include the change in the next release. Unfortunately I don't have a possibility to test this at the moment.

[cormullion]

Yes, that works OK for me here, if I add the AUTH PLAIN authentication step. (But then the last version did too.)



I'm still puzzled by the difference between my local copy and the official one:

Code Select Expand

(define (net-send-get-result str conf)
   (set 'send-str (append str "rn"))
   (if debug-flag (println "sent: " send-str))
   (net-send socket 'send-str)
   (if conf (confirm-request conf) true))

(define (net-send-get-result str conf)
   (set 'send-str (append str "rn"))
   (if debug-flag (println "sent: " send-str))
   (net-send socket send-str)
   (if conf (confirm-request conf) true))

The first version is in /usr/share/newlisp/modules/smtp.lsp. How could (net-send socket 'send-str) work? Or have I accidentally modified it...? What's your copy say, Lutz?

[cormullion]

PS: hello Kirill!

[Lutz]

Up to version 9.3.1 the quirky syntax (net-send socket 'send-str) was allowed but since 9.3.2 the (net-send socket send-str) the quote must be omitted (which most people did anyway).

[Tim Johnson]

If somebody can test this, I can include the change in the next release. Unfortunately I don't have a possibility to test this at the moment.

Besides my localhost, I have accounts with two mail servers that require

authentication. I would be happy to test, but being a newbie I would

need the entire code base with modifications for authentication.



tim

[cormullion]

Without newlispdoc comments, my version of smtp.lsp is like this:

Code Select Expand

(context 'SMTP)

(set 'debug-flag nil)

(define (send-mail mail-from mail-to mail-subject mail-body SMTP-server user-name password)
    (and
        (set 'from-hostname (nth 1 (parse mail-from "@")))
        (set 'socket (net-connect SMTP-server 25))
        (confirm-request "2")
        (unless (and (empty? user-name) (empty? password))
           (mail-authorize user-name password))
        (net-send-get-result (append "HELO " from-hostname) "2")
        (net-send-get-result (append "MAIL FROM: <" mail-from ">") "2")
        (net-send-get-result (append "RCPT TO: <" mail-to ">") "2")
        (net-send-get-result "DATA" "3")
        (mail-send-header)
        (mail-send-body)
        (confirm-request "2")
        (net-send-get-result "QUIT" "2")
        (or (net-close socket) true)))

(define (confirm-request conf)
   (and
    (net-receive socket 'recvbuff 256 "rn")
    (if debug-flag (println recvbuff) true)
    (starts-with recvbuff conf)))
 
(define (net-send-get-result str conf)
   (set 'send-str (append str "rn"))
   (if debug-flag (println "sent: " send-str))
   (net-send socket send-str)
   (if conf (confirm-request conf) true))

(define (mail-authorize user-name password)
   (net-send-get-result (append "AUTH PLAIN " (base64-enc (append "00" user-name "00" password))) "235"))

(define (mail-send-header)
    (net-send-get-result (append "TO: " mail-to))
    (net-send-get-result (append "FROM: " mail-from))
    (net-send-get-result (append "SUBJECT: " mail-subject))
    (net-send-get-result (append "X-Mailer: newLISP v." (string (nth -2 (sys-info))))))

(define (mail-send-body )
    (net-send-get-result "")
    (dolist (lne (parse mail-body "rn"))
        (if (starts-with lne ".")
            (net-sent-get-result (append "." lne))
            (net-send-get-result lne)))
    (net-send-get-result "."))

(define (get-error-text)
    recvbuff)

(context 'MAIN)

; eof


and is tested like this:

Code Select Expand

(load "...smtp.lsp")

(set 'SMTP:debug-flag true)

(SMTP:send-mail
 "from@example.com"
 "to@example.com"
 "title"
 "body"
 "smtp.example.com"
 "user.name"
 "password"))


[Lutz]

Thanks Cormuliion, here is a merged version of yours and the last of the distribution. If it checks out well in testing, we can put it in the next release.



also posted here: http://www.newlisp.org/smtp.lsp">http://www.newlisp.org/smtp.lsp

and syntax highlighted: http://www.newlisp.org/syntax.cgi?http://newlisp.org/smtp.lsp">http://www.newlisp.org/syntax.cgi?http: ... g/smtp.lsp">http://www.newlisp.org/syntax.cgi?http://newlisp.org/smtp.lsp

Code Select Expand


;; @module smtp.lsp
;; @description Send mail using SMTP protocol
;; @version 2.0 - 2006-10-08 cormullionx added AUTH PLAIN authentication
;; @author Lutz Mueller 2001 , Cormullion 2008
;;
;; <h2>Routines for sending mail</h2>
;; This module implements routines to communicate with a SMTP mail server
;; for sending email. To use this module include the following 'load' statement
;; at the beginning of the program file:
;; <pre>
;; (load "/usr/share/modules/newlisp/smtp.lsp")
;; </pre>
;; To see debugging information:
;; <pre>(set 'debug-flag true)</pre>

(context 'SMTP)

(set 'debug-flag nil)

;; @syntax (SMTP:send-mail <str-from> <str-to> <str-subject> <str-message> <str-server>i [<str-usr> str-pass>]])
;; @param <str-from> The email address of the sender.
;; @param <str-to> The email address of the recipient.
;; @param <str-subject> The subject line of the email.
;; @param <str-message> The message part of the email.
;; @param <str-server> The address of the SMTP server.
;; @param <str-user> Optional user name for authentication
;; @param <str-pass> Optional password for authentication
;; @return On success 'true', on failure 'nil'.
;; In case the function fails returning 'nil', the function
;; 'SMTP:get-error-text' can be used to receive the error text.
;;
;; @example
;;(SMTP:send-mail "jdoe@asite.com" "somebody@isp.com" "Greetings"
;;   "How are you today? - john doe -" "smtp.asite.com" "jdoe" "secret")

;; This logs in to the server, tries to authenticate using the username 'jdoe' and password 'secret' (if supplied),
;; and sends an email with the format:
;; <pre>
;;  From:    jdoe@asite.com
;;  To:      somebody@isp.com
;;  Subject: Greetings
;;  Message: How are you today? - John Doe -
;; </pre>


(context 'SMTP)

(set 'debug-flag nil)

(define (send-mail mail-from mail-to mail-subject mail-body SMTP-server user-name password)
    (and
        (set 'from-hostname (nth 1 (parse mail-from "@")))
        (set 'socket (net-connect SMTP-server 25))
        (confirm-request "2")
        (unless (and (empty? user-name) (empty? password))
           (mail-authorize user-name password))
        (net-send-get-result (append "HELO " from-hostname) "2")
        (net-send-get-result (append "MAIL FROM: <" mail-from ">") "2")
        (net-send-get-result (append "RCPT TO: <" mail-to ">") "2")
        (net-send-get-result "DATA" "3")
        (mail-send-header)
        (mail-send-body)
        (confirm-request "2")
        (net-send-get-result "QUIT" "2")
        (or (net-close socket) true)))

(define (confirm-request conf)
    (net-receive socket 'recvbuff 256 "rn")
    (if debug-flag (println recvbuff) true)
    ; Empty out pipe. According to SMTP spec, last line has valid code.
    ; added for 1.8 for newLISP 9.2.0
    (while (< 0 (net-peek socket))
        (net-receive socket 'recvbuff 256 "rn")
        (if debug-flag (println recvbuff)))
    (starts-with recvbuff conf))
 
(define (net-send-get-result str conf)
   (set 'send-str (append str "rn"))
   (if debug-flag (println "sent: " send-str))
   (net-send socket send-str)
   (if conf (confirm-request conf) true))

(define (mail-authorize user-name password)
   (net-send-get-result
       (append "AUTH PLAIN "
               (base64-enc (append "00" user-name "00" password))) "235"))

(define (mail-send-header)
    (net-send-get-result (append "TO: " mail-to))
    (net-send-get-result (append "FROM: " mail-from))
    (net-send-get-result (append "SUBJECT: " mail-subject))
    (net-send-get-result (append "X-Mailer: newLISP v." (string (nth -2 (sys-info))))))

(define (mail-send-body )
    (net-send-get-result "")
    (dolist (lne (parse mail-body "rn"))
        (if (starts-with lne ".")
            (net-sent-get-result (append "." lne))
            (net-send-get-result lne)))
    (net-send-get-result "."))

(define (get-error-text)
    recvbuff)

(context 'MAIN)

;  test

; (set 'SMTP:debug-flag true)

; (SMTP:send-mail
;  "from@example.com"
;  "to@example.com"
;  "title"
;  "body"
;  "smtp.example.com"
;  "user.name"
;  "password"))

; eof



[Tim Johnson]

Thanks folks. I will test that out later today. I'm 4 hours "behind" Lutz

(alaska DST), so results will be late.

Tim

[Tim Johnson]

Here are results:

Case 1) Sending authentication user-name and password to

server requiring authentication:

debug info

Code Select Expand

220 sj1-dm103.mta.everyone.net ESMTP EON-AUTHRELAY2 sent: AUTH PLAIN AHRqb2huc29uAGNoaW1heQ== 503 Need HELO/EHLO first
 
Case 2)Sending message thru localhost (no authentication required) with

empty string for username and password

Code Select Expand

220 bart ESMTP Postfix (Ubuntu) - No message sent

Case 3)Sending message thru localhost (no authentication required) with

empty string for username and password - and - the 'unless sexp

commented out

Code Select Expand

220 bart ESMTP Postfix (Ubuntu) sent: HELO johnsons-web.com 250 bart sent: MAIL FROM: 250 2.1.0 Ok sent: RCPT TO: 250 2.1.5 Ok sent: DATA 354 End data with . sent: TO: tim@johnsons-web.com sent: FROM: test@johnsons-web.com sent: SUBJECT: Testing Newlisp sent: X-Mailer: newLISP v.9300 sent: sent: Line one line two sent: . 250 2.0.0 Ok: queued as 162BE80C114 sent: QUIT 221 2.0.0 Bye
- I. E. successful transmission

Observation: In addition to the "Need HELO/EHLO" issue, it appears

to me that a nil return from 'unless is breaking out of the 'and sexp?

Inserting some stubs seems to confirm that....

Thanks - this is fun.

Tim

[cormullion]

Yes, that authentication code is rubbish - who on earth added that? :)



I think there should be a true as the other result of unless:

Code Select Expand

(unless (and (empty? user-name) (empty? password))
    (mail-authorize user-name password)
     true)

because if the two strings are empty this returns nil which stops the and.



As for the HELO, perhaps that should be before the authentication:

Code Select Expand

(net-send-get-result (append "HELO " from-hostname) "2")
(unless (and (empty? user-name) (empty? password))
           (mail-authorize user-name password)
            true)


That seems to work too.

[Lutz]

Thanks for testing and correcting, I also added defaults for user-name and password. This way both new parameters can be left out completely if required.



New version for testing here:



http://www.newlisp.org/smtp.lsp">http://www.newlisp.org/smtp.lsp



and highlighted here:



http://www.newlisp.org/syntax.cgi?http://newlisp.org/smtp.lsp">http://www.newlisp.org/syntax.cgi?http: ... g/smtp.lsp">http://www.newlisp.org/syntax.cgi?http://newlisp.org/smtp.lsp