Sending and Receiving Email - Using Caché Internet Utilities

docs.intersystems.com

Using Caché Internet Utilities

Sending and Receiving Email

InterSystems: The power behind what matters

Caché Web Development > Using Caché Internet Utilities > Sending and Receiving Email

This chapter describes how you can use Caché to send and receive MIME email messages. It discusses the following topics:

Also see the class documentation for examples and extensive comments.

Note:

The examples in this chapter are organized so that the methods for managing email messages can be used with different email servers, which is useful during testing and demonstrations. This is not necessarily the code organization that is most suitable for production needs.

Supported Email Protocols

Email sends messages across the Internet using standard protocols. Caché supports three of these protocols, as follows:

Caché provides an object representation of MIME email messages. It supports text and non-text attachments, single-part or multipart message bodies, and headers in ASCII and non-ASCII character sets. (For more general support of MIME parts, see the chapter “Creating and Writing MIME Messages.”)
You can send email via an SMTP server. SMTP (Simple Mail Transport Protocol) is the Internet standard for sending email.
You can also retrieve email from an email server via POP3, the most common standard for retrieving email from remote servers.

Note:

Caché does not provide a mail server. Instead, it provides the ability to connect to and interact with mail servers.

How Caché Represents MIME Email Messages

First, it is useful to understand how Caché represents MIME email messages.

In general, a multipart MIME message consists of the following pieces:

A set of message headers, each of which contains information such as the address to which the message is sent. This also includes the Mime-Type header and the Content-Type header for the entire message.

For a multipart message, the Content-Type header must be multipart/mixed or some other subtype of multipart; the MIME standard has many variants.
Multiple message parts, each of which consists of the pieces:
- A set of content headers, which includes the Content-Type header and other headers specific to this part.
- A body, which is either text or binary, and which can be in a different character set than the bodies of other parts.

Caché uses two classes to represent email messages: %Net.MailMessage and %Net.MailMessagePart, the superclass of %Net.MailMessage. The following graphic shows the relationship between these classes:

In general:

To represent an ordinary, one-part message, you use %Net.MailMessage
To represent a multipart message, you use %Net.MailMessage as the parent message and you use multiple instances of %Net.MailMessagePart as its parts.

The following sections provide details.

Creating Single-part Email Messages

To create a single-part email message, you use the %Net.MailMessage class. To create a mail message, do the following:

Create an instance of %Net.MailMessage.

Tip:
You can specify a character set as the argument to %New(); if you do so, that sets the Charset property for the message. For information on how this affects the message, see “Automatic Encoding and Character Translation.”
Set the To, From, and Subject properties of your instance.
- To — The list of email addresses to which this message will be sent. This property is a standard Caché list class; to work with it, you use the standard list methods: Insert(), GetAt(), RemoveAt(), Count(), and Clear().
- From — The email address this message is sent from.
- Subject — The subject of the message, if this is required by the SMTP server you are using.
Optionally set Date, Cc, Bcc, and other properties. For details, see “Specifying Basic Email Headers.”
If the message is not plain text, set the following properties to indicate the kind of message you are creating:
- If this is an HTML message, set the IsHTML property to 1.
- If this is a binary message, set the IsBinary property to 1.
To specify the character set of the message and its headers, set the Charset property as needed. (For details on how this affects the message, see “Automatic Encoding and Character Translation.”)

Important:
It is important to specify the character set before you add the contents of the message.
Add the contents of the message:
- For plain text or HTML, use the TextData property, which is an instance of %FileCharacterStream. You do not need to specify the TranslateTable property of this stream; that occurred automatically when you specified the character set of the mail message.
- For binary data, use the BinaryData property, which is an instance of %FileBinaryStream.
Tip:
When you specify the Filename property of the stream, be sure to use a directory to which the users will have write access.

To work with these properties, use the standard stream methods: Write(), WriteLine(), Read(), ReadLine(), Rewind(), MoveToEnd(), and Clear(). You can also use the Size property of the stream, which gives you the size of the message contents.

Note:

You should be aware of the requirements of the SMTP server that you are using. For example, some SMTP servers require that you include a Subject header. Similarly, some SMTP servers do not permit arbitrary From headers.

Similarly, some SMTP servers recognize the Priority header and others recognize X-Priority instead.

Also see “Creating Multipart Email Messages.”

Example 1: CreateTextMessage()

The following method creates a simple message and specifies the addresses for it:

ClassMethod CreateTextMessage() As %Net.MailMessage
{
    Set msg = ##class(%Net.MailMessage).%New()
    set msg.From = "test@test.com"
    Do msg.To.Insert("xxx@xxx.com")
    Do msg.Cc.Insert("yyy@yyy.com")
    Do msg.Bcc.Insert("zzz@zzz.com")
    Set msg.Subject="subject line here"
    Set msg.IsBinary=0
    Set msg.IsHTML=0
    Do msg.TextData.Write("This is the message.")
    
    Quit msg
}

Example 2: SimpleMessage()

You may instead prefer to specify the addresses when you actually send the message (see “Example 3: SendMessage()” in “Using an SMTP Server to Send Email”). The following variation of the preceding example generates a text message with no addresses:

ClassMethod SimpleMessage() As %Net.MailMessage
{
  Set msg = ##class(%Net.MailMessage).%New()
  Set msg.Subject="Simple message "_$h
  Set msg.IsBinary=0
  Set msg.IsHTML=0
  Do msg.TextData.Write("This is the message.")
  Quit msg
}

There are other examples in the SAMPLES namespace. To find them, search for %Net.MailMessage in that namespace.

Creating Multipart Email Messages

To create a multipart email message:

Create an instance of %Net.MailMessage and set its To, From, and Subject properties. Optionally set other properties to specify other message headers.
Set the IsMultiPart property to 1.
Set the MultiPartType property to one of the following: "related", "alternative", or "mixed". This affects the Content-Type header of the entire message.
For each part that the message should contain, create an instance of %Net.MailMessagePart and specify its properties as described in “Creating Single-part Email Messages” — starting with step 4.
For the parent email message, set the Parts property, which is an array. Insert each child message part into this array.

When you send the message, the %Net.SMTP class automatically sets the Content-Type header for the message as appropriate (given the value of the MultiPartType property).

Specifying Email Message Headers

As noted previously, both the message itself and each part of a message has a set of headers.

The %Net.MailMessage and %Net.MailMessagePart classes provide properties that give you easy access to the most commonly used headers, but you can add any header you need. This section provides information on all the headers as well as how to create custom headers.

The headers of a given message part are in the character set specified by the Charset property of that part.

Note:

Similarly, some SMTP servers recognize the Priority header and others recognize X-Priority instead.

Specifying Basic Email Headers

Set the following properties (only in %Net.MailMessage) to set the most commonly used headers of the message itself:

To — (Required) The list of email addresses to which this message will be sent. This property is a standard Caché list; to work with it, you use the standard list methods: Insert(), GetAt(), RemoveAt(), Count(), and Clear().
From — (Required) The email address this message is sent from.
Date — The date of this message.
Subject — (Required) A string containing the subject for this message.
Sender — The actual sender of the message.
Cc — The list of carbon copy addresses to which this message will be sent.
Bcc — The list of blind carbon copy addresses to which this message will be sent.

Content-Type Header

When you send the message, the Content-Type header for the message and for each message part is automatically set as follows:

If the message is plain text (IsHTML equals 0 and IsBinary equals 0), the Content-Type header is set to "text/plain".
If the message is HTML (IsHTML equals 1 and IsBinary equals 0), the Content-Type header is set to "text/html".
If the message is binary (IsBinary equals 1), the Content-Type header is set to "application/octet-stream".
If the message is multipart, the Content-Type header is set as appropriate for the value of the MultiPartType property.

Both %Net.MailMessage and %Net.MailMessagePart provide the ContentType property, which gives you access to the Content-Type header.

Content-Transfer-Encoding Header

Both %Net.MailMessage and %Net.MailMessagePart provide the ContentTransferEncoding property, which provides an easy way to specify the Content-Transfer-Encoding header of the message or the message part.

This property can be one of the following: "base64" "quoted-printable" "7bit" "8bit"

The default is as follows:

For a binary message or message part: "base64"
For a text message or message part: "quoted-printable"

Also see “Automatic Encoding and Character Translation.”

Custom Headers

With both %Net.MailMessage and %Net.MailMessagePart, you can set or get custom headers by accessing the Headers property, which is an array with the following structure:

Array Key	Array Value
Name of the header, such as "Priority"	Value of the header

You use this property to contain additional headers such as X-Priority and others. For example:

 do msg.Headers.SetAt(1,"X-Priority")
 do msg.Headers.SetAt("High","X-MSMail-Priority")
 do msg.Headers.SetAt("High","Importance")

Different email servers and clients recognize different headers, so it can be useful to set multiple similar headers to be sure that the server or client receives a message with a header it can recognize.

Adding Attachments to a Message

You can add attachments to an email message or message part (specifically, to an instance of %Net.MailMessagePart or %Net.MailMessage). To do so, use the following methods:

Each of these methods adds the attachment to the Parts array of the original message (or message part), and automatically sets the IsMultiPart property to 1.

AttachFile()

method AttachFile(Dir As %String, 
                  File As %String, 
                  isBinary As %Boolean = 1, 
                  charset As %String = "", 
                  ByRef count As %Integer) as %Status

Attaches the given file to the email message. By default the file is sent as a binary attachment, but you can specify instead that it is text. You can also specify the character set that the file uses if it is text.

Specifically, this method creates an instance of %Net.MailMessagePart and places the contents of the file in the BinaryData or TextData property as appropriate, and sets the Charset property and TextData.TranslateTable properties if needed. The method returns, by reference, an integer that indicates the position of this new message part within the Parts array.

This method also sets the Dir and FileName properties of the message or message part.

AttachStream()

method AttachStream(stream As %Stream.Object, 
                    Filename As %String, 
                    isBinary As %Boolean = 1, 
                    charset As %String = "", 
                    ByRef count As %Integer) as %Status

Attaches the given stream to the email message. The attachment is considered a file attachment if Filename is specified. Otherwise it is considered an inline attachment. See the comments for AttachFile().

AttachNewMessage()

method AttachNewMessage() as %Net.MailMessagePart

Creates a new instance of %Net.MailMessage, adds it to the message, and returns the newly modified parent message or message part.

AttachEmail()

method AttachEmail(mailmsg As %Net.MailMessage)

Given an email message (an instance of %Net.MailMessage), this method adds it to the message. This method also sets the Dir and FileName properties of the message or message part.

Note:

This method sets ContentType to "message/rfc822". In this case, you cannot add any other attachments.

Example: MessageWithAttachment()

The following example generates a simple email message with one hardcoded attachment. It does not provide any addresses for the message; you can provide that information when you actually send the message (see “Example 3: SendMessage()” in “Using an SMTP Server to Send Email”).

ClassMethod MessageWithAttachment() As %Net.MailMessage
{
  Set msg = ##class(%Net.MailMessage).%New()
  Set msg.Subject="Message with attachment "_$h
  Set msg.IsBinary=0
  Set msg.IsHTML=0
  Do msg.TextData.Write("This is the main message body.")

  //add an attachment
  Set status=msg.AttachFile("c:\", "GNET.pdf")
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status)
    Quit $$$NULLOREF
  }
  
  Quit msg
}

For other examples, see the class reference for the %Net.MailMessagePart class.

Using an SMTP Server to Send Email

If you have access to an SMTP server, you can send email messages. The SMTP server must be running and you must have the needed permissions to use it. To send email, do the following:

Create an instance of %Net.SMTP and set its properties as needed, especially the following:
- smtpserver is the name of the SMTP server you are using.
- port is the port you are using on the SMTP server; the default is 25.
- timezone specifies the time zone of the server, as specified by RFC 822, for example "EST" or "-0400" or "LOCAL". If this is not set, the message uses universal time.
This object describes the SMTP server you will use.
If the SMTP server requires authentication, specify the necessary credentials. To do so:
1. Create an instance of %Net.Authenticator.
2. Set the UserName and Password properties of this object.
3. Set the authenticator property of your %Net.SMTP instance equal to this object.
4. If the message itself has an authorized sender, set the AuthFrom property of your %Net.SMTP instance.
To use an SSL/TLS connection to the SMTP server:
1. Set the SSLConfiguration property to the name of the activated SSL/TLS configuration to use.
  
  For information on creating and managing SSL/TLS configurations, see “Using SSL/TLS with Caché” in the Caché Security Administration Guide. The SSL/TLS configuration includes an option called Configuration Name, which is the string to use in this setting.
2. Set the UseSTARTTLS property to either 0 or 1.
  
  In most cases, use the value 0. Use the value 1 for the case in which the server interaction begins on a normal TCP socket and then switches to TLS on the same port as the normal socket. For details, see RFC 3207.
3. Optionally set the SSLCheckServerIdentity property to 1. Do this if you want to verify the host server name in the certificate.
Create the email message to send (as described in “Creating Single-part Email Messages” and “Creating Multipart Email Messages”).
Call the Send() method of your SMTP instance. This method returns a status, which you should check.
If the returned status indicates an error, check the Error property, which contains the error message itself.
Check the FailedSend property, which contains a list of email addresses for which the send action failed.

The examples in the following sections use a couple of different free SMTP services that were available at the time this manual was written. No particular endorsement is implied by the selection of these services. Also note that the examples do not show the actual passwords.

There are other examples in the SAMPLES namespace. To find them, search for %Net.SMTP in that namespace. Also see the class documentation for %Net.SMTP.

Important:

%Net.SMTP writes the message body into a temporary file stream. By default, this file is written to the namespace directory and if the directory requires special write permissions, the file is not created and you get an empty message body.

You can define a new path for these temporary files and choose a path that does not restrict write access (for example, /tmp). To do so, set the global node %SYS("StreamLocation",namespace ) where namespace is the namespace in which your code is running. For example:

Set ^%SYS("StreamLocation","SAMPLES")="/tmp"

If %SYS("StreamLocation",namespace ) is null, then Caché uses the directory specified by %SYS("TempDir",namespace ). If %SYS("TempDir",namespace ) is not set, then Caché uses the directory specified by %SYS("TempDir")

Example 1: HotPOPAsSMTP() and SendSimpleMessage()

This example consists of two methods that you use together. The first creates an instance of %Net.SMTP that uses a test account that has already been set up on the HotPOP SMTP server:

ClassMethod HotPOPAsSMTP() As %Net.SMTP
{
  Set server=##class(%Net.SMTP).%New()
  Set server.smtpserver="smtp.hotpop.com"
  //HotPOP SMTP server uses the default port (25)
  Set server.port=25
  
  //Create object to carry authentication
  Set auth=##class(%Net.Authenticator).%New()
  Set auth.UserName="isctest@hotpop.com"
  Set auth.Password="123pass"
  
  Set server.authenticator=auth
  Set server.AuthFrom=auth.UserName
  Quit server
}

The next method sends a simple, unique message, using an SMTP server that you provide as the argument:

ClassMethod SendSimpleMessage(server As %Net.SMTP) As %List
{
  Set msg = ##class(%Net.MailMessage).%New()
  Set From=server.authenticator.UserName
  Set:From="" From="xxx@xxx.com"
  Set msg.From = From
  
  Do msg.To.Insert("xxx@xxx.com")
  //Do msg.Cc.Insert("yyy@yyy.com")
  //Do msg.Bcc.Insert("zzz@zzz.com")
  Set msg.Subject="Unique subject line here "_$H
  Set msg.IsBinary=0
  Set msg.IsHTML=0
  Do msg.TextData.Write("This is the message.")
  
  Set status=server.Send(msg)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status)
    Write server.Error
    Quit ""
  }
  Quit server.FailedSend
}

Example 2: YPOPsAsSMTP()

This example creates an instance of an instance of %Net.SMTP that uses YPOPs, which is client software that provides SMTP and POP3 access to a Yahoo email account. It uses a test account that has already been set up for this purpose:

ClassMethod YPOPsAsSMTP() As %Net.SMTP
{
  Set server=##class(%Net.SMTP).%New()
  //local host acts as the server
  Set server.smtpserver="127.0.0.1"
  //YPOPs uses default port, apparently
  Set server.port=25
  
  //Create object to carry authentication
  Set auth=##class(%Net.Authenticator).%New()
  //YPOPs works with a Yahoo email account
  Set auth.UserName="isc.test@yahoo.com"
  Set auth.Password="123pass"
  
  Set server.authenticator=auth
  Set server.AuthFrom=auth.UserName
  Quit server
}

You can use this with the SendSimpleMessage method shown in the previous example.

Example 3: SendMessage()

The following, more flexible method accepts both an SMTP server and an email message. The email message should already include a subject line (if required by the SMTP server), but does not have to include addresses. This method then sends the email message to a set of hardcoded test destinations:

ClassMethod SendMessage(server As %Net.SMTP, msg as %Net.MailMessage) as %Status
{
  Set From=server.authenticator.UserName
  //make sure From: user is same as used in authentication
  Set msg.From = From
  
  //finish addressing the message
  Do msg.To.Insert("xxx@xxx.com")
  //send the message to various test email addresses
  Do msg.To.Insert("isctest@hotpop.com")
  Do msg.To.Insert("isc_test@hotmail.com")
  Do msg.To.Insert("isctest001@gmail.com")
  Do msg.To.Insert("isc.test@yahoo.com")
  
  Set status=server.Send(msg)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status)
    Write server.Error
    Quit $$$ERROR($$$GeneralError,"Failed to send message")
  }
  Quit $$$OK
}

This example is meant for use with the example methods SimpleMessage and MessageWithAttachment described in “Adding Attachments to a Message”.

Other Properties of %Net.SMTP

The %Net.SMTP class also has some other properties that you might need, depending on the SMTP server you are using:

AllowHeaderEncoding specifies whether the Send() method encodes non-ASCII header text. The default is 1, which means that non-ASCII header text is encoded as specified by RFC 2047.
ContinueAfterBadSend specifies whether to continue trying to send a message after detecting a failed email address. If ContinueAfterBadSend is 1, the system will add the failed email address to the list in the FailedSend property. The default is 0.
ShowBcc specifies whether the Bcc headers are written to the email message. These will normally be filtered out by the SMTP server.

Fetching Email from a POP3 Server

This section discusses how to use the %Net.POP3 class to fetch email messages. It includes the following topics:

Also see the class documentation for %Net.FetchMailProtocol for examples and extensive comments. %Net.FetchMailProtocol is the abstract superclass of %Net.POP3.

Communicating with a POP3 Server

If you have the needed permissions and if the mail server is running, you can download and process email messages from it using the POP3 protocol. In general, to communicate with a POP3 server, you log in, perform a series of actions that affect a mailbox, and then either commit or roll back any changes. To do this in Caché:

Create an instance of %Net.POP3. This object describes the POP3 server you will use.
Optionally specify the following properties of your instance of %Net.POP3:
- port — Specifies the port you will use; the default is 110.
- timeout — Specifies the read timeout in seconds; the default is 30 seconds.
- StoreAttachToFile — Specifies whether to save each attachment to a file, when a message is read (when the message includes the content-disposition; attachment header). The default is false.
  
  Note that this setting does nothing unless AttachDir is also set.
- StoreInlineToFile — Specifies whether to save each inline attachment to a file, when a message is read (when the message includes the content-disposition; inline header). The default is false.
  
  Note that this setting does nothing unless AttachDir is also set.
- AttachDir — Specifies the directory into which the attachment are saved. There is no default. Make sure to terminate the name of the directory with a slash (/) or backslash (\), as appropriate for the operating system. Also make sure that this is directory already exists, and the users have write access to it.
- IgnoreInvalidBase64Chars — Specifies whether to ignore invalid characters found during base–64 decoding. The default is false (and invalid characters result in an error). Note that RFC 2045 is ambiguous about whether unexpected characters should be ignored or should result in an error during base–64 decoding.
To use an SSL/TLS connection to the POP3 server:
1. Set the SSLConfiguration property to the name of the activated SSL/TLS configuration to use.
  
  For information on creating and managing SSL/TLS configurations, see “Using SSL/TLS with Caché” in the Caché Security Administration Guide. The SSL/TLS configuration includes an option called Configuration Name, which is the string to use in this setting.
2. Set the UseSTARTTLS property to either 0 or 1.
  
  In most cases, use the value 0. Use the value 1 for the case in which the server interaction begins on a normal TCP socket and then switches to TLS on the same port as the normal socket. For details, see RFC 2595.
3. Optionally set the SSLCheckServerIdentity property to 1. Do this if you want to verify the host server name in the certificate.
Call the Connect() method of your instance. This method takes three arguments, in order:
1. The name of the POP3 server
2. A username
3. A password
Use the methods of your instance to examine the mailbox, retrieve messages, and delete messages. The following sections provide details.
Optionally, to prevent the connection from timing out, call the Ping() method of your %Net.POP3 instance.
Optionally, if you have marked messages for deletion but now choose not to delete them, call the RollbackDeletes() method of your %Net.POP3 instance.
When you are done making changes to the mailbox, call one of the following methods:
- QuitAndCommit() — Commits your changes and logs out of the mail server.
- QuitAndRollback() — Rolls back your changes and logs out of the mail server.

Each of these methods returns a status, which you should check before continuing. Also see the class reference for %Net.POP3 for complete method signatures.

The examples in the following sections use two different free POP3 services that were available at the time this manual was written. No particular endorsement is implied by the selection of these services. Also note that the examples do not show the actual passwords.

Example 1: HotPOPAsPOP3()

The following method logs into the HotPOP POP3 server using an account that was previously set up for this purpose:

ClassMethod HotPOPAsPOP3() As %Net.POP3
{
  Set server=##class(%Net.POP3).%New()
  
  //HotPOP POP3 server uses the default port
  //but let's set it anyway
  Set server.port=110

  //just in case we plan to fetch any messages
  //that have attachments
  Set server.StoreAttachToFile=1
  Set server.StoreInlineToFile=1
  Set server.AttachDir="c:\DOWNLOADS\"

  Set servername="pop.hotpop.com"
  Set user="isctest@hotpop.com"
  Set pass="123pass"
  
  Set status=server.Connect(servername,user,pass)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Quit server
}

This method returns the %Net.POP3 server instance. Many of the examples later in this chapter accept the %Net.POP3 instance as an argument.

Example 2: YPOPsAsPOP3()

The following method also returns a %Net.POP3 server instance. In this case, we are using YPOPs, which is client software that provides SMTP and POP3 access to a Yahoo email account. It uses a test account that has already been set up for this purpose:

ClassMethod YPOPsAsPOP3() As %Net.POP3
{
  Set server=##class(%Net.POP3).%New()
  
  //YPOPs uses the default port 
  //but let's set it anyway
  Set server.port=110

  //just in case we plan to fetch any messages
  //that have attachments
  Set server.StoreAttachToFile=1
  Set server.StoreInlineToFile=1
  Set server.AttachDir="c:\DOWNLOADS\"
  
  //local host acts as the server
  Set servername="127.0.0.1"
  //YPOPs works with a Yahoo email account
  Set user="isc.test@yahoo.com"
  Set pass="123pass"
  
  Set status=server.Connect(servername,user,pass)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Quit server
}

Getting Information about the Mailbox

While you are connected to a POP3 server, you are logged into a user account and have access to the mailbox for that user account. Use the following methods to find what the mailbox contains:

GetMailBoxStatus()

Returns, by reference, the number of messages in the mailbox and the number of bytes that the mailbox uses.

GetMessageUIDArray()

If given an empty string as the first argument, this method returns, by reference, an array of information about the messages in the mailbox (excluding any that are currently marked for deletion). Each element in this array contains the following information about one message:

Array Key	Array Item
Number of the message, within the mailbox in its current state. The first message is number 1, and so on. The message number of a given message is not guaranteed to be the same in all sessions.	Unique message identifier (UID), which is the permanent identifier of this message available in all sessions. UIDs are unique to each mailbox.

GetSizeOfMessages()

Array Key	Array Item
Number of the message, within the mailbox in its current state.	Size of this message, in bytes.

Each of these methods returns a status, which you should check before continuing. Also see “Other Message Retrieval Methods” for more details on these methods.

Example: ShowMailbox()

For example, the following method writes information about the mailbox that we are currently accessing:

ClassMethod ShowMailbox(server as %Net.POP3) 
{
    Set status=server.GetMailBoxStatus(.count,.size)
    If $$$ISERR(status) {
       Do $System.Status.DisplayError(status) 
       Quit 
    }
    Write "Mailbox information *****",!
    Write "Number of messages in mailbox: ",count,!
    Write "Size of messages: ",size,!

    Set status=server.GetMessageUIDArray(,.uids)
    Set status=server.GetSizeOfMessages(,.sizes)
    
    //iterate through messages, get info, and write it
    For i=1:1:count {
        Set uid=uids.GetAt(i)
        Set size=sizes.GetAt(i)
        Write "Msg number:", i,"   UID:",uid, "   size:",size,!
    }

}

This method generates output similar to the following:

Mailbox information *****
Number of messages in mailbox: 4
Size of messages: 18634
Msg number:1   UID:6ef78df6fd660391   size:7245
Msg number:2   UID:7410041a6faf4a87   size:5409
Msg number:3   UID:5555af7fa489e406   size:5121
Msg number:4   UID:299ad2b54c01a6be   size:859

Fetching Messages from the Mailbox

To simply get a message, use one of the following methods of the %Net.POP3 class:

Fetch()

Given a message number as the first argument, this method returns (by reference, as the second argument) an instance of %Net.MailMessage that contains that message.

FetchMessage()

Given a message number as the first argument, this method returns (by reference) information such as the From and To and other common headers, an array containing all the headers (including the common ones), and the message contents themselves

Each of these methods returns a status, which you should check before continuing. Note that these methods return an error status if the message is currently marked for deletion.

Also see “Other Message Retrieval Methods,” which shows the complete method signatures for Fetch() and FetchMessage()

Example: FetchMailbox()

The following example is a variation of the ShowMailbox example described in “Getting Information about the Mailbox”. This method uses the Fetch() method, examines each message, and writes the subject line of each message:

ClassMethod FetchMailbox(server As %Net.POP3)
{
  Set status=server.GetMailBoxStatus(.count,.size)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Write "Mailbox information *****",!
  Write "Number of messages in mailbox: ",count,!
  Write "Size of messages: ",size,!

  Set status=server.GetMessageUIDArray(,.uids)
  Set status=server.GetSizeOfMessages(,.sizes)
  
  //iterate through messages, get info, and write it
  For i=1:1:count {
    Set uid=uids.GetAt(i)
    Set size=sizes.GetAt(i)
    Set status=server.Fetch(i,.msg)
    If $$$ISERR(status) {
      Set subj="***error***"
    } else{
      Set subj=msg.Subject
    }
    Write "Msg number:", i,"  UID:",uid, "  Size:",size
    Write "  Subject: ",subj,!
  }
}

Saving Attachments as Files

The Content-Disposition header might specify attachment, with or without a filename. For example:

Content-Disposition: attachment; filename=genome.jpeg;

If the Content-Disposition header does specify attachment, your %Net.POP3 instance can save all attachments in the message to files. To make this happen:

Specify the following properties of your %Net.POP3 instance:
- Specify StoreAttachToFile as 1.
- Specify StoreInlineToFile as 1.
- Specify a valid directory for AttachDir. Make sure to terminate the name of the directory with a slash (/) or backslash (\), as appropriate for the operating system. Also make sure that this is directory already exists, and the users have write access to it.
Call Fetch() or FetchMessage() of your %Net.POP3 instance.

Each filename is determined as follows:

If the Content-Disposition header specifies a filename, that filename is used.
Otherwise, if the Content-Type header specifies a filename, that filename is used.
Otherwise, the system creates a name of the form ATTxxxxxx.dat.

Note the following points:

If the file already exists, the attachment is not downloaded.
There is no default for AttachDir.
The size of the attachment is not limited by Caché but might be limited by the file system.
The Dir and FileName properties are not used here. They are relevant only when you upload an attachment to a mail message, as described in “Adding Attachments to a Message”.

Example: GetMsg()

The following example method retrieves an entire message, given an instance of %Net.POP3 and a message number:

ClassMethod GetMsg(server as %Net.POP3,msgno as %Integer) as %Net.MailMessage 
{
    Set status=server.Fetch(msgno,.msg)
    If $$$ISERR(status) {
       Do $System.Status.DisplayError(status) 
       Quit $$$NULLOREF
    }
    Quit msg
}

If the message had attachments, and if you specified the StoreAttachToFile, StoreInlineToFile, and AttachDir properties of the %Net.POP3 server, those attachments would be saved to the given directory when you called this method.

Getting Attached Email Messages

While you are connected to a mailbox, you can download any email messages that are attached to the email messages in the inbox. To do so, use the GetAttachedEmail() method of your %Net.POP3 instance to retrieve the contents of the enclosed email.

Given an instance of %Net.MailMessagePart, this method returns a single-part message that has contents of that message part. Specifically, it returns (as an output parameter) an instance of %Net.MailMessage initialized with the data taken from the attached email message.

Other Message Retrieval Methods

This section lists all the methods of %Net.POP3 that you can use to examine and retrieve messages.

Fetch()

method Fetch(MessageNumber As %Integer, 
             ByRef MailMsg As %Net.MailMessage,
             Delete As %Boolean = 0,
             messageStream As %BinaryStream) as %Status

Returns (by reference) the message indicated by MessageNumber and optionally marks the message for deletion. Note that this method returns an error status if the message is already marked for deletion.

If messageStream is specified, then the original message is written to this binary stream.

FetchFromStream()

method FetchFromStream(messageStream As %BinaryStream, ByRef Msg As %Net.MailMessage) as %Status

This method is for use when you specify the messageStream argument for Fetch().

Retrieves a single email message from the given binary stream. messageStream must be a binary stream containing the message. The message is returned by reference in Msg. This could be a multipart message.

FetchMessage()

method FetchMessage(MessageNumber As %Integer, 
                    ByRef From As %String, 
                    ByRef To As %String, 
                    ByRef Date As %String, 
                    ByRef Subject As %String, 
                    ByRef MessageSize As %Integer, 
                    ByRef MsgHeaders As %ArrayOfDataTypes, 
                    ByRef MailMsg As %Net.MailMessage, 
                    Delete As %Boolean = 0) as %Status

Returns (by reference) specific message headers, the message size, the message header array, and the message itself and optionally marks the message for deletion. Note that this method returns an error status if the message is already marked for deletion.

FetchMessageHeaders()

method FetchMessageHeaders(MessageNumber As %Integer, 
                           ByRef MsgHeadersArray As %String) as %Status

Given a message number, this method returns (by reference) an array containing all the headers of that message. This method returns an error status if the message is currently marked for deletion.

FetchMessageInfo()

method FetchMessageInfo(MessageNumber As %Integer, 
                        Lines As %Integer, 
                        ByRef From As %String, 
                        ByRef To As %String, 
                        ByRef Date As %String, 
                        ByRef Subject As %String, 
                        ByRef MessageSize As %Integer, 
                        ByRef MsgHeaders As %ArrayOfDataTypes, 
                        ByRef MessageText As %String) as %Status

Given a message number, this method returns (by reference) specific message headers, the message size, the message header array, and the given number of lines of text from this message. This method returns an error status if the message is currently marked for deletion.

GetAttachedEmail()

method GetAttachedEmail(msgpart As %Net.MailMessagePart, 
       Output mailmsg As %Net.MailMessage) as %Status

Given a message part, this method returns (as an output parameter) a single-part email message that is initialized with the data from the message part.

GetMessageUID()

method GetMessageUID(MessageNumber As %Integer, 
                     ByRef UniqueID As %String) as %Status

Returns, by reference, the UID of a message, given a message number. See the previous section for details on message numbers and UIDs. This method returns an error status if the message is currently marked for deletion.

GetMessageUIDArray()

method GetMessageUIDArray(MessageNumber As %String = "", 
                          ByRef ListOfUniqueIDs As %ArrayOfDataTypes) as %Status

Array Key	Array Item
Number of the message, within the mailbox in its current state. The first message is number 1, and so on. The message number of a given message is not guaranteed to be the same in all sessions.	Unique message identifier (UID), which is the permanent identifier of this message available in all sessions. UIDs are unique to each mailbox.

Or, given a message number, this method returns a one-element array that contains the UID of that message. In this case, the method returns an error status if the message is currently marked for deletion.

GetSizeOfMessages()

method GetSizeOfMessages(MessageNumber As %String = "", 
                         ByRef ListOfSizes As %ArrayOfDataTypes) as %Status

Array Key	Array Item
Number of the message, within the mailbox in its current state.	Size of this message, in bytes.

Or, given a message number, this method returns a one-element array that contains the size (in bytes) of that message. In this case, this method returns an error status if the message is currently marked for deletion.

Deleting Messages

While you are connected to a mailbox, you can mark messages for deletion in the mailbox that you are logged into. You can do this in a couple of ways.

You can use the DeleteMessage() method. This method takes one argument, the message number to delete.
When you retrieve a message with the Fetch() or FetchMessage() method, you can specify an optional argument that tells the POP3 server to mark the message for deletion after you have retrieved it.

Remember the following points:

These methods do not delete a message; they mark it for deletion. The message is not deleted until you complete the POP3 transaction with QuitAndCommit(). If you simply disconnect from the server, your changes are discarded.
You can call the RollbackDeletes() method to change the messages so that they are no longer marked for deletion.
Each of these methods returns a status, which you should check.

Example: GetMsgAndDelete() and CommitChanges()

The following example retrieves a message and marks it for deletion:

ClassMethod GetMsgAndDelete(ByRef server As %Net.POP3, msgno As %Integer) As %Net.MailMessage
{
  //third argument to Fetch says whether to 
  //mark for deletion
  Set status=server.Fetch(msgno,.msg,1)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  
  Quit msg
}

Note that this message returns (by reference) an altered version of the %Net.POP3; the altered version contains the information about which message is marked for deletion.

You would use the preceding method with a method like the following:

ClassMethod CommitChanges(server As %Net.POP3) As %Status
{
  //commit all changes and log out
  Set status=server.QuitAndCommit()
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$ERROR($$$GeneralError,"Failed to commit changes")
  }
  Quit $$$OK
}

Alternatively you would roll back the changes with RollbackDeletes() or QuitAndRollback().

Working with a Received Email Message

This section describes how you can work with an email message (%Net.MailMessage) that you have retrieved via %Net.POP3.

Message Basics

After you retrieve an email message (%Net.MailMessage), you generally start by determining what kind of message it is and how to read it; that is, whether it is a multipart message and whether the parts are binary. In this step, you can use the ContentType property. Or you can use the IsBinary, IsHTML, and IsMultiPart properties, which indirectly provide the same information as ContentType.

If the message is a multipart message, each part is an instance of %Net.MailMessagePart.

Message Headers

Both the message itself and each part of a message has a set of headers.

The %Net.MailMessage and %Net.MailMessagePart classes provide properties that give you easy access to the most commonly used headers. For example, %Net.MailMessage provides properties such as To, From, Subject, and Date. The Headers array property lets you access any custom header; see “Specifying Email Message Headers.”

Also, if you have retrieved a message via %Net.POP3, you can use the GetAttribute() method. Given a header name and an attribute, this method returns the value of that attribute.

Message Contents

Once you know what the general message structure is, use the following techniques to retrieve the contents:

For a multipart message, use the Parts property, which is an array of the parts. Parts.Count() gives you the number of parts. The key for each part is an integer, starting with 1. Use the GetAt() method to retrieve a given part. A message part is an instance of %Net.MailMessagePart.

For information on the relationship of %Net.MailMessage and %Net.MailMessagePart, see “How Caché Represents MIME Email Messages.”
For a binary message (or message part), use the BinaryData property.
For a text message (or message part), use the TextData property.
- If IsHTML is 0, the TextData property is an ordinary text string.
- If IsHTML is 1, the TextData property is an HTML text string.

Note that the email client that sends a message determines any wrapping in the message. The mail server has no control over this; nor does Caché.

Other Message Information

The MessageSize property indicates the total length of the message, apart from any attached email messages.

The following methods provide additional information about the message:

GetLocalDateTime()

Returns the date and time when the message was retrieved, converted to local time in $HOROLOG format.

GetUTCDateTime()

Returns the date and time when the message was retrieved, converted to UTC in $HOROLOG format.

GetUTCSeconds()

Returns the date and time when the message was retrieved, in seconds since 12/31/1840.

The following class methods are also available for time/date conversion:

HToSeconds()

A class method that converts a date/time in $HOROLOG format to seconds since 12/31/1840.

SecondsToH()

A class method that converts seconds since 12/31/1840 to a date/time in $HOROLOG format.

Example 1: ShowMsgInfo()

Given an instance of %Net.MailMessage, the following method writes information about the message to the current device:

ClassMethod ShowMsgInfo(msg as %Net.MailMessage)
{
    Write "Message details *****",!
    Write "To (count): ", msg.To.Count(),!
    Write "From: ", msg.From,!
    Write "Cc (count): ", msg.Cc.Count(),!
    Write "Bcc (count): ", msg.Bcc.Count(),!
    Write "Date: ", msg.Date,!
    Write "Subject: ", msg.Subject,!
    Write "Sender: ", msg.Sender,!
    Write "IsMultipart: ", msg.IsMultiPart,!
    Write "Number of parts: ", msg.Parts.Count(),!
    Write "Number of headers: ", msg.Headers.Count(),!
    Write "IsBinary: ", msg.IsBinary,!
    Write "IsHTML: ", msg.IsHTML,!
    Write "TextData: ", msg.TextData.Read(),!
    Write "BinaryData: ", msg.BinaryData.Read(),!
}

This method produces output similar to the following:

Message details *****
To (count): 1
From: "XXX XXX" <XXX@XXX.com>
Cc (count): 0
Bcc (count): 0
Date: Fri, 16 Nov 2007 11:57:46 -0500
Subject: test 5
Sender:
IsMultipart: 0
Number of parts: 0
Number of headers: 16
IsBinary: 0
IsHTML: 0
TextData: This is test number 5, which is plain text.
BinaryData:

Example 2: ShowMsgPartInfo()

The following method writes information about a part of a message:

ClassMethod ShowMsgPartInfo(msg as %Net.MailMessage, partno as %Integer)
{
    Set part=msg.Parts.GetAt(partno)
    Write "Message part details *****",!
    Write "Message part: ", partno,!
    Write "IsMultipart: ", part.IsMultiPart,!
    Write "Number of parts: ", part.Parts.Count(),!
    Write "Number of headers: ", part.Headers.Count(),!
    Write "IsBinary: ", part.IsBinary,!
    Write "IsHTML: ", part.IsHTML,!
    Write "TextData: ", part.TextData.Read(),!
    Write "BinaryData: ", part.BinaryData.Read(),!
}

This produces output similar to the following (given a different message than previously shown):

Message part details *****
Message part: 1
IsMultipart: 0
Number of parts: 0
Number of headers: 2
IsBinary: 0
IsHTML: 0
TextData: 1 test string
 
 
BinaryData:

Example 3: ShowMsgHeaders()

The following method writes information about the headers of a message; you could write a similar method that did the same for a message part.

ClassMethod ShowMsgHeaders(msg as %Net.MailMessage)
{
    Set headers=msg.Headers
    Write "Number of headers: ", headers.Count(),!
    
    //iterate through the headers
    Set key=""
    For {
        Set value=headers.GetNext(.key) 
        Quit:key=""  
        Write "Header:",key,!
        Write "Value: ",value,!!
    }
}

This produces output similar to the following:

Number of headers: 16
Header: content-class
Value: urn:content-classes:message
 
Header: content-type
Value: multipart/alternative; boundary="----_=_NextPart_001_01C8286D.D9A7F3B1"
 
Header: date
Value: Fri, 16 Nov 2007 11:29:24 -0500
 
Header: from
Value: "XXX XXX" <XXX@XXX.com>
 
Header: message-id
Value: <895A9EF10DBA1F46A2DDB3AAF061ECD501801E86@Exchange1_backup>
 
Header: mime-version
Value: 1.0
 
...

Automatic Encoding and Character Translation

A email message part contains information about both the character sets used and the content-transfer-encoding used (if any). For reference, this section describes how this information is used.

For background information on character translation in Caché, see “Localization Support” in the Caché Programming Orientation Guide.

Outgoing Email

%Net.SMTP checks the Charset property of each part and then applies the appropriate translation table.

If you do not specify the Charset property for a given part, when you send the message, the following defaults are used for this part:

On Unicode systems, the default is UTF-8.
On 8–bit systems, the default is the system default character set.

%Net.SMTP also checks the ContentTransferEncoding property. If this property is "base64" or "quoted-printable", then when it creates the message, %Net.SMTP encodes the body as needed. (If the content transfer encoding is "7bit" or "7bit", no encoding is needed.)

Incoming Email

%Net.POP3 checks the Content-Transfer-Encoding header of each message part and decodes the body as needed.

Then %Net.POP3 checks the Content-Type header of each message part. This affects the Charset property of the message part and also controls the translation table used when the message part is created in Caché.