Retrieving Email via POP3

Overview

If you have the needed permissions and if the mail server is running, you can download and process email messages from it using the POP3 protocol. In general, to communicate with mail server via POP3, you log in, perform a series of actions that affect a mailbox, and then either commit or roll back any changes. To do this in InterSystems IRIS:

1. Create an instance of %Net.POP3Opens in a new tab. This object describes the mail server you will use.

2. Optionally specify the following properties of your instance of %Net.POP3Opens in a new tab:

   • port — Specifies the port you will use; the default is 110.

   • timeout — Specifies the read timeout in seconds; the default is 30 seconds.

   • StoreAttachToFile — Specifies whether to save each attachment to a file, when a message is read (when the message includes the content-disposition; attachment header). The default is false.

     Note that this setting does nothing unless AttachDir is also set.

   • StoreInlineToFile — Specifies whether to save each inline attachment to a file, when a message is read (when the message includes the content-disposition; inline header). The default is false.

     Note that this setting does nothing unless AttachDir is also set.

   • AttachDir — Specifies the directory into which the attachment are saved. There is no default. Make sure to terminate the name of the directory with a slash (/) or backslash (\), as appropriate for the operating system. Also make sure that this is directory already exists, and the users have write access to it.

   • IgnoreInvalidBase64Chars — Specifies whether to ignore invalid characters found during base–64 decoding. The default is false (and invalid characters result in an error). Note that RFC 2045Opens in a new tab is ambiguous about whether unexpected characters should be ignored or should result in an error during base–64 decoding.

3. To use an SSL/TLS connection to the server:

   1. Set the SSLConfiguration property to the name of the activated SSL/TLS configuration to use.

      For information on creating and managing SSL/TLS configurations, see InterSystems TLS Guide. The SSL/TLS configuration includes an option called Configuration Name, which is the string to use in this setting.

   2. Set the UseSTARTTLS property to either 0 or 1.

      In most cases, use the value 0. Use the value 1 for the case in which the server interaction begins on a normal TCP socket and then switches to TLS on the same port as the normal socket. For details, see RFC 2595Opens in a new tab.

      Note that if UseSTATRTLS is set to 1, connect to the server over port 110, instead of the default 995.

   3. Optionally set the SSLCheckServerIdentity property to 1. Do this if you want to verify the host server name in the certificate.

4. Call the Connect() method of your instance. This method takes three arguments, in order:

   1. The name of the POP3 server

   2. A username

   3. A password

5. Use the methods of your instance to examine the mailbox, retrieve messages, and delete messages. The following sections provide details.

6. Optionally, to prevent the connection from timing out, call the Ping() method of your %Net.POP3Opens in a new tab instance.

7. Optionally, if you have marked messages for deletion but now choose not to delete them, call the RollbackDeletes() method of your %Net.POP3Opens in a new tab instance.

8. When you are done making changes to the mailbox, call one of the following methods:

   • QuitAndCommit() — Commits your changes and logs out of the mail server.

   • QuitAndRollback() — Rolls back your changes and logs out of the mail server.

Each of these methods returns a status, which you should check before continuing. Also see the class reference for %Net.POP3Opens in a new tab for complete method signatures.

Enabling XOAUTH2 Authentication

%Net.POP3Opens in a new tab supports XOAUTH2 as an authentication method.

To use XOAUTH2 with %Net.POP3Opens in a new tab, pass an access token as a parameter to %Net.POP3.Connect()Opens in a new tab. If passed an access token, this method will attempt to use XOAUTH2 regardless of whether or not a password is supplied. Users who do not wish to use XOAUTH2 should be careful not to pass in an access token parameter.

Connection Examples

The following examples use two different free POP3 services that were available at the time this content was written. No particular endorsement is implied by the selection of these services. Also note that the examples do not show the actual passwords.

Example 1: HotPOPAsPOP3()

The following method logs into the HotPOP POP3 server using an account that was previously set up for this purpose:

ClassMethod HotPOPAsPOP3() As %Net.POP3
{
  Set server=##class(%Net.POP3).%New()
  
  //HotPOP POP3 server uses the default port
  //but let's set it anyway
  Set server.port=110

  //just in case we plan to fetch any messages
  //that have attachments
  Set server.StoreAttachToFile=1
  Set server.StoreInlineToFile=1
  Set server.AttachDir="c:\DOWNLOADS\"

  Set servername="pop.hotpop.com"
  Set user="isctest@hotpop.com"
  Set pass="123pass"
  
  Set status=server.Connect(servername,user,pass)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Quit server
}

This method returns the %Net.POP3Opens in a new tab server instance. Many of the examples later in this topic accept the %Net.POP3Opens in a new tab instance as an argument.

Example 2: YPOPsAsPOP3()

The following method also returns a %Net.POP3Opens in a new tab server instance. In this case, we are using YPOPs, which is client software that provides SMTP and POP3 access to a Yahoo email account. It uses a test account that has already been set up for this purpose:

ClassMethod YPOPsAsPOP3() As %Net.POP3
{
  Set server=##class(%Net.POP3).%New()
  
  //YPOPs uses the default port 
  //but let's set it anyway
  Set server.port=110

  //just in case we plan to fetch any messages
  //that have attachments
  Set server.StoreAttachToFile=1
  Set server.StoreInlineToFile=1
  Set server.AttachDir="c:\DOWNLOADS\"
  
  //local host acts as the server
  Set servername="127.0.0.1"
  //YPOPs works with a Yahoo email account
  Set user="isc.test@yahoo.com"
  Set pass="123pass"
  
  Set status=server.Connect(servername,user,pass)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Quit server
}

Getting Information about the Mailbox

While you are connected to a POP3 server, you are logged in to a user account and have access to the mailbox for that user account. Use the following methods to find what the mailbox contains:

Each of these methods returns a status, which you should check before continuing. Also see Other Message Retrieval Methods for more details on these methods.

Example: ShowMailbox()

For example, the following method writes information about the mailbox that we are currently accessing:

ClassMethod ShowMailbox(server as %Net.POP3) 
{
    Set status=server.GetMailBoxStatus(.count,.size)
    If $$$ISERR(status) {
       Do $System.Status.DisplayError(status) 
       Quit 
    }
    Write "Mailbox information *****",!
    Write "Number of messages in mailbox: ",count,!
    Write "Size of messages: ",size,!

    Set status=server.GetMessageUIDArray(,.uids)
    Set status=server.GetSizeOfMessages(,.sizes)
    
    //iterate through messages, get info, and write it
    For i=1:1:count {
        Set uid=uids.GetAt(i)
        Set size=sizes.GetAt(i)
        Write "Msg number:", i,"   UID:",uid, "   size:",size,!
    }

}

This method generates output similar to the following:

Mailbox information *****
Number of messages in mailbox: 4
Size of messages: 18634
Msg number:1   UID:6ef78df6fd660391   size:7245
Msg number:2   UID:7410041a6faf4a87   size:5409
Msg number:3   UID:5555af7fa489e406   size:5121
Msg number:4   UID:299ad2b54c01a6be   size:859

Retrieving Messages

To simply get a message, use one of the following methods of the %Net.POP3Opens in a new tab class:

Each of these methods returns a status, which you should check before continuing. Note that these methods return an error status if the message is currently marked for deletion.

Also see Other Message Retrieval Methods, which shows the complete method signatures for Fetch() and FetchMessage()

Example: FetchMailbox()

The following example is a variation of the ShowMailbox example described in Getting Information about the Mailbox. This method uses the Fetch() method, examines each message, and writes the subject line of each message:

ClassMethod FetchMailbox(server As %Net.POP3)
{
  Set status=server.GetMailBoxStatus(.count,.size)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  Write "Mailbox information *****",!
  Write "Number of messages in mailbox: ",count,!
  Write "Size of messages: ",size,!

  Set status=server.GetMessageUIDArray(,.uids)
  Set status=server.GetSizeOfMessages(,.sizes)
  
  //iterate through messages, get info, and write it
  For i=1:1:count {
    Set uid=uids.GetAt(i)
    Set size=sizes.GetAt(i)
    Set status=server.Fetch(i,.msg)
    If $$$ISERR(status) {
      Set subj="***error***"
    } else{
      Set subj=msg.Subject
    }
    Write "Msg number:", i,"  UID:",uid, "  Size:",size
    Write "  Subject: ",subj,!
  }
}

Encoding and Character Translation

A email message part contains information about both the character sets used and the content-transfer-encoding used (if any). For reference, this section describes how this information is used.

For information on character sets and translation tables, see Translation Tables.

%Net.POP3Opens in a new tab checks the Content-Transfer-Encoding header of each message part and decodes the body as needed.

Then %Net.POP3Opens in a new tab checks the Content-Type header of each message part. This affects the Charset property of the message part and also controls the translation table used when the message part is created in InterSystems IRIS.

Saving Attachments

The Content-Disposition header might specify attachment, with or without a filename. For example:

Content-Disposition: attachment; filename=genome.jpeg;

If the Content-Disposition header does specify attachment, your %Net.POP3Opens in a new tab instance can save all attachments in the message to files. To make this happen:

1. Specify the following properties of your %Net.POP3Opens in a new tab instance:

   • Specify StoreAttachToFile as 1.

   • Specify StoreInlineToFile as 1.

   • Specify a valid directory for AttachDir. Make sure to terminate the name of the directory with a slash (/) or backslash (\), as appropriate for the operating system. Also make sure that this is directory already exists, and the users have write access to it.

2. Call Fetch() or FetchMessage() of your %Net.POP3Opens in a new tab instance.

Each filename is determined as follows:

1. If the Content-Disposition header specifies a filename, that filename is used.

2. Otherwise, if the Content-Type header specifies a filename, that filename is used.

3. Otherwise, the system creates a name of the form ATTxxxxxx.dat.

Note the following points:

• If the file already exists, the attachment is not downloaded.

• There is no default for AttachDir.

• The size of the attachment is not limited by InterSystems IRIS but might be limited by the file system.

• The Dir and FileName properties are not used here. They are relevant only when you upload an attachment to a mail message, as described in Adding Attachments to a Message.

Example: GetMsg()

The following example method retrieves an entire message, given an instance of %Net.POP3Opens in a new tab and a message number:

ClassMethod GetMsg(server as %Net.POP3,msgno as %Integer) as %Net.MailMessage 
{
    Set status=server.Fetch(msgno,.msg)
    If $$$ISERR(status) {
       Do $System.Status.DisplayError(status) 
       Quit $$$NULLOREF
    }
    Quit msg
}

If the message had attachments, and if you specified the StoreAttachToFile, StoreInlineToFile, and AttachDir properties of the %Net.POP3Opens in a new tab server, those attachments would be saved to the given directory when you called this method.

Retrieving Attached Messages

While you are connected to a mailbox, you can download any email messages that are attached to the email messages in the inbox. To do so, use the GetAttachedEmail() method of your %Net.POP3Opens in a new tab instance to retrieve the contents of the enclosed email.

Given an instance of %Net.MailMessagePartOpens in a new tab, this method returns a single-part message that has contents of that message part. Specifically, it returns (as an output parameter) an instance of %Net.MailMessageOpens in a new tab initialized with the data taken from the attached email message.

Summary of Methods to Examine and Retrieve

This section lists all the methods of %Net.POP3Opens in a new tab that you can use to examine and retrieve messages.

Fetch()

method Fetch(MessageNumber As %Integer, 
             ByRef MailMsg As %Net.MailMessage,
             Delete As %Boolean = 0,
             messageStream As %BinaryStream) as %Status {}

Returns (by reference) the message indicated by MessageNumber and optionally marks the message for deletion. Note that this method returns an error status if the message is already marked for deletion.

If messageStream is specified, then the original message is written to this binary stream.

FetchFromStream()

method FetchFromStream(messageStream As %BinaryStream, ByRef Msg As %Net.MailMessage) as %Status {}

This method is for use when you specify the messageStream argument for Fetch().

Retrieves a single email message from the given binary stream. messageStream must be a binary stream containing the message. The message is returned by reference in Msg. This could be a multipart message.

FetchMessage()

method FetchMessage(MessageNumber As %Integer, 
                    ByRef From As %String, 
                    ByRef To As %String, 
                    ByRef Date As %String, 
                    ByRef Subject As %String, 
                    ByRef MessageSize As %Integer, 
                    ByRef MsgHeaders As %ArrayOfDataTypes, 
                    ByRef MailMsg As %Net.MailMessage, 
                    Delete As %Boolean = 0) as %Status {}

Returns (by reference) specific message headers, the message size, the message header array, and the message itself and optionally marks the message for deletion. Note that this method returns an error status if the message is already marked for deletion.

FetchMessageHeaders()

method FetchMessageHeaders(MessageNumber As %Integer, 
                           ByRef MsgHeadersArray As %String) as %Status {}

Given a message number, this method returns (by reference) an array containing all the headers of that message. This method returns an error status if the message is currently marked for deletion.

FetchMessageInfo()

method FetchMessageInfo(MessageNumber As %Integer, 
                        Lines As %Integer, 
                        ByRef From As %String, 
                        ByRef To As %String, 
                        ByRef Date As %String, 
                        ByRef Subject As %String, 
                        ByRef MessageSize As %Integer, 
                        ByRef MsgHeaders As %ArrayOfDataTypes, 
                        ByRef MessageText As %String) as %Status {}

Given a message number, this method returns (by reference) specific message headers, the message size, the message header array, and the given number of lines of text from this message. This method returns an error status if the message is currently marked for deletion.

GetAttachedEmail()

method GetAttachedEmail(msgpart As %Net.MailMessagePart, 
       Output mailmsg As %Net.MailMessage) as %Status {}

Given a message part, this method returns (as an output parameter) a single-part email message that is initialized with the data from the message part.

GetMessageUID()

method GetMessageUID(MessageNumber As %Integer, 
                     ByRef UniqueID As %String) as %Status {}

Returns, by reference, the UID of a message, given a message number. See the previous section for details on message numbers and UIDs. This method returns an error status if the message is currently marked for deletion.

GetMessageUIDArray()

method GetMessageUIDArray(MessageNumber As %String = "", 
                          ByRef ListOfUniqueIDs As %ArrayOfDataTypes) as %Status {}

If given an empty string as the first argument, this method returns, by reference, an array of information about the messages in the mailbox (excluding any that are currently marked for deletion). Each element in this array contains the following information about one message:

Array Key	Array Item
Number of the message, within the mailbox in its current state. The first message is number 1, and so on. The message number of a given message is not guaranteed to be the same in all sessions.	Unique message identifier (UID), which is the permanent identifier of this message available in all sessions. UIDs are unique to each mailbox.

Or, given a message number, this method returns a one-element array that contains the UID of that message. In this case, the method returns an error status if the message is currently marked for deletion.

GetSizeOfMessages()

method GetSizeOfMessages(MessageNumber As %String = "", 
                         ByRef ListOfSizes As %ArrayOfDataTypes) as %Status {}

If given an empty string as the first argument, this method returns, by reference, an array of information about the messages in the mailbox (excluding any that are currently marked for deletion). Each element in this array contains the following information about one message:

Array Key	Array Item
Number of the message, within the mailbox in its current state.	Size of this message, in bytes.

Or, given a message number, this method returns a one-element array that contains the size (in bytes) of that message. In this case, this method returns an error status if the message is currently marked for deletion.

Deleting Messages

While you are connected to a mailbox, you can mark messages for deletion in that mailbox. You can do this in a couple of ways.

• You can use the DeleteMessage() method. This method takes one argument, the message number to delete.

• When you retrieve a message with the Fetch() or FetchMessage() method, you can specify an optional argument that tells the POP3 server to mark the message for deletion after you have retrieved it.

Remember the following points:

• These methods do not delete a message; they mark it for deletion. The message is not deleted until you complete the POP3 transaction with QuitAndCommit(). If you simply disconnect from the server, your changes are discarded.

• You can call the RollbackDeletes() method to change the messages so that they are no longer marked for deletion.

• Each of these methods returns a status, which you should check.

Example: GetMsgAndDelete() and CommitChanges()

The following example retrieves a message and marks it for deletion:

ClassMethod GetMsgAndDelete(ByRef server As %Net.POP3, msgno As %Integer) As %Net.MailMessage
{
  //third argument to Fetch says whether to 
  //mark for deletion
  Set status=server.Fetch(msgno,.msg,1)
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$NULLOREF
  }
  
  Quit msg
}

Note that this message returns (by reference) an altered version of the %Net.POP3Opens in a new tab; the altered version contains the information about which message is marked for deletion.

You would use the preceding method with a method like the following:

ClassMethod CommitChanges(server As %Net.POP3) As %Status
{
  //commit all changes and log out
  Set status=server.QuitAndCommit()
  If $$$ISERR(status) {
    Do $System.Status.DisplayError(status) 
    Quit $$$ERROR($$$GeneralError,"Failed to commit changes")
  }
  Quit $$$OK
}

Alternatively you would roll back the changes with RollbackDeletes() or QuitAndRollback().

See Also

• Email Support in InterSystems IRIS

• %Net.FetchMailProtocolOpens in a new tab (for examples and extensive comments)

• Working with Received Email