Getting the mail in: receiving in JavaMail

DJ Walker-Morgan, 22 November 2005

uk.builder.com

While sending email from an application may be more common, you'll probably want to fetch mail from a server at some point as well. Here we show you JavaMail's receiving functions

In the , we looked at sending mail with JavaMail and Apache Commons Email. The one thing Commons Email doesn't cover is receiving email, being focused on sending email. For receiving mail, we need to look at JavaMail's capabilities.

Where sending email was relatively simple with only one protocol (SMTP) to handle, reception involves two protocols, POP3 and IMAP. POP3 is the older protocol, which offers a single queue of mail messages as a single inbox. IMAP is the more modern protocol, which presents mail messages as entries in a hierarchy of folders, one of which will be an inbox. There are also two more protocols to allow for: POP3 and IMAP over a secure connection. JavaMail abstracts this into the concept of a mail Store which has a hierarchy of Folders. That abstraction means for many tasks, we only need to know how to navigate through a Store's Folders. The dirty work of handling the actual mail protocol is done by what JavaMail calls a Provider.

JavaMail comes with Provider implementations for POP3 and IMAP, and the secure versions of those as POP3S and IMAPS. It's worth remembering that you can plug additional Providers into the JavaMail API to handle other protocols like NNTP or locally stored mail; Sun's web site lists some of the third party providers out there.

Let's look at a practical example; MailPage. The task at hand is to produce a servlet which creates a web page based on the most recent mail sent to a mail account, allowing a remote user to update a status page without having to have direct access to the web server; all they have to do is send a mail to a particular email account.

At the core of the solution is the . This needs four bits of data before it gets going; the name of the email user account, the password for that account, the host name of the server and the name of provider to use - "pop3", "pop3s", "imap", "imaps" being the available standard ones. Now we can go get mail in the getMail method. The first thing we need, as with sending mail, is a Session. In this case, we get a distinct Session:

Properties props=System.getProperties();
session=Session.getInstance(props,null); 

From this Session we can get a Store which is appropriate to the provider:

store=session.getStore(emailprovider);

And connect to it by calling the Store's connect method, with the server name, username and password:

store.connect(emailserver,emailuser,emailpassword);

Now we are ready to navigate the Folder hierarchy. To get to the root of the tree, we ask the store for the default Folder.

folder=store.getDefaultFolder();

Each Folder can carry a list of other folders it contains. For our example, we want to navigate to the inbox which we can do with the getFolder method:

inboxfolder=folder.getFolder("INBOX");

For IMAP, this is navigating to the folder in the IMAP tree. With POP3 this is all a facade to map the single queue as an inbox. The good news is these differences are hidden.

To actually work with the messages in a folder, you need to open the folder:

inboxfolder.open(Folder.READ_ONLY);

We're opening the folder in READ_ONLY mode because we don't want to make any changes to the state or content of the folder. Opening with READ_WRITE would let us make changes, if the underlying provider allows it. We can now get an array of the messages within the folder:

Message[] msgs=inboxfolder.getMessages();

We could start iterating through it but this wouldn't be that efficient. Messages start out as empty references to the remote message, with the message headers and content retrieved on demand, and as we are dealing with a remote system, this would mean a round trip on the network for each message we iterated over. What we need is a way to hint to the API that we're really interested in particular fields in the header so it can get those headers up front. Enter the FetchProfile, a class which allows us to define a hint:

FetchProfile fp=new FetchProfile();
fp.add("Subject"); 

We are just interested in the Subject header for this example, so we add that to the FetchProfile. You can have a whole range of fields, or you can use the two shortcuts, FetchProfile.Item.ENVELOPE (for the common From, To, CC and subject fields) or FetchProfile.Item.CONTENT_INFO (for information about the size and type of the message content). Now we can ask the folder instance to populate that field:

inboxfolder.fetch(msgs,fp);

and we can get to iterating over those subject headers. For MailPage, we're looking for a specific string, "MailPage:" at the start of the subject. We'll start at the top end of the array and work down till we find an appropriate message;

for(int j=msgs.length-1;j>=0;j--) {
	if(msgs[j].getSubject().startsWith("MailPage:")) {
		setLatestMessage(msgs[j]);<
		break;
	}
}

We'll come back to the setLatestMessage and what it does in a moment. All that's left for us to do is to close the folder and store:

inboxfolder.close(false);
store.close(); 

The parameter on the folder close sets whether the folder is purged of deleted mail when closed; we set it to false here because we are trying to ensure we don't change the state of the mailbox.

That completes the actual getting of the mail; now we have to parse the mail. This is done in the example's setLatestMessage method. The first thing you need to remember is that mail comes in a variety of formats; the most basic of which is plain text. How do you identify what kind of format? First stop is checking the content type of the message. A plain text message is identified by a content type of "text/plain":

void setLatestMessage(Message message) {
	…
	if(message.getContentType().startsWith("text/plain")) {
	latestMessage=new RenderablePlainText(message);
	} else {
		latestMessage=new RenderableMessage(message);
	}

Note the use of startsWith() to compare with the type; This is because content type strings can have more information about the content type in the string, but we only need to worry about the initial part. For the example, we've created an interface called Renderable which is a simplistic view of a decoded mail; it comprises the subject line, the main text of the message and an array of attachments; we then have a RenderablePlainText class which just decodes text/plain messages; the attachments can be ignored for a plain text message. If we look in the constructor for RenderablePlainText, we can see how to read the content:

public RenderablePlainText(Message message) throws MessagingException,
IOException {
	subject=message.getSubject().substring("MailPage:".length());
	bodytext=(String)message.getContent();
}

The subject is easy to get, being directly exposed. We make some assumptions on the content though. Remember the Part interface from the previous article, where we assembled BodyPart and MimeBodyPart to build a message. When we decode the mail, we are dealing with a tree of Parts; with the Message at the top of the tree. For text/plain it is just the text content as a string.

 

It gets more complex when we have a MultiPart message. MultiPart denotes the Part is composed of multiple parts, each of which we have to examine to get the content. For this example, we have a RenderableMessage class, which will disassemble a message with HTML text and attachments.

public RenderableMessage(Message m) throws MessagingException,IOException {
	subject=m.getSubject().substring("MailPage:".length());
	attachments=new ArrayList();
	extractPart(m);
}

RenderableMessage is a very simple handler which just recurses through Parts for which we have the extractPart method. The first thing this does is check if the Part given is a MultiPart part, and if it is it steps through those sub parts and calls extractPart on them:

private void extractPart(final Part part) throws MessagingException,
	IOException {
	if(part.getContent() instanceof Multipart) {
		Multipart mp=(Multipart)part.getContent();
		for(int i=0;i

We can then process any content which is text/html in a similar way to how we handled the plain text earlier.

	if(part.getContentType().startsWith("text/html")) {
		if(bodytext==null) {
			bodytext=(String)part.getContent();
		} else {
			bodytext=bodytext+"
"+(String)part.getContent();
		}
	}

Nothing too special there; if there are multiple text/html parts, the raw text from them is stuck together into one long string. Now we get to core of the task:

	else if(!part.getContentType().startsWith("text/plain")) {
		Attachment attachment=new Attachment();
		attachment.setContenttype(part.getContentType());
		attachment.setFilename(part.getFileName());

If the content isn't plain text, we treat it as an attachment, and create our own Attachment instance (which is just a holder for filename, content type and a byte array of content) and fill in the content type and filename. The actual content is available as an InputStream which we have to read. We'll create a ByteArrayOutputStream to write it to:

		InputStream in=part.getInputStream();
		ByteArrayOutputStream bos=new ByteArrayOutputStream();

and then copy the content's input stream into it:

		byte[] buffer=new byte[8192];
		int count=0;
		while((count=in.read(buffer))>=0) bos.write(buffer,0,count);
		in.close();

and finally set the content of our Attachment instance from the ByteArrayOutputStream and add the Attachment to an ArrayList of attachments:

		attachment.setContent(bos.toByteArray());
		attachments.add(attachment);
	}
}

And that's about it for extracting the content of attachments. There's a main method in MailRetriever which lets us give MailRetriever a run; just give it the username, password, server name and provider type as parameters when you run it and it'll retrieve the latest message and dump the text and a list of attachments to the console. Here's the output from a run:

Subject:Mail with attachments
Body Text:
Here's some images of the Space Shuttle.






2 attachments
9603752.jpg 81312 bytes of (image/jpeg; x-unix-mode=0644; name=9603752.jpg)
1976_04380L.jpg 32089 bytes of (image/jpeg; x-unix-mode=0644; 
name=1976_04380L.jpg)

In this case, two images are attached. As an aside, notice that up in the HTML text of the message, there's references not to the filenames of the images, but to cid: URLs. We'll look at handling them next month as we incorporate the MailRetriever code into a servlet.

Before you dash off to try the example code, a warning though. Talking to mail servers even with JavaMail has pitfalls even with the FAQ to hand. For example, when I was writing this article, I decided initially to use GMail, Google's mail service and its POP3 support. Strangely though, Google's POP3 service is not has properties which control timeouts, authentication and how the provider actually interacts with the mail server. This in turn may resolve those compatibility issues mentioned above. These properties are provider implementation specific. Take the standard POP3 provider though; some POP3 providers automatically delete read messages when you disconnect, but the POP3 RSET command can reset the mailbox to its original read state. The property mail.pop3.rsetbeforequit set to true can enable that behaviour in the POP3 provider; the "pop3" part of the property name reflecting the provider's name, so change it to "pop3s" if you are using secure POP3. Here's code from the example which enables that for both and turns debugging on:

Properties props=System.getProperties();
props.setProperty("mail.pop3s.rsetbeforequit","true");
props.setProperty("mail.pop3.rsetbeforequit","true");
session=Session.getInstance(props,null);
session.setDebug(true);	

And if you are wondering, no, this has no effect on Google Mail's POP3 implementation; once you've read it, it's gone. Next month, we'll look at using the MailRetriever as a servlet, complete with displaying those attachments, and look at handling some other types of mail beyond simple attachments.

You can download

DJ Walker-Morgan is a consulting developer, specialising in Java and user-to-user messaging and conferencing.