Redhatter (VK4MSL)

I’ll be posting up a series of notes on various aspects of OpenERP. This is largely as a brain-dump for my own reference, but might prove interesting for others who may be facing the same problems I have faced in my maintenance of my work’s OpenERP instance.

OpenERP’s mail gateway system is a generic interface for manipulating objects according to the reception of incoming emails. The system can either receive emails via a script ran by the mail server, or by periodic collection of a POP3/IMAP mail account. Each dedicated account is allocated for processing of one kind of object. You can also configure outbound email SMTP servers for sending email traffic — these can be accessed and utilised by any object.

Sending emails from OpenERP objects

Configuring outbound email

Configuring outbound email is a fairly straightforward affair. After installing the ‘mail’ module (or one of its dependents) you should be able to see “Outgoing Mail Servers” under Settings/Configuration/Email.

The configuration here is not much different to any email client you might otherwise configure; give the host name of the SMTP server, port and connection security options, click save and you’re done. Unless you’ve got a fairly specialised set-up, you should only need to configure the one mail server.

Sending arbitrary messages: mail.message

As the saying goes, there is more than one way to skin a cat, and here is no different. There are in fact, two interfaces for sending emails. The first of these is mail.message, which is useful for sending arbitrary messages. It defines a method, schedule_with_attach which, as the name suggests, schedules an email for delivery, optionally with an attachment or two.

def schedule_with_attach(self, cr, uid, email_from, email_to, subject,
        body, model=False, email_cc=None, email_bcc=None,
        reply_to=False, attachments=None, message_id=False,
        references=False, res_id=False, subtype='plain', headers=None,
        mail_server_id=False, auto_delete=False,
        context=None):
    """ Schedule sending a new email message, to be sent the next time the
        mail scheduler runs, or the next time :meth:`process_email_queue` is
        called explicitly.

    :param string email_from: sender email address
    :param list email_to: list of recipient addresses (to be joined with commas)
    :param string subject: email subject (no pre-encoding/quoting necessary)
    :param string body: email body, according to the ``subtype`` (by default, plaintext).
        If html subtype is used, the message will be automatically converted
        to plaintext and wrapped in multipart/alternative.
    :param list email_cc: optional list of string values for CC header (to be joined with commas)
    :param list email_bcc: optional list of string values for BCC header (to be joined with commas)
    :param string model: optional model name of the document this mail is related to (this will also
        be used to generate a tracking id, used to match any response related to the
        same document)
    :param int res_id: optional resource identifier this mail is related to (this will also
        be used to generate a tracking id, used to match any response related to the
        same document)
    :param string reply_to: optional value of Reply-To header
    :param string subtype: optional mime subtype for the text body (usually 'plain' or 'html'),
        must match the format of the ``body`` parameter. Default is 'plain',
        making the content part of the mail "text/plain".
    :param dict attachments: map of filename to filecontents, where filecontents is a string
        containing the bytes of the attachment
    :param dict headers: optional map of headers to set on the outgoing mail (may override the
        other headers, including Subject, Reply-To, Message-Id, etc.)
    :param int mail_server_id: optional id of the preferred outgoing mail server for this mail
    :param bool auto_delete: optional flag to turn on auto-deletion of the message after it has been
        successfully sent (default to False)"""

So in your method, you might call it with something like this:

def send_mail(cr, uid, ids, ... context=None):
    if not isinstance(ids, list):
        ids = [ids]

    msg_pool = self.pool.get('mail.message')
    for object in self.browse(cr, uid, ids, context=context):
        # Constructing the body text. This can be done a variety of ways
        body_text = '''An example email body using some fields from the object.
Object name: %s
Object ID: %s
''' % (object.name, object.id)

        msg_pool.schedule_with_attach(cr, uid, email_from='some@email.address',
                email_to=[  'a@list.of',
                            'people@for.the',
                            'to@list' ],
                subject='Your Spam',
                body=body_text, # Constructed above
                email_cc=['same@deal'], # or False for no CCs
                email_bcc=['etc@somewhere'],
                ..., context=context)

If you trigger this method, then look in your outbound message queue (Settings/Email/Messages) you’ll see the message queued for delivery. At some point the scheduler in OpenERP will send this message to the destination address. This is done every few minutes or so.

Sending templated messages: email.template

This sort of hard-coded message is all very well, but they aren’t pretty, and they certainly aren’t user customisable. The project module gets around the user customisability by providing fields for a header and footer with a fixed set of fields, but if you want to do something nicer, you’re out of luck. We had a need to be able to notify people when tasks were assigned to them, or whenever the task was closed. I initially used the above approach, but then investigated the email.template method.

Under “Settings/Configuration/Email/Templates” you’ll see a list of such templates. Essentially you can fill any field from the source object in, at any position. In each field, any text inside ${ } blocks is executed as Python code. There seems to be more options here for formatting, but so far that’s what I definitely know. The object being formatted gets passed in as object, and the context dict is exposed as ctx. You’ve got the ability to customise both HTML and plain-text versions, and to attach a report or existing files if you so choose.

So for my project task notification, I took the following approach. Firstly, to save me a lot of messy code, I augmented the task object with a function field which enumerated all the people that were to be notified. The idea was to enhance the “Warn Manager”/”Warn Customer”, replacing that implementation with this one. So in a custom module, I code up a derived model like so:

class project_task(osv.osv):
    '''
    Project Task notification hooks.  This allows us to send an email
    when a task is created or updated.
    '''
    _name = "project.task"
    _inherit = "project.task"

    def _get_mailing_list(self, cr, uid, ids, field_name, arg, context=None):
        '''Fetch the list of email addresses that will be sent notifications.'''

        ret = {}
        for task in self.browse(cr, uid, ids, context=context):
            mail_list = set()

            if task.user_id and task.user_id.notify_on_task \
                    and task.user_id.user_email:
                mail_list.add(task.user_id.user_email)

            if task.manager_id and (task.manager_id.notify_on_task or      \
                     (task.project_id and task.project_id.warn_manager))  \
                        and task.manager_id.user_email:
                mail_list.add(task.manager_id.user_email)

            if task.project_id and task.project_id.warn_customer:
                for partner in filter(lambda p : p, [task.partner_id,
                        task.project_id.partner_id]):
                    if partner.address and partner.address[0].email:
                        mail_list.add(partner.address[0].email)

            # We have a set of relevant email addresses, convert to a sorted
            # list and put it in the returned output.
            ret[task.id] = u','.join(sorted(mail_list))
        return ret

    _columns = {
        'notification_mailing_list': fields.function(_get_mailing_list, type='text',
                string='Email addresses to notify', store=False),
    }

You’ll notice that here, res.users has an additional field; notify_on_task, this is a per-user flag, configurable in the user preferences that lets users decide whether they’ll personally get nagged or not. I use a set, as we only want to add each address once.

We can now use this field in the template to populate the To: field, rather than a very messy lambda expression to do the same.

Next, we create the template. Now, you can (and I did) create this in the user interface, the following shows how to do it inside the XML files for a module. This was lifted from the edi demo and tweaked.

    <!-- Mail template and workflow bindings are done in a NOUPDATE block
         so users can freely customize/delete them -->
    <data noupdate="1">
        <!--Email template -->
        <record id="email_template_task" model="email.template">
            <field name="name">Automated Task Update Notification Mail</field>
            <field name="email_from">${ctx.get('uid_email') or 'no-body@localhost.localdomain'}</field>
            <field name="subject">Task #${object.id}: ${object.name} [${object.state.title()}]</field>
            <field name="email_to">${object.notification_mailing_list}</field>
            <field name="model_id" ref="project.model_project_task"/>
            <field name="body_html"><![CDATA[
            <p>Hello, The following task has recently been updated:</p>
            <ul>
                <li>${'<i>(Updated)</i> ' if 'name' in ctx.get('changed_fields',[]) else ''}Name: ${object.name} [Task ID #${object.id}]</li>
                <li>${'<i>(Updated)</i> ' if 'state' in ctx.get('changed_fields',[]) else ''}State: ${object.state.title()}</li>
                <li>${'<i>(Updated)</i> ' if 'project_id' in ctx.get('changed_fields',[]) else ''}Project: ${object.project_id.name if object.project_id else 'No project'}</li>
                <li>${'<i>(Updated)</i> ' if 'user_id' in ctx.get('changed_fields',[]) else ''}Assignee: ${object.user_id.name if object.user_id else 'Nobody'}</li>
                <li>Manager: ${object.manager_id.name if object.manager_id else 'Nobody'}</li>
                <li>${'<i>(Updated)</i> ' if 'total_hours' in ctx.get('changed_fields',[]) else ''}Total Hours: ${object.total_hours}.
                        ${'<i>(Updated)</i> ' if 'remaining_hours' in ctx.get('changed_fields',[]) else ''}Remaining Hours: ${object.remaining_hours}
                        Progress: ${object.progress}%</li>
                <li>${'<i>(Updated)</i> ' if 'date_start' in ctx.get('changed_fields',[]) else ''}Start Date: ${object.date_start or 'None'}</li>
                <li>${'<i>(Updated)</i> ' if 'date_end' in ctx.get('changed_fields',[]) else ''}End Date: ${object.date_end or 'None'}</li>
                <li>${'<i>(Updated)</i> ' if 'date_deadline' in ctx.get('changed_fields',[]) else ''}Deadline: ${object.date_deadline or 'None'}</li>
            </ul>
            <h2>${'<i>(Updated)</i> ' if 'description' in ctx.get('changed_fields',[]) else ''}Description:</h2>
            <pre>${object.description or ''}</pre>
            <h2>${'<i>(Updated)</i> ' if 'notes' in ctx.get('changed_fields',[]) else ''}Notes:</h2>
            <pre>${object.notes or ''}</pre>
            ]]></field>
            <field name="body_text"><![CDATA[
Hello, The following task has recently been updated:

 * ${'(Updated) ' if 'name' in ctx.get('changed_fields',[]) else ''}Name: ${object.name} [Task ID #${object.id}]
 * ${'(Updated) ' if 'state' in ctx.get('changed_fields',[]) else ''}State: ${object.state.title()}
 * ${'(Updated) ' if 'project_id' in ctx.get('changed_fields',[]) else ''}Project: ${object.project_id.name if object.project_id else 'No project'}
 * ${'(Updated) ' if 'user_id' in ctx.get('changed_fields',[]) else ''}Assignee: ${object.user_id.name if object.user_id else 'Nobody'}
 * Manager: ${object.manager_id.name if object.manager_id else 'Nobody'}
 * ${'(Updated) ' if 'total_hours' in ctx.get('changed_fields',[]) else ''}Total Hours: ${object.total_hours}  ${'(Updated) ' if 'remaining_hours' in ctx.get('changed_fields',[]) else ''}Remaining Hours: ${object.remaining_hours}  Progress: ${object.progress}%
 * ${'(Updated) ' if 'date_start' in ctx.get('changed_fields',[]) else ''}Start Date: ${object.date_start or ''}
 * ${'(Updated) ' if 'date_end' in ctx.get('changed_fields',[]) else ''}End Date: ${object.date_end or ''}
 * ${'(Updated) ' if 'date_deadline' in ctx.get('changed_fields',[]) else ''}Deadline: ${object.date_deadline or ''}

${'(Updated) ' if 'description' in ctx.get('changed_fields',[]) else ''}Description:
${object.description or ''}

${'(Updated) ' if 'notes' in ctx.get('changed_fields',[]) else ''}Notes:
${object.notes or ''}
            ]]></field>
        </record>

You’ll notice I use ctx.get('uid_email') for the From address. When I call the template up, I can pass in the From address via the context, calling the key uid_email, and it will be filled in. I also have a list of fields that have changed; changed_fields — this is as simple as grabbing vals.keys() from within a write method. To whistle up the template and send the email, I defined a method which I can call.

    def _send_email(self, cr, uid, ids, context=None):
        '''
        Send an email relating to the given task IDs.
        '''
        if not context:
            context = {}

        # Grab the tasks
        tasks = self.browse(cr, uid, ids, context=context)

        # Filter out the tasks for which no emails will be sent.
        tasks = filter(lambda t : \
                (t.user_id and t.user_id.notify_on_task) or \
                (t.manager_id and t.manager_id.notify_on_task) or \
                (t.project_id and \
                    (t.project_id.warn_manager or \
                     t.project_id.warn_customer)), tasks)
        if not tasks:
            # If we've eliminated them all, stop here.
            return None

        # Mail message template handler
        model_pool = self.pool.get('ir.model')
        template_pool = self.pool.get('email.template')
        template_id = template_pool.search(cr, uid, [
            ('model_id','in',model_pool.search(cr, uid,
                                               [('model','=',self._name)],
                                               context=context)),
        ], context=context)
        if template_id:
            # Use the first one
            template_id = template_id.pop(0)

            # Get the current user's email address
            user_pool = self.pool.get('res.users')
            user = user_pool.browse(cr, uid, uid, context=context)
            context['uid_email'] = user.user_email

            # Send the emails for each task
            map(lambda t : template_pool.send_mail(cr, uid, template_id, \
                        t.id, context=context), tasks)

This method performs the following steps:

1. It filters out the tasks for which no emails will be sent; if no one has elected to receive notifications for a task, then let’s not waste our time.
2. We then look for the template used to format tasks. To do this, we must first know what ID corresponds to a project.task, so we do a search, then
   use its result in the domain expression to search the templates.
3. If we find a template ID; there should only be one, so we pick the first of the list. searchalways returns a list.
4. We look up the current user, and try to find their email address, so that the email will be sent from the person making the changes.
5. The map function passes each individual task in to the send_mail method.

The send_mail method of email.template has the following structure:

    def send_mail(self, cr, uid, template_id, res_id, force_send=False, context=None):
        """Generates a new mail message for the given template and record,
           and schedules it for delivery through the ``mail`` module's scheduler.

           :param int template_id: id of the template to render
           :param int res_id: id of the record to render the template with
                              (model is taken from the template)
           :param bool force_send: if True, the generated mail.message is
                immediately sent after being created, as if the scheduler
                was executed for this message only.
           :returns: id of the mail.message that was created 
        """

Above, res_id was passed the ID of the task. We also do some magic injecting values into context so they’ll appear in the template. The final step in our mail template integration is to actually make use of the new method. I wanted to send an email when the task was updated, much like Bugzilla’s updates. So I hooked the create and write methods of project.task:

    # Override the action handlers so that we can slip our email out.
    def create(self, cr, uid, vals, context=None):
        id = super(vrt_project_task, self).create(cr, uid, vals, context=context)
        self._send_email(cr, uid, [id], context=context)
        return id

    # What fields do we notify on?  TODO: make this configurable?
    NOTIFY_ON   =   {   'user_id':  lambda val : True,
                        'state':    lambda val : val in ('cancelled','done'),
    }

    def write(self, cr, uid, ids, vals, context=None):
        res = super(vrt_project_task, self).write(cr, uid, ids,
                vals, context=context)
        notify = False
        for field, do_check in self.NOTIFY_ON.iteritems():
            if (field in vals) and do_check(vals[field]):
                # We found a field of interest
                notify = True
                break

        # Send an email describing what changed
        if notify:
            if not context:
                context = {}
            else:
                context = context.copy()
            # List the fields that changed so we can highlight them
            # in the email.
            context['changed_fields'] = vals.keys()
            if isinstance(ids, (int,long)):
                ids = [ids]
            self._send_email(cr, uid, ids, context=context)
        return res

If you process the tasks one-by-one, it is possible to read the state of the task first, do the write, then send an email off for that message telling the recipients how everything changed, but that was not necessary here, thus hasn’t been implemented. One thing that irritates me is the hard-coded list of notification fields, but it’ll do for now.

Receiving emails in OpenERP

Sending emails is only half the story however. It’d be nice, for example, to have a form on a website that people can use to contact sales staff and make enquiries, and for this form to turn into a lead. One such approach would be to use one of many off-the-shelf form posting scripts, and to parse the email within OpenERP. The approach I am specifically looking at, is using Python-code Server Actions to parse the email.

For now I’ve focussed my attention on bringing the email in to OpenERP and doing some processing on it without actually creating leads, as this is more an exploration of how the processing all works.

Setting up the inbound email account

To start off with, you must have a dedicated email address that will receive these notifications. OpenERP then checks this via IMAP or POP3 periodically, and for each new message, will either create a new document or run a server action.

Under “Settings/Configuration/Email/Incoming Mail Servers”, configure a new account, specifying the usual login details. Below this, you’ll notice the fields below the heading “Actions to Perform on Incoming Mails”. This is where OpenERP is told what to do. “Create a new record” defines what sort of object is to be processed.

I used mail.message initially, then discovered I got some funny errors. For everything to work, the object chosen needs to inherit mail.thread. In fact, for testing you can even use mail.thread, and this is what I later used. For CRM stuff, crm.lead inherits mail.thread so I’ll probably use that in the final implementation.

Creating the server action

When the server creates the object it can trigger a server action at the same time. This server action can take a number of forms, but the one of most interest is the Python Code server action. As a test, I made a server action that took a mail.thread, then posted a response email for each message seen, a bit like the autoanswer alias on vger.kernel.org. Creating the server action, I specified mail.thread as the object and used the following Python code:

if context.get('active_id'):
    thread = self.browse(cr, uid, context.get('active_id'), context=context)
    for msg in thread.message_ids:
        msg_body = '''Test server action -- email received:
From: %s
To: %s
CC: %s
Subject: %s
Headers:
  %s
TEXT:
  %s
HTML:
  %s
''' % (
    msg.email_from,
    msg.email_to,
    msg.email_cc,
    msg.subject,
    msg.headers,
    msg.body_text,
    msg.body_html,
)
        pool.get('mail.message').schedule_with_attach(cr, uid, (msg.email_to or 'me@mydomain.com.au').split(',')[0], [msg.email_from or 'me@mydomain.com.au'], 'Test email reply [Re: %s]' % msg.subject, msg_body, context=context)

Probably worth noting, I suspect the self.browse() bit and the context.get('active_id') bit is not necessary, as that is what I’d imagine object is for, but initially I didn’t have the right model chosen, so I suspect that bit of code could be re-worked.

The plan now, is to use the YAML module to parse the message body so that the fields can be sent from the website in a human-readable and unambiguous form.

I’ll be doing some further experimentation, but for now that’s where I’ve gotten. No doubt, there’ll be updates as I discover more about what goes on here.