Postmortem on the incoming email delay issue on January 24th and 25th, 2018

Postmortem on the incoming email delay issue on January 24th and 25th, 2018

Last Thursday morning on January 24th and Friday morning on January 25th, incoming emails to all Helpmonks mailboxes were delayed by two hours, respectively on the 25th for up to four hours.

The root cause was that we a recent update to the “message id” of an email (each email has a unique message id) and the reply-to-id were not correctly set. This in return, caused messages from another Helpmonks mailbox to be accepted, and if there was an auto-reply, it created an endless loop for some emails.

The endless loop created thousands of emails that came back to our mail server. As these emails got larger each time in size, it brought our parsing engine to its knees. Besides, our search server started to index all those massive emails causing a backlog in searches and an unaccounted overhead on our database.

We were actively testing and working on this update for several days and felt confident to roll it out to all of our customers. However, we failed to properly account for how to parse internal emails with auto-replies correctly.

In hindsight, it is clear to us that we could have avoided the problem. It’s bad enough that our customers had to experience a significant delay, but knowing that this could have been prevented is challenging to accept. I cannot express my apologies deeply enough.

Even though we had issues with relaying emails to our customers promptly, we did not lose a single email, as all emails automatically get stored on different incoming servers that are independently setup. That was another major upgrade we conducted in the recent months.

We work hard every day to bring the best team email collaboration platform for you. We have a weekly uptime record of 99.98% and in some weeks even a 100% uptime. The delay in delivering emails to you does not live up to our standards and we will take this incident and will discuss what we can do better to avoid this happening again.

I want to thank our support team who responded to each customer individually and also to our customers who despite the issue and the inconvenience this caused showed their understanding and encouragement.

If you have any questions, or if we can help in any way, please reach out to us. Thank you.

Helpmonks All-in-one Customer Service Platform
Features and enhancements that were added to Helpmonks in February 2019

Since the beginning of 2019, we've been working on many enhancements for your team emails. Here are the ones we made available in February. You can also follow public Helpmonks RoadMap to get updated and vote on features. Global shortcut to compose a new message Many of you like the…

Helpmonks All-in-one Customer Service Platform
Activity stream and notification panel

It’s been a while since we’ve had a chance to write on here. We’ve been working hard to build the next great destination for email collaboration. The surge of new customers in the last few months is proof that we are on the right track. Behind the scenes, we’ve been…

Helpmonks All-in-one Customer Service Platform
Helpmonks Update: Things we've added in the recent weeks

We've been busy here at Helpmonks and enhanced your email collaboration platform with many useful features. Without further ado, here are some of the most notable ones: User settings There is now an account-wide setting available to set the expiration time for your users. Also, we've added a permission setting…

Helpmonks website 2019
Helpmonks in 2019: the year in review

2019 was an outstanding year for the Helpmonks email platform. I'll take you on a short journey to highlight some of the most notable features and enhancements that arrived for Helpmonks in 2019; An enhanced Dashboard At the start of 2019, we've introduced a new way for our customers to…

Helpmonks Features Overview
Here are a few updates you might have missed

Here at Helpmonks, we live and breathe our shared mailbox product daily, and as such, our team continues to improve and add features to the product. So much so, that we almost release something new weekly (you can check this on the Helpmonks RoadMap). Here are some enhancements you might…

Helpmonks what is email marketing
What is Email Marketing and How Does it Work?

Email marketing uses email to promote products or services while developing relationships with potential customers or clients. Read about the perks of email marketing, methods to do it right, and how it compares to other marketing forms. Key Takeaways Email marketing is one of the most cost-effective ways to spread…

Trusted by over 2,000 organizations with millions of emails

Helpmonks helps us answer customer emails at lighting speed
We couldn't do our job without Helpmonks!
Helpmonks, the best thing since sliced bread
Excellent option for collaborating email amongst your team
A good platform that is here to grow and help you grow
Helpmonks is a great tool that our company utilizes and depends on
Great shared inbox for the team
Fantastic email support service for any business
Easy to navigate and catalog data for specific users
Solid Helpdesk software w/ responsive customer support
A really good solution to manage customer support mailbox
Good support, useful features

Hosted, dedicated cloud server, or self-hosted

Helpmonks is the only vendor with a multi-deployment model

Hosted

Use our hosted shared mailbox software for your business. Everything is safe, secure and password-protected. Works with any email account and email client.

Over 2,000 organizations use Helpmonks daily on our platform. Get up and running within minutes.

See why over 2000 organizations trust us

Cloud Server

Want your own server with unlimited mailboxes? Our cloud based servers are deployed with the highest security in mind and are GDPR compliant.

Dedicated Helpmonks shared mailbox servers are the perfect companion to your team workflow.

Explore the benefit of the cloud server mailbox

Self-Hosted

Scalable and enterprise ready. The Helpmonks self-hosted edition grows with your company and has been deployed by some of the largest Fortune 500 companies with millions of emails

Secure and protected. Have full control over all data.

Check out the on-premise shared mailbox

One more thing...

Helpmonks gives you the ability tocollaborate as a groupin your shared mailbox, features aTeam To-Do App, powerfulCustomer Relationship Management (CRM)capabilities, integratedLive-Chatand Email Marketing Tools for email newsletter and automated email campaigns, collision detection, email tracking, reminders, mentions, custom fields, custom applications, customer satisfaction ratings, Single-Sign-On (SSO) & SAML authentication, Trello, Zapier, Slack, an API, and much more.

Check outthe complete feature listor sign up for afree 30-day shared inbox trial now