I have just completed sessions of usability testing of Thunderbird.
This time, I had the pleasure of working with Andreas Nilsson, who came to London to observe the sessions. It was very useful to get his feedback and to work collaboratively with him on the analysis and implications of the findings. In addition to these benefits of our work together, there is an added one: since he observed participants struggling with certain aspects of the interface, he will no doubt be a very effective user experience advocate with his team.
Andreas, thanks for your time!
The Test
Twelve participants were recruited from the general public – one turned out to be a no-show. They represented a mix of gender and age. Special consideration was given to heavy email users. Of the 11 participants, 5 were exclusively Windows users, 3 were exclusively Mac users, and 3 used both Windows and Mac.
In preparation for the sessions, we set up 2 test email accounts. A few days prior to the sessions, I sent messages to these accounts and also subscribed them to mailing lists. When participants signed up, they had already received a sizable quantity of emails, allowing us to ask them to manage the messages the way they generally do in their own email boxes, to find specific messages, to create filters, and more.
Thunderbird was tested on Maverick and Unity.
Between sessions, Thunderbird was removed and all hidden files were deleted, so the next participant got to start from scratch.
The Methodology
Over the 60 minutes of each session, I went through as many features of Thunderbird as possible with each participant. Participants were asked to:
- Install Thunderbird from the Software Centre
- Create an account
- Sign up
- Create filters
- Set up alerts
- Manage emails in folders
- Create a signature
- Change the colour of the font
- Create a contact list
- Search for a specific email discussing a form (which I had sent prior to the session)
- Respond to an email that contained an attachment: in particular, open the attachment, modify it and send it back to the original sender
What participants liked
There were many aspects of Thunderbird that participants enjoyed, and many tasks at which they succeeded.
Participants commented positively on the tab system, which makes the navigation between messages easy and immediate, and which provides visibility on multitasking. The tagging of messages also got positive evaluation. Many participants commented on the simplicity and usefulness of the contacts. Filtering was perceived as effective, although, as we will see below, the majority of participants experienced some challenges here.
Participants found the activities which they carry out most often – opening, reading, responding to and deleting emails – easy and straightforward.
Where the trouble is
Critical issues
Participants encountered few critical usability issues – by ‘critical’, I mean issues that would make it difficult or even impossible to use the application on a regular basis. These issues need to be addressed if we are not to lose users to alternative products.
Install
After installation from the Ubuntu Software Centre, participants could not find Thunderbird to start using it. They did not see, in the product description, the bread crumb indicating the location of the download when provided.
Observation: After having installed a new application, users generally are excited about using their new software. The user experience would flow much better if, as the process of installation ends, the application opens automatically in the main window, allowing users to deal with their settings and messages right away. We need to keep users excited about Thunderbird. As it stands, it is a bit of a let down to not be able to find the new toy!
Create folders
One of the main challenges for participants was managing their many emails by creating folders.
Most participants did manage to create a folder by right clicking on the folder area. However, they could not find the folder once they’ve created it, and so couldn’t drop messages into it. This was because they had in fact not created a ‘folder’ (as promised by the menu label) but a ‘sub-folder’. The sub-folder was not visible, because it was hidden under a folder.
Those participants who did eventually find the sub-folder they had created wanted to make it into a folder, but were not able to do so. Users normally organise their folders in a way that facilitates their use of emails. They tried to drag their sub-folder out of the parent folder and relocate it.
There are 4 main folders they want visible: inbox, sent, junk and draft. It is worth noting that the ‘sent’ folder in Thunderbird is a sub-folder of gmail; this was confusing to participants. As a case in point, several participants failed at checking if a message they had sent me had really been sent because they couldn’t find the ‘sent’ folder at all.
Participants also expressed a preference for ordering their folders. In addition to the point mentioned just above, some indicated that they like to create a work folder and a personal folder. They place these folders next to each other. They were not able to do this in Thunderbird.
Observations: Users manage their mailbox by customising folders. The level of customisation they need goes beyond creating and naming sub-folders. They want to create their own hierarchy of folders and sub-folders as well as to order them for convenience and visibility.
One more thing on this topic: participants were not clear about some of the words used to describe folders. For example, they did not know the difference between a ‘folder’ and a ‘local folder’.
Create filters
Most participants failed at creating a filter.
First, they didn’t know where to look to set filters up. Most participants looked under preferences and account settings. After looking generally at the menus they gave up.
Second, participants were unsure of the meaning of the dialogue boxes and of what was expected of them. They found the process of setting a filter unduly complex and they needed more feedback to measure their progress.
After participants managed to create a filter in the filter rules dialogue box, they clicked OK but didn’t know if the filter was actually set up or not. Additionally, they couldn’t figure out how to run a filter they had created. The issue was that, once having created a filter, when participants came back to the message filters dialogue box, the last filter set up is not selected – thus the run now option is not enabled. At the same time the enabled check-box is selected indicating that the item has been selected.
Observation: After setting up a filter, users would like to run it to confirm that it works. Make the command ‘run now’ the next step in the process without users having to specifically select the filter to run it.
Find open and modify an attachment
None of our participants was able immediately to find the attachment in a message. They expected the attachment to be visible at the top of the message. While most participants eventually found the attachment, some didn’t, and consequently could not open and modify it.
When participants did not find the attachment, they consulted help, but were not provided correct information.
After some participants found the attachment, I asked them to edit it. They did not expect that the attachment would be in a read-only mode and tried to edit it without saving it first. The message warning them that the document is read-only only appeared after many attempts. It would have been friendlier for the message to be shown at the first attempt.
A few participants, after they attached a document, were not clear if the document was in fact attached to the message. They needed a stronger visual cue.
Observation: Sending, finding and reading attachments are fundamental activities on email. The user experience would be greatly improved if the attachments were located where users expect them, at the top of a message and/or if they would be more visible by changing the appearance of the link or using a colourful icon. Additionally, users would benefit from some immediate feedback on ‘read only’ documents as well as from a confirmation that a document has been successfully attached to an email.
In this case, for Thunderbird to be user-friendly, it would need to anticipate users’ needs, mainly need for visibility and for feedback at the first occurrence of an error. This anticipation of users’ needs would show the willingness of Thunderbird to collaborate with its users and to recognize their goals.
Search
Participants were unclear about the differences between the 2 search boxes at the top left of the screen.
Often they didn’t get results because the global search bar doesn’t suggest anything other than names.
Search doesn’t take into account misspellings – and so, when a word was misspelled, participants got no results.
Every time a participant performed a search, a tab opened automatically even if the search provided no results. As a result, participants opened many tabs that were not useful or wanted. They found that the tabs cluttered the interface and made it difficult to find such things as the inbox.
Observation: Users should know, before searching, what the fields will be actually searching. The area dedicated to filters is interpreted as a search and not a filter by participants. In part, the issue for users is that the boxes look virtually identical, and thus, from their point of view, should be interchangeable. A different visual treatment would greatly improve the usability of the different search boxes.
Less-than-critical issues
Participants also highlighted usability issues that were not critical, but that compromised their enjoyment of Thunderbird.
Mail account setup
Participants did not understand the message contained in the mail account setup dialogue box. They had to make a choice between:
IMAP – Access folders and messages from multiple computers
POP – Download all messages onto this computer, folders are local only
Uniformly, they did not understand the implications of this choice and went for the ‘recommended option’ – just because it was recommended. Most participants said that they would not read the message anyway and would just accept and move onto the next screen.
One participant chose the POP option, which caused her problems with search later.
In addition, the mail setup message has a button that says “create account”. This was confusing for some participants who thought they had already created an account and now were in doubt. Some wanted to go back to the signup page to check. There is no way to come back to the signup page, however.
Observation: While users are setting up their account, they are most eager to get the process over with. This is in part, because they want to see the application but also because they need to see what they will get, so to speak, before they can understand the pertinence of the various options proposed to them. In this case, it is good practice to make a recommendation – which simplifies the process. However, the choice should be clear, from the user needs perspective (so users don’t just choose what is recommended because it is simpler and don’t foresee the consequences of their choice).
Set up alerts
2 participants expected to be able to set up alerts in tools, but were not able to. Many participants were not able to find a way to set up alerts at all.
Create a signature
The majority of participants expected to be able to create a signature under ‘composition’. When that failed, they looked under tools, add-ons, preferences, insert and write. Most who wanted to did not succeed at creating a signature.
Some minor issues
‘Minor’ usability issues don’t compromise the main usage of an application or the integrity of the user experience. However, they can be annoying and irritating, particularly when the application is used on a regular basis.
Change the colour of the font of their message
Most participants either failed at changing the colour of the font for all their messages or were not sure they had succeeded after selecting a colour from the palette of ‘display’ in ‘preferences’.
In part, participants could not find the option to change colours. For those who found it, when they selected a new colour, the new selection was not reflected in the messages they wrote just afterwards. But also, after they selected a different colour, they were not sure if the chosen colour would appear in their message.
Observation: Many users like to personalise their communication. Playing with colours and lay-outs should be easier for them, with relevant options more visible. In addition, users need some feedback that their change will be immediately implemented. An ‘apply’ button or a confirmation that the new selection has been registered would reassure them.
Navigational issues
Participants did not know how to get back to their inboxes from either the address book or the ‘write’ screen. They didn’t understand that in these specific cases new windows were opened, instead of tabs, and that they needed to close them to go back to their in-boxes.
And more…
Participants had further suggestions for new features. They wished for:
- A calendar on the side so they can see messages and their commitments at the same time
- A way to compress large files directly from the email account
- Some social networking, at least so that they could see that their friends are online
Excellent report. I wonder what’s next… Are you papercutting TB as well?
I also would like to see the calendar (Lightning) integrated on TB by default. Also I miss a lot something like the auto-add account (when you add an account and TB searches for imap servers, pop, smtp, etc). I miss on Lightning the feature to say, enter your email account and it would fetch the existing calendars for that account. That would rock, wouldn’t it?
Cheers.
I’m excited to see the science of Ubuntu replace the religion of Linux.
Scientific end-user experimentation and observation, prediction, directed change. This is why I am Ubuntu.
The search functions in the new thunderbird are completely nonsense.
It is a pity that TB was so downgraded.
Thank you.
Thanks for this very interesting study. I feel also concern by the complexity of setting up an imap account.
For the particular concern “It is worth noting that the ’sent’ folder in Thunderbird is a sub-folder of gmail”, it is identified as bug 80858 at https://bugzilla.mozilla.org/show_bug.cgi?id=80858 for which you can vote or try to solve ;-)
About the automatic launch post-install, wouldn’t it be a disaster when installing multiple applications at a time? (note that I am a Fedora user, I don’t even know if the Ubuntu Software Center allows that).
Gnome-PackageKit has this nice feature of presenting a window populated with the newly installed applications, right after completing a transaction, and offering the user to launch them right away.
It seems like it would answer your use case.
About the “sent folder is inside the GMail folder3, it works exactly the same on Evolution, so I’m pretty sure that’s due to GMail’s weird setup of not really having folders, but rather everything in a huge “All Mails” folder, and tags (“Inbox”, “Sent” or custom user tags) for each email.
Those tags are then mapped to folders when synchronized over IMAP, and everything starts being funny (like if you apply several tags to the same email, the email appears in two folders. Remove it in one, the email is still in the other folder… until the next IMAP sync :)
GMail is really a poor implementation of IMAP, but unfortunately it appears to be the most used. :(
Excellent report and work, this has to be the way forward for all open source projects.
But I do feel a number of the comments are “can we make it work like the one I am used to”.
Personally I prefer attachments at the end of the message.
If the commands are well thought out and logical users will soon get used to them, if they are awkward to use, however familiar, they will annoy after a time.
I hope Mozzilla take on board what you have found.
David,
I am not arguing that any software should copy any other and, as you say, “make it like I am used to”. The point is even if you make important changes, you need to preserve the user experience and make it flow so your users can enjoy the innovations you introduced. If you think it is better to show attachments at the bottom of the message, then you need to make sure that they are immediately visible to your users.
I have a small suggestion. Instead of opening the app after installation finishes, just add it to the launcher automatically. It could do the wiggle thing to get the user’s attention or something similar.
Hi,
just a user-specific feedback regarding “bad” effects in TB 3.x
While creating a new account the assistant tries to auto-detect server-adress etc without asking before he does that.
how about letting the user decide if he is willing to use that assistant which is guessing wrong settings in my case several times.
Right now users land in that “strange” autoworkflow – must cancel that and do have a nice chance canceling the entire account-creation process while trying to cancel just the auto-part.
greetings
fidel
> Gnome-PackageKit has this nice feature of presenting a window populated with the newly installed applications, right after completing a transaction, and offering the user to launch them right away.
How do you find the app the second time you want to launch it after that “newly installed apps window” has gone away?
This is exactly the kind of work I want to help FLOSS get more of done. I imagine that this kind of testing could be done quite easily without a usability lab, nor with formal recruitment. Would you be willing to share the usability test script you used, so we could leverage for other testing? Perhaps a template of sorts to help frame similar tests more easily?
Kirk,
Yes and no. Yes, testing can be done without a formal usability lab – actually ours is quite portable! As for recruitment, it needs to be formal to a certain extent because the results of your testing will depend a lot on your participants. Just to illustrate, if my participants had never use email before, the problem areas for them would have been different than for the participants who matched my profile. This said, you can decide on a profile and do the recruitment yourself more informally as long as the participants correspond to your criteria.
Every test protocol is tailored to an application. There are no templates. However, I will start to discuss the process of setting up usability testing, writing protocols and conducting testing in this blog over the next few weeks and will be happy to share my protocols as well.
Hi, I’m happy to see Ubuntu trying to improve the user experience.
the issue about launching an application right now after it being installed is
IMHO not so important because, not every one want this behavior.
Software Center already shows where we could find our installed application.
We could just imphasis to the user that Software shows them how to find their application. or another option it to indicate via indicator buble where to find them (both in Unity if we are running Unity and in the Classic desktop if we are
running this last one.
Great Job Canonical
Will you be filing bugs in bugzilla.mozilla.org (and Launchpad for the Software Center issue) and updating this article to reference them?
fwiw, personally, wrt the Software Center post install problem, I’d expect the Windows behavior (i suppose it’s a Vista behavior, but I use 7 — who uses Vista?). New items appear with a highlight background color in the Start menu until you use them (actually, the logic that Windows uses for dismissing the highlight is too complicated for me to entirely understand, but…). This makes it very easy to find them.
Some minor issues – Navigational issues:
This doesn’t surprise me at all. I’m helping an old man with his computer and he has the same problems – even I would like it to have it in a Tab instead of a new window.
PS:
cyrildz wrote “the issue about launching an application right now after it being installed is
IMHO not so important because, not every one want this behavior.
Software Center already shows where we could find our installed application.”
Thats my point of view too. Maybe make the installation place on Software-Center better visual (bigger or a colored box around) but this should be enough. New user will learn fast how to launch Software in Ubuntu.
OT – Usability problem in Software-Center (10.10):
1. When looking at the ‘History’ it shows all installed Software >> updates included<< but this is not what the user is expecting – he expect to see all software that he installed manually.
2. When (right) clicking in 'History' at an installed program there is no option to remove it – this would make the 'History' function useful for users (read only history make no sense to me).
I am vehemently against launching applications immediately after install. It’s sort of rude. Why not have a “launch program” button embedded in the software-center page after install (only that one time, though!)
I’m not sure if it’s better or worse for Outlook exiles, but Thunderbird 3.3 is (probably) going to display attachments differently: https://bugzilla.mozilla.org/show_bug.cgi?id=282068
I dislike TB because of one simply reason: it does not support splitted messages produced by MS Outlook or TheBat.
Content-Type: message/partial;
Thunderbird (and many other clients) can not handle and recombine such messages. Mozilla promised that these messages will never be handled properly, so, the TB is worst choice for users, who receive such messages on regular basis.
Why did you only test for one hour? Why don’t you let the users use thunderbird for two days, and check what they’d learned until then?
Nobody can expect to be familiar with a program after an hour of usage, neither can it be the aim to make a program such that the user understands everything with in the first minutes, rather should it be designed to be effective to use on the long term.
Marian,
In a traditional usability test, like this one, we are testing for ease of use, learnability, effectiveness and enjoyment. Our test questions are: how quickly can users make the product do what they need to get done with it? Is the product communicating what its creators intended or the affordances of the product? How does using the product make people feel? You are right that long exposure will increase the knowledge and familiarity with a product. We didn’t do it in this case because for an email client, testing over a period of days would not reflect users real world experience (users don’t struggle over days to use a new email) or answer our basic questions.
We do conduct longitudinal studies sometimes, 1) in order to examine learning and appropriation of technology, 2) when the software is fairly new and complex or needs to be integrated with many user activities not possible in the lab context, 3) when we have a very specific issue or user behaviour we need to understand.
In the case of TB, I think the 60 minute test revealed its strengths and weaknesses and ease of use. None of the activities we tested should take 2 days to learn but they should be immediately available.
Very interesting to see other users’ experiences with a previously unknown email application. As a recent switcher to Ubuntu (and not very tech-y one)(switched from Win Vista and it’s default mail software) I have to say I struggled with Evolution and have ended up on Thunderbird, but it too is far from perfect. What I’d like to see (and someone somewhere may know if this is possible) is a product that (a) downloads to the pc (i.e. cleans off server when onto pc but not when onto Blackberry like POP3 does, but can be easily configured)
(b) when archiving into folders saves the message as an individual html or txt file, with its attachment, into a ‘real’ folder that I can then archive onto another drive but find again easily in the future and open with a web browser (html) or notepad (txt). The compressing of email into a hidden box deep inside the file structure somewhere is, for me, the most frustrating thing about ALL email software. In conclusion, all email products I’ve used over the years have many failings and it’s a poor state for something that is used so much, so universally !
When using the Mac App store once the install button for an application is pressed the applications icon appears in the dock with a progress bar showing the how far the installation process is and once its installed the icon remains in the dock so that the user can use the application straight away and knows exactly where it is. Maybe this is something that Ubuntu could do with the Software Centre and Unity.
Charline,
It was a great to look at Thunderbird from the eyes of first time users. It brought me back to my first experiences with Thunderbird, as well as playing with new features in Thunderbird 3.
As a long-time involved Mozilla and Ubuntu user, I’d like to let you know how nice it was to read this study. I have been an advocate for making Thunderbird the default mail client in Ubuntu, having better integration between Lightning (calendar add-on) the system calendar [1], as well as better integration with the system [2]. After this interesting read, now comes time to provide feedback.
1) Address Book could have used more testing:
Other than the additions of pictures, the address book code in TB3 has not been rewritten and it seems to me that it has become quite buggy. The address book is a feature that is often placed in the background, however I beleive that it deserves an involved code review. Address Book issues include the following symptoms: unexpected behavior when dragging and dropping contacts – results duplicate or ignored contacts (Bug #711314), not being able to copy address block from card (Bug #235101). Having a large base for contacts to start off (perhaps ask users to import contacts from a csv, vcf*, and ldif file) as this would be helpful in testing the address book and identifying unexpected behavior.
2) Software Center – Run after install
This is a feature that was available in “Applications > Add/Remove” in previous releases of Ubuntu (before software-center). I would suggest that after downloading the application, double-clicking its icon in software-center would open that application.
3) Filters – “Most participants failed”
This brought me back to before the time I knew about email message headers. I use these heavily to filter Launchpad email notifications. However, I agree that without help in discovering this possibility, I would still be filtering via email address and/or subject key words. A more intuitive method of creating filters (“Filter messages like these” contextual menu option comes to mind) as well as an example of what would be filtered (preview) are great ideas. From time to time, I still have an issue where filters based on headers do not work after the email has been received and is in the inbox (Bug #119899) – it only work on mail as it is being received.
4) Attachment Icons
I had this problem as well. It seems that the package thunderbird-gnome-support (which supplied the icons needed) is only a suggestion and not a dependency of the thunderbird package. There is a report (Bug #137221) which deals with this issue. Unfortunately it seems to be marked as Fixed/Wishlist even though it has not been made a dependency. The necessary package was simply identified.
5) Permission issues with attachments
Opening (not saving) an attachment directly from Thunderbird downloads the file in /tmp and makes the file permission 400 meaning the owner can read only. There may be a technical reason for this, however you have the ability to work around this by saving a copy somewhere else than /tmp (from TB or the app which handles the file, such as OOo, gedit) or changing permission of the file in /tmp (using nautilus properties or chmod). Most users will not know how to do the latter. Is there any harm in saving the file with 600 permissions (owner-read/write). Perhaps the user will make changes, save to /tmp and then when /tmp is cleared the documents changes will be lost. Perhaps it has to do with execution from attachments. We should definitely look into this.
6) Search VS. Filter
This was a big change for me when trying TB3 for the first time. I could not get the hang of it until I used it a few times and understood that top right of folder list view is filter and extreme top right of thunderbird window is global search (i.e. results new tab). Overall I like the feature, but it took a while until I understood the gray text withing these two fields.
Thanks again for the great post!
[1] http://blueprints.launchpad.net/ubuntu/+spec/lightning-extention-panel
[2] https://bugs.launchpad.net/thunderbird/+bug/711305
* Tricky since this is not a default import feature
That’s awesome. :)
Even if the process is tailored to a specific application, as it should, all usability test have a great degree of overlap. I think it would be helpful that you share the test protocol to:
1) Show to the community an example of a real usability test protocol.
2) To help the community provides feedback and improvements for future test improvements.
As a side effect, sharing this protocol would allow, when a new version of Thunderbird is released, any member of the community to test the new versions under the same assumptions.
Also, you talk a great deal about choosing the right participants but the information about how you recruited and what you searched in participants seems a bit scarce.
As a final note, i’m really pleased to see all this preoccupation about improving the usability and the ux of OSS reaching beyond the “works for me” mentality.
One thing nobody in this testing thought to do is include a blind person in the test. I am working with a blind lady using NVDA (Non-Visual Desktop Access, an open-source screen-reader), and Thunderbird, while more accessible than the web interface of Gmail, is proving to be quite challenging without a mouse or a display.
In particular, the search dialog is WAY too complex.
David,
I do just that in my upcoming posts.
“The user experience would flow much better if, as the process of installation ends, the application opens automatically in the main window”
I disagree. This introduces an ‘initial’ and a different ‘subsequent’ process to running an application. The underlying issue will remain with your proposed fix. Further, people may get into the habit of running applications by navigating to them in the software centre.
The real root cause here is that it’s not obvious how to run applications.
Perhaps it should be easier to find applications – if software centre popped open the Applications->Internet main menu and flashed the ‘Thunderbird’ option, now that would be cool.
Perhaps software centre and the main menu should even be combined – the download/install step being an incidental/background action.
For me Thunderbird is : unusable – for the simple reason that my ISP’s auth parameters are TLS with its specific #995 port for POP, and smtp #25 with username auth.
There is no ways,no where in Thunderbird that I can know to setup specific parameters for email account.
Finaly I found a place (here) to report this non-sens failure, because I am not willing to create accounts every where for bugzilla and family just to report that failure for something I will not use anyway.
Thanks,
Good luck.
Sorry for my insistance on TLS:
I wonder how KDE’s KMail email account setup does automagicaly detects the protocol’s params for me transparently.
This is something Thunderbird developers should look…
I thought Thunderbird did that automatically now. You put in an email address, and then checks common ports on that domain to see which ones to use.
Another issue: Thunderbird is very very slow (like Firefox). Just compare Firefox to Chromium/Chrome Browser, the same for Firefox.
Another thing is missing x64 support from mozilla – they keep ignoring x64 completely…
As a mutt user I have to say that it is not about usability but about _efficiency_. People use email applications daily and figure out everything eventually.
Thanks for posting this study. It’s a usual way to conduct and to analyze that topic. Some remarks about methodology:
1. Many aspects of your observations refer to setup and data handling. I wouldn’t expect a Windows users to be confirm with Ubuntu’s kind of “appstore”. And even Mac users are, as far as I know, not confirm with folders like /home//…
2. You were testing 11 people but report just one “observation”. Unfortunately, this is not unusual but one must not treat results as final. Who cares about 2 MB limit of attachments, how knows how to mark a signature corrrectly, or posts a reply at the right position? If one, some, or even all subjects want the attachment to be inserted into the text like Lotus Notes it does, never Mozilla should (and will, I hope so) implement this.
3. I don’t agree that e.g. “changing a font colour” is a key feature that needs special care. Usability starts with a persona, with basic navigation, with core functions. A good program is not that one with most features or funniest goodies.
For a future test you should consider to compare TB against other programs like Kmail, Outlook, Lotus, Googlemail and let users perform some easy tasks. Measure time, failures, repeatedly started functions etc. And ask for fun :-). All setup questions should be let to experts like most readers here.
After all I’m very pleased to read about Usability tests for open source programs. Thanks a lot!
Why such an extensive test with Thunderbird instead of Evolution?
Is Ubuntu planning to replace Evolution with TB?
I think same test should be carried out by Canonical for Evolution especially because it supports Exchange accounts through MAPI & exchange plugins.
I was trying to replace Windows with Ubuntu at my workplace, but failed miserably as Evolution is not even 1% efficient in dealing with MS Exchange accounts as Outlook.
To make Ubuntu a successful candidate in corporate environment, Exchange integration is a must and Evolution holds the key!
This is a poorly conceived and poorly observed study, Comparing apples with oranges and complaining about them both having pips.
Many of the complaints about TB are either unrelated or also true of Outlook, also many are of the “it isn’t Outlook ” variety.
Outlook doesn’t launch itself after install, nor does it appear in a sensible location- “Programs > Microsoft Office > Outlook” is no easier to find than “Applications > Internet > Thunderbird”, in fact, I would contend that Outlook is harder to find.
The Google mail IMAP problem is a Google issue, not TB, gmail’s IMAP implementation is poor on both platforms, mainly because it is a webmail app.
The read only attachment issue is a Linux issue, not a TB one.
I know from experience that most Windows users struggle to create signatures in Outlook.
“Users expected the attachment at the top”. Why? Because that’s where Outlook puts it?
Just because TB is not the same as Outlook is not grounds for criticising it.
The problems were compounded by using a different OS and a different mail platform to the common set.
I could go on.
It is fairly obvious that most if not all of the subjects had previously used Outlook in a corporate exchange server environment where the IT dept puts shortcuts for Outlook on the desktop and pre-configures it for them.
I have users happily using Evolution on Ubuntu in an Exchange environment alongside Outlook users.
I also have 2 users using TB on Windows (to separate their email from the main company, as it uses an external server) with little issue.
Did any of the testers notice that they were not able to drag attachments from a message to the desktop or a folder? https://bugzilla.mozilla.org/show_bug.cgi?id=377621
Nice test. I just had some problems w redirecting my mail. People (meaning me) are lazy – if there are some problems, they just go with the old (which might be worse) version.
Charline, are you open to making your data publicly available for further analysis? It’s tough to find good open source usability data and I wasn’t sure if it was an option for this project since TB is open source and for Canonical since you are open-source aware.