Mozilla History
Mozilla (stylized as moz://a) is a free software community founded in 1998 by members of Netscape.
Jamie Zawinski from Netscape registered mozilla.org
He was a founder of Mozilla.org, personally registering its domain name on the day of Netscape’s open source announcement and helping design and run the organization through its first year.
He developed the Unix release of Netscape Navigator 1.0 and later, Netscape Mail, the first mail reader (or Usenet reader) to natively support HTML.
The project took its name “Mozilla”, after the original code name of the Netscape Navigator browser.
Mozilla’s most known products include the Firefox web browser and Thunderbird e-mail client.
How does Thunderbird emails storage work ?
There are several ways to store emails on disk. Two of the most common are mbox files and Maildir directories.
Unless you explicitly use optional support for Maildir which allows a single unique file per email (using the EML data format), Thunderbird stores emails content in two distinct files.
The first one is an mbox file (a text file without extension), which is a standard format to store email content.
All messages from a mailbox folder are concatenated and stored as plain text in a mbox file.
It was first implemented in Fifth Edition Unix.
mbox database files sometimes have an .mbox extension, but this is not required nor expected.
Each message starts with the four characters “From” followed by a space (the so-called “From_ line”).
RFC 4155 (2005) The application/mbox Media Type : https://datatracker.ietf.org/doc/html/rfc4155.html
The second is an .msf file, which stands for Mail Summary File.
This is an index file, which contains only headers and summary of emails.
See Mail Summary Files writeup explaining Mozilla mbox and .msf file design choices.
Jamie Zawinski
"In this modern world, the only sensible thing for someone implementing a mail reader to do is use BSD mbox files to store mail messages, because that is the de-facto standard used by almost every popular mail reader, on Unix, Windows, and Mac.
…
The mbox file format is basically one big text file containing every message of the folder sequentially, with separator lines between them. This has obvious efficiency problems: you don't want to spend a lot of time doing linear searches through the file, and you don't want to ever pull the whole file into memory at once, and deletions are expensive.
So, you have to summarize it, and be lazy about updates.
My approach, when designing this for Netscape Mail in the 2.0 and 3.0 timeframe, was to identify the pieces of information that needed to be gotten at quickly and repeatedly, and store those in a cache.
Most other information was only needed on a per-message basis (not when operating on many messages at once) so that latter sort was simply parsed from the message on demand, as the message was being displayed.
…
So, we have these summary files, one summary file for each folder, which provide the necessary info to generate that message list quickly.
Each summary file is a true cache, in that if it is deleted (accidentally or on purpose) it will be regenerated when it is needed again (which might take a little while, but which only needs to be done once.)
…"Mail Summary Files (.msf) use the Mork format.
Mork is a database file format invented by David McCusker for the Mozilla code since the original Netscape database information was proprietary and could not be released open source.
It is not very human-readable, it has been phased out in favor of SQLite, a more widely-supported file format.
As you may notice it is still used by Thunderbird application.
The first line in the .msf file is the header:
// <!-- <mdb:mork:z v="1.4"/> -->It is structured as a comment so it can be safely ignored by the parser.
NOTES
EML (Electronic Mail Format) files typically store each message as a single file (unlike MBOX which concatenates all the messages from a folder into one file), and attachments may either be included as MIME content in the message or written off as a separate file, referenced from a marker in the EML file.
RFC 5322 (2008) Internet Message Format : https://datatracker.ietf.org/doc/html/rfc5322
Christopher J.Prom (2011)
"The IMF serves as the basis for two of the most common storage and exchange formats: MBOX and EML.
In reality, neither of these methods constitutes a storage format per se, since each server or client implements them somewhat differently.
MBOX (sometimes known as Berkeley format) is a set of four slightly different storage formats, developed originally for Unix systems.
Generally, a single file with the extension .mbox or .mbx contains the contents of an entire folder, with MIME content stored directly in the file.
Files can and do grow to astronomical sizes, and even slight file corruption may affect the ability of certain clients to access individual messages or even the entire folder.
MBOX files also include the attachments in their MIME format, meaning that action will likely need to be taken to migrate them, if they are to remain accessible in the future.
EML files typically store each message as a single file, and attachments may either be included as MIME content in the message or written off as a separate file, referenced from a marker in the EML file.
In spite of these issues, MBOX and EML have achieved a certain status as de facto standards.
Most modern email clients and servers can import and export one or both of the formats"Where does Thunderbird store data ?
Thunderbird saves all the data such as emails, passwords, user preferences and settings in a set of files called a profile.
The profile is stored in a separate location from the Thunderbird program files.
The location of the profile folder varies according to the Operating System.
With Debian it is located in /home/<username>/.thunderbird/
It is possible to have several profiles, each profile is stored in a dedicated folder.
A profile folder is identified using the following convention: <random_string>.<profile_name>
where <random_string> is eight alphanumeric characters randomly generated by Thunderbird and <profile_name> is the name you assigned to the profile.
Thunderbird uses the /home/<username>/.thunderbird/profiles.ini file to store profiles information
The default profile is set via
[Install<ID>]
Default=<profile_folder>[General]
StartWithLastProfile=<boolean_value>
0=Thunderbird will ask you to select a profile (and make the one selected the default profile)
1=Thunderbird won't ask you to select a profile, it will use the default profile
This option corresponds to Profile Manager 'Use the selected profile without asking at startup'Get Thunderbird’s Default Profile (Debian)
$ sed -n '/^\[Install/,/^Default=/p' $HOME/.thunderbird/profiles.ini | sed -n 's;^Default=;;p'NOTES
When you open Thunderbird for the first time, it creates a default profile. This profile is used automatically.
While it is possible to have multiple profiles, most users just use the default one which may contain several email accounts.
The Profile Manager is used to create/rename/delete profiles and to select the profile to use for a Thunderbird session.
It is not displayed by default, you have to start Thunderbird with the option -ProfileManager
$ thunderbird -ProfileManagerAll non-default settings: prefs.js file
/home/<username>/.thunderbird/<random_string>.<profile_name>/prefs.js
Thunderbird stores any settings you change or create using the GUI via ‘Account Settings’ and/or ‘Settings’ in the prefs.js file
The recommended way to add or modify a setting which is not available via the User Interface is to use ‘Config Editor‘, equivalent to Firefox’s about:config.
OR
You can add/modify a setting by editing prefs.js directly using a text editor.
However, prefs.js does not contain all of the settings, it only contains any settings that have been modified (not set with their default value).
In fact, if you add a setting with a default value Thunderbird will remove it from the file the next time you run it.
Be careful when you edit prefs.js while Thunderbird is running since it may overwrite some of the settings when it exits, losing your edits.
user.js file
/home/<username>/.thunderbird/<random_string>.<profile_name>/user.js
It does not exist by default, you have to create it. This is a user-set overriding preferences (prefs.js)
It is mainly used by administrators to set the same settings in several profiles.
Unless you know what you are doing, it is recommended that you don’t use it since any settings you add to it will overwrite prefs.js when Thunderbird starts, preventing permanent changes using the Config Editor.
Whenever Thunderbird starts it will overwrite any settings in prefs.js with the corresponding settings from user.js
ImapMail: IMAP Folders containing mbox .msf .sbd files
/home/<username>/.thunderbird/<random_string>.<profile_name>/ImapMail/
When you set up an IMAP account Thunderbird creates a folder named after the IMAP server name.
So if your IMAP server is imap.mozilla.org Thunderbird will create a directory
/home/<username>/.thunderbird/<random_string>.<profile_name>/ImapMail/imap.mozilla.org/
You will find inside: the mbox and .msf files according to the standard folders (Inbox, Draft, Sent, Junk, Trash, etc.) and the specific ones you have created on the IMAP server.
The .sbd subdirectories are used to store the folders in a hierarchy
Subfolders are obtained with the .sbd extension
If you created subfolders within your Inbox folder on the IMAP server, you will find an extra file Inbox.sbd which will also contain mbox and .msf files (and eventually other .sbd files for each subfolder)
Be aware though that you may get a different folder name from what is shown in the application and the actual matching mbox .msf files
This renaming option is set within .msf files
Local Folders
/home/<username>/.thunderbird/<random_string>.<profile_name>/Mail/Local Folders
The name says it all : it is local folders (on your computer) meaning there is no direct interaction with any mail server.
Local Folders provides a convenient place to store messages locally when desired, ideal for backups.
The only space limit is the amount of space available on your disk.
By default you have a ‘Trash’ and ‘Outbox’ empty folder
Trash folder:
/home/<username>/.thunderbird/<random_string>.<profile_name>/Mail/Local Folders/Trash
/home/<username>/.thunderbird/<random_string>.<profile_name>/Mail/Local Folders/Trash.msf
Outbox folder:
/home/<username>/.thunderbird/<random_string>.<profile_name>/Mail/Local Folders/Unsent Messages
/home/<username>/.thunderbird/<random_string>.<profile_name>/Mail/Local Folders/Unsent Messages.msf
Outbox is a special folder used to queue messages when you use ‘Send Later’ rather than ‘Send’
When you compose a message and save it to be sent later (File > Send Later), it is saved in the ‘Unsent Messages’ mbox file in ‘Local Folders’.
That folder is also used to send a message in the background if you set mailnews.sendInBackground true
NOTES
emails can be stored on the client, on the server side, or in both places.
Standard formats for mailboxes include Maildir and mbox.
msgFilterRules.dat (Message filters)
Message filters allow you to set up Thunderbird to organize your messages automatically.
Each account has its own set of filters.
Message filters are useful if you routinely want to perform certain actions on messages:
moving messages to folders, deleting them, forwarding them to other email addresses, etc.
Filters can be applied automatically to incoming mail, or you can run them manually when desired.
The message filters for each account are stored in a msgFilterRules.dat file
IMAP Backup Solution
If you want to back up/archive your emails from an IMAP account you may use Local Folders
Keep in mind that this simple solution exists and is appropriate in several use cases:
– You are running out of space on your email account and need to free up server space
– You want to create a snapshot of your emails (copy/backup)
In order to proceed, create a new folder yyyymmdd-backup-emailaccount into ‘Local Folders’
via the GUI: right-click on Local Folders > New Folder… > Create as a subfolder of Local Folders
It will create 2 files (mbox and .msf) :
/home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/yyyymmdd-backup-emailaccount
/home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/yyyymmdd-backup-emailaccount.msf
IMPORTANT
Make sure Thunderbird has fully synchronized all your IMAP folders and subfolders (downloaded all emails)
Set mail.server.default.check_all_folders_for_new
File > Offline > Download/Sync Now
Download and/or sync the following : Select ‘Mail messages’ only and click on ‘Select’
Tick any folder you want to sync and click on ‘OK’ twice
Depending on the email account and the network capacity, it may take a few minutes…
Your IMAP folders should be synchronized now
Via the GUI select the IMAP folders you want to backup (Ctrl+Left Click)
Drag and Drop to the folder yyyymmdd-backup-emailaccount in ‘Local Folders’
Here too, it may take a few minutes to complete…
You have now a local copy of your emails (snapshot):
/home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/yyyymmdd-backup-emailaccount
/home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/yyyymmdd-backup-emailaccount.msf
/home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/yyyymmdd-backup-emailaccount.sbd
The good thing is that you get exactly the same IMAP folder<>file names match.
It is recommended that you save/move the mbox .msf and .sbd files to a directory ‘yyyymmdd-backup-emailaccount’ on a separate backup device
Do NOT create this directory into /home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/
Close and reopen Thunderbird application
If you want to restore your backup:
– Copy the three files back into /home/<username>/.thunderbird/<profile_folder>/Mail/Local Folders/
– Close and reopen Thunderbird application
Thank You Mozilla