PGOffline 4 Walkthrough (Desktop PC)*
Updated November 3, 2019
(Web Formatted Version of this Document Here)
How to Install PGOffline and Add a New Group
How To Download & Export Messages
How To Download Files and Photos
Will I Be Able To Import The Messages Into A New Mailing List?
There Is A Lot Of Spam! What To Do?
OK, I Am Done Downloading, What Should I DoWith The Files?
Preface
On or about October 16, Yahoo announced that it would be permanently closing all Yahoo Groups on December 14, 2019, and deleting all hosted content. Yahoo Groups has played an important role in the history of fandom, hosting everything from zines to discussion groups to fanfics and fanart.
Now is a great opportunity to dust off your old log-in credentials, take a trip down memory lane, and rescue a bit of fandom history. If you wrote or posted anything on Yahoo Groups, why not visit groups.yahoo.com and re-post it to AO3? You can even backdate on AO3 so the date matches when you originally composed it.
Please note: If you haven’t used your Yahoo account for twelve months, Yahoo may have deactivated it due to inactivity. If your account name has not been claimed, you may be able to create it again, and regain access. (see: “What to Do if Yahoo Mail Deletes Your Inactive Account”)
Open Doors - the fanworks-rescuing committee of the Organization for Transformative Works - would be happy to help import fanworks from Yahoo Groups to Archive of Our Own. But they can only do so with the permission of a group administrator. If you are such an administrator, please contact the Open Doors Project. If you’re not, feel free to encourage the admins of your favorite fan groups to do so.
Please note that on AO3, unless you are importing works from a community of which you were the administrator, you can only publish works that you own the copyright to.
That being said, anyone can download and archive a mailing list that you belong to. You do not have to wait for a moderator to archive. In fact, the more that people download, the more fanworks and fandom meta we can save. This is a group effort. Archive Team has already archived a large number of groups, but many private groups may have been hidden or inaccessible, and thus missed. You can stop and start downloads at any time and do them in small chunks.
To see the mailing lists that have been archived or are in the process of being archived: List of Yahoo Groups Being Archived
To submit the name of mailing lists that you are planning to archive, use this Google Form
PGOffline only works for Desktop PCs running Windows. If you are a non-Windows user, see the Mac/Linux User Tools section below
How to Install PGOffline and Add a New Group
How to download all the contents of a Yahoo Group using PG Offline 4. You will need Windows 7/8/10 and a valid Yahoo account for this to work. PGOffline offers a free 14 day trial but as of October 31, 2019, the trial limits you to only 1000 messages - the cost is $25 after that. Be certain to test the software first before purchasing.
Download and Install PG Offline 4 from:
http://www.personalgroupware.com/downloads.htm
Enter your Yahoo username and password. Please note that if you have 2-factor authentication enabled on your account, this program will not work. You will need to turn it off - but beware that this may reduce the security of your account. If you have problems logging in check this post by the PGOffline developer.
The developer has a video tutorial that will cover most of these next steps (link)
You also have to have an active Yahoo Alias, distinct from your email. Without this, the program will give all sorts of connection errors. Try to join a group and make sure your credentials are accepted on Yahoo’s side. If you are getting connection errors when you try to download something (“Your email address is not linked to a Yahoo ID" or “Login failed. Either you have the wrong username & password OR you have two factor authentication switched on.”) try entering your Yahoo alias wherever PGOffline prompts you to.[1]
If you are encountering other login problems see the Login Problems section in “Yahoo Groups Is Shutting Down.”
PGOffline will only work for groups that you have access to, so you will need to “Join” any community you are trying to download from, even if it’s a Public group.
Once you have joined your Yahoo Groups, the next step is to tell PGOffline what Groups you now belong to. There are two ways to do this: Automatic List or Manual List Groups
Automatic
Click on ‘List’ It will pull up a list of Groups. This may take a while.
You will see your list of groups
Click on the Group you’d like to Download (you can click on more than one)
Then click “Add”
You should see the Group in the right column. Then click “OK” when you are done adding.
Manual Method of Adding Groups To Download
Enter the Group Name and select “OK”.
How To Download & Export Messages
On the main page, highlight a Group you previously added and then select Messages --> Receive Messages.
Messages should be appearing in batches of 25, from oldest to newest. You can stop and start downloads at any time.
From there, make sure you Export everything. For free Trial users, the best method is to choose to export the messages in a giant .txt file or save the entire database. Either hit the big “Export” button, or right-click on the Group and select “Export Messages”
After choosing the path, change the default file extension from .txt to .html.[2]
Open the file in your browser of choice. It’s ugly, but it works.
BONUS SUGGESTION: Export the PG4 file before closing out. This file can be included along with the messages and files to serve as a safeguard. Go to “File” ---> “Export Group”
Store the PG4 file along with the messages, files and photos.
WARNING ABOUT SHARING YOUR DB3 FILES: Do not share your DB3 files (these are the files that the software stores inside its Loca/App folder). Instead, if you want someone else to be able to read the messages using their copy of PGOffline, export the files as PG4 files (File-->Export To Folder). If you are sharing the DB3, be aware that it saves your password in plain text. Be certain to change your Yahoo password. You can do this first before exporting for sharing purpose and then change it back to your regular password. Or you can export, then change your password to a new one.
IMPORTANT EXPORT CAVEAT: ( “Will I Be Able To Import The Messages Into Another Mailing List?”).
Paid Users: If you have paid for the full version, you can export nicely formatted messages using “Multiple HTML with 50 messages per File”. Otherwise, the free trial versions cap this type of export to only 1000 messages total.
There are other formats you can select with a paid subscription like MySQL that may be good for future archiving.
How To Download Files and Photos
Select the Group, and then click “Files” from the header bar.
Click “List” and then “Download”
The files will then save somewhere weird. You can access them by clicking on the fandom and then “Files”, or navigate in Explorer to where they’re stored, e.g.:
C:\Users\XXXX\AppData\Local\PG Offline 4\Files\Firefly_Fanfiction
You can follow the same method to download Photos, Attachments and Links.They will each be stored separately, in their own folder
C:\Users\XXXX\AppData\Local\PG Offline 4\Photos\Firefly_Fanfiction
C:\Users\XXXX\AppData\Local\PG Offline 4\Links\Firefly_Fanfiction
C:\Users\XXXX\AppData\Local\PG Offline 4\Attachments\Firefly_Fanfiction
Copy & zip the folders.
Don’t Get Locked Out
Yahoo will locked you out if you download too many messages or files too quickly. There is no rhyme or reason to these lock-outs - they can last from two hours to a whole day.
To reduce the chance of lockouts use these settings:
Under File-->Options, select Download and use the configuration in the image below
Then click on the next Tab --> Workflow and use the settings in the image below
Setting the speed limit is especially important when the downloads are running overnight. If you have set your speed limit in the Download and Workflow tabs you will eventually see this:
Will I Be Able To Import The Messages Into A New Mailing List?
None of these methods (Plain Text with HTML delimited, HTML, or MYSQL) are guaranteed to import your messages into another mailing list platform. What you are aiming for is a format that can be easily read and searched by your average user.
There are import services that charge $ like Group.io or a customized import into a self-hosted Wordpress forum - before hiring anyone, keep in mind that both Groups.io, Google Groups and Wordpress limit explicit material. Self-hosted WordPress installs will be governed by your web-host’s terms of service for explicit material.
There Is A Lot Of Spam! What To Do?
PGOffline Workflow to deal with Spam: Go to the Yahoo Groups Message Archive overview (found by adding “info” to the end of the Group name. Ex: https://groups.yahoo.com/neo/groups/comicsfanatic/info
Look for the year with a huge increase in the number of messages, spot check a few, then go to the most recent month, spot check again. Decide what year to stop downloading (for me it is usually around 2008-2011). I still download all the files and photos but if you do ....WARNING!! Make certain your anti-virus app is running and consider selecting an external drive as the default PGOffline download location
OK I Am Done Downloading, What Can I Do With The Files?
The OTW has made an announcement. Their Open Doors preservation program is open to
providing storage of fandom related Yahoo Groups backups that are assembled by moderators
and non-moderators alike. Details here.
If you are a moderator of a Group and want to submit your mailing list to the public Internet
Archives, you can do so directly.
Help! I Have Questions!
PGOffline has a user Forum: http://personalgroupware.com/smf/index.php?board=5.0
You can also visit the Save Yahoo Group Discord Channel - it is is by fans, for fans and volunteer based.
Mac/Linux User Tools
Mac Users: try this Chrome based plugin and Instructions on how to use
Alternatively, if you are familiar with Python https://www.archiveteam.org/index.php?title=Yahoo!_Groups
*Prepared by James Kruk and Morgan Dawn, fellow fans and volunteers.
[1] Getting this part to work has tripped up a number of new PGOffline users, including one of the authors. Don’t give up!
[2] Unless you want to read the raw HTML code.