FetchYahoo

Current Version: .1.7

Older Versions: .1.6, .1.5, .1.4, .1.3, .1.2, .1.1, .1.0, .0.9, .0.8, .0.7, .0.6, .0.5

Note

Other Versions - untested and unsupported

• This version sets up a POP server on the computer so that it can be used with any mail client supporting POP fetching (most windows clients). I am still not sure whether I want to do this in the main version and if I do, in what form. Credit to Murtaza Nooruddin for this. This is from a very early version of FetchYahoo and many features are not available.
• This version contains https support and a few extra options for entering the username/password. I have already integrated the https patch (as of v.1.1) and am going to work on integrating the others very soon. Credit to Thanh Nguyen for this patched version. Also given here is a useful shell script which can process multiple username/password pairs, stored in an encrypted file. This functionality will almost certainly not be integrated into FetchYahoo for a while.
• YahooPOPs! is a Windows based utility which does something similar to FetchYahoo program. The application works as a POP3 emulator to download emails from Yahoo and make it available via POP. A *nix port is in progress. I have heard good things about this program !
• Hotmail users - This program (gotmail) (written by Peter Hawkins) does the same thing as FetchYahoo for Hotmail webmail. It has a new maintainer and is being actively developed again with a new webpage.
• Windows users can try out this program (web2pop) which downloads from webmail services and transparently transmits them through a pop server on the local machine (similar to the pop-version given above) . This is very cool, but unfortunately the software costs $20 US.
• Yahoo Mail Sucker (YoSucker) is a Perl script that does almost the same thing as FetchYahoo (and in almost the same way).

Description

Requirements

• A mail client that can read from a mail spool or an mbox-formatted file (almost all Unix mail clients, Netscape Mail for Windows)
• A working installation of Perl (comes with almost all Unix installations, needs to be installed for Windows - try ActivePerl)
• Several Perl modules are required. The following module-collections contain the required modules: libwww-perl, html-parser, MIME-tools, libnet (for mail forwarding), Crypt-SSLeay (for https). Some users report additionally requiring the following: URI, MailTools, IO-stringy, MIME-Base64. There are several methods for getting these :
  • If you have an RPM-based distribution, search at RPMfind.net for perl-MIME-tools, perl-html-parser, perl-libwww-perl, perl-libnet (for mail forwarding) and perl-Crypt-SSLeay (for https) and get the required RPMs for your distribution
  • For deb-based systems (properly configured), you can just do "apt-get -u install libnet libhtml-parser-perl libmime-perl libwww-perl libcrypt-ssleay-perl"
  • For windows, you can use the Perl Package Manager which comes with ActivePerl
  • If you have the CPAN perl module installed and working you can run "perl -MCPAN -e shell" and then install each of the following separately: HTML::Entities, LWP::UserAgent, MIME::Entity, Net:SMTP (for mail forwarding - use the libnet version), Crypt::SSLeay (for https)
  • Go to the CPAN repository. Go to the first search choice (Perl core and CPAN modules). Search for MIME-tools, libwww-perl, html-parser, libnet (for mail forwarding) and Net_SSLeay (for https). Follow the appropriate installation instructions.

Usage notes

• Download fetchyahoo.pl
• Chmod a+rx fetchyahoo.pl
• Edit fetchyahoo.pl or the configuration file
  • put in your Yahoo! username and password and path to your mail spool/mbox file (usually /var/spool/mail/username )
    OR copy fetchyahoorc to /etc/fetchyahoorc or ~/.fetchyahoorc and edit that
  • fill in your smtp mail server, forwarding-to address and forwarding-from address if you want messages to be forwarded
  • fill in your web proxy host and port if you are using one (and your proxy username and password if necessary)
  • set use-https to 1 if you want to login securely. This may require the Crypt::SSLeay Perl module.
• Run fetchyahoo.pl
  • --newonly to download new msgs only
  • --nodelete to not delete messages after downloading
  • --configfile=/path/to/configfile to specify an alternate configuration file
  • --username=yahoousername to specify a Yahoo! username
  • --password=yahoopassowrd to specify a Yahoo! password
  • --spoolfile=/path/to/spoolfile to specify the file to send messages to
  • --quiet to suppress regular (non-error) messages
  • --noerrors to suppress error messages
  • --nodownload to only delete messages, do not download them
  • --inlinehtml to transmit HTML messages inline
  • --emptytrashbefore to empty trash before downloading messages
  • --emptytrashafter to empty trash after downloading messages
  • --logout to log out of Yahoo! when done
  • --repeatinterval=N to check for mail every N minutes
  • --nofromline to leave out the leading From_ line
  • --leaveunread to leave messages unread on the server
• Use your mail client to get mail from your spool or mbox file
• e-mail author with bugs and feature requests

Procmail instructions

• use "/path/to/procmail" as your mail spool name
• set spoolMode to "pipe"

Windows Instructions

• Download Perl for Windows (e.g. ActivePerl)
• Edit fetchyahoo.pl (or the appropriate configuration file) as shown above
• Use Netscape Mail to get your mail from the mailbox you specified
• e-mail author with bugs and feature requests

• For Eudora specify a new or existing Eudora mailbox as the spool file.
• For Outlook or Outlook Express, specify a temporary (existent but empty) file as the spool file, and then after fetchyahoo has completed, choose File -> Import -> Messages from Outlook (or Outlook Express) and choose the temporary file you specified. This has been tested successfully.

Changelog

• fix to work with the beta of yahoo's new layout (English only) (v.1.7)
• daemon mode should fork into background (v.1.7)
• fix configuration parsing bug (v.1.7)
• mailbox locking while spooling (v.1.6)
• add proxy authentication (patch by David Maslen) (v.1.6)
• get proxy URL from http_proxy/HTTP_PROXY env. variable if it exists (v.1.6)
• --quiet outputs erors and warnings (v.1.6)
• --noerrors suppresses errors and warnings (v.1.6)
• Non-fatal errors are now warnings instead of failures (v.1.6)
• fix urgent SSL bug (v.1.5)
• --leaveunread option (v.1.5)
• --nofromline option (v.1.5)
• better error handling if unnecessary modules are missing (v.1.5)
• Be more rigorous, don't die so much for daemon (v.1.5)
• use user-agent string from Mozilla linux (v.1.5)
• fix overwrite-mode bug (v.1.5)
• much simplified structure (Hubert's patch) (v.1.4)
  This makes several other patches unnecessary. (inlineHTML, content-IDs, real date in From_ line)
• negative options where they make sense (v.1.4)
• messages not being deleted after empty-trash bug (v.1.4)
• Repeat mailchecking with a specified interval (v.1.3)
• --emptytrashbefore --emptytrashafter (v.1.3)
• --inlineHTML option for HTML-capable mail clients (v.1.3)
• --nodownload option to not download messages (v.1.3)
• --logout option to logout after downloading messages (v.1.3)
• --quiet option to suppress regular (non-error) output (v.1.3)
• Make it easy to add new translations (v.1.2)
• Add translations for us, fr, es, de, it, br (v.1.2)
• Include all the message headers, not just recognized ones (v.1.2)
• Better parsing for config file - values can now include whitespace (v.1.2)
• Use SSL to connect securely (v.1.1)
• Header field parsing for Portuguese (v.1.1)
• Specify an alternate config file on the commandline (v.1.1)
• More commandline options (v.1.1)
• Config option to not spool (v.1.0)
• Usability fix for improper configs (v.1.0)
• Use of a configuration file (v.1.0)
• Header field parsing for Italian (v.1.0)
• Bugfix for mail forwarding (v.1.0)
• Patch to correctly specify single-part text/html messages (v.0.9)
• Configuration to use a web proxy (v.0.9)
• Better configuration of spoolMode (v.0.9)
• Forward to another e-mail address (v.0.9)
• Progress indicator showing how many messages have been downloaded (v.0.8)
• Include more message header fields (including German headers) (v.0.8)
• If there is only one part, send as a single-part message (v.0.8)
• Download attachments and include them in the e-mail (v.0.7)
• Better error messages for wrong username/password (v.0.6)
• Fix bug encountered when using procmail (v.0.6)
• Make sure (internal and header) From_ lines are RFC 822 compliant (v.0.5)
• Add a --nodelete option which specifies no deleting of messages (v.0.5)
• Make parsing of the From_ line more rigorous (make one if necessary) (v.0.6)
• Get options working, option to d/load new messages only (v.0.4)
• Remove use of the specific us.f149 webmail server (v.0.4)
• Modify for use of procmail (v.0.4)
• Send message to spool (v.0.3)
• Delete successfully grabbed msgs (v.0.3)
• Code main loop to parse each msgid from yahoo (v.0.2)
• Parse message page for raw message text (v.0.2)
• Get login working (v.0.1)
• Download summary page (v.0.1)
• Parse summary page for number of messages (v.0.1)

To-Do/Bugs

• multiple yahoo accounts (1)
• Download from different folders (esp Bulkmail folder) (1)
• Improve usage (prompt for username/password/spool and save to file) (1)
• --verbose otion (2)
• --maxsize option (2)
• Parse for % of quota used / quota warning (2)
• Use cookies from a file. Save cookies to a file. (3)
• Send mail via webmail (3)
• encrypted passwords (3)
• modularize (loadPage, MessageMangle, logIn etc) (3)
• Built-in POP3 server (4)
• Per-message download and deletion (5)
• remove use of MIME perl package (5)
• fast check via Yahoo IM (5)

E-mail author