I am looking for a software tool that can automatically download my
credit card and bank statements from thier respective websites every
month and save them to HTML files on my local hard drive.

The tool needs to be smart enough to be able to enter password and
navigate the sites to get to the right pages. It also should be
programmable to run automatically according to a defined schedual.

I wan't to keep all this pages on my local hard drive becuase (1) it
typically is only available on the web sites for a few months, and (2)
I'll be able to search though all the data effifiently on my local
machine.

Thanks!

---

Request for Question Clarification by hedgie-ga on 09 Jan 2006 13:12 PST

bigjosh2

            You've gotten quite a few suggestions in comments. Some
(most) are goofy  but it can be done in several ways. That means that
you can specify additional requirements, such as cost, OS, etc.

There is no  software which you just install and it does everything
you want automatically, for a general (arbitrary) account.
 The best you can expect is to have the program repeat the same
procedure every month, for each account. You will have to describe the
procedure to the program. If the bank changes procedures or rearranges
pages, you will need give it some attention and revise things.

 It may require some computer skills, not any 'real programming, like
C++' but the ability to 'describe a  procedure of downloading a file'
in a prescribed way. If you do not have such 'advanced user skills' ,
you would have to hire a local consultant for a few hours or days of
work to set it all up.
 
Be aware that whichever way you do the particular task you have
described, yourself, or with help, it is a security risk. Passwords
are not to be recorded inside of machines. The risks can be minimized,
but then there is some extra work.

To do what you want you need 2 or 3 different programs.

1) A  program which will execute any  specified program of your
choice, periodically (the scheduler).
    Scheduler will run 'the downloader'  every month. Chances are you
already have the Scheduler.

2)   The 'Downloader' must  have the ability to simulate human
actions, such as  wait for prompt, enter proper response, wait for
another response from the machine, etc.  There is such program. It was
developed for 'program testing' . It is free (open software) BUT it
normally uses CLI, not GUI. Meaning, it was not developed to work via
graphical interface. That is not an insoluble problem, but using it
would require the ability to work in the command mode (try Lynx
browser to download your statement to get a feel for what it may look
like).

3) Search is another task. A separate question if you go with programs
I am suggesting: Once you have the monthly statements in your box, you
may load them to a spreadsheet or some personal finance or accounting
program or (preferably
a data base) or accounting program with a database.

Alternative would be to automatise
Quicken Download Procedure or MsMoney Download Procedure
https://www.esl.org/onlineserv/ibhelp.html
is a poplar commercial Personal Finnances Program. That is a similar
task as described above, similar dificulty and  I would not want to go
into that.

Another alternative would be to wait antil these people (MS, Intuit)
do add this feature (automatic dowload) to their product. It would be
an easy and natural extension.
 
If you do not like the approach (scheduler and downloader) and its
limitations, please let me know now so I do not  invest more time on
this approach. If it sounds like what you want,  I will elaborate, but
the task of setting it all up will be up to you (and perhaps an
eventual consultant).

Do you want to proceed?


Hedgie

---

Request for Question Clarification by hedgie-ga on 09 Jan 2006 13:26 PST

Correcting a typo

is a poplar commercial

should be

in some popular commercial

---

Clarification of Question by bigjosh2-ga on 10 Jan 2006 06:21 PST

I already use Quicken and Paytrust and neither of these really provide
what I am looking for - a PDF version of my entire credit card and
bank account monthly statement stored on my hard drive. There is lots
of information that is contained in the statements that is not
captured by Quicken or Paytrust.

I understand that it would be reliatively straightforward to write a
program to do all the HTTP gets to log in and download these files
every month, but I was hoping to find a better solution.

I once saw a scripting tool a long time ago that did exactly what I
want - you'd open the tool (which had an embeded browser), then
execute the steps needed to get to the file you wanted. It would
record a macro of the steps and then give you the option to customize
parts of the macro that would change (i.e. It would see the string
"FEB" in the macro and ask if you wanted to substiute the current
month here), then it would let you set up triggers for the macro. I
can't remeber what the software was called and can't find it anywhere.
This is the software I am looking for with this google answers
question.

The closest I've found so far is RoboForm, which could do what I want
with only a little extra functionality.

Thanks!

Hi,
 I don't think there is any tool which automatically does that. There
is one work around which you can acheive what you want upto certain
extent.  There is a tool called "WinRunner" from Mercury interactive.
Basically it is a web application testing tool which does check all
the pages and their functionality.
http://www.mercury.com/us/products/quality-center/functional-testing/?swf_src=us_opt-center-diagram_functional-testing

You need to learn this tool little bit,even though it is easy if you
are from computer back ground. You can record the predefined sequence
of events you want to do every month , and this software stores that
events as a script and you can schedule this script every week to do
those steps.

I don't know whether you have heard of www.yodlee.com. It consolidate
all your credit card, bebit card, and bank information and so much
more with one single sign-on, even though it won't keep a backup of
your statements. Only first time you give log-in of each site, and
from next time onwards it displays overall change in your finnaces and
due dates.

Hope this information help you in some way. Let me know if you have
any questions on this.

Sounds to me like you need to use quicken or microsoft money. 

I use quicken with my bank, which allows me to download all my
transactions directly into the program.  Most banks either support
direct downloading of statement information, or allow you to download
files which you can import into quicken or microsoft money.

Quicken or microsfot money doens't automattically login into the web
site. you need to log in and import the statements to quicken or
money. Only solution that nears ur requirement is the one above I
mentioned.

If my solutions resolves your issue, tips will be appreciated.

You can do this if you had a Apple Computer. The way it would work is
by writing a script using "AppleScript" and basically you would go
through the routine once and AppleScript would record your actions,
then you just set the script to run automatically or run the script
yourself and this would perform all your actions and save your
statements to your computer.

I would also appreciate it if you found my answer useful to send me a
couple of bucks. You can send me some money through paypal. Send it to
jimmyl@best.com. Thanks.

What you are looking for is referred to as a "Web Scraper" / Web Robot
/ Account Aggregator / or Web Page Clipper (depending on the context).

Although this appears to be a straight-forward concept... it is, in
fact, extremely complicated. Data url and web site html changes can
kill a previously working script and web pages that use framesets,
popups, flash and active-x plugins are difficult to scrape.

Most companies that previously competed in this market have since been
acquired by high-end portal vendors to supply Portlet engine
technology(e.g. OnePage - Sybase, Octopus - Ask Jeeves).

If the statements you are interested in downloading can be accessed
using a consistent url (e.g. currentstatement.yourname.yourbank) then
you can use the built-in "offline" capability of MSIE. To make this
work, go to your MSIE browser and traverse the Web until you get to
the link required to retrieve your statement. Add this to your
"Favorites" list, then select it from your favorites list and right
click it, and then select "Make Available Offline". MSIE will then
prompt you via a wizard to configure the account / password info and
the synchronisation method (Manual or scheduled). NOTE: This is a
convenient but insecure approach since anyone using your PC and
browser can access your financial records. Additionally, this method
is likely to breach the Terms & Conditions you have with your
financial institutions regarding the recording of your account
details. Your decision.

Your best compromise is to subscribe to the free services that are
endorsed by financial institutions themselves (check with one of your
financial institutions... most subscribe to the Yodlee aggregation
services):

www.yodlee.com (US centric)
www.ewise.com (supports multiple countries - Australia centric)

Note that Yodlee stores your account details in a secure central store
while ewise stores your details in an encrypted form on your PC.

There are other tools available but I would suggest they are too
expensive and complex for your requirements (www.clickmarks.com or
www.kapowtech.com).

I finally found a product called iOpus. It is not perfect and you
really need to have some programming experience to use it effectively,
but it will do what I need.

http://www.iopus.com

If you do not have programming experience, you might want to check out...

http://botmakers.com/

...it is a company that claims that they will hand code a script to
automate a web task for you. I don't know how much they cost, but
seems like a good service.