I'm looking into the possibility of having a program written which
would grab text and identify images from the windows of another
program.  It would also fill in text boxes and effect button-presses
in those windows.  The problem?  I don't know how all this can be
done...lol.

To distinguish this not-yet-written program from the windows it works
with, I will call the program which is to be written the "master
program".  The master program would read the text from the windows of a
3rd-party program, make decisions based upon the what it sees, and
then fill in text boxes and initiate button-presses in those windows. 
The master program would essentially be a user of the
3rd-party program.

The 3rd-party program is not wholly text-based though -- there are
also a number of predefined images which are shown, which a user must
respond to.  That is why the master program must not only be able to
read text, but also be able to recognize when one of the predefined
images has been painted, and respond to it as well.  The master
program would have bitmaps of all the predefined images.

My question is: How can I accomplish the interaction needed with the
3rd-party program windows?  Specifically, for given positions within
a window, how do I know what text (if any) is there?  How do I know
which predefined image (if any) is there?  How do I have the master
program enter text into textboxes?  And how do I effect
button-presses?  I don't have the source code for the 3rd-party
program, so modifying it is not an option.  The 3rd-party program runs
on Windows operating systems (I'm using XP Pro).

If some of the desired functionality is available as a software
package or library, then I would certainly be open to using that
software as part of the solution.  Programming language is open. 
Thanks!

---

Request for Question Clarification by sublime1-ga on 29 Mar 2006 14:31 PST

I know zip about programming, but if you're want to explore
the methodology of such programming, I'm thinking you might
benefit from having a peek at a program called PTFB, or Push
The Freakin Button, which, when programmed by the user using
drag and drop, detects the script of an open window title of
another program in order to uniquely identify it, and then 
pushes the button, again based on the text found on the button:

PTFB:
http://www.tlhouse.co.uk/PTFBPro/mouseautoclicker.shtml


You can examine the programming of this and other software
using a wonderful freeware tool called Resource Hacker:
http://www.angusj.com/resourcehacker/

Let me know if this is useful enough to consider as an answer.

sublime1-ga

One Windows App can control another App

Generally this is done by sending Windows Messages to the other App,
to read and write text in Edit controls and to 'click' Buttons

One can also move the mouse and simulate clicks and key presses

Sometimes it is fairly easy if all the controls are 'Windowed' eg:
they have a Handle and an identifiable name.
Sometimes it is pretty difficult if the Form is 'painted' or it uses
unconventional controls. Grids are very difficult, as are Browsers
(but you get at Browsers via another route).

The first step is to inspect it with a copy of Spy, a utility program
that comes with most languages.  Personally I prefer using my own
little utility.

Those images might be a nuisance, but one can always read them Pixel by Pixel.

There are probably utilities out there that do this sort of thing
(under DOS there was something called PC Automator), but from what
I've seen most people write their own, because the actual logic is
very specific to the target Application. The programming News Groups
generally throw up one of these problems a day.

The simple answer is that it can certainly be done, it might be fairly
easy, and it is possible that it will be a nightmare involving OCR.

From what you have said I think it should be pretty manageable.

Yes, what you are suggesting is possible and does exist. But I am
guessing from what you have written that the app you are seeking to
use/delevop is to be used for some form of cheating at online gaming,
my guess would be online poker specifically.

Before you go any further in trying to obtain/write said software, I
would suggest thinking about whether you really want to do it... The
poker sites - especially the largest ones - are getting very smart to
such issues and I would imagine that if you do not have sufficient
knowledge yourself to write the software, you probably are not good
enough to avoid being caught. Just learn to play better.  =)

If I have been mistaken in thinking this is your aim, I apologise.
However, if online poker is your target, I am happy to offer lessons
if you are interested. I am a long time winning player making a
comfortable living from the game. Let me know if you'reinterested.