I want to control an application (an online game) automatically using
some decision making software that I have written. I've refined the
software that plays the game, but I need a way for it to interface
with the game's client. For this application I have decided that the
best way to do this is by capturing what is in the client's graphical
user interface, interpreting the images, having my software make a
decision of what to do, and then generating mouse clicks within the
client to take the desired action.
I suspect that a software library exists to facilitate this task.
Perhaps library/toolkit is free, or perhaps it is a commercial product
-- I don't know. If these products exist, I want to know what they
are.
Please tell me what software development tools exist for interfacing
with an application's GUI. Note that I need more than just a
scripting language for generating a sequence of mouse clicks. So
something like AutoIT is not sufficient.
Please don't suggest another type of solution, like snooping on the
application's network traffic. |
Request for Question Clarification by
maniac-ga
on
18 Oct 2004 16:16 PDT
Hello Dogbite,
Hmm. The solution to this problem will vary by operating system / type
of interface.
- Which type of system will the game client run on?
- For image capture, is any format OK or do you have a preferred format?
--Maniac
|
Clarification of Question by
dogbite-ga
on
18 Oct 2004 17:23 PDT
Hello,
The application runs on Windows. I'll probably be running Windows XP
when I run it, but it also runs on other versions of Windows.
I do not care about what format the images are captured. They're only
captured so that I can write software that looks to see what has
happened and if it is my turn to take an action. JPEG images would be
fine, I imagine.
I should also add that some people have pointed me to QA automation
tools like IBM Rational's Robot, but after a quick look at that tool's
documentation, it wasn't clear to me that it would do what I wanted.
In particular, it looks more geared to automated execution of by rote
behavior, but perhaps that is simply how they market their product.
Thank you for starting to think about my question.
|
Request for Question Clarification by
maniac-ga
on
19 Oct 2004 16:11 PDT
Hello Dogbite,
I don't see how a QA automation tool would meet your needs. The image
capture requirement is somewhat unique. An approach that appears to
fit better is a remote control program such as:
Anyplace Control - http://www.anyplace-control.com/
pcAnywhere - http://www.symantec.com/pcanywhere/
GoToMyPC - https://www.gotomypc.com/
and so on. They capture images (and display them remotely) as well as
drive the keyboard and mouse on the remote machine.
The problem using one of these is they are not set up to expose those
interfaces to other programs.
Another alternative is to load drivers to capture the images / act as
mouse and keyboard. I will look into this approach as well.
--Maniac
|
Clarification of Question by
dogbite-ga
on
20 Oct 2004 00:33 PDT
I agree that the image capture requirements -- so the software can
figure out what is going on in the game and decide what action to take
-- makes it tricky for a QA automation tool. At least that's what I
determined from reading about them a bit.
dogbite
|
