After turning on my computer but before going online (I use AOL), I
have started getting a pop up that says "The web page you requested is
unavailable offline"  I have two options, "Connect" and "Stay Offline"
 An IE icon is on the pop up with an x in the corner of the icon. 
This is annoying.  I have run adware, spyware and malware programs and
mcafee virus program but it continues to occur

Check this page. It has pictures that explain what settings you need to look at. 

<a href="http://www.jegsworks.com/Lessons/web/basics/step-openie.htm">http://www.jegsworks.com/Lessons/web/basics/step-openie.htm</a>

This is not caused by a virus or spyware. Internet Explorer will show
you this message if you are in 'Work Offline' mode. Work Offline mode
instructs Internet Explorer not to attempt to use an open internet
connection, or to open an internet connection.

What do you do when this window appears? If you click Connect then
Internet Explorer will exit Work Offline mode and should offer to
connect you. If when you click Connect you have already connected to
AOL then Internet Explorer should load the page normally. If, however,
you click Stay Offline or click the close button then you will remain
in Work Offline mode.

If clicking connect does not connect you automatically you can check
your settings. From within Internet Explorer click Tools -> Internet
Options then the connections tab. Ensure that Always dial my default
connection is selected and then click OK to save your changes.

In short then, clicking Connect should solve your problem. Internet
Explorer should then exit 'Work Offline' mode and remember that
setting. Work Offline mode can be reenabled by selecting File -> Work
Offline.