KingLeechHog

Short:        Simplify mass WWW download AWEB3/DOPUS5
Author:       fisherking@softhome.net (fisherking)
Uploader:     fisherking softhome net (fisherking)
Type:         comm/www
Architecture: m68k-amigaos



                  K I N G   L E E C H   H O G   V 1 . 0b

                    ®   F I S H E R K i N G    2 0 0 0


                      email : fisherking@softhome.net


Short : KingLeechHog is a util/script/rexx for managing mass downloads/leech using
Dopus5.x, Aweb 3 and Httpresume.

Dont bother using this for 4-5 files, its purpose is to hold a que for aweb3/httpresume
and feed files/url's until end.

I've had this util/script handle and download 1000's of files in one go and then disconnect
from internet in the end. Go rent a movie, goto bed, whatever....


KingLeechHog is freeware, use at own risk, bla,bla....u know the drill..


Features : 

		Uses Dopus5.x for 'gui'. All files/actions made are/can be controlled
		via dopus button bank (included ofcoz).

		Selection of files/filters/toggles/suffix/prefix uses dopus functions.

		Files listed in a own Dopus lister, with own toolbar.

		Having a destination lister will download files to dest (req. btw)

		By parsing to Httpresume, authorization/resume/timeout probs disapear.

		Good handling/solving of HTML page indexes, KingLeechHog will try
		extracting all HREF pointers, and if one failed it tryes another
		approatch.



shittystuff :

		Needs a clitool (wrap) for wordwrapping lines with more that 1000 chars
		coz arexx cant handle reading from a files with more than 1000 chars in one line.
		
		Wrap tool included (Found on aminet) Should be in KingLeechHog:

Install :


1. Make a KingLeechHog directory somewhere and assign KingLeechHog: there

         ex. assign KingLeechHog: DH1:Programs/Internet/KingLeechHog

2. Copy/extract all files to KingLeechHog:

3. Edit 'httpresume.rx' and change path to where you have httpresume exefile


4. Think thats it.....you can dblclick on KingLeech_Hog file (its a Dopus buttonbank)
   bottons pop up, and the you have control...pretty much..


***Now, with Aweb running and page with links loaded, switch to dopus screen and click 'Rescan'
button.....this will start main arexx script, and if not that html page contains 100's
of links/files a new dopus lister should pop up, displaying files within a few secs.
Now you can select what files to download and select download with aweb or httpresume.

So in a surf session, you can browse around and when you find files you want, click rescan
button, urls will pop up in a lister, and then browse around more, when you have files enough
or dont find more files, start the download and goto bed (disconnect ONLY works with resume
button and amitcp/genesis)

Downloads with aweb will leave modem online when done, if you dont calculate approx. online time
needed and use a timer (c:wait) and a reboot tool to reboot after timer 

example

download que in aweb of 30 mb and 64k isdn line
aprox 6000 chars per second

30.000.000 / 6000 = 5000 secs

5000 secs / 60 secs a minute = 84 mins

c:wait 90 mins
c:reboot

usally works, sometimes not :)

havent checked aweb docs for a cmd who checks for downloads in progress


Buttons :

		Rescan   - look behind ***(above) in so lazy writin....

		Getfiles - will parse all selected files to aweb for download
			   files will be saved in your destination dopus lister
			   No destination lister selected a msg will pop up.

		Resume   - Will parse files to httpresume, one by one
			   Good for bigger files (500k and up)
			   Also a requester will ask if you want to disconnect
			   modem from internet after download.

			   A second requester asking if files require authorization
			   and if you select yes, a 3'rd requester prompts for username/password
			   in u:p format (if username=amiga & password=rulez then enter amiga:rulez)

			   For disconnection after download you must use Amitcp/Genesis as tcp stack
				(script executes amitcp:bin/stopnet)
			   with another tcp stack, you gotta edit 'httpresume.rx' script and
			   replace address command('execute amitcp:bin/stopnet') with whatever
			   command your tcp stack need, or just email me. fisherking@softhome.net

			   Destination for files are your destination lister !REQUIRED!

		@webgo   - will parse the first selected file in the KingLeechHog lister
			   into aweb who loads it as if you entered file as url.
			   dunno why someone should find this useful, guess i make script for test purpose
			   maybe remove for a future update of KingLeechHog.....?!

		Addlog   - Saves URL of selected files into a logfile (KingLeech_Hog:log/Leech_Hog.log)
			   Appends to file if earlyer log exists. you must delete this file manually
			   if you dont want log to grow with files you maybe already have downloaded.

			   Useful if you find some files, but wanna download later without loading
			   page into aweb again and then exeute main KingLeech_Hog script.


		Loadlog  - Loads entire logfile into your current KingLeech_Hog lister, you can then download
			   these....


buttons have a rightbutton feature aswell..white text on black background bottom of buttons.



rescan button rightclick (Toggle Selected) toggle selected/unselected files in KingLeech_Hog lister
getfiles      rightclick (Select by #?) Select files from wildcard (#?.lha)

send mailto:  disabled, n/a
netstat       disabled, n/a



Hint/tips :



	problem 	Its not always KingLeechHog find links/files in html pages using multiple frames
			but in aweb links/files show.

	solve		i've had this shit a few times, usally its a small menu frame in left of
			browser with links to pages who loads into the 'main' frame in right of browser.

			Best solved by holding mouse over link of a page and rightclick mouse and load
			link into new aweb window, and then restart/rescan with KingLeechHog script


 Pages with LOTS of banners and links to firms or buy this/that you should NOT use this util for that
 coz you'll get most of the files/gif's/logos into dopus lister , and with lot of lines this script
 slows down massive much...and lister will appear after 5-10-15 secs.....



  I've been working on this thing since sometime in 1998....I used Ahmatti (same kinda script for aweb)
  for a while but wanted more control/options and a better html scan engine. this is the result..

  Been using it ALOT and its really useful for me, so why not release it for others to enjoy.



  Button images are ugly i know...made in a hurry....draw your own or use/make textbuttons
  if you draw some, send me :)