Introduction: Web Page Scraping With a Gui.

So far we have done web page scraping via the command line with https://www.instructables.com/id/Web-page-scraping-via-Linux/ and then we did it with the web with https://www.instructables.com/id/Web-page-scraping-fromto-a-web-page/. Now let's do it to the desktop. I will only do one example and you can figure out the rest of them. We will be using zenity again which we used in the instructable: https://www.instructables.com/id/Text-to-speech-with-linux/. Have fun.

Note: Microsoft Windows Server 8 will be mostly command line. If you want to be an admin or advanced user now is a good time to get used to the command line. BSD, Linux, and etc, you have always been able to take advantage of the command line.

Step 1: What's Needed.

Linux box with all the files from the previous projects (see intro).
Zenity installed (use the package manager or from the command line:)
$ sudo apt-get install zenity

Step 2: Original Code.

$ ./ghp.sh virgo
--------------------------------------------
   _
  ' `:--.--.
     |  |  |_     Virgo-  The Virgin
     |  |  | )
     |  |  |/
          (J

Today's date: 10/04/11
Today's horoscope for:
   VIRGO (Aug. 23-Sept. 22). You think you could have done
better, but you're wrong. Consider the forces that were
weighing on you at the time. Next time, you'll do it
differently, but for now, take peace in the knowledge that
you did the best you could.
--------------------------------------------


ghp.sh
[code]
# Get today's horoscope
echo "--------------------------------------------"
# character width
cw=60
hsign=$1
hsign="`echo $hsign|tr '[a-z]' '[A-Z]'`"
cat $hsign
echo -n "Today's date: "
date +%D
echo "Today's horoscope for:"
lynx -width 1000 -dump "http://www.creators.com/lifestylefeatures/horoscopes/horoscopes-by-holiday.html" | grep $hsign | fold -sw $cw
echo "--------------------------------------------"
[/code]

Step 3: Revised Code.

Save the code to a file in the data dir. (You may want to rename your original file to "ghp.sh.bak").

ghp.sh:
[code]
#=================================
# Get today's horoscope
#-------------------------------------------------
# Get horoscope sign, but have a default.
# get sign
hsign=""
hsign="virgo"
hsign=$(zenity --entry \
        --title="Daily Horoscope" \
        --text="Enter your _sign:" \
        --entry-text "$hsign")
#-------------------------------------------
# output to dialog box settings
# character width required for information box
cw=38
#-----------------------------------------------------------
#create data file (datadir and file name can be changed to your needs.
datadir="/home/eddie/signs"
filename="$datadir/th"
# make sure hsign is uppercase
hsign="`echo $hsign|tr '[a-z]' '[A-Z]'`"
cat $datadir/$hsign > $filename
echo -n "Today's date: " >> $filename
date +%D >> $filename
echo "Today's horoscope for:" >> $filename
lynx -width 1000 -dump "http://www.creators.com/lifestylefeatures/horoscopes/horoscopes-by-holiday.html" | grep $hsign | fold -sw $cw >> $filename
#------------------------------------------------------------------------------------------------
#output
zenity --text-info --filename=$filename
#remove  unneeded file
rm $filename
#================================================================
[/code]

In the datadir, you can run the program to get the following dialog boxes.

$./ghp.sh
....

Step 4: Easier (using a Launcher).

Now we need to create a launcher (shortcut for mswindows folks).
With the mouse, right click on the open desktop and click on create launcher.
You will get a new window. Enter the desktop name for the launcher. We used "Horoscope".
Do not say OK or press enter yet.
Click on browse.
Witht he new windows go to the directory where the batch file is. Click on ghp.sh on our case.
You can add a comment if you want.
Now press enter or click pn OK.

You should have a new icon on the desktop. So now all you have to do is double click on it.
Enter horoscope sign information you are looking for and then click OK.

You will be presented with your answer in a new window. Tada!

Step 5: Create a Web Page.

See report.sh.txt

$ rename to report.sh

$ chmod +x report.sh

edit as needed.

$ ./report.sh

Then load report.html into your browser.

Note: gwhtml.sh and ghphtml.sh will be here soon. changed computers and the files are on the old drive.Be sure to change the horoscope sign and the zip code for the weather as needed.

Comments

About This Instructable

1,858views

9favorites

License:

Bio: computoman.blogspot.com Bytesize articles instead of a trilogy in one post.
More by Computothought:Coffeepot mealsEasy sun hatEasy cakes and pies.
Add instructable to: