|
|||||||||||||
|
Configuring CGI Scripts (For Beginners)
First off, you should be some what familiar with UNIX and TELNET before reading this tutorial. If you've never downloaded or uploaded anything, or even attempted to Telnet in to Frognet then you are not yet ready for this tutorial.
Click Here To Read FrogNet's Telnet Tutorial
What is CGI and SSI anyway? An excellent definition for the terms CGI and SSI can be found at webopedia.com.
"CGI - Abbreviation of Common Gateway Interface, a specification for transferring information between a World Wide Web server and a CGI program. A CGI program is any program designed to accept and return data that conforms to the CGI specification. The program could be written in any programming language, including C, Perl, Java, or Visual Basic. "
"SSI - Short for server-side include, a type of HTML comment that directs the Web server to dynamically generate data for the Web page whenever it is requested."
Contents:
- Finding The Location of Essential Programs on FrogNet's Server
- File Permissions for CGI Scripts
- Server Side Included Naming Conventions
- The Almighty "cgi-bin" directory
- Prep Command
- Example: Setting Up a Guest Book
- Debugging a script
Finding The Location of Essential Programs on FrogNet's Server
Smartly written CGI scripts usually ask you to plug in some basic information about the server on which the script resides. Many of the free CGI scripts written in PERL that are available at Matt's Script Archive require you to change one or two variables.
If you want to find the location of a program on FrogNet's server, establish a TELNET connection to FrogNet and log on to FrogNet's command shell. The command you want to use to display the paths of common program like sendmail and the PERL interpreter is called "which"
At the command prompt type which followed by a space and the name of the program you wish to find then hit return. The graphic below shows several which statements along with their output.
File Permissions
One of the most common stumbling blocks for beginner CGI users is the concept of "file permissions" All servers maintain a level of security through the usage of permissions. Typically, only the person who creates a file should have permission to alter or remove it. This prevents some random guy from erasing all your hard work or stealing private information.
CGI programs will not function unless they have the proper permissions set. 75% of the time this is the reason why your CGI script has failed.
FrogNet uses the Apache web server. We've installed an add-on feature called "SU-EXEC" which requires your CGI scripts to have a permission setting of 700.
What does 700 mean? Every file has three sets of permissions: one for the owner of the file, another for users in the same group as the owner, and the third is for strangers. 700 is not "seven hundred" but rather "seven zero zero"
|
7
|
0
|
0
|
|
owner's permissions |
group permissions (other frognet users) |
strangers (guests and stray people) |
The first 7 is the permission setting for the owner of the file. If you uploaded the CGI script to your directory then you are the owner of that file. The second digit, the "zero" is the permission setting for other members in your group. Generally, all FrogNet dial-up account users are grouped together under the label "users." Finally, the last zero is the permission setting for strangers to the system.
Each digit represents a level of security. Any file has the potential to be read, written to, or executed. Setting the permissions from zero (no access at all) to 7 (total access) determines who can use the file. Here is a small chart that explains the digit system.
|
Number |
Permission |
Meaning |
|
0 |
none |
All turned off |
|
1 |
--x |
Execute only |
|
2 |
-w- |
Write only |
|
3 |
-wx |
Write and Execute |
|
4 |
r-- |
Read only |
|
5 |
r-x |
Read and Execute |
|
6 |
rw- |
Read and Write |
|
7 |
rwx |
Read, Write and Execute |
You can set the permissions for any file that you own manually by using the command chmod. An example of how to use chmod is provided in the graphic below.
In the above example. "ls -al" is used to display a detailed list all the files in the current directory. If the permissions start with a "d" that means that particular file is actually another directory (or "folder" for you pc and mac users) The first three characters after the initial "d" or "-" are the permissions for the owner of the files. The next three characters represent the permissions for people in the same group as you. The final three characters are for guests. Compare the letters to the table listed above and you can see that we are changing the permissions on the PERL file called test.pl from 644 (--rw-x-r-x) to 700 (-rwx------)
Why Does it have to be 700? My cousin Bob's ISP has them set permissions to 755 and it works fine for cousin Bob! Well, cousin Bob's files are a lot less secure than they should be. It used to be that in order for a CGI script to work, it had to be 755. Now, with the su-exec patch no one but you and the web server can read, write and execute your CGI scripts, text files, or any other sensitive file on your account.
If cousin Bob was storing anything private on his account, any old fool with an account on your cousin Bob's ISP can log on to the shell and read every line of code Bob's program, copy it, steal it, and say he wrote it. Or, if your cousin Bob was storing people's personal information in a text file. Stuff like phone numbers and credit cards for a shopping cart, then Bob's probably going to get sued when all of his customers credit card numbers get stolen.
What about permissions for directories? They should still be 755.
What about regular Web Pages? Regular old HTML files should be 644.
What if I have a CGI script that writes to a web page? Should I change the web page's permissions to 777? No, they should still be 644. Permissions of 777 allow anyone to write to your html page. It's a good way to get your web page hacked by malicious users.
What if my CGI script needs to access another directory? Should I change the directory being referenced to 777? No, all directories should be 755.
You know, the instructions for this CGI script tell me to set the permissions for the script itself to 755... do I realy have to set them to 700? YES!
I don't know, are you sure the script will still work? YES!
Are you pulling my leg here? NO!
Man, I dunno, who should I belive here? Believe US... we're running the server!
OK, You convinced me.
To recap: CGI, SSI and any TEXT files they create should have permissions of 700. HTML files should always be 644 (even if they are being written to) and all directories should have permissions of 755.
Server Side Include Naming Conventions
The Apache web server will only execute call to a server side include if the HTML document it is located in ends in .shtml
You must change the extension of all pages that include server side includes from .htm or .html to .shtml
The actual Server Side Include script itself can be named anything that ends in .ssi, .pl, or .cgi on the HTML document calling the script must have a file name that ends with .shtml
The Almighty "CGI-BIN" directory
A lot of free CGI scripts that you find on the web (like the ones on Matt's Script Archive) state in their instructions that they need to be installed in the web server's "cgi-bin" directory. This is not the case at FrogNet. There is no centralized location on FrogNet's server where all cgi-scripts reside. You can create your own directory anywhere on your account and store as many CGI scripts as you desire. This means that you may have to edit your CGI script so that it can find YOUR personal cgi-bin directory.
Does the directory where my cgi scripts are located have to be called cgi-bin? No, you could call it "jabber-wookie" if you wanted. In fact, it should NOT be called cgi-bin, especially if your account is a virtual domain.
Can I name my script directory CGI-BIN? NO! You can call it anything but cgi-bin! call it joe, tony, alice, or even cgibin (cgi-bin without the minus... tricky, eh?) but you cannot call it "cgi-bin"
Do I even need a CGI-BIN directory? Technically, no. You could put all your html files, images and cgi scripts in your html directory. However, this makes for a messy folder. You may find it extremely hard to find stuff. Making separate subfolders for your media, cgi scripts, Java scripts, style sheets, real audio and other things is a good idea. Especially when you are trying to debug things.
How Do I Make My scripts Directory? You should be know how to connect to FrogNet via TELNET. If "TELNET" is a foreign phrase for you, then you should read our Telnet Tutorial first before proceeding.
Connect to FrogNet via Telnet. This is called "logging on to the shell." An example of what logging on to the shell would look like with Window's built in Telnet application is shown below.
Once your "in the shell", you should change to your html directory. This is done by typing "cd html" and hitting return (Note: Don't type the quotes, silly) CD stands for change directory. Next, type "mkdir blah" and hit return. Substitute the word blah with the name you want to assign your new cgi sub-directory.Just in case you missed our warning earlier... DO NOT NAME your directory cgi-bin!!!! You can call it anything else you want, except cgi-bin. Call it perl, fred, scripts, joe, alice, myscripts, perlstuff, cgi, poop, binaries, or anything else you want... EXCEPT cgi-bin.
If you do call it "cgi-bin" your script will run into problems--especially if this is a virt domain. The phrase "cgi-bin" is a reserved word assigned to point to FrogNet's cgi-bin directory (/var/www/http/cgi-bin). Virtual domain owners do not have permission to access FrogNet's cgi-bin directory and therefore will have problems. Not only that, but your scripts aren't even located in FrogNet's cgi-bin directory. They are located in yours (/home/joefrog/mydomain.com/html/). So to avoid the confusion, don't name your new directory "cgi-bin"
The dos2unix, mac2unix Commands
O.K. here is some oversimplified background info that you should know about.
There are three major types of computer operating systems. There are UNIX based computers, Windows based computers and Macintosh based computers. Unfortunately, all three of these computer operating systems have different ways of representing what happens when you press the return or enter key on your keyboard.
Lets say, you are writing a CGI script on your PC using notepad. When you come to the end of a line of code, you hit the return key to move to the next line. PC's use \r\n to represent the end of a line. \r = carriage return and \n = newline. These are special characters that you don't normally see. An old, pre OSX Mac represents the end of a line with two carriage returns (\r\r). Newer Mac's are actually UNIX based.
Well, UNIX computers represent the end of a line as with single newline character (\n). If you save a cgi script on your PC and Mac, and upload it to FrogNet's UNIX computer. Your CGI script will not run unless you reformat it to have "UNIX terminated lines"
There is a command that you can run via TELNET that will convert your files for you. If you have a Mac, you'd want to use... mac2unix. If you have a PC, you'd want to use dos2unix.
Log into the FrogNet command shell via TELNET. "cd" to the directory where your CGI scripts are located and then type: dos2unix (or mac2unix if you have a mac) followed by a space and then the name of your cgi script. Hit return and the CGI script will be converted from a PC or Mac formatted file to a UNIX file.
Some newer text editors like editplus for the PC allow you to save a file as Unix, Pc, or Mac and convert between all the types. If you don't like to telnet, convert the file in editplus and then upload the Unix converted file to your account.
Check the permissions for the file and make sure they are still 700. You may have to chmod them back to 700 otherwise your script will not run.
Example: Set Up A Guest Book
The following is a basic example of how to set up a guest book script on FrogNet user account. For those of you who skipped all of the text above and came directly here, skip right on back to the top and read ALL of the information. Otherwise this example will not make sense.
This tutorial is for intermediate useres. This tutorial is also aimed for regular FrogNet account users. If you are trying to add this guestbook to your virtual domain, the following information will wrong.
Tools You'll Need:
-
An FTP program to transfer files on your computer to your account on Frognet. If you don't know what an FTP program, go back to the help section and read about ftp.
-
An archive proram to un-zip files. If you don't know what "un-zipping" or "un-compressing" a file means, then you should go to webopedia.com and read the definition of zip. There are some helpful links on the page to even more zip resources.
-
A TELNET Program so that you can log on to the shell and change the permissions of some files. If that previous sentence read like a bunch of nonsense words, then you need to read our tutorial on telnet
Step 1) Download the file.
Matt's script archive is a fantastic place for finding easy to use, basic CGI scripts. They are all written in a language called PERL and since PERL is a very portable language, his scripts can be run all all sorts of web servers. Go to the MSA web site and download the guestbook file.
Step 2) Unzip The File
Pick a folder on your computer to temporarily hold the 5 files archived in the guestbook.zip file.
Step 3) Edit The Files
guestbook.pl - Using the text editor of your choice (notepad or bbedit for example) open the guestbook.pl file. Right around the 25th line of the script you will see the following.
$guestbookurl = "http://your.host.com/~yourname/guestbook.html";
$guestbookreal = "/home/yourname/public_html/guestbook.html";
$guestlog = "/home/yourname/public_html/guestlog.html";
$cgiurl = "http://your.host.com/cgi-bin/guestbook.pl";
$date_command = "/usr/bin/date";
Note: phrases like "yourname" and "yourlogin" are not literal. They were put there by the program for demonstration purposes and are meant to be replaced. You should replace phrases like "yourlogin" and "yourname" with your actual login information. For example: if your login was joefrog, you'd change the first line to read...
$guestbookurl = "http://home.frognet.net/~joefrog/guestbook.html";
Change the second line to read...
$guestbookreal = "/home/joefrog/html/guestbook.html";
Change the third line to read...
$guestlog = "/home/joefrog/html/guestlog.html";
Change the fourth line to read...
$cgiurl = "http://home.frognet.net/~joefrog/scripts/guestbook.pl";
For the last line, we should do something special. You should connect to FrogNet via telnet and type in the command "which date" to find the location of the "date" command on FrogNet's server. I'll just save you the time and tell you what it is right now.
$date_command = "/bin/date";
Next you'll see the following lines of code...
# Set Your Options:
$mail = 0; # 1 = Yes; 0 = No
$uselog = 1; # 1 = Yes; 0 = No
$linkmail = 1; # 1 = Yes; 0 = No
$separator = 1; # 1 = <hr>; 0 = </p>
$redirection = 0; # 1 = Yes; 0 = No
$entry_order = 1; # 1 = Newest entries added first; # 0 = Newest Entries added
last.
$remote_mail = 0; # 1 = Yes; 0 = No
$allow_html = 1; # 1 = Yes; 0 = No
$line_breaks = 0; # 1 = Yes; 0 = No
You can leave the above lines of code the way they are, or if you want to change some of the options, you can do that too. It's up to you and fairly self explanitory. If you don't want to use the log file, for example, change the value of "$uselog" to equal zero.
Now for some more code...
# If you answered 1 to $mail or $remote_mail you will need to
fill out
# these variables below:
$mailprog = '/usr/lib/sendmail';
$recipient = 'you@your.com';
Again, by loging into the shell and using the command "which sendmail" you can find the correct location of the sendmail program. But, to save you time, I'll tell you what it is...
$mailprog = '/usr/sbin/sendmail';
The recipient, being you, should be your email address...
$recipient = 'joefrog@frognet.net';
Your done here, so save the file close it.
addguest.html - Open the addguest.html file in your favorite text editor. right around line 12 of the file, you should see the following line...
<form method=POST action="http://your.host.com/cgi-bin/guestbook.pl">
You should change the line to read the same as the value for the $cgiurl variable in the guestbook.pl file. Remeber to substitute your login name for joefrog!
<form method=POST action="http://home.frognet.net/~joefrog/scripts/guestbook.pl">
About 10 lines down from line 12 (on line 22) you should see the following bit of html code...
* <a href="http://your.host.com/guestbook.html">Back
to the Guestbook Entries</a>
change this line to read...
* <a href="http://home.frognet.net/~joefrog/guestbook.html">Back to the Guestbook Entries</a>
Step 4) Make the "scripts" Directory
Connect to frognet via telnet. If you don't know how to do so, then you should read the telnet tutorial.
After connecting to frognet, type in the following lines...
cd
cd html
mkdir scripts
Keep your telnet window open.
Step 5) FTP the Guest Book Files to Your Account
Open up your FTP program. We suggest using wsftp-le if you own a PC or Fetch if you own a MAC. If you don't know how to use wsftp-le or Fetch, then you should read our tutorial (Macintosh Users Click Here, PC users Click Here)
Upload all the files that end in .html to your html directory (/home/joefrog/html). Upload the guestbook.pl file to the scripts directory which you created in step 4 (/home/joefrog/html/scripts). Please remember to substitute your login for "joefrog."
Step 6) Convert the File and Set The Permissions
If you read all the information above about su exec on FrogNet, then you know might already know how to set the permissions. If you skipped all that stuff and came directly here, we HIGHLY reccomend that you go back to the top of this page and read everything.
There are two commands used to convert your text files into UNIX formatted files. If your CGI scripts are not UNIX formatted, they will not run. If you use a mac, then the command you want to remember is mac2unix. If you use a pc, then dos2unix is your command.
Go back to your telnet window, which should still be open, and type the following commands... (If you edited your script on a Macintosh Computer, use mac2unix INSTEAD OF dos2unix in the following example)
cd
cd html/scripts
dos2unix guestbook.pl
chmod 700 guestbook.pl
You've successfully changed the format and permissions of your guestbook.pl and the scripts directory. That's it! You're done. To use your guestbook, open your favorite web browser (Netscape or Microsoft Internet Explorer) and go to the following url...
http://home.frognet.net/~joefrog/guestbook.html
have fun...
Debugging a CGI Script
Lets go over the check list:
-
Did you upload the file to the correct directory? You should telnet to your account and use cd and ls to explore your directories to make sure the file is actually where you think it is.
-
Did you use dos2unix or mac2unix to convert your CGI script to a UNIX formated file?
-
Are the permissions for the CGI script set to 700?
-
Are the permissions of the directory where your CGI script resides set to 755?
If you've done all of the above, and your script still doesn't work. It is time for some advanced debugging. First, try running the script directly from your telnet session. Lets say I am user joefrog and I have a script named mailtest.pl in my /home/joefrog/html directory. I log in using telnet, CD to my /home/joefrog/html directory and attempt to run my CGI script from the shell by typing in..
adenine-> ls
mailtest.pl
adenine-> ./mailtest.pl
Scalar found where operator expected at ./mailtest.pl line 4, at end of line
(Missing semicolon on previous line?)
syntax error at ./mailtest.pl line 4, near "$message "
Execution of ./mailtest.pl aborted due to compilation errors.
It would seem that there is an error on line 4 of joefrog's script. It would seem that he might be missing a semicolon at the end of a line of code. This is a very common PERL, C, and PHP error that happens a lot. So I need to edit my file, re-upload and convert it to UNIX and run it again.
Another thing you can do it tail the error logs.
Our web server maintains error logs in a publicly accessible file. All frognet users with a shell account can look at the error logs. A trick I like to do it watch the error logs in real time as they are being updated with fresh errors.
Open a telnet session to FrogNet and log in with your username and password. Our error logs are kept at the following location...
/etc/httpd/logs/error_log
Open up a telnet session.
We keep a lot of logs. The log you are concerned with is called error_log as it contains all the errors that occured today. The file is constantly being written to as people's scripts and web pages always seem to be generating errors. To view the latest entries to the log file as they are being written, you have to use the tail unix command.
If you havent already done so, open up a telnet session and then type..
tail -f /etc/httpd/logs/error_log
The command "tail" show the end of a text file. The option "-f" means, "follow the tail" and is used to see the end of a text file as the file is being written to. So as new errors occur you can see them being added to the log in real time.
Open your web browser, and surf to your failing CGI script. Now, quickly switch back to your telnet window and hit Ctrl-C to stop tail from tailing the log file. Now, search the last few lines for your error message.
Here is an example of some errors that you might find in the log
/home/arnoldss/top.pl: premature end of script headers
/home/gooberhead/html/mypage.html: No such file or directory
/home/gooberhead/html/stuffpage.html: such file or directory
/home/joefrog/mailtest.pl: premature end of script headers
Looks like user arnoldss is having trouble with a CGI script he wrote called top.pl and it looks like user gooberhead has a link to a non-existant file. Dead links and crashed CGI scripts are very common and appear in the error logs.
In our sample error log above, you can see that joefrog's mailtest.pl script crashed and was reported to the log.
"premature end of script headers" is a generic
way of stating that your script had a major error. If you see this, cd to your
directory and try running the script directly. Error logs don't give as specific
an error message as running the script directly from the shell.
What's next?
You've created a guestbook on your personal frognet account. Try your hand at some other of Matt's scripts. Each of them will have different variables to configure and other new challenges. You should keep all your scripts in the "scripts" directory that you created. That way they are in a centralized location and not spread throughout your FrogNet account.
Many of Matt's scripts are very general and are often modified by individuals to make the pages prettier. If you decide to change the PERL code or the HTML make sure to keep a backup of everything just in case you break something. ALWAYS KEEP BACKUPS! Otherwise you are asking for trouble.
We hope you had fun? Send questions or comments to