File Transfer Protocol

[TwinsX2Dad]

FTP (File Transfer Protocol) was developed at a time when typical modem speeds were 110 to 300 bits per second. Since ASCII only used 7 bits, long files could be transmitted more quickly by not sending all the unused bits. Binary was reserved for files which require 8 bits per byte.

The big drawback is that if a binary file (one that uses all 8 bits in each byte) is accidentally sent using ASCII transfer, it will lose 1/8 of its information content. In most files, even the loss of one bit is enough to make it invalid, and losing 1/8 makes them totally unreadable. On the other hand, if a ASCII file is sent using binary transfer, bits will be appended to each byte, adding one bit and making the file again full of errors.

The following is a list of the most common file types (additions are welcomed) you will encounter, separated by how they need to be transferred:

ASCII

• .asp
  
• .bat
  
• .c
  
• .cf
  
• .cfm
  
• .cgi
  
• .cgi
  
• .conf
  
• .cpp
  
• .css
  
• .csv
  
• .dat
  
• .dhtml
  
• .diz
  
• .h
  
• .hpp
  
• .hta
  
• .htaccess
  
• .htm
  
• .html
  
• .inc
  
• .js
  
• .mak
  
• .mht
  
• .nfo
  
• .pas
  
• .patch
  
• .php
  
• .phtml
  
• .pinerc
  
• .pl Perl scrips must be transferred in ASCII
  
• .por
  
• .ps
  
• .rtf
  
• .sh
  
• .shtml
  
• .sql
  
• .svg
  
• .tcl
  
• .tp
  
• .tpl
  
• .txt (ASCII text - see Unicode note below)
  
• .vbs
  
• .xml
  

Binary

• .bmp
  
• .com
  
• .dll
  
• .doc
  
• .dta
  
• .exe
  
• .gif
  
• .gz
  
• .gzip
  
• .jpeg
  
• .jpg
  
• .mdb
  
• .mp
  
• .mp2
  
• .mp3
  
• .mp4
  
• .odt
  
• .pdf
  
• .png
  
• .ra
  
• .sav
  
• .sea
  
• .stc
  
• .swf
  
• .sxc
  
• .sxi
  
• .sys
  
• .tar
  
• .txt (Unicode text - see note below)
  
• .wav
  
• .wk
  
• .wpd
  
• .wma
  
• .wmv
  
• .xls
  
• .zip
  

All audio and/or video files, as well as any compressed format, must be transferred in binary mode. Most files created with an office suite should also be transferred in binary mode.

Note about Unicode

Unicode is the system which was designed to ultimately replace ASCII, because it has the potential to include all the characters of all human languages. It can do this because it uses 8, 16, or 32 bits instead of 7.

Since it uses all the bits in each byte, it must be transferred in binary mode. Unfortunately, the extension .txt is routinely used for Unicode text files, making it impossible to tell from the extension whether to transfer it in ASCII or binary.

When sending or receiving files via FTP, it is important to know whether the files are ASCII or binary. ASCII is the default and should be used with text files. Binary files are image files and this setting should be used with most software-specific files.

The Help section for WS-FTP offers this advice:

• A file that is text, HTML, postscript, other ASCII format (.txt, .html, .htm, .cgi), or EBCDIC should be transferred in ASCII mode. Note: Transferring a binary file in ASCII format can damage the file.
  
• A file that is a bitmap, an executable, or compressed (.bmp, .jpg, .exe, .zip) should be transferred in binary mode. (Binary mode should also be used if there is no file extension or the file extension is not registered.)
  

The last one is where we often come into problems with FTP in uploading SMF (or other message board scripts) that are unzipped first, since we tend to choose the AUTO setting in the transfer process.

Some FTP programs will default to binary mode for unrecognized file types. A commonly unrecognized file extension for these programs is .php, meaning they will be transferred in binary mode. This will add all sorts of meaningless information to your php files, which comprise a large portion of the SMF & TP software.

Other FTP programs will default to ASCII mode for unrecognized file types. A commonly unrecognized file extension for these programs is .png, meaning they will be treated like a text document. Your various .png graphics will be corrupted and useless. SMF & TP both utilize .png files.

Your solution is to double check the file transfer settings of the FTP client you choose. One of the most capable and popular, WS-FTP, has many versions which do not have the most common php related file types entered into settings. This means you, the user, will need to configure the settings before you use it. Usually this means you will need to enter in the file types which require ASCII transfer. WS-FTP and some other clients will, by default, create a log file into each folder. These are harmless, but do take space.

If you are uncertain as to which file tranfer protocol to use, see if you can open the file in a text editor (like Notepad). If you can read the contents of the file, you should probably use ASCII mode. If the result is either unreadable or will not open in your text editor, it is probably a binary file.

Remember, transferring ASCII files in binary protocol, or vice versa, will result in corrupted files. If in doubt, ask someone knowledgeable in FTP for advice or Google the file extension.

Keeping these points in mind will help minimize any problems installing or upgrading your message board.

[G6Cad]

This was very useful reading 

Thanks for charing this information TwinsX2Dad, I just had a peek in my Flash FXP and i realised that the auto transfer in settings not always manage to get the extensions right, But now i can set them up myselfe as they should be in the skiplist with more

[IchBin]

A plethera of great information Twins. Thank you for clarifying stuff I had questions about, and adding to my book of internet knowledge.

[TwinsX2Dad]

It is my pleasure.

I've found that FTP errors account for a number of failed installs. I help a lot of friends and acquaintances get their own boards running - they upload and get errors, I reupload and no errors. When I check their FTP settings, there always seems to be a file extension that wasn't handled correctly.

But, to be truthful, the biggest issues are outside of SMF. While many still mishandle php & javascript files, the biggest errors come in installs of the likes of phpBB and UBB, with their .tpl and other less common file extensions.

Still, with all of the problems we continue to attribute to corrupted files, this info should be helpful.

[Nokonium]

Could I copy this and pass it onto some fellow SMF board admins please?

[TwinsX2Dad]

Could I copy this and pass it onto some fellow SMF board admins please?

Be my guest - just please include the TP site url so they know where it came from.

Any growth in the TP world is good growth. 

[Nokonium]

Thankyou.

No worries about spreading the word. The Our-Local main site is TP, I am close to going live with one of my other test sites using TP and one of my co-admins runs two other boards both using TP, TP rocks!

[PhilH]

Perhaps a reference to this source is appropriate.

http://courses.wccnet.edu/computer/mod/na36c.htm

[TwinsX2Dad]

Or many other .edu sites - I find a few dozen - most of these are various grades of improvements/designs.

Much of the information and format actually originated from AT&T/Western Electric/Bell Labs (the originators of UNIX & FTP) and dates back to the sixties. I know this well, as I worked for them for over 20 years.

[eldacar]

Hmm nice. I'll suggest it on the OM.