Official Gmail Blog: New in Labs: Canned Responses

Gmail dashing new feature. For complete detail visit
Official Gmail Blog: New in Labs: Canned Responses.

If anyone asks ya to work on a weekend...

Blogged with the Flock Browser

LHC

Blogged with the Flock Browser

Ramadan Day Three Prayer

ALLAH, on this day, grant me wisdom and awareness, keep me away from foolishness and pretension, grant me a share in every blessing You send down, by You generosity, O the most Generous.

Blogged with the Flock Browser

Ramadan Day Two Prayer

ALLAH, on this day, take me closer towards Your pleasure, keep me away from Your anger and punishment, grant me the opportunity to recite Your verses (of the Qur'an), by Your mercy, O the most Merciful.

Blogged with the Flock Browser

Ramadan Day One Prayer

ALLAH, on this day make my fasts the fasts of those who fast (sincerely), and my standing up in prayer of those who stand up in prayer (obediently), awaken me in it from the sleep of the heedless, and forgive me my sins , O God of the worlds, and forgive me, O one who forgives the sinners.

Blogged with the Flock Browser

I Proudly become part of world record

Blogged with the Flock Browser

The "Last lecture" professor passed away

Generate Sitemap Using Google Webmaster Tools

Contents

Before you begin
Downloading the Sitemap Generator program files
Creating a configuration file
Uploading the files to your web server
Running the Sitemap Generator script
Submitting your Sitemap to Google
Troubleshooting

Before you begin

The Google Sitemap Generator is a Python script that creates a Sitemap for your site using the Sitemap Protocol. This script can create Sitemaps from URL lists, web server directories, or from access logs. In order to use this script:

• You must be able to connect to and run scripts on your web server.
• Your web server must have Python 2.2 or later installed.
• You must know the command that launches Python. (Generally, this is python, but may vary by installation. For instance, if the web server has two versions of Python installed, the earlier version may be invoked by the command python and the later version may be invoked by the command python2.)
• You must know the directory path to your site. If your web server hosts one site, this may be a path such as var/www/html. If you have a virtual server that hosts multiple sites, this may be a path such as home/virtual/site1/fst/var/www/html.
• You must be able to upload files to your web server (for instance, using FTP).
• If you will be generating a list of URLs based on access logs, you must know the encoding used for those logs and the complete path to them.

If you aren't sure about any of this, you can check with your web hosting company.

Now you’re ready to get started. Here’s an overview of what you’ll need to do.

1. Download the Sitemap Generator program files. Extract the files to a local directory.
2. Create a configuration file for your site using the provided example_config.xml file as a template. Modify this file as needed for your site and save it.
3. Upload the necessary files to your web server.
4. Run sitemap_gen.py.
5. Add the generated Sitemap to your Google webmaster tools account.
6. Set up a recurring script. (optional)

If you are unable to use the Sitemap Generator, you can add a Sitemap to your Google webmaster tools account in another format, such as a simple text file. Third-party programs supporting the Sitemap Protocol.

For News Sitemaps: The Sitemap Generator is not recommended for use in creating Google News Sitemaps at this time, due to the special requirements of News Sitemaps. News Sitemaps are intended to be dynamic lists of only the most recently published news articles (rather than the entire website), and they are updated frequently.

1. Downloading the Sitemap Generator program files

The Sitemap Generator files are available in ZIP and GZ archive formats from the following location:

http://sourceforge.net/project/showfiles.php?group_id=137793&package_id=153422

Once you download the archive, extract it into a local directory. Locate the following files:

• README —contains the latest information about this tool
• sitemap_gen.py —the python script that generates your Sitemap
• example_config.xml —the template configuration file you’ll use to specify the configuration for your site
• example_urllist.txt —the template URL list you can use if you wish to create a Sitemap based on a set of URLs that you specify

2a. Creating a configuration file

This section provides step-by-step instructions for creating a configuration file. It also provides a complete reference of the options available. If you are creating Mobile Sitemaps, see the additional mobile guidelines.

In order to create a configuration file for your site, you must have the following information:

• The base URL for your site (such as http://www.example.com/). Ensure that you include the protocol (such as http://). For instance, http://www.google.com is a valid base url, but www.google.com is not.
• The web server path to the location where you want to store the Sitemap. Generally, this is the path to the base URL as the Sitemap cannot contain URLs that are in a higher-level directory from the location of the Sitemap. When you run the Google Sitemap Generator, it creates the Sitemap and places it in the location you specify.
• The methods you want the Sitemap Generator to use to create your Sitemap. You can use any combination of methods. The following methods are available:

• URL —list individual URLs in this section of the configuration file, along with information about each URL. You would generally use this method in conjunction with another method to manually include additional URLs that other methods wouldn't pick up.
• URL list —point the configuration file to a text file that contains a list of URLs. You might want to use this method if this text file already exists or if you use a script to generate a list of URLs.
• Directory paths —specify the directory paths for your site and corresponding URLs to those paths. The Sitemap Generator will create a list of URLs based on the contents of those directories. You might want to use this method if your site consists of static HTML files.
• Access logs —point to the path to your log files. The Sitemap Generator will create a list of URLs based on the URLs included in the logs. You might want to use this method if your site consists of dynamic pages.
• Sitemap —point to existing Sitemaps that you have created with the Sitemap Generator. The Sitemap Generator will create a single Sitemap that includes the URLs contained in each Sitemap. You could use this method if you have already created several smaller Sitemaps that you want to combine into one larger Sitemap.

Create the configuration file as follows:

1. Open the example_config.xml file in a text editor. Save it as a new file (such as config.xml or mysite_config.xml).
2. Locate the site definition section:

<site 
base_url="http://www.example.com/" 
store_into="/var/www/docroot/sitemap.xml.gz"
verbose="1">
				

3. Change the base_url value to the URL for your site.
4. Change the store_into value to the path on your web server where you want to store the Sitemap and the filename you want to use for the Sitemap. Generally, this is the path to the base URL since Google can only accept URLs that are at the same level as or subdirectories of the directory that holds the Sitemap. You can specify a relative path from the directory where you upload the script or a complete path from the root of your web server. If you upload the script to your base URL directory, you can simply specify the filename.
5. Locate the generation method sections that begin with ** MODIFY or DELETE **. Each of these sections corresponds to a method for generating a Sitemap.
6. Delete the sections for the methods you aren’t going to use.
7. Follow the instructions below for the methods you are going to use.

URL

Locate the following section:

<!-- ** MODIFY or DELETE ** 
"url" nodes specify individual URLs to include in the map. <br>

Required attributes: 
href - the URL

Optional attributes: 
lastmod - timestamp of last modification (ISO8601 format) 
changefreq - how often content at this URL is usually updated
priority - value 0.0 to 1.0 of relative importance in your site 
--> 

<url href="http://www.example.com/stats?q=name" /> 
<url 
href="http://www.example.com/stats?q=age" 
lastmod="2004-11-14T01:00:00-07:00" 
changefreq="yearly" 
priority="0.3"
/> 

This section gives two examples: the first includes only the required attribute and the second includes the required attribute as well as the optional attributes.

Use this format for each of the URLs you want to include. The changefreq attribute gives Google a general idea of how often the URL is updated. This helps Google know how often to visit the page for new content. The priority attribute gives Google information about the relative importance of this page compared to the other pages of your site. This attribute has no effect on how Google compares your page to pages on other sites, it just helps Google know which pages of your site that you think are most important.

URL list

Locate the following section:

<!-- ** MODIFY or DELETE **
"urllist" nodes name text files with lists of URLs. 
An example file "example_urllist.txt" is provided. 

Required attributes: 
path - path to the file 

Optional attributes: 
encoding - encoding of the file if not US-ASCII 
--> 
<urllist path="example_urllist.txt" encoding="UTF-8" /> 
				

Use this format to point to the path and name of the text file that contains your list of URLs. You can use the provided example_urllist.txt file as a template for that text file. You can specify either a relative or complete path to your web server. For instance, if the Sitemap Generator and urlist.txt file are located in the same directory, you can simply specify the filename of the .txt file, If you create a text file with an encoding other than UTF-8, you can use the encoding attribute to indicate this encoding. If you have multiple .txt files, you can use wildcards. For instance:

<urllist path="example_urllist*.txt" encoding="UTF-8" /> 

For each URL you include in the text file, you can specify the last modification date, change frequency, and priority. See the URLlist text file reference section for complete information about the structure of this file.

Directory paths

Locate the following section:

<!-- ** MODIFY or DELETE ** 
"directory" nodes tell the script to walk the file system and 
include all files and directories in the Sitemap.

Required attributes:
path - path to begin walking from 
url - URL equivalent of that path 

Optional attributes:
default_file - name of the index or default file for directory URLs

--> 
 <directory  path="/var/www/icons"    url="http://www.example.com/images/" />
  <directory
     path="/var/www/docroot"
     url="http://www.example.com/"
    default_file="index.html"
  />
 

This section gives two examples. If all of your pages are contained in subdirectories of one path, then you only need to include one entry. However, if you have multiple paths to pages on your site, include an entry for each.

Remember that each URL must begin with the base URL you specified in step 3. For instance, the examples given in the example_config.xml file both have URLs that begin with http://www.example.com/. Therefore, both URLs are valid.

Replace the example entries with entries for your site. Many sites will only have one entry that points to the base URL. Ensure that path value is the complete path to the directory on your web server. Ensure that the url value is the complete URL, including the protocol (such as http) and a trailing slash, if required.

You can use the default_file parameter to specify the filename that your server uses as the default page for a directory. In the above example, /var/www/docroot resolves to http://www.example.com/index.html. You are not required to specify this. However, if you do, the Sitemap Generator will include the page that maps to each subdirectory only once (rather than list both the directory URL and filename URL) and will use the last modified date of the file (rather than the directory) to extract the lastmod attribute for that page.

Access logs

Locate the following section:

<!-- ** MODIFY or DELETE **
"accesslog" nodes tell the script to scan webserver log files to
extract URLs on your site.  Both Common Logfile Format (Apache's default 
logfile) and Extended Logfile Format (IIS's default logfile) can be read.
				
Required attributes:
 path - path to the file
Optional attributes:
 encoding - encoding of the file if not US-ASCII
  -->
<accesslog path="/etc/httpd/logs/access.log" encoding="UTF-8" />
<accesslog path="/etc/httpd/logs/access.log.0" encoding="UTF-8" />
<accesslog path="/etc/httpd/logs/access.log.1.gz" encoding="UTF-8" />

This section gives three examples. You should replace these entries and include an entry for each log file. Ensure that the path value is the complete path and filename on your web server. If the log files are not encoded as US-ASCII or UTF-8, then use the optional encoding attribute to specify the encoding. Rather than list each log file, you can use wildcards. For instance, in the above example, you could include the following entry that would include all three log files:

<accesslog path="/etc/httpd/logs/access.log*" encoding="UTF-8" /> 

The Sitemap Generator assigns priority to URLs it finds in the logs based on how often each URL is accessed. For instance, a URL that has been accessed 100 times will be given a higher priority than a URL that has been accessed twice. The actual priority assignment is relative and depends on each URL as compared to other URLs in the site.

sitemap

Locate the following section:

 <!-- ** MODIFY or DELETE **
    
"sitemap" nodes tell the script to scan other Sitemap files.  This can    
be useful to aggregate the results of multiple runs of this script into
a single Sitemap.
				 
Required attributes:
  path - path to the file
  -->
<sitemap path="/var/www/docroot/subpath/sitemap.xml" />

This section gives one example. You should replace this entry and include an entry for each Sitemap you want to include. Ensure that the path value is the complete path and filename on your web server. You can list gzipped Sitemaps as well, as long as they have a .gz extension. Rather than list each Sitemap, you can use wildcards. For instance, the following entry would include any Sitemaps that begin with the word "sitemap" and have an .xml extension:

<sitemap path="/var/www/docroot/subpath/sitemap*.xml" /> 

The Sitemap Generator extracts all URLs and the optional data listed for each URL for every Sitemap you list and creates one Sitemap with this information. At this time, we can't guarantee that this method will work Sitemaps created with tools other than the Sitemap Generator.

8. Locate the filter definition section:

<!-- ********************************************************         
FILTERS
				
Filters specify wild-card patterns that the script compares
against all URLs it finds. Filters can be used to exclude
certain URLs from your Sitemap, for instance if you have
hidden content that you hope the search engines don't find.

Filters can be either type="wildcard", which means standard
path wildcards (* and ?) are used to compare against URLs,
 or type="regexp", which means regular expressions are used
to compare.

Filters are applied in the order specified in this file.
An action="drop" filter causes exclusion of matching URLs.
An action="pass" filter causes inclusion of matching URLs,
shortcutting any other later filters that might also match.
If no filter at all matches a URL, the URL will be included.
Together you can build up fairly complex rules.

The default action is "drop".
The default type is "wildcard".

You can MODIFY or DELETE these entries as appropriate for
your site. However, unlike above, the example entries in
this section are not contrived and may be useful to you as
they are.
********************************************************* -->

<!-- Exclude URLs that end with a '~' (IE: emacs backup files) -->
<filter action="drop" type="wildcard" pattern="*~" />

<!-- Exclude URLs within UNIX-style hidden files or directories -->
<filter action="drop" type="regexp" pattern="/\.[^/]*" />

You can use filtering to exclude specific URLs from the generated Sitemap. You might want to do this to create a cleaner list, to reduce redundant listings, or to keep certain URLs from being indexed. Note that if you use a robots.txt file to keep URLs from being indexed, then even if the URLs are included in your Sitemap, Google will not search or index them.

You can use any or all of the filtering methods. You can delete the entries you don’t need and can create additional entries, if desired. Below are sample usages.

<filter action="drop" type="wildcard" pattern="*.jpg" />

This filter excludes URLs that end in .jpg. You might want to include a similar filter if all of your site’s images are embedded within HTML pages and should not be accessed as standalone URLs.

<filter action="pass" type="wildcard" pattern="*.htm*" />
<filter action="drop" type="wildcard" pattern="*" />

This filter includes all .htm* files but excludes everything else.


9. Once you have made all the changes for your site, save the file.

Config File Syntax Reference

A complete explanation of the config file syntax is below. Each tag begins with a code sample, followed by a description of the attributes.

site
Required tag at the beginning of each config file.

<site
base_url="http://www.example.com/"
store_into="/var/www/html/sitemap.xml.gz"
verbose="1"
supress_search_engine_notify="1"
default_encoding="UTF-8">

base_url	required	The HTTP path of the base of your website - only URLs that begin with this base can be included in the Sitemap
store_into	required	The web server path to the desired output file. The script will create this file - there's no need to create the file before running the script.
verbose	optional	Enter a number from 0-3, with higher numbers corresponding to increased debug information
suppress_search_engine_notify	optional	Disable search engine notification by entering "1" for testing purposes
default_encoding	optional	Specify a character encoding to be applied to file system paths and URLs

url
Optional tag that you can use to list each URL in your site.

<url
href="http://www.example.com/stats?q=age" 
lastmod="2004-11-14T01:00:00-07:00" 
changefreq="yearly" 
priority="0.3"
/> 

href	required	The HTTP path of the base of your website - only URLs that begin with this base can be included in the Sitemap
lastmod	optional	The time the URL was last modified in W3C Datetime format (YYYY-MM-DDThh:mm:ss+00:00). You may omit the time portion. Examples:   "2005-02-21T18:00:15+00:00"   "2005-02-21"
changefreq	optional	The frequency with which the URL is likely to change. This is considered a hint and not a command. The value must be one of "always", "hourly", "daily", "weekly", "monthly", "yearly", or "never".
priority	optional	The priority of this page relative to other pages on the same site. The value is a number between 0.0 and 1.0, where 0.0 is the lowest priority and 1.0 is the highest priority. The priority can affect the order that search engines select URLs to explore on your site. Since the priority is relative, it is only used to select between URLs within your own site; the priority of your pages will not be compared to the priority of pages on other sites.

urllist
Optional tag that you can use to point to a text file that contains a list of the URLs in your site.

<urllist path="/var/www/html/urllist.txt" encoding="UTF-8" />

path	required	The path and filename of the .txt file. You can specify either a relative or complete path.
encoding	optional	The encoding of the file, if not UTF-8.

The urllist.txt file is a simple text file containing a list of URLs to map. You can also include optional attributes for each URL. Attributes are entered on the same line as the URL and are separated by a single space. For example:

http://www.example.com/abc/something
http://www.example.com/abc/xyy.pdf lastmod=2001-12-31T14:05:06+00:00
http://www.example.com/abc/def?x=12&y=23 changefreq=weekly priority=0.3

lastmod	optional	The time the URL was last modified in W3C Datetime format (YYYY-MM-DDThh:mm:ss+00:00). You may omit the time portion. Examples:   "2005-02-21T18:00:15+00:00"   "2005-02-21"
changefreq	optional	The frequency with which the URL is likely to change. This is considered a hint and not a command. The value must be one of "always", "hourly", "daily", "weekly", "monthly", "yearly", or "never".
priority	optional	The priority of this page relative to other pages on the same site. The value is a number between 0.0 and 1.0, where 0.0 is the lowest priority and 1.0 is the highest priority. The priority can affect the order that search engines select URLs to explore on your site. Since the priority is relative, it is only used to select between URLs within your own site; the priority of your pages will not be compared to the priority of pages on other sites.

directory
Optional tag that you can use to specify directories in your site so the Sitemap Generator can create a list of URLs from the files found in those directories.

--> 
 <directory  path="/var/www/icons"    url="http://www.example.com/images/" />
  <directory
     path="/var/www/docroot"
     url="http://www.example.com/"
    default_file="index.html"
  />
 

path	required	States the initial path. Sitemap Generator will traverse this directory and all subdirectories.
url	required	Specifies the URL equivalent of the path value.
default_file	optional	Specifies the default file for a directory on the server.

accesslog
Optional tag that you can use to specify the path and filename of IIS and Apache-style access logs so the Sitemap Generator can automatically pick up URLs from them.

<accesslog path="/etc/httpd/logs/access-0.log" encoding="UTF-8"/>

path	required	States the path to the file.
encoding	optional	Specifies encoding of the file, if not UTF-8.

sitemap
Optional tag that you can use to specify the path and filename of existing Sitemaps that you have created with the Sitemap Generator. The Sitemap Generator will create a single Sitemap that includes the URLs contained in each Sitemap.

<sitemap path="/var/www/docroot/subpath/sitemap.xml" />

path

required

States the path to the Sitemap file.

filter
Optional tag that you can use to build rules that include or exclude specific files. Filters are obeyed in the order in which they appear in the config.xml file. However, intermixing filter entries and input entries (url, urllist, directory, or accesslog) has no additional effect - every URL the Sitemap Generator adds to the Sitemap is first compared against every filter. If no filter matches a URL, the default is to include the URL in the Sitemap.

<filter action="drop" type="wildcard" pattern="*/internal/*" />

action	optional	The action the filter should take. Valid values are: drop - excludes matching URLs. This is the default action, so if no action is specified, the generator assumes "drop". pass - includes matching URLs.
type	optional	The type of filtering. Valid values are: wildcard - standard path wildcards (? and *) are used to compare against URLs. This is the default type, so if no type is specified, the generator assumes "wildcard". regexp - regular expressions are used.
pattern	required	Specifies the pattern to match against.

Encodings

Files referenced by your configuration file, either URL lists or web server logs, can use encodings other than the default UTF-8. You can specify alternate encodings in config.xml to affect how the Sitemap Generator reads your files. Some common encodings are:

• encoding="utf-8" is the assumed default
• encoding="ascii" is a subset of UTF-8 so you don't have to specify it
• encoding="iso-8859-1" is common for many west European languages

2b. Additional information for creating a mobile configuration file

You create a configuration file for a Mobile Sitemap in the same way as for a non-mobile Sitemap. However, you must create a separate config file for each markup language and run the Sitemap Generator with each config file separately so that you create a separate Sitemap for each.

Each config file must:

• Specify a different filename for the store_into value.
• Use filters to specify the URLs to exclude and include for the markup language. Remember that each Sitemap should include URLs for only one markup language. This means that the same URL may be included in multiple Sitemaps, if those URLs serve multiple markup languages.

Examples of filtering

Below are some examples of how you can use extension-based filters to generate Mobile Sitemaps for different markup languages. The specific filtering you use should be based on the types of markup languages used in your site, and how you specify each type. If you have implemented the details of your site differently (for instance, you may organize URLs with different markup languages in separate folders), you should filter based on the specifics of your site implementation. Remember that filters are applied in the order you list them in the config file. So, the first filter you should list is a "pass" action that specifies the URLs you want to include in the Sitemap.

To create a Sitemap for WML (WAP 1.2) content:

<filter action="pass" type="wildcard" pattern="*.wml" />
<filter action="drop" type="wildcard" pattern="*.*" />

To create a Sitemap for XHTML mobile profile (WAP 2.0) content:

<filter action="pass" type="wildcard" pattern="*.xhtml" />
<filter action="drop" type="wildcard" pattern="*.*" />

			

3. Uploading the files to your web server

You should upload the following files to your web server in a location you can access from a command line:

• config.xml —this is the configuration file you just created using example_config.xml.
• sitemap_gen.py —this is the Python script that generates your Sitemap.
• urllist.txt —this file is optional; you only need to include it if you used the text file method of generating a Sitemap.

The method you use to upload these files depends on your environment. Common methods include FTP and SCP. For more information, contact your web host.

4. Running the Sitemap Generator script (sitemap_gen.py)

In order to run the Sitemap Generator, you’ll need to connect to your web server. The method you use to connect depends on your environment. For instance, you can generally access a UNIX-based server using SSH. For more information on connecting to your web server and running scripts, talk to your web host.

Once you have copied the files to your web server, you'll need to run the Sitemap Generator script. Connect to your web server and run the following command (replace <path/config.xml> with the path to and filename of your configuration file; if you have uploaded this file to the same location as the Python script, you can exclude the path):

python sitemap_gen.py --config=<path/config.xml>

For instance, a UNIX-based command line might look similar to this:

A MS-DOS-based command shell might look similar to this:

Tip: If you're testing your configuration and are not ready to submit your Sitemap, the following syntax will prevent Sitemap Generator from contacting Google:

	$ python sitemap_gen.py --config=config.xml --testing

You'll see the status of your request in the command prompt:

	
	Reading configuration file: /path/config.xml
	Opened URLLIST "/path/urllist.txt"
	Walking DIRECTORY "/var/www/html/dir"
	Walking DIRECTORY "/var/www/html/dir2"
	Opened ACCESSLOG "/etc/httpd/logs/access-0.log"
	Sorting and normalizing collected URLs.
	Writing Sitemap file "/path/sitemap.xml.gz" with 1092 URLs
	Notifying search engines.
	Notifying www.google.com
	Count of file extensions on URLs:
		208  .html
		574  .jpg
		...
		Number of errors: 0
		Number of warnings: 0

If you don't see very much output like this, remember that the verbose setting in your configuration file affects how much information is printed on the screen. This example is representative of setting verbose to "1".

Any errors in the file will also be returned. For instance, if you leave the url= attribute off a directory entry, the script will output the following:

	
	[ERROR] Directory entries must have both "path" and "url" attributes
	Number of errors: 1

Correct any errors in your config.xml file and re-run the script. If no errors are present, the Sitemap Generator will create a new sitemap.xml.gz file in the location you specified in the config file.

5. Submitting your Sitemap to Google

The Sitemap Generator creates a sitemap.xml.gz file in the location you specified in the config file. Once this file is successfully created, make sure it is accessible through a web browser. Then, add it to your Google Sitemaps account. This enables Google to provide you with useful status and statistical information. If Google reports problems with your Sitemap, you can correct the problems and resubmit it. You only have to add the Sitemap manually once. After that, you can use an HTTP request to notify Google about changes to your Sitemap (although you can also resubmit it through your Google webmaster tools account).

6. Setting up a recurring script

We suggest setting up Sitemap Generator to run as often as your content is changed, to a maximum frequency of once per hour.

Webmasters with a UNIX web server may consider setting this up as a cron job.

Webmasters using other platforms should contact their system administrator for help in configuring recurring scripts. You may also benefit from peer advice in the Google Sitemaps Group on Google Groups.

You can use an HTTP request to let Google know about changes to your Sitemap. However, please make sure that you log in to Google Webmaster Tools with your Google Account once to manually add your Sitemap to your Google Webmaster Tools account.

Troubleshooting

In this section we talk about some common questions or issues that some people run into while using Sitemap Generator, and what you can do if you come across one of these.

Web-accessible

Issue: Sitemap Generator returns the following error and warning:

	
	[ERROR] When attempting to access your generated Sitemap at the following URL:
	http://www.example.com/sitemap.xml.gz
	we failed to read it.  Please verify the store_into path you specified in
	your configuration file is web-accessible.  Consult the FAQ for more
	information.
	[WARNING] Proceeding to notify with an unverifiable URL.

What just happened? Sitemap Generator created the file where you specified, then attempted to retrieve it using HTTP just as a search engine would, and failed. It went ahead and notified search engines anyway, but it's outputting the error and warning to let you know that your Sitemap may be not readable by search engines.

Sitemap Generator creates your Sitemap file at the path specified in the store_into attribute of your config.xml file. Sitemap Generator then builds a URL to that file using the base_url attribute, and reports the URL to search engines. For instance, if you set the configuration to:

	<site base_url="http://www.example.com/"
	      store_into="/var/www/html/sitemap.xml.gz">

Sitemap Generator will notify search engines to look for your Sitemap at:

	http://www.example.com/sitemap.xml.gz

The file must be accessible through this URL. If the file can not be retrieved using this URL, search engines have no way of finding your Sitemap.

What can you do? You need to verify that your config.xml is specifying the correct base_url and store_into paths for your Sitemap. You also need to confirm that web browsers can retrieve the Sitemap file off of the base_url. If you find errors in the path or URL, you may need to re-run Sitemap Generator, or just move the Sitemap file to the correct place and notify search engines manually through their web sites.

Support for XML

Issue: Sitemap Generator gives the following error:

	
	Some installs of Python 2.2 do not include complete support for XML. 
	Please try upgrading your version of Python and re-running the script.

What just happened? As the message says, some platforms have a version of Python (the language the Sitemap Generator script is written in) that is missing support libraries needed for processing XML files. This script requires full XML support in order to run.

What can you do? Try upgrading your installed version of Python to a newer version. You will probably need to contact your system administrator to do this.

Note that this just affects the Sitemap Generator, not Sitemaps overall. If you have another method or tool for creating Sitemaps, you can certainly use it and submit your Sitemaps to search engines.

What are all the extra files in the .gz or .zip?

When you extract the sitemap_gen.py script, you'll probably see quite a few more files than we refer to above. The full file list looks closer to this:

	
	AUTHORS
	ChangeLog
	COPYING
	example_config.xml
	example_urllist.txt
	PKG-INFO
	README
	setup.py
	sitemap_gen.py
	test_sitemap_gen.py

The extra files tend to be information on the package and licensing terms. You are encouraged to look through these files.

The one exception is test_sitemap_gen.py, which is a unit-test script that other developers may find useful if they wish to contribute to this open source project. If you intend to use Sitemap Generator without modifying any of the source code - this is the expected case for nearly everyone - you probably don't need this test script.

If you are interested in helping with this project, please visit http://sourceforge.net/projects/goog-sitemapgen. Third-party programs that support the Sitemap Protocol.

Non-ASCII characters

Issue: Your site domain name or URLs within it contain non-ASCII characters.

Generally, non-ASCII URLs should be encoded using UTF-8 before being percent-escaped. However, some webservers only respond correctly if URLs are encoded using an encoding other than UTF-8. All URLs within your Sitemap, as well as the URL of the Sitemap itself must be encoded for readability by the web server on which they are located. Within the site definition section, use the optional default_encoding attribute to specify the encoding used by your webserver. If you don't use this tag and your webserver uses an encoding other than UTF-8, we can't know which encoding to use.

If your URLs contain non-ASCII characters, we recommend that you run the Sitemap Generator script using Python 2.3 or higher. This version of Python has increased non-ASCII support. If your domain name contains non-ASCII characters, you must use Python 2.3 or later, as Internationalizing Domain Names in Applications (IDNA) support wasn't added until this version. Without IDNA support, the Sitemap Generator can't correctly encode a non-ASCII domain name.

Blogged with the Flock Browser

Thirty day challenge flock settings

Embedded Video

Blogged with the Flock Browser

Announcing AJAX Libraries API: Speed up your Ajax apps with Google’s infrastructure

Category: Ajax, JavaScript, Library, Google

I just got to announce the Google AJAX Libraries API which exists to make Ajax applications that use popular frameworks such as Prototype, Script.aculo.us, jQuery, Dojo, and MooTools faster and easier for developers.

Whenever I wrote an application that uses one of these frameworks, I would picture a user accessing my application, having 33 copies of prototype.js, and yet downloading yet another one from my site. It would make me squirm. What a waste!

At the same time, I was reading research from Steve Souders and others in the performance space that showed just how badly we are doing at providing these libraries. As developers we should setup the caching correctly so we only send that file down when absolutely necessary. We should also gzip the files to browsers that accept them. Oh, and we should probably use a minified version to get that little bit more out of the system. We should also follow the practice of versioning the files nicely. Instead, we find a lot of jquery.js files with no version, that often have little tweaks added to the end of the fils, and caching is not setup well at all so the file keeps getting sent down for no reason.

When I joined Google I realised that we could help out here. What if we hosted these files? Everyone would see some instant benefits:

• Caching can be done correctly, and once, by us... and developers have to do nothing
• Gzip works
• We can serve minified versions
• The files are hosted by Google which has a distributed CDN at various points around the world, so the files are "close" to the user
• The servers are fast
• By using the same URLs, if a critical mass of applications use the Google infrastructure, when someone comes to your application the file may already be loaded!
• A subtle performance (and security) issue revolves around the headers that you send up and down. Since you are using a special domain (NOTE: not google.com!), no cookies or other verbose headers will be sent up, saving precious bytes.

This is why we have released the AJAX Libraries API. We sat down with a few of the popular open source frameworks and they were all excited about the idea, so we got to work with them, and now you have access to their great work from our servers.

Details of what we are launching

You can access the libraries in two ways, and either way we take the pain out of hosting the libraries, correctly setting cache headers, staying up to date with the most recent bug fixes, etc.

The first way to access the scripts is simply be using a standard <script src=".."> tag that points to the correct place.

For example, to load Prototype version 1.6.0.2 you would place the following in your HTML:

PLAIN TEXT

HTML:

1.  
2. <script src="http://ajax.googleapis.com/ajax/libs/prototype/1.6.0.2/prototype.js"></script>
3.  

The second way to access the scripts is via the Google AJAX API Loader's google.load() method.

Here is an example using that technique to load and use jQuery for a simple search mashup:

PLAIN TEXT

HTML:

1.  
2. <script src="http://www.google.com/jsapi"></script>
3. <script>
4.   // Load jQuery
5.   google.load("jquery", "1");
6.  
7.   // on page load complete, fire off a jQuery json-p query
8.   // against Google web search
9.   google.setOnLoadCallback(function() {
10.     $.getJSON("http://ajax.googleapis.com/ajax/services/search/web?q=google&;v=1.0&;callback=?",
11.  
12.       // on search completion, process the results
13.       function (data) {
14.         if (data.responseDate.results &&
15.             data.responseDate.results.length>0) {
16.           renderResults(data.responseDate.results);
17.         }
18.       });
19.     });
20. </script>
21.  

You will notice that the version used was just "1". This is a smart versioning feature that allows your application to specify a desired version with as much precision as it needs. By dropping version fields, you end up wild carding a field. For instance, consider a set of versions: 1.9.1, 1.8.4, 1.8.2.

Specifying a version of "1.8.2" will select the obvious version. This is because a fully specified version was used. Specifying a version of "1.8" would select version 1.8.4 since this is the highest versioned release in the 1.8 branch. For much the same reason, a request for "1" will end up loading version 1.9.1.

Note, these versioning semantics work the same way when using google.load and when using direct script urls.

By default, the JavaScript that gets sent back by the loader will be minified, if there is a version supported. Thus, for the example above we would return the minified version of jQuery. If you specifically want the raw JavaScript itself, you can add the "uncompressed" parameter like so:

PLAIN TEXT

JAVASCRIPT:

1.  
2. google.load("jquery", "1.2", {uncompressed:true});
3.  

Today we are starting with the current versions of the library, but moving forward we will be archiving all versions from now onwards so you can be sure they are available.

For a full listing of the currently supported libraries, see the documentation.

Here I am, talking about what we are doing in two short slides:

The Future

This is just the beginning. We obviously want to add more libraries as you find them useful. Also, if you squint a little you can see how this can extend even further.

If we see good usage, we can work with browser vendors to automatically ship these libraries. Then, if they see the URLs that we use, they could auto load the libraries, even special JIT'd ones, from their local system. Thus, no network hit at all! Also, the browser could have the IP addresses for this service available, so they don't have the hit of a DNS lookup. Longer lived special browser caches for JavaScript libraries could also use these URLs.

The bottom line, and what I am really excited about, is what this could all mean for Web developers if this happens. We could be removed of the constant burden of having to re-download our standard libraries all the time. What other platform makes you do this?! Imagine if you had to download the JRE everytime you ran a Java app! If we can remove this burden, we can spend more time flushing out functionality that we need, and less time worrying about the actual download bits. I am all for lean, but there is more to life.

Acknowledgements

I want to acknowledge the other work that has been done here. Some libraries such as jQuery and Dean Edwards Base were already kind of doing this by hot linking to their Google Code project hosting repository. We thought this was great, but we wanted to make it more official, and open it up to libraries that don't use our project hosting facilities.

Also, AOL does a great job of hosting Dojo already. We recommend using them for your Dojo needs, but are proud to also offer the library. Choice is good. Finally, Yahoo! placed the YUI files on their own CDN for all to use.

Blogged with the Flock Browser

Enable code coloring in Dreamweaver for Drupal by adding file extensions recognized by Dreamweaver

Introduction

This TechNote applies to the following related symptoms in Adobe Dreamweaver:

• When opening a file, you see this error message: "Can't find a valid editor for this file extension."
• The color-coding in Code view isn't matching the color-coding for the file type you want.
• The server-model functionality isn't available for your file type.
• You cannot open a page in Design view. It only opens in Code view.
• When you go to File > Save, or File > Open, you don't see the file type you want.

Dreamweaver ships with the ability to edit and display many different file types. This TechNote focuses on customizing Dreamweaver to recognize new file types by editing the Extensions.txt and MMDocumentTypes.xml files, and setting the Open In Code View settings to see the new file types in Design view.

• The Extensions.txt file
  The Extensions.txt file is a text file listing all file types recognized by Dreamweaver. This file can be edited in any external text editor like Notepad or Text Editor to add to the list of recognized file types. Do not modify the Dreamweaver configuration files in Dreamweaver itself.
   
• The MMDocumentTypes.xml file
  The MMDocumentTypes.xml file contains definitions of document types as well as a list of file extensions associated with each recognized document type. Adding file extensions to the Extensions.txt file allows Dreamweaver merely to recognize a file extension and allow files of this type to be opened in Dreamweaver. Adding file extensions to the MMDocumentTypes.xml file tells Dreamweaver how to treat this new file extension and to which document type this file extension belongs.

  Note: For the purpose of this TechNote, only basic aspects of editing the MMDocumentTypes.xml file will be discussed. For a detailed overview of editing this file, see Extending Dreamweaver (from within Dreamweaver, choose Help > Extending Dreamweaver, and search on MMDocumentTypes.xml).

• Open in Code View
  The Open in Code View preference identifies file extensions that do not allow Design view. After following the instructions in this TechNote, be sure to remove the file extension (if it is listed) from the Open In Code View preference to allow file editing in Design view. To set this preference, choose Edit > Preferences (Windows) or Dreamweaver > Preferences (Mac OS), and locate the File Types / Editors category.

Editing the Extensions.txt file

When opening a file that Dreamweaver doesn't recognize, you see this error message: "Can't find a valid editor for this file extension." New extensions can be added to the list of extensions that Dreamweaver recognizes. To add new extensions, edit the Extensions.txt file. Entries in the Extensions.txt file also control the order in which files are listed in the File > Open and File > Save dialog boxes. For the purpose of demonstration, we will add the .inc file type to Dreamweaver.

Note: This procedure requires you to locate hidden files, hidden folders and extensions for known file types. By default, Windows Explorer doesn't show hidden files, hidden folders, and file name extensions that it recognizes.

To show hidden files, hidden folders, and extensions for known file types:

1. In Windows Explorer, do one of the following:
   • In Windows XP, choose Tools > Folder Options.
   • In Windows Vista, choose Organize > Folder and Search Options.
2. Click the View tab in the Folder Options dialog box.
3. In Advanced Settings, select Show Hidden Files And Folders.
4. Deselect Hide Extensions For Known File Types.
5. Click OK.

To add file types to the Extensions.txt file:

1. Locate the Extensions.txt file within the Dreamweaver user configuration folder. The location of this folder depends on the operating system and the version of Dreamweaver:
   • Dreamweaver CS3 on Windows Vista:
     C:\Users\[username]\AppData\Roaming\Adobe\Dreamweaver 9\Configuration
   • Dreamweaver 8 on Windows Vista:
     C:\Users\[username]\AppData\Roaming\Macromedia\Dreamweaver 8\Configuration
   • Dreamweaver CS3 on Windows XP:
     C:\Documents and Settings\[username]\Application Data\Adobe\Dreamweaver 9\Configuration
   • Dreamweaver 8 on Windows XP & 2000:
     C:\Documents and Settings\[username]\Application Data\Macromedia\Dreamweaver 8\Configuration
   • Dreamweaver CS3 on Mac OS X:
     Mac HD:Users:[username]:Library:Application Support:Adobe:Dreamweaver 9:Configuration
   • Dreamweaver 8 on Mac OS X:
     Mac HD:Users:[username]:Library:Application Support:Macromedia:Dreamweaver 8:Configuration
   Note: Edits to this file will only be available to the current user. Changes made to files within the user profile will not be available to other users. For more information about manually editing Dreamweaver configuration files in a multi-user environment, please see Customizing Dreamweaver.
   
2. Open Extensions.txt in any external text editor like Notepad or Text Editor. Do not modify the Dreamweaver configuration files in Dreamweaver itself.
   
3. Add the extension to the All Documents line. Extensions must be in uppercase letters and separated by commas. (INC should already be listed by default. If not, add it.)

   Example:  HTML,HTM,SHTM,SHTML,INC

4. To add extensions to an existing document group, add the extension to the extension list for the desired group. To add a new document group, add a new line that contains the new document type as well as a listing of associated extensions.

   Example:  INC:Include Files

   (This can also be added to the Server Side Include list already within the file.)

5. Save the file and restart Dreamweaver.
6. To see the changes, open Dreamweaver and open a file with the newly added file extension. It should now open in Code view only in Dreamweaver. Select File > Save As, and click the Save As Type pop-up menu. You should now see an option for Include Files with *.inc as the recognized file extension. It's the same for the Files Of Type pop-up menu in File > Open.

Note:  The order in which extensions are listed is the order in which they will appear in the File > Open and File > Save dialog boxes.

To get Design view, color coding, and server behavior functionality (if applicable) working, you need to edit the MMDocumentTypes.xml file, as described below.

The MMDocumentTypes.xml file

Where the Extensions.txt file contains the list of extensions recognized by Dreamweaver, the MMDocumentTypes.xml file contains information about the server model, color coding style, descriptions, and so forth for each known document type.

In addition to adding extensions to the Extension.txt file, new extensions must also be added to the file extension lists for the associated document type. If not, Dreamweaver will not know how to display this file type and will open it in Code view by default. If this extension is not associated with any defined document type, a new document type should be created. To create a new document type, see Extending Dreamweaver (from within Dreamweaver, choose Help > Extending Dreamweaver, and search on MMDocumentTypes.xml).

Note:  For the purpose of this TechNote, 'document type definition' refers to files that contain information about documents recognized by Dreamweaver and are not to be confused by 'DTDs' mentioned in XML documentation.

Steps to add extensions to a document type definition in the MMDocumentTypes.xml file:

1. Locate the MMDocumentTypes.xml file in the main Dreamweaver configuration folder (not the Dreamweaver user configuration folder). The location of this folder depends on the operating system and the version of Dreamweaver:
   • Dreamweaver CS3 on Windows Vista & XP:
     C:\Program Files\Adobe\Adobe Dreamweaver CS3\Configuration\DocumentTypes
   • Dreamweaver 8 on Windows Vista, XP & 2000:
     C:\Program Files\Macromedia\Dreamweaver 8\Configuration\DocumentTypes
   • Dreamweaver CS3 on Mac OS X:
     Mac HD:Applications:Adobe Dreamweaver CS3:Configuration:Document Types
   • Dreamweaver 8 on Mac OS X:
     Mac HD:Applications:Macromedia Dreamweaver 8:Configuration:Document Types
2. Open MMDocumentTypes.xml in any external text editor like Notepad or Text Editor. Do not modify the Dreamweaver configuration files in Dreamweaver itself.
3. Find the tag set for the desired document type. Below is the default document type definition for HTML documents from the MMDocumentTypes.xml file:
   
   <documenttype id="HTML" internaltype="HTML" winfileextension="html,htm,shtml,shtm,stm,tpl,lasso,xhtml" macfileextension="html,htm,shtml,shtm,tpl,lasso,xhtml,ssi" file="Default.html" writebyteordermark="false">
4. Add the desired extension to the  winfileextension  and  macfileextension  attributes of the  documenttype  tag. For example:
   winfileextension="html,htm,shtml,shtm,stm,tpl,lasso,xhtml,inc"
   

   The code above adds the INC file extension to the list of files that Dreamweaver treats as HTML files. Adding extensions to the Mac OS and Windows lists for this file types allows Dreamweaver to open this file type in Design view as other files with defined HTML file extensions.

5. Save MMDocumentTypes.xml and restart Dreamweaver.
6. To see the changes, open Dreamweaver and open a file that has the newly added file extension. You should see that Design view, code coloring and server behavior functionality (if applicable) are working now.

Note:  The order in which extensions are listed is important. The first extension in the extensions lists will be the default extension used for this file type.

Note:  If you made a backup of the MMDocumentTypes.xml and left it in the Configuration/DocumentTypes folder, you will see multiple warning messages pop up when you launch Dreamweaver that look like this:

'The Document Type "HTML" will not be added because it uses a file extension that is already associated with a prior Document Type.'

To stop these warning messages, move your backup copy of the MMDocumentTypes.xml file outside the Configuration/DocumentTypes folder. For example, move it to the desktop. You will also get this error message if you have the same file type listed more than once in the MMDocumentTypes.xml.

Edit in Code View preference

The final step is to check the File Types / Editors preference settings (Edit > Preferences) and make sure that this file extension is not listed in the Open In Code View field. By default, the INC file extension appears on this list. Remove ".inc" from the Open in Code View list and files with this extension should now be viewable in Design view.

Blogged with the Flock Browser

Fighting the Android: Nokia buys and open sources Symbian

Nokia announced plans today to transform the Symbian mobile phone operating system into an open source platform. Nokia, which already owns roughly half of all shares of Symbian, is in the process of obtaining the other half from various holders for €264 million. The company has partnered with several other major handset makers to launch the new nonprofit Symbian Foundation, which will facilitate the liberation of the platform.

Several core Symbian Foundation members are contributing their own Symbian-based technologies so that the best third-party enhancements can be converged into a single unified stack. This will include Nokia's own S60 platform, the UIQ graphical user interface layer which is jointly owned by Sony Ericsson and Motorola, and NTT DoCoMo's MOAP. These components will be available soon under a royalty-free license to all members of the foundation. Within two years, the platform will be completely open source and will be distributed under the Eclipse Public License (EPL), a moderately permissive license that requires downstream patent grants.

"Establishing the Foundation is one of the biggest contributions to an open community ever made," said Nokia CEO Olli-Pekka Kallasvuo in a statement. "Nokia is a strong supporter of open platforms and technologies as they give the freedom to build, maintain and evolve applications and services across device segments and offer by far the largest ecosystem, enabling rapid innovation. Today's announcement is a major milestone in our devices software strategy."

Symbian is currently the most widely-used smartphone platform, but its popularity and relevance have been declining as handset makers moved towards other platforms, including Windows Mobile and Apple's iPhone, as well as Linux and other open source alternatives that reduce licensing costs and offer more flexibility. The Symbian's business model and development strategy were far out of step with the direction in which the industry was collectively moving. Nokia's efforts to completely open the platform will change that and make Symbian a much stronger contender in the next-generation mobile space.

This could give Symbian a competitive boost against other open phone operating systems such as the one created by the LiMo Foundation and Google's Android platform which has reportedly run into a few snags with major carriers.
Fighting the Android: Nokia buys and open sources Symbian

Don’t forget to be part of Firefox’s Download Day!

Mark your calendar, wrap a ribbon around your finger or write a little sticky yellow note. Download Day will start on June 17, 2008.

Blogged with the Flock Browser

The Internet Star

Blogged with the Flock Browser

Design Stencils - Yahoo! Design Pattern Library

Download a Stencil KitYahoo! Design Stencil Kit version 1.0 is available for OmniGraffle, Visio (XML), Adobe Illustrator (PDF and SVG), and Adobe Photoshop (PNG), and covers the following topics: * Ad Units * Calendars * Carousels * Charts and Tables * UI Controls * Form Elements * Grids * Menus and Buttons * Mobile - General * Mobile - iPhone * Navigation and Pagination * OS Elements * Placeholder Text * Screen Resolutions * Tabs * Windows and ContainersDOWNLOADOmniGraffleVisio (XML)PDFPNGSVGCreative Commons LicenseThis work is licensed under a Creative Commons Attribution 2.5

Design Stencils - Yahoo! Design Pattern Library

Blogged with the Flock Browser

Webby Awards 2008 announced.

Webby Awards 2008 are announced. Visit the site to see the latest.

Blogged with the Flock Browser

developer destiny

Incredible singing skills

Reggie Watts: Out Of Control from Jakob Lodwick on Vimeo.

Adobe Integrated Runtime

Adobe Integrated Runtime (AIR), codenamed Apollo,[1] is a cross-operating system runtime environment for building rich Internet applications using Adobe Flash, Adobe Flex, HTML, and Ajax, that can be deployed as a desktop application.Adobe made a public preview release of AIR (then called Apollo) along with a software development kit and extension for developing Apollo applications with the Flex framework, on March 19, 2007. On June 10, 2007, Apollo was renamed to AIR and a public beta release of the runtime was launched. Public beta 2 of AIR SDK was released on October 1, 2007. Public beta 3, was released on December 12, 2007, and version 1.0 was released on February 25, 2008 (2008-02-25). [2][3]A Linux alpha version was released on March 31, 2008.[4]

Blogged with the Flock Browser

immigration crackdown hurts

Embedded Video

Amazing pictures from CERN

My first flock blog

Flock gives innumerable optional to be connected with internet. It really is web 2.0 browser. I recommend whether you are social networking crazy or news lover.

Blogged with the Flock Browser

YSlow!

Yslow is new exciting plugin for firfox which gives you details Why Your May be Slow.
I optimized my sites with it. For reference visit.

Usual Planning in Software Industry