Thursday, February 20, 2014

All About The New Google “Hummingbird” Algorithm

What’s a “search algorithm?”
That’s a technical term for what you can think of as a recipe that Google uses to sort through the billions of web pages and other information it has, in order to return what it believes are the best answers.

What’s “Hummingbird?”
It’s the name of the new search algorithm that Google is using, one that Google says should return better results.

Why is it called Hummingbird?
Google told us the name come from being “precise and fast.”




What type of “new” search activity does Hummingbird help?
“Conversational search” is one of the biggest examples Google gave. People, when speaking searches, may find it more useful to have a conversation.

“What’s the closest place to buy the iPhone 5s to my home?” A traditional search engine might focus on finding matches for words — finding a page that says “buy” and “iPhone 5s,” for example.

Hummingbird should better focus on the meaning behind the words. It may better understand the actual location of your home, if you’ve shared that with Google. It might understand that “place” means you want a brick-and-mortar store. It might get that “iPhone 5s” is a particular type of electronic device carried by certain stores. Knowing all these meanings may help Google go beyond just finding pages with matching words.

In particular, Google said that Hummingbird is paying more attention to each word in a query, ensuring that the whole query — the whole sentence or conversation or meaning — is taken into account, rather than particular words. The goal is that pages matching the meaning do better, rather than pages matching just a few words.

Wednesday, October 2, 2013

COMMON SEARCH ENGINE PRINCIPLES


To understand SEO, we should have knowledge about ARCHITECTURE of search engines.
Note: As we have various search engines, each search engine have its own principle components.
Here are the main components of Google search engine:
  • Spider
  • Crawler
  • Indexer
  • Data Base
  • Results Engine
  • Web Server
Spider also known as Search Engine Robots.
  • This is a program , it downloads web pages (just like a web browser) .
  • The difference is that the browser displays the information presented on each page(text, graphics, etc) while a spider doesn’t.
  • A Spider works directly with the underlining HTML code of the page.
Crawler
  • This is a program, finds all the links on each page. Its task is to determine where the Spider should go either by evaluating the links (or) according a pre-defined list of addresses(Home Page, Contact page).
  • The Crawler follows these links and tries to find documents not already known to search engines.
In SHORT,
  • Crawler finds the links.
  • Crawler guides Spider to follow the links.
  • Mainly Crawler finds the documents which are New (unknown things).
Note: Crawler reads from left to right, top to bottom.
Indexer
  • This program analysis each page and analysis various elements such as text, headers, structure of stylistic features, special HTML tags (like all HTML tags)etc.
Data Base
  • This is a storage area for the data that the search engine downloads and analysis. Sometimes it s called the “Index of the Search Engine”.
Results Engine
  • Results Engines, ranks pages which it determines which pages best match a users query and in what order the pages should be listed. This is done according to the ranking algorithm of search engines.
  • It follows the Page Rank, which is a valuable and interesting property and any SEO specialist is most interested in it when trying to improve his site search results.
Web Server
  • The search engine Web Server usually contains a HTML page “with a input field” where the users can specify the search query he/she interested in.
  • A Web Server is also responsible in displaying search results to the users in the form of a HTML page.
Note: More page rank indicates trusted and user friendly website.


Static Analysis

Basic SEO Factors for Instant Analysis:
  • Domain Extension
  • Page Loading Time
  • Page Rank
  • Alexa Rank
  • Google, Bing & Yahoo Indexed Pages and Back Links
  • Domain Age
  • Canonical Redirection
  • Title & Meta Tags
  • Header Tags
  • Image Tags
  • Xml sitemap
  • Html sitemap
  • Robots.txt
  • Google analytics account
  • Google webmaster tools account

SEO Process Tips to Rank

white hat SEO tips to follow according to the latest search engine algorithms in 2011.
  1. Research and pick the right keywords which drive quality traffic, leads and conversions
  2. Analyze competitors who are running with success in similar business and checkout the methods they are following and the keywords they are optimizing
  3. Pick the right Domain extension based geo – targeted business like ‘.in’ for India, ‘.com’ for USA, ‘.co.uk’ for UK etc.
  4. Plan a user& search engine friendly design & navigation with good link architecture
  5. Hire a professional content writer to write good quality content to catch user as well as search engine attention
  6. Plan different sub sections with relevant content for most targeted keywords
  7. Make sure that site has no broken links, timed out urls, high loading time etc. Tools like Xenu and extensions like Pagespeed, Yslow will help us to develop technically clean site.
  8. Use robots.txt file to block all unnecessary pages which are indexing by the search engines.
  9. Create Xml & HTML sitemap for users and search engines
  10. Submit Xml sitemap to search engines webmaster tools account and fix the errors if any
  11. Build only quality back links using White hat link building methods.
  12. Analyze traffic and plan or change the strategies to drive quality traffic which will convert to sales

Tuesday, September 17, 2013

SEO Glossary


A
Alt Attribute: Used in images, this HTML tag provides alternative text or description of the image.

Alt Tags:Used in images, this HTML tag provides the image name or description that you see if you hover your mouse over the image. This is often used to make your page more accessible to individuals with disabilities such as people who may have a vision impairment and use a device that reads the contents of a webpage to them.

Anchor Text: Anchor text is the text portion of a link that you click on when an address isn’t displayed.Google uses anchor text to determine what keywords your site may focus on, and using appropriate anchor text is an important search engine optimization technique.

Authority: A site that is deemed as an “authority” site will generally have better search engine result placement and page-rank.Authority is primarily determined by relevant content and incoming links.

Average Page Views: The amount of pages within a site a user visits within one session (one session is equal to one visit, meaning if the user leaves and returns later this counts as two sessions)

Average Time on Site:The average amount of time a user spends on your website.



B

Backlinks: Backlinks are links from other sites that point to your site.

Bait & Switch: This is a technique used by black hat SEO’s and involves displaying one web page to search engines and a completely different page for other user agents at the same URL.It generally creates an optimized page targeting specific keywords and submits this page to a search engine or directory, but replaces that page with the regular page as soon as the optimized page has been indexed.

Banned: If your website has been penalized and removed from the search engine results, it is considered to be banned.

Black Hat SEO: This is a term used in the type of SEO you want to avoid and is against search engine guidelines.Using Black Hat techniques can get your site banned from Google – and chances are you won’t be able to get it back into search results.
Following are Black Hat tactics to avoid:
  • Hidden text or Hidden links – Links or text which is the same color as the background of your page.
  • Artificially increasing the number of links to your site: Sites which hold are wide repository of links are considered by Google as “link farms” and receiving a wide number of incoming links from “link farms” will have a negative effect on your site.
  • Duplicated content:Content copied from other sites and used on yours.
  • Excessive pop ups
  • Spamming other websites or forums to place your link on their site.
  • Keyword stuffing:Using a keyword so many times that the context makes little to no sense.
  • Cross linking: An excessive amount of cross linking with sites to increase your websites popularity.
Blacklist: Black lists are created by organizations to create a database of websites, ip addresses or users who are known for black hat tactics, hacking attempts or other shady practices.

Bot: Another name for Agent, Crawler or Spider

C
Cache: Copies of your web pages that are stored locally (on a visitors computer).This helps web pages to load quicker as it will show the “cached” version of the webpage

Canonical URL: Choosing a URL structure that will be your primary structure, and notifying search engines to ignore all others.Not having a proper canonical URL setup results in duplicate content penalties, as search engines will view all URL versions of a webpage as different websites.Examples would be:
http://abc.com
http://www.abc.com
www.abc.com
abc.com

Cannibalization: This is often referred to as either “Keyword Cannibalization” or “SEO Cannibalization”.This occurs when certain pages of your website compete against one another for certain keywords.

Cloaking: Cloaking is another illegal technique, which partially involves content separation because spiders see one page (highly-optimized, of course), and everybody else is presented with another version of the same page.

Conversion Rate:The percentage of your website visitors which convert – or perform a “goal” or intended action.This could be purchases, newsletter sign-ups, free trial downloads, etc.

Cookie: A small piece of code, or information that is stored on a users computer.This is generally used for websites to remember who you are, such as storing passwords, email addresses, etc.It can also be used to track a visitors navigation through the site and how many times they have visited.

Crawler: Same as a spider, Bot or Agent

Custom error pages: This generally refers to customizing your 404 page, or page that a user sees when the content they were searching for could not be found.

D

Directory: A website that is a directory of other websites.These can often be grouped into specific categories, or the directory itself could focus on a specific niche or category.

Domain Trust: Domain trust is calculated by various factors such as who you link to (reputable sites, spammy sites, etc) who links to you, your domain age and your domain registration information.

Domain Authority: The number of root domains which link to different pages on your website (rather than getting all links pointing directly to your homepage).

Doorway pages: This is also known as a “bridge page” or a “gateway” page.This is basically a webpage created with a lot of keyword rich content without much useful information.The aim is to trick spiders that your site is a highly-relevant one when it is not, is another way to get the kick from search engines.

Dynamic:Content or URL’s that are generated on the fly from a database



E
Error page: A web page that cannot be displayed due to an error, such as a 404 or “file not found” error.
F
Flash: Allows webmasters to display interactive media on webpages. This is often seen in website intros, games and animated navigation.The use of flash is generally not a recommended SEO practice, although search engines have made recent developments to now read flash.

Frames: Frames combine two web pages into one, allowing you to display content from one webpage within another.They may each have their own scrollbar, or part of the page my scroll while the rest of the page does not move. Search engines don’t like frames, as the content within the frame is difficult to crawl, though they do support them.You can also use a <noframes> tag in HTML to help crawlers index the page normally.

Funnel: A path which visitors follow before arriving at a “goal” page or completing a specific action.In Google Analytics you can create funnels which have up to 10 pages



G
Goal Abandonment Rate:The percentage of users who exited your website or your funnel before reaching your goal page.

Google Bombing: Google Bombing, or a Google Bomb refers to when a lot of webmasters will get together and agree to link to a specific website using an unflattering term.This takes advantage of the importance of anchor text within a link, and will generally cause a page to rise in search results for an unflattering term. An example of this is when George Bush’s White House bio was displayed when anyone Googled the term “miserable failure”.This has since been changed, and it has become harder to Google Bomb a website.

Google Bowling: Google bowling is a black hat technique used to knock pages out of the search engine results, or significantly lower them.People sabotage websites this way by pointing hundreds of low quality links to a competitor’s site.Newer sites are generally more susceptible to this as older sites are better established with a range of high quality links.

Google Dance: This refers a time when Google’s indexes are being updated.During this time, you may notice drastic differences in the rankings of your site and/or keywords.

Googlebot: A search bot used by Google.

Grey Hat SEO:SEO that users a combination of both white hat and black hat tactics.

H
Heading Tag: This can also be referred to as {H1, H2, etc}.This is an HTML reference that explains the section on a web page.Titles are usually placed in heading tags.The text placed within these tags are often considered slightly more important to the content on the page than normal text would be.

Hidden Keywords:Keywords which are placed on a webpage and are hidden to visitors, but viewable by spiders.

Hidden Text: The practice of hiding text on a webpage so that spiders can see and read it, but users cannot.The reason this is done is to create more content and keywords for a page.This however is a black hat tactic and can harm your rankings.

HTML: This stands for “Hyper Text Markup Language” and this is a programming language.

HTML Source: Raw programming code.You can view this by right clicking on a page and choosing “view source”

HTTP:Hyper Text Transfer Protocol.This is the protocol used when a web browser communicates with a web server to request and retrieve data.

HTTPS: A secure Hyper Text Transfer Protocol

Hyperlinks: Links that you click on within a webpage to be brought to another page or website.

I
Inbound Links: Links that point to your website from other websites.

Index: A search engines database that stores information about websites.

Internal Links: This is a link that points to another page on the same site.



K
Key Performance Indicators (KPI’s): Helps organizations achieve goals through defining the measurement and progress of a campaign.

Key Phrase (Keyword Phrase): A search made up of keywords

Keyword: A single word that a search engine user would search for in hopes of bringing up relevant useful website results for that term.Your website should target Keywords and Keyword Phrases that searchers will use to find you.

Keyword Density: The no. of times a keyword phrase(N) in a web page mutilply by no. of keywords in that phrase(M) divided by total word count of a web page.
                         Keyword Density = N*M/Total Word Count

Keyword Proximity measures how close in the text the keywords are

Keyword Prominence:Refers to the fact that the keywords placed in important parts of a webpage are given priority by the search engines


Keyword Stuffing: The act of including an excessive amount of keywords in your page text and HTML.If the keywords are listed so often that it affects the readability of your site, this can have a negative effect on your search results.And is affected by Google Penguin

L
Link Bait: Link Bait is designed specifically to gain attention or encourage others to link to the website.

Link Building: The process of requesting links from other webmasters who own sites that are relevant to yours.This includes directory submissions, blog commenting, press releases.. etc.Sites also offer “paid links” promising to do the link building for you, which is not an advisable practice to participate in.

Link Farm: A group of websites that are all interlinked with the purpose of inflating their link popularity.This is a form of spam.

Link Popularity: How many other websites have linked to your website.The more pages linking to you, the better.

Linkwheel: A group or “hub” of websites that link to one another in a circular fashion.

LSI stands for Latent Semantic Indexing. Latent Semantic Indexing keywords can make your content look more natural to search engines and users. Instead of using the exact phrase repeatedly throughout your text, you can use synonyms and other terms that are related to your content and have the same meaning with your target key phrase.

M
Meta Data: The HTML area toward the “head” of a site that displays various Meta Tags

Meta Description: Meta Descriptions how search engines list your site in engines. Generally not an influence in ranking on most engines, it still gets shown as text to support a listing in natural search and does influence click through rate and relevance.

Meta Keywords: Meta Keywords – List of keywords used to indicate what your web page is about.Used in SEO but no longer being used by Google to categorise web page content.

Meta Tags: HTML tags of information associated with a website, such as title, description, keywords and author.

Microblogging:Microblogging is providing group of people with news/information using a very short amount of text/words to describe it.Sites like Twitter are considered microblogging sites.

O
Off Page Optimization: The area of search engine optimization that applies to building backlinks (link building)

On Page Optimization: The area of search engine optimization that applies to the content and structure of the website

Outbound Links:
Links that are directed to another website from your website.

P
Page Rank (PR): Page rank is a score Google gives a web page based on how trustworthy and relevant the website appears to be.The main factor of page rank is your link popularity.Google consider many factors into account when judging your links and gives some links a higher score while other links may negatively affect your score.
The scale ranges from 0-10 with 0 being the lowest and 10 being the highest. Each rank is harder to achieve the higher you get
There is an ongoing debate over the importance of Page Rank to SEO.

Portal: A separate site used to function as an access point to another website.These can be authoritive hubs that link to a website.
Q
Query: A keyword or phrase typed into a search engine to return results.

Query String: A portion of a URL which often appears after a question mark( ? )

QDF: “Query Deserves Freshness”, sometimes also referred to as “Quality Deserves Freshness”.For example, if a blog post is created about a specific trending topic, that post may be added to the search index quicker as it directly relates to the current trend.

R
Reciprocal Linking: Trading links between websites.

Redirect: Redirecting one site to another.This happens automatically without the need of user to click on anything.There are two main types of redirects, the 301 which lets the search engines know your site has been permanently moved to another, and the 302 which tells search engines the website has been “temporarily moved”.

Referrer: If a user finds your website by clicking on a link provided on another website, the website they arrived from would be considered the referrer.

Relevance: How relevant your website is to a search term entered, and the likelihood of your website appearing in results for that term.

Robot: Same as Googlebot, crawler or spider.

ROI:Return on investment.The benefit gained from investing budget into advertising, SEO or Social Media.Total revenues generated from the campaign minus total costs.
S
Search Engine Results Page (SERP): The search results provided after you type a query, keyword or phrase.

SEM: Search Engine Marketing. Involves strategies to increase the number and quality of leads generated by search engines.

SEO:Search engine optimization.The practice of optimizing your website for better placement in search engine results. Involves three steps including technical optimization, on-page optimization and off-page optimization.

SMO: Social Media Optimization:Strategy and methods for social media marketing.

Spider: Also known as a bot, robot, crawler or agent. These are programs used by search engines to explore the web and your website and retrieve content about what that site is about.

T
Theme: The main focus of your website or webpage.

Title Tag:The text displayed at the very top of your browser window, above the “back” “forward” and “refresh” buttons.If you are using Google Chrome you may not see this information.
A very important line of text for search engine optimization.The words in this text are given more weight, and this text is generally what is displayed in search engine results.

U
Usability: How friendly a website is to visitors, and the ease of use in which they can find their way around your site, or perform an action or task.

USG (User Generated Content): Content created and published (or added to) by the end user.This consists of videos and podcasts as well as sites like forums, wiki’s, blogs and social media sites.

V
Visibility: How well placed your website is in search engine results.

W
Web 2.0: Refers to the new generation of web-based services and communities that are generally characterized by participation, collaboration and sharing of information.

Web 2.0 applications include wiki’s, folksonomies, blogs, and other social networking sites.
White Hat SEO: Ethical SEO that follows search engine guidelines.

X
XML: Extensible Markup Language – this is a scripting language that allows a programmer to define the properties of a document.
XML Sitemap: A sitemap generated in Google’s required XML format.A sitemap is a map to all pages on your sites.



Friday, February 8, 2013

What is SEM?

If you have a website, you need to make sure your website is found!

It’s not enough to rely on your offline marketing to drive traffic to your website. Audiences often forget website addresses or have forgotten about your marketing campaign by the time they actually need it! Online users want quick and relevant answers to their search queries. Search engines are focused on driving and delivering the most relevant search results to their users.
SEM (Search engine marketing) involves a range of marketing techniques required to maximize a visibility of the website on search engines so that it will increase visitor traffic. This typically includes:
-> Search Engine Optimization (SEO) of your website
->Pay per Click Keyword Advertising (PPC)

The key benefit of SEM for your business is the opportunity to put yourself in front of a prospect that's already looking for what you have to sell. It's not low hanging fruit, it's in the basket waiting for you. All you have to do is be there to pick it up.

What is SEO?
SEO is a fundamental part of search engine marketing (SEM). It requires constant updating and continuous attention of the website content for solid, long-term effective results. 'Natural' Search Engine Optimization (SEO) is the practice of guiding the design and development of a website so that it will naturally attract visitors by winning top rankings on the major search engines for selected search key word phrases.

It is common that search engine rankings go up and down all the time. Many webmasters are unsure what they have to do to keep their web sites at the top of the search engine result pages. There are two main factors that influence the rankings of your web pages. This typically includes:

-> On Page Optimization
-> Off Page Optimization

What is on page Optimization?
The term On Page Optimization refers to just that: optimization that is done directly on a specific webpage to achieve better search engine rankings. This could mean on the visible parts of the page (what the viewer or customer can see) or in the code or invisible parts of the page. This is all done directly on the .html ‘file’ or ‘page’ which is the reason for the name On Page Optimization.
(OR)
On-page optimization refers to factors that have an effect on your Web site or Web page listing in natural search results. These factors are controlled by you or by coding on your page. Examples of on-page optimization include actual HTML code, meta tags, keyword placement and keyword density,etc.

What is off Page Optimization?
Off-Page optimization refers to the ranking factors that are usually out of your control. The most important off-Page factor is the links to your web site. Link Buildings are part of Off Page Optimization.Search engines use these two main factors to determine the rankings of your web pages. SEO Process is a combination of on page and off page optimization in which off page optimization is most crucial to rank a website on major search engines. It helps to achieve, maintain and improve the ranking by chasing down major competitive keywords. This is why most of the companies go for off page optimization.

What is Link Building?
In terms of traffic building, organic search engine rankings and internet sales, link building is one of the most important aspects of search engine optimization. Link popularity or building simply refers to the mechanism of securing text links pointed at your website and strategic landing pages in order to increase rankings. This is often referred to as Off Page Optimization. Currently, link building - developing quality relevant one way links (also known as backlinks or inbound links) - to your website is one of the most important factors in SEO and achieving high rankings in the search engines.

The different types of links are, but not limited to:

->One way link building - Convincing any given website to link back to you by various methods are including Directory Submission, Article writing and content distribution and publication and Web 2.0 - social bookmarking and blog / forum links.

->Two way link building (Reciprocal linking) - you link to me from a page in your website, and I will link back.

->Three way link building - Three Way Link Building helps you build links while avoid reciprocal link building and other types of penalties resulting from it.
They way it works is that you give Website (A) a link to Website (B).
Website (B) gives a link to Website (C).
Website (C) then gives a link to Website (A).
In the end, (C) websites have links to each other, and yet none of them are reciprocal. Three waylinks are a subset of reciprocal links.

->Paid links - links secured on any given webpage by monthly or yearly payment.

What is PPC?
Pay-Per-Click (PPC) is a search engine marketing technique that requires you to pay a fee every time someone clicks to your website from an advert you have placed in a search engine’s results. Therefore, you only pay each time someone visits your site.
(OR)
Pay-Per-Click (PPC) is a performance-based online marketing model in which businesses bid for enhanced placement in search results on terms that are relevant to their business, and pay on a per-click basis. You set the price you're willing to pay for each sales lead and pay only when customers click through to your site, leading to a high return on your investment (ROI).

Cost per click (CPC) is the amount of money an advertiser pays search engines and other Internet publishers for a single click on its advertisement that brings one visitor to its website.

Adwords: Name of the Google pay per click program. The Adwords advertisements appear in the top two positions above the natural listings and in boxes on the right side of the page. Google Adwords are visible in Google as well as their partner sites including AOL and AskJeeves. If “content match” is activated the ads will also show up next to magazine articles and other pages throughout the web.

Adsense: An advertising and money-making program run by Google where website owners can show Google ads on their site and generate revenue based on the Pay Per Click model. The website owner gets a percentage of the PPC revenue generated by clicks from their site

SEO Services

Wednesday, January 2, 2013

Focus on Target Audience to Improve SEO Efforts


If you want to perform well in the search engines then you need to stop focusing on pleasing the search engines. That doesn’t seem to make much sense does it? Let’s explain.

The search engines want to provide users with the best possible results. If the results aren’t relevant to a search, don’t meet a user’s needs, or are filled with low quality sites the user isn’t going to have a very good experience. The primary revenue source of the search engines comes from visitors clicking on paid listings in the Sponsored section. If search engine users aren’t getting good results, they will find alternate ways to find what they are looking for, and won’t be clicking on ads meaning that the search engines won’t be generating revenue. Like any other business, the search engines aim to please their customers, or users. For this reason, the search engines (more specifically- Google) are changing their algorithms to attempt to “think” more like a human. SEO is no longer about pleasing search engine robots, it’s about pleasing actual people.

Think about all of the shifts that have taken place in the SEO industry even just within the last few years. One of the first major changes was the Google Panda update. The targets of the Panda update were websites that produced what was considered to be low quality content. Website owners knew that in order to get recognized by the search engines they had to produce content across the web that included targeted keywords and links. In many cases, how an actual visitor viewed the content was an after thought. Content distribution was done mostly for link building purposes. Quantity was the focus over quality. The recent Google Penguin update was a reminder that Google is serious about cleaning up and cleaning out the spam that lives amid its search results. Penguin went after sites that were guilty of too much SEO, or “over optimization”. For a human visitor, there is no need to include 5 keyword anchor text links in a 500 word article or launch keyword rich domain micro sites. That kind of behavior is clearly meant for the search engines only, and the search engines have made their stance loud and clear- stop doing it already!

Another example of how the search engines are focusing more on the user experience to rank websites is the emphasis on social search. People want suggestions from other people. If certain content performs well in social media, it must be of good quality. The search engines no longer have to assume using backlink data what kind of content people want to see. The evidence is right there within the re-tweets, shares, likes, comments, etc.

The bottom line is to stop thinking so much about the search engine spiders. Think about your target audience members. What kind of content would they like to see? This doesn’t mean that SEO best practice is dead. It’s still important to conduct keyword research, optimize your website, and build inbound links but the focus has shifted. Write content that appeals to your target audience and build links from places that they might actually visit. Keep it natural and don’t over optimize anything. By focusing on your target audience you have the best chance of succeeding in the search engines and online in general for the long term.

Tuesday, December 18, 2012

How Search Engine Works ?

The first basic truth you need to know to learn SEO is that search engines are not humans. While this might be obvious for everybody, the differences between how humans and search engines view web pages aren't. Unlike humans, search engines are text-driven. Although technology advances rapidly, search engines are far from intelligent creatures that can feel the beauty of a cool design or enjoy the sounds and movement in movies. Instead, search engines crawl the Web, looking at particular site items (mainly text) to get an idea what a site is about. This brief explanation is not the most precise because as we will see next, search engines perform several activities in order to deliver search results – crawling, indexing, processing, calculating relevancy, and retrieving.

First, search engines crawl the Web to see what is there. This task is performed by a piece of software, called a crawler or a spider (or Googlebot, as is the case with Google). Spiders follow links from one page to another and index everything they find on their way. Having in mind the number of pages on the Web (over 20 billion), it is impossible for a spider to visit a site daily just to see if a new page has appeared or if an existing page has been modified, sometimes crawlers may not end up visiting your site for a month or two.

What you can do is to check what a crawler sees from your site. As already mentioned, crawlers are not humans and they do not see images, Flash movies, JavaScript, frames, password-protected pages and directories, so if you have tons of these on your site, you'd better run the Spider Simulator below to see if these goodies are viewable by the spider. If they are not viewable, they will not be spidered, not indexed, not processed, etc. - in a word they will be non-existent for search engines.

After a page is crawled, the next step is to index its content. The indexed page is stored in a giant database, from where it can later be retrieved. Essentially, the process of indexing is identifying the words and expressions that best describe the page and assigning the page to particular keywords. For a human it will not be possible to process such amounts of information but generally search engines deal just fine with this task. Sometimes they might not get the meaning of a page right but if you help them by optimizing it, it will be easier for them to classify your pages correctly and for you – to get higher rankings.

When a search request comes, the search engine processes it – i.e. it compares the search string in the search request with the indexed pages in the database. Since it is likely that more than one page (practically it is millions of pages) contains the search string, the search engine starts calculating the relevancy of each of the pages in its index with the search string.

There are various algorithms to calculate relevancy. Each of these algorithms has different relative weights for common factors like keyword density, links, or meta tags. That is why different search engines give different search results pages for the same search string. What is more, it is a known fact that all major search engines, like Yahoo!, Google, Bing, etc. periodically change their algorithms and if you want to keep at the top, you also need to adapt your pages to the latest changes. This is one reason (the other is your competitors) to devote permanent efforts to SEO, if you'd like to be at the top.

The last step in search engines' activity is retrieving the results. Basically, it is nothing more than simply displaying them in the browser – i.e. the endless pages of search results that are sorted from the most relevant to the least relevant sites.