Search Engine Optimization
S.E.O. - General information
Search engine optimization (SEO) as a subset of search
engine marketing seeks to improve the number and quality of visitors to a web site from "natural" ("organic" or
"algorithmic") search results. The quality of
visitor traffic can be measured by how often a visitor using a specific keyword
leads to a desired conversion action, such as making a purchase or
requesting further information. In effect, SEO is marketing by appealing first
to machine algorithms to increase
search engine relevance
and secondly to human visitors. The term SEO can also refer to "search engine
optimizers", an industry of consultants who carry out optimization projects on
behalf of clients.
Search engine optimization is available as a stand-alone service or as a part
of a larger marketing campaign. Because SEO often requires making changes to the
source code of a site, it is often most effective when incorporated into the
initial development and design of a site, leading to the use of the term "Search
Engine Friendly" to describe designs, menus, Content
management systems and shopping carts that can be optimized easily and
effectively.
A range of strategies and techniques are employed in SEO, including changes
to a site's code (referred to as "on page factors") and getting links from other
sites (referred to as "off page factors"). These techniques include two broad
categories: techniques that search engines recommend as part of good design, and
those techniques that search engines do not approve of and attempt to minimize
the effect of, referred to as spamdexing. Some industry commentators classify
these methods, and the practitioners who utilize them, as either "white hat SEO", or "black hat SEO".[1] Other SEOs
reject the black and white hat dichotomy as an over-simplification.
History
Origin: Early search engines
Webmasters and content providers began optimizing sites for search engines in
the mid-1990s, as the first search engines
were cataloging the early Web. Initially, all a webmaster needed to do was submit a site to the
various engines which would run spiders, programs that "crawled" a page and stored
the collected data in a database.
By 1996, SEO related email spam was commonplace.[2][3] The earliest known use of
the phrase "search engine optimization" was a spam posted on Usenet on July 26, 1997.[4]
The process involves a search engine spider downloading a page and storing it
on the search engine's own server, where a second program, known as an indexer,
extracts various information about the page, such as the words it contains and
where these are located, as well as any weight for specific words, as well as
any and all links the page contains, which are then placed into a scheduler for
crawling at a later date.
At first, search engines were supplied with information about pages by the
webmasters themselves. Early versions of search algorithms relied on webmaster-provided information
such as the keyword meta tag, or
index files in engines like ALIWEB.
Meta-tags provided a guide to each page's content. But indexing pages based upon
meta data was found to be less than reliable, mostly because webmasters abused
meta tags by including keywords that had nothing to do with the content of their
pages, to artificially increase page impressions for their Website and increase
their Ad Revenue. Cost Per Impression was at the time the
common means of monetizing content websites. Inaccurate, incomplete, and
inconsistent meta data in meta tags caused pages to rank for irrelevant
searches, and fail to rank for relevant searches. [5] Search engines responded by
developing more complex ranking algorithms, taking into account additional factors
including:
Today the only major search engine that says it considers meta keywords in
its ranking algorithms is Yahoo, though most experts feel that even there the
attention paid to meta keywords is minimal.[citation needed] Explicit facts
about the effectives of meta keywords are, however, not readily known, because
of the secrecy used during the ranking of algorithms by the search engines. One
could therefore recommend the use of meta keywords in webpages, and limit them
to 20 keywords or less.[citation needed] For example,
the source code of this page shows that Wikipedia uses meta keywords. The
"description" tag is, however, claimed by most SEO-experts to be more important.
One could therefore say to use both meta tags in webpages.
Web content providers also manipulated a number of attributes within the HTML
source of a page in an attempt to rank well in search engines.[6]
By relying extensively on factors that were still within the webmasters'
exclusive control, search engines continued to suffer from abuse and ranking
manipulation. In order to provide better results to their users, search engines
had to adapt to ensure their SERPs showed
the most relevant search results, rather than useless pages stuffed with
numerous keywords by unscrupulous webmasters using a bait-and-switch lure to
display unrelated web pages. This led to the rise of a new kind of search
engine.
More sophisticated ranking algorithms
Google brought a new concept to
evaluating web pages. This concept, called PageRank, has been important to the Google algorithm
from the start.[7] PageRank is an algorithm that weights
a page's importance based upon the incoming links. PageRank estimates the likelihood that a given
page will be reached by a web user who randomly surfed the web, and followed
links from one page to another. In effect, this means that some links are more
valuable than others, as a higher PageRank page is more likely to be reached by
the random surfer.
The PageRank algorithm proved very effective, and Google began to be
perceived as serving the most relevant search results. On the back of strong
word of mouth from programmers, Google became a popular search engine. Off-page
factors weighted more heavily than on-page factors as Google identifed the
manipulation of off-page to be more difficult.
Despite being difficult to game, webmasters had already developed link
building tools and schemes to influence the Inktomi search engine, and these methods proved
similarly applicable to gaining PageRank. Many sites focused on exchanging,
buying, and selling links, often on a massive scale.
Inktomi, an earlier search engine
using similar off-page factors, had forced webmasters to develop link building
tools and schemes to infuence searches; these same tools proved applicable to
Google's PageRank system. Thus an online industry spawned focused on selling
links designed to improve PageRank and link popularity. To drive human site
visitors, links from higher PageRank pages sell for more money.
A proxy for the PageRank metric is still displayed in the Google Toolbar, though
the displayed value is rounded to the nearest integer, and the toolbar is
believed to be updated less frequently than the value used internally by Google.
In 2002 a Google spokesperson stated that PageRank is only one of more than 100
algorithms used in ranking pages, and that while the PageRank toolbar is
interesting for users and webmasters, "the value to search engine optimization
professionals is limited" because the value is only an approximation.[8] Many experienced SEOs recommend
ignoring the displayed PageRank.[9]
Google — and other search engines — have, over the years, developed a wider
range of off-site factors they use in their algorithms. The Internet was
reaching a vast population of non-technical users who were often unable to use
advanced querying techniques to reach the information they were seeking and the
sheer volume and complexity of the indexed data was vastly different from that
of the early days. Combined with increases in processing power, search engines
have begun to develop predictive, semantic, linguistic and heuristic algorithms. Around the same time as the
work that led to Google, IBM had begun work on
the Clever Project [10], and Jon Kleinberg was developing the HITS algorithm.
As a search engine may use hundreds of factors in ranking the listings on its
SERPs; the factors themselves and the weight each carries can change
continually, and algorithms can differ widely, with a web page that ranks #1 in
a particular search engine possibly ranking #200 in another search engine, or
even on the same search engine a few days later.
Google, Yahoo, Microsoft and Ask.com do not disclose the algorithms they use to rank
pages. Some SEOs have carried out controlled experiments to gauge the effects of
different approaches to search optimization. Based on these experiments, often
shared through online forums and blogs, professional SEOs attempt to form a
consensus on what methods work best, although consensus is rarely, if ever,
actually reached.
SEOs widely agree that the signals that influence a page's rankings
include:[11]
- Keywords in the title tag.
- Keywords in links pointing to the page.
- Keywords appearing in visible text.
- Link popularity.
- (PageRank for Google) of the page.
- Keywords in Heading Tag H1,H2 and H3 Tags in webpage.
- Linking from one page to inner pages.
- Placing punch line at the top of page.
There are many other signals that may affect a page's ranking, indicated in a
number of patents held by various search engines, such as historical data[12].
More than Just Rankings
Search engine optimization often involves more than just rankings. By
improving the quality of a page's search listings, more users will select that
page. Factors that may improve search listing quality include: an
attention-grabbing title,[citation needed] an interesting
description,[13] and a domain and URL that reinforce the
legitimacy of the site. Some commentators have noted that domains with lots of
hyphens look spammy and may discourage click throughs.[citation needed]
Relationship between SEO and search engines
The first mentions of Search Engine Optimization do not appear on Usenet
until 1997, a few years after the launch of the first Internet search engines.
The operators of search engines recognized quickly that some people from the
webmaster community were making efforts to rank well in their search engines,
and even manipulating the page rankings in search results. In some early search
engines, such as Infoseek, ranking
first was as easy as grabbing the source code of the top-ranked page, placing it
on your website, and submitting a URL to instantly index and rank that page.
Due to the high value and targeting of search results, there is potential for
an adversarial relationship between search engines and SEOs. In 2005, an annual
conference named AirWeb[14] was created to discuss bridging the gap and
minimizing the sometimes damaging effects of aggressive web content
providers.
Some more aggressive site owners and SEOs generate automated sites or employ
techniques that eventually get domains banned from the search engines. Many
search engine optimization companies, which sell services, employ long-term,
low-risk strategies, and most SEO firms that do employ high-risk strategies do
so on their own affiliate, lead-generation, or content sites, instead of risking
client websites.
Some SEO companies employ aggressive techniques that get their client
websites banned from the search results. The Wall Street Journal profiled a company that
allegedly used high-risk techniques and failed to disclose those risks to its
clients.[15] Wired reported the same company sued a blogger
for mentioning that they were banned.[16]
Google's Matt Cutts later
confirmed that Google did in fact ban Traffic Power and some of its clients.[17]
Some search engines have also reached out to the SEO industry, and are
frequent sponsors and guests at SEO conferences and seminars. In fact, with the
advent of paid inclusion, some
search engines now have a vested interest in the health of the optimization
community. All of the main search engines provide information/guidelines to help
with site optimization: Google's, Yahoo!'s, MSN's and Ask.com's. Google has a
Sitemaps program [18] to help webmasters learn if Google
is having any problems indexing their website and also provides data on Google
traffic to the website. Yahoo! has Site Explorer that
provides a way to submit your URLs for free (like MSN/Google), determine how
many pages are in the Yahoo! index and drill down on inlinks to deep pages.
Yahoo! has an Ambassador Program[19] and Google has a program for
qualifying Google Advertising Professionals[20].
Getting into search engines' databases
Today's major search engines, by and large, do not require any extra effort
to submit to, as they are capable of finding pages via links on other sites.
However, Google and Yahoo offer submission programs, such as Google Sitemaps,
for which an XML type feed can be created and submitted. Generally, however, a
simple link from a site already indexed will get the search engines to visit a
new site and begin spidering its contents. It can take a few days or even weeks
from the acquisition of a link from such a site for all the main search engine
spiders to begin indexing a new site, and there is usually not much that can be
done to speed up this process.
Once the search engine finds a new site, it uses a crawler program to retrieve and index the pages on
the site. Pages can only be found when linked to with visible hyperlinks.
However, some search engines, such as Google, are starting to read links created
within Flash.
Search engine crawlers may look at a number of different factors when crawling a site, and many pages
from a site may not be indexed by the search engines until they gain more
PageRank, links or traffic. Distance of pages from the root directory of a site
may also be a factor in whether or not pages get crawled, as well as other
importance metrics. Cho et al.[21] described some standards for those decisions as
to which pages are visited and sent by a crawler to be included in a search
engine's index.
A few search engines, such as Yahoo!, operate paid submission services that
guarantee crawling for either a set fee or CPC. Such programs usually guarantee
inclusion in the database, but does not guarantee specific ranking within the
search results.
Blocking robots
Webmasters can instruct spiders not to crawl certain files or directories
through the standard robots.txt
file in the root directory of the domain. Additionally, a page can be explicitly
excluded from a search engine's database by using a robots meta tag.
When a search engine visits a site, the robots.txt located in the root folder
is the first file crawled. The robots.txt file is then parsed, and only pages
not disallowed will be crawled. As a search engine crawler may keep a cached
copy of this file, it may on occasion crawl pages a webmaster does not wish
crawled.
Pages typically prevented from being crawled include login specific pages
such as shopping carts and user-specific content such as search results from
internal searches.
"White hat" methods
An SEO tactic, technique or method is considered "White hat" if it conforms
to the search engines' guidelines and/or involves no deception. As the search
engine guidelines[22][23][24][25][26] are not written as a series of rules or
commandments, this is an important distinction to note. White Hat SEO is not
just about following guidelines, but is about ensuring that the content a search
engine indexes and subsequently ranks is the same content a user will see.
White Hat advice is generally summed up as creating content for users, not
for search engines, and then make that content easily accessible to their
spiders, rather than game the system. White hat SEO is in many ways similar to
web development that promotes accessibility[27], although the two are not
identical.
"Black hat" methods
-
"Black hat" SEO are methods to try to improve rankings that are disapproved
of by the search engines and/or involve deception. This can range from text that
is "hidden", either as text colored similar to the background or in an invisible
or left of visible div, or by redirecting users from a page that is built for
search engines to one that is more human friendly. A method that sends a user to
a page that was different from the page the search engined ranked is Black hat
as a rule. One well known example is Cloaking, the practice of serving one version of a
page to search engine spiders/bots and another version to human visitors.
Search engines may penalize sites they discover using black hat methods,
either by reducing their rankings or eliminating their listings from their
databases altogether. Such penalties can be applied either automatically by the
search engines' algorithms or by a manual review of a site.
One infamous example was the February 2006 Google removal of both BMW Germany and Ricoh Germany for use of deceptive practices.[28]. Both companies, however, quickly
apologized, fixed the offending pages, and were restored to Google's list. [citation needed]
SEO and marketing
There is a considerable sized body of practitioners of SEO who see search
engines as just another visitor to a site, and try to make the site as
accessible to those visitors as to any other who would come to the pages. They
often see the white hat/black hat dichotomy mentioned above as a false dilemma. The focus of
their work is not primarily to rank the highest for certain terms in search
engines, but rather to help site owners fulfill the business objectives of their
sites. Indeed, ranking well for a few terms among the many possibilities does
not guarantee more sales. A successful Internet marketing campaign may drive
organic search results to pages, but it also may involve the use of paid
advertising on search engines and other pages, building high quality web pages
to engage and persuade, addressing technical issues that may keep search engines
from crawling and indexing those sites, setting up analytics programs to enable
site owners to measure their successes, and making sites accessible and
usable.
SEOs may work in-house for an organization, or as consultants, and search
engine optimization may be only part of their daily functions. Often their
education of how search engines function comes from interacting and discussing
the topics on forums, through blogs, at popular conferences and seminars, and by
experimentation on their own sites. There are few college courses that cover
online marketing from an ecommerce
perspective that can keep up with the changes that the web sees on a daily
basis.
SEO, as a marketing strategy, can often generate a good return. However, as
the search engines are not paid for the traffic they send from organic search,
the algorithms used can and do change, there are no guarantees of success,
either in the short or long term. Due to this lack of guarantees and certainty,
SEO is often compared to traditional Public Relations (PR), with PPC advertising
closer to traditional advertising. Increased visitors is analogous to increased
foot traffic in retail advertising. Increased traffic may
be detrimental to success if the site is not prepared to handle the traffic or
visitors are generally dissatisfied with what they find. In either case
increased traffic does not guarantee increased sales or success.
While endeavoring to meet the guidelines posted by search engines can help
build a solid foundation for success on the web, such efforts are only a start.
SEO is potentially more effective when combined with a larger marketing campaign
strategy. Despite SEO potential to respond to the latest changes in market
trends, SEO alone is reactively following market trends instead of pro-actively
leading market trends. Many see search engine marketing as a larger
umbrella under which search engine optimization fits, but it's possible that
many who focused primarily on SEO in the past are incorporating more and more
marketing ideas into their efforts, including public relations strategy and
implementation, online display media buying, web site transition SEO, web trends
data analysis, HTML E-mail campaigns, and business blog consulting making SEO
firms more like an ad
agency.
In addition, whilst SEO can be considered a marketing tactic unto itself,
it's often considered (in the view of industry experts) to be a single part of a
greater whole.[citation needed] Marketing
through other methods, such as viral, pay-per-click, new media marketing and other related means
is by no means irrelevant, and indeed, can be crucial to maintaining a strong
search engine rank.[citation needed] The part of
SEO that simply insures content relevancy and attracts inbound link activity may
be enhanced through broad target marketing methods such as print, broadcast and out-of-home advertising as well.
|