Search engine optimization for today's search engine robots requires that sites be well-designed and easy-to-navigate. To a great degree, organic search engine optimization is simply an extension of best practices in web page design. SEO's relationship with web design is a natural one. By making sites simple and easily accessible, you are providing the easiest path for the search engine robots to index your site, at the same time that you are creating the optimum experience for your human visitors.
This approach ties well into the notion of long-term search engine marketing success. Rather than trying to "psych out" the ever-changing search engine algorithms, build pages that have good text and good links. No matter what the search engines are looking for this month or next, they will always reward good content and simple navigation.
Search Engine Robots
Search engine robots are automated programs that go out on the World Wide Web and visit web pages. They read the text on a page and click through links in order to travel from page to page. What this really means is that they "read" or collect information from the source code of each page. Depending on the search engine, the robots typically pick up the title and meta description. The robots then go on to the body text of the page in the source code. They also pay attention to certain tags such as headings and alt text. Search engine robots have capabilities like first-generation browsers at best: no scripting, no frames, no Flash. When designing, think simple.
Search Engine Friendly Design
Creating search engine friendly design is relatively easy. Cut out all the bells and whistles and stick to simple architecture. Search engine robots "understand" text on the page and hyperlinks, especially text links. The relationship of SEO and web design makes sense when you start with good design techniques for your visitor. The easier the navigation and the more text on the page, the better it is not only for the visitor but also for the search engine robots.
Obstacles For Indexing Web Pages
Search engine robots cannot "choose" from drop down lists, click a submit button, or follow JavaScript links like a human visitor. In addition, the extra code necessary to script your pages or create those lists can trip-up the search engine robots while they index your web page. The long JavaScript in your source code means the search engine robots must go through all this code to finally reach the text that will appear on your page. Offload your JavaScript and CSS code for quicker access to your source code by the search engine robots, and faster loading time for your online visitors. Some search engine robots have difficulty with dynamically-generated pages, especially those with URLs that contain long querystrings. Some search engines, such as Google, index a portion of dynamically generated pages, but not all search engines do. Frames cause problems with indexing and are generally best left out of design for optimum indexing. Web pages built entirely in Flash can present another set of problems for indexing.
Depth Of Directories
Search engine robots may have difficulty reaching deeper pages in a website. Aim to keep your most important pages no more than one or two "clicks" away from your home page. Keep your pages closer to the root instead of in deeply-nested subdirectories. In this way you will be assured the optimum indexing of your web pages. Just as your website visitor may become lost and frustrated in too many clicks away from your homepage, the robots may also give up after multiple clicks away from the root of your site.
Solutions And Helpful Techniques
If there are so many problems with indexing, how will you ever make it work?
The use of static pages is the easiest way to ensure you will be indexed by the search engine robots. If you must use dynamically-generated pages, there are techniques you can use to improve the chances of their being indexed. Use your web server's rewrite capabilities to create simple URLs from complex ones. Use fixed landing pages including real content, which in turn will list the links to your dynamic pages. If you must use querystrings in your page addresses, make them as short as possible, and avoid the use of "session id" values.
When using Flash to dress up your pages, use a portion of Flash for an important message, but avoid building entire pages using that technology. Make sure that the search engine robots can look at all of the important text content on your pages. You want your message to get across to your human visitor as well. Give them enough information about your product to interest them in going the next step and purchasing your product.
If you must use frames, be sure to optimize the "no frames" section of your pages. Robots can't index framed pages, so they rely on the no frames text to understand what your site is about. Include JavaScript code to reload the pages as needed in the search engine results page.
Got imagemaps and mouseover links? Make sure your pages include text links that duplicate those images, and always include a link back to your homepage.
Use a sitemap to present all your web pages to the search engine robots, especially your deeper pages. Make sure you have hyperlink text links on your page, and a sentence or two describing each page listed, using a few of your keyword phrases in the text.
Remember that the search engine robots "read" the text on your web page. The more that your content is on-topic and includes a reasonable amount of keyword-rich text, the more the search engine robot will "understand" what the page is about. This information is then taken back to the search engine database to eventually become part of the data you see in the search engine results.
Last of all, it is very important to test your pages for validation. Errors from programming code and malformed html can keep the search engine robots from indexing your web pages. Keep your coding clean.
Check List For Success
Include plenty of good content in text on your web pages
Incorporate easy to follow text navigation
Serve up dynamically generated pages as simply as possible
Offload JavaScript and other non-text code (style sheets, etc.) to external files
Add a sitemap for optimum indexing of pages
Validate your pages using the World Wide Web Consortium's validation tool, or other html validator
On Your Way To Indexed Pages
The best way to assure that your pages will be indexed is to keep them simple. This type of architecture not only helps the search engine robots, but makes it easier for your website visitors to move throughout your site. Don't forget to provide plenty of good content on your pages. The search engine robots and your visitors will reward you with return visits.
Resources
To learn more about how to work around optimization problems with JavaScript, dynamically-generated pages, Frames and Flash, read the following articles:
Optimizing Pages with JavaScript and Style Sheets for Search Engines
http://www.searchinnovation.com/optimizing-pages-with-javascript.asp
Optimizing Dynamic Pages (Part I)
http://www.searchinnovation.com/optimize-dynamic-pages-1.asp
Optimizing Dynamic Pages (Part II)
http://www.searchinnovation.com/optimize-dynamic-pages-2.asp
Optimizing Frames for Search Engines
http://www.searchinnovation.com/optimizing-frames-for-search-engines.asp
Html validation tool
http://validator.w3.org/
Stylesheet validation tool
http://jigsaw.w3.org/css-validator/
SEM - Research Measures Success
Search engine marketing success comes from good research. By applying research to understand your competition and target audience, your optimization efforts will succeed. Remember when homework from school often required some research on your part to complete? It is much the same scenario for search engine marketing: you need to apply yourself by researching in order to understand your competition and target audience. Your visitors need to relate to you and understand your message and what you want them to do.
Know Your Industry
You must spend time understanding the industry you are trying to make a profit from. If you sell widgets, know everything you can about widgets: where they come from, how they are typically sold, why people need them, etc. You can learn a great deal from your competitors. Research using industry communications such as online magazines, forums, newsletters and blogs. Read articles written by industry leaders and keep up on the latest news about your industry. The more you know, the better your basis for building an authoritative website.
What's Your Point?
Ok, so you've got this product you want to sell online. What are you saying to your audience? Make sure you write in a clear, concise way so your message does not get lost in the process. Just because everyone in your company understands your message doesn't mean the visitors to your site will. Have a few test scenarios set up; ask a few objective readers what they understand from your website and see just how much your message is getting across. You'd be surprised that what may be obvious to you is not necessarily obvious to your website visitors. If you are having trouble creating a clear message for your website, consider hiring a copywriter to convey what you want to say.
Know Your Target Audience
Who buys your product and why? Who needs the information you have on your website? Who would you like to have visit your website that isn't already there? Who is visiting you? Are they professionals who understand your technical terms or visitors of varying levels who all need the same information from you? What do you offer that a certain market would want from you? Take the time to get a good look at what is out there and how your competitors are presenting their information to online visitors. Use your log statistics reports to track who comes to your website. Get familiar with the keyword terms they are using in the search engines to find you. Research the domains that most visit you. Find out why visitors are clicking away from certain pages before going deeper into the content of your website. Is it a lack of information? Too many choices to click on? Is the language used or instructions given easy to understand? Don't give your visitors a reason to leave before they understand your message.
Know Your Competition
Take a good look at your top competition and see what they are offering online. Even looking at websites that are not direct competitors may give you an idea of what to offer your visitors. Think of it this way: someone put effort into creating those websites. Visit your competitor's website. Search in the major search engines for your most important keyword phrases and see if your competitors show up in the top thirty search engine results. Learn what you can and see if what they are doing is something you should be doing. It's always good to know if your competitors are using SEO, Paid Inclusion, PPC, Link Building and other means to rank well.
Make Your Website Accessible
There's nothing worse than muddling through a website looking for what you want and clicking so much you finally give up. Use easy navigation, make sure your information or products are easily accessible to your visitors. Create written text that is easily understood in order to get your message across readily. Give your visitors plenty of written information. There's no such thing as "too much text" when it comes to search engine robots "understanding" your web pages. What's good for the visitor is often good for the search engine robots.
Do The Work
Research is the cornerstone to your success. The more you know about your subject, the better you will be able to inform your visitors. By informing your visitors you build trust and interest in learning more about your website. Do the math - get searching!
Search Engine Wars - Innovate To Survive in 2004
Search Engine Innovation For 2004
After being blind-sided by the Google Florida update, many webmasters and SEO's were reeling from the results. The message is clear: you can't rely on just one search engine for all of your traffic. You must use all your wits to emerge victorious from the search engine wars. Google is important, but it is not everything. Keep your eyes and ears open to new opportunities and old standbys: other search engines and directories, paid placement and pay-per-click, newsletters, and even more traditional channels.
Wait To Change
So were you an innocent bystander caught in the onslaught of sites dumped in the Google Florida update? Many people lost their hard-earned ranking, even though they did nothing "wrong". Many websites that follow Google's rules for optimization to the letter were still caught up in the carnage. Unfortunately, many businesses were devastated by these changes, especially heading into the holiday months.
What to do? As difficult as it may have been to make sense of Google's changes, for many, the simplest course of action was to simply do nothing. While perhaps contrary to a normal "it's broken so I need to fix it" approach, for many webmasters "do nothing" has proven to be the correct course of action. Since the update, many sites that were exiled to search engine Siberia have returned to nearly their former ranking, shaken but intact. From all appearances, Google simply changed their algorithm and may not have gotten it quite right. Additional "tweaks" subsequent to the Florida update seem to have brought some sanity back to their results.
Who Will Stay Tops In The Search Engines?
You never know who will become the leader in search engines. It was only a few years ago that directories were the major force--until the upstart search engine Google came along. Google got its start about five years ago and hasn't looked back. As long as Google provides good results for its users, it is in a good position to stay on top. However, with MSN working on the creation of its own search engine and Yahoo's acquisition of Overture (which includes AllTheWeb and AltaVista), things could get interesting in 2004. Microsoft is always a force to be reckoned with, and Yahoo certainly has the tools to become a major competitor to Google.
Inktomi's New Role
Inktomi may play an important role in this growth since it is now owned by Yahoo. Keep an eye on this engine: it provides secondary results for MSN and will probably replace Google in supplying primary results in Yahoo. Inktomi's importance may also increase in MSN once the Microsoft property stops using LookSmart for its primary results.
To see which pages you have listed in Inktomi, use the Inktomi Pure Search function from Positiontech (http://search.positiontech.com/InktomiSearch/PositionTechSearch.jsp). Inktomi often adds a few free pages to its databases. Check first to see which pages you may already have in their database for free before using Paid Inclusion for your most important pages.
Other Ways To Promote Your Website
Keep your eye on search engine news. Google was an up and coming engine a few years ago, you never know what will happen in the industry so stay on your toes. Continue to promote your website through links in topical directory listings. Search for websites that contain topics related to yours. Link when it "makes sense". Don't forget traditional means of marketing your website: print ads, brochures, magazine articles and more may help to make a difference. One of the best ways to promote yourself online and increase your link popularity is to write articles on your subject. Find websites that accept free content and submit your ezine, articles or newsletters to those websites to build your link popularity. Newsletters, forums, FAQ's, blogs and tips on your subject are all viable means to inform your visitors and bring in new traffic to your website. Don't forget to archive your newsletters and articles on your website, which works to build your site size and increase link popularity through your authoritative knowledge of your subject. You aren't a writer? Consider working with a copywriter to help build your good content.
Paid Inclusion And Pay Per Click
If you haven't ventured into using Paid Inclusion or PPC services, consider using them to help balance the changes in your traffic. Use a Paid Inclusion subscription for your most important pages, or submit dynamically generated pages that aren't being picked up by the search engine robots so they will appear regularly in the search engine database. You can start your PPC bidding in small doses. Look for some of the secondary smaller terms that don't cost as much but will still bring in traffic your competitors may miss. Take a look at some of the smaller PPC engines available out there, a little traffic from a lot of places can add up.
For more information on choosing keyword phrases, read our article Finding Targeted Keyword Phrases Your Competitors Miss (http://www.searchinnovation.com/keyword-phrases-competitors.asp).
Content, Content, Content
The biggest mistake I see webmasters make is creating a website with little content. Don't rely on a few paragraphs of text with optimization to convince search engine robots to stick around. A skeleton website does not make a good impression on anyone. Build the content of your website. Google's new algorithm may be a sign of search engine robots getting a little smarter when it comes to understanding what your website content is about. Build information that will keep your visitors at your website. Become an authority on your subject so other websites will naturally link to you because your information is invaluable. Remember, Google is interested in serving those who use its search capabilities, just as you should be interested in serving your visitors. Give as much real content information as able to your visitors, they will thank you with return visits.
And In The End...
In the end, the information you give is often equal to the response you receive. Make the effort to become an authority site on your subject. Building the groundwork of your website with quality information and broadening your methods of marketing will help sustain you during the search engine wars upcoming.