When it comes to search engines, there are really only two that matter: Google and Bing. 97% of all searches in North America are conducted through these two search engines. As for Yahoo!? Yahoo! gets its search results from Bing. AOL, Earthlink, and Comcast all get their search results from Google.

What Does “Optimizing a Web Site” Mean?

Optimizing a web site means making it easier for search engines to find and index your content. By structuring your site and creating searchable content on each page, you can help improve your site's position in the search results. Keep in mind that there are over a trillion web pages indexed by Google and Bing, so it's important to focus on creating relevant, targeted content.

Two main factors to focus:

Text in the Website

- Where are the keywords on the web page? URLs, title tags, page headings, body text? - How are the words formatted? Bold text, near the top or bottom of the page?

Links pointing to the web site

- Are the keywords in the links pointing to the site? - Where do the links to the site come from? Search engines look at hundreds of criteria and change their rules regularly. However, most of the foundations of good search engine optimization remain constant.

Understanding how Google and Bing approach crawling and indexing websites is essential for technical SEO and useful in developing strategies for improving search visibility.

How Search Engines Index your website

Let’s look at the nuts and bolts of how search engines operate.

Web Indexing

Indexing is the middle child right in between crawling and ranking, and it refers to adding a webpage’s content to Google so it can be considered for rankings. There are several ways to index a new page on your site. The simplest method is to do nothing and let Google's crawlers discover the new content. However, you can also submit your website to Google for indexing.

How To Get A Page Indexed Faster

If you want Googlebot to access your page faster, there are a few things you can do. This is especially important if the content on your page is timely or if you've made an important change to the page.

You can use faster methods when a critical page is optimized or when you adjust the title and description to improve click-through rates. This way you get to specifically know when they were picked up and displayed in the SERPs so that you can measure where the improvement starts.

In these instances, there are a few additional methods you can use to speed up Googlebot's access to your page.

1. XML Sitemaps

XML sitemaps are a good way to get a search engine's attention to your content. They provide a list of all the pages on your site, as well as additional details about each page, such as when it was last modified. You can submit a sitemap to Bing via Bing Webmaster Tools, and you can also submit it to Google via Search Console. However, XML sitemaps are not the most reliable method for getting a page indexed immediately.

To build an XML sitemap from scratch, please follow these steps:

Step 1: Gather all of your site’s URLs

There are a few ways to get a list of URLs for your website:

Manually looking at your site and collecting all URLs

Perform a site search in Google to identify the URLs that Google has in its index

Look at Google Analytics landing page report to find URLs

The most important thing to keep in mind when gathering URLs for a sitemap is to determine if you want Google to index that specific page. For example, your homepage is most likely a page you want in the index but not your privacy policy or terms and conditions page.

Step 2: Code the URLs

Once you have a list of URLs that you want to include in your sitemap, it is time to code the URL in an XML format.

Now that you have your XML sitemap created, it’s time to submit it to Google through Search Console. Here’s how you can do that in a few easy steps:

1. Sign into Google Search Console and click “Sitemaps” in the left sidebar

2. Add the URL of your sitemap at the top of the page where it says “Add a new sitemap”

3. Click submit and Google will crawl your newly created XML sitemap.

2. Request Indexing With Google Search Console

In order to "Request Indexing" from Google, simply:

Click on the top search field in Search Console, which defaults to "Inspect and URL in domain.com."

Enter the URL you want Google to index.

Press enter.

You'll know if the page is already known to Google by the wealth of information presented to you. If you're just looking to request that Google understands a recent change, the important button appears whether the page has been indexed or not.

Within a few seconds to a few minutes, you can search the new content or URL in Google and find the change or new content picked up.

How to See Which of Your Web Site’s Pages are Being Found by the Search Engines?

Search engines index web pages based on the content of the page, not on the page’s location on the web. This means that a page’s visibility on the web is not related to how often it is visited. A search engine might index a page even if no one has ever visited it, if it finds important information on that page.

Google’s Indexed Pages Tool

To view the number of web pages indexed by Google, go to google.com and type in “site:your-domain-name.com”, press the search icon, and a list of indexed pages for your site will appear.

Bing’s Indexed Pages Tool

To see the number of indexed pages in Bing, go to bing.com and type in the same url: “site:your-domain-name.com”. The number of indexed pages may vary between Google and Bing.

What if None of Your Web Site’s Pages are Indexed?

If none of your web pages are being indexed, check your site's robots.txt file. This file, located in the root of your website, tells crawlers which directories can or cannot be crawled. Make sure it’s current for your site so that you don’t accidentally block Google’s and Bing’s crawlers. Your site’s pages may not be “readable” to the search engines. Or, there are so few links to your site’s web pages, that the search engines haven’t found your site or consider it unimportant. This is where search engine optimization on your site comes into play and can help to increase the number of indexed pages.

How to create and update a robots.txt file?

How Web Page Content is Seen by the Search Engines

Search engines have a bias for text. If you are in a competitive field and are up against many other companies online for the same keywords, your web site’s pages needs to have keyword-rich text. Images, Flash animation, and PDF files won’t cut it.

Images, Flash Files, & PDF Files

Images, Flash animation, and PDF files are not easily readable by the search engines and do not help your search engine optimization efforts. They can be very useful elements for humans. However, when trying to get your web site’s pages indexed with the search engines, the content that matters most is text. If you want the text to be indexed and rank well, it should not be embedded in images, PDFs or Flash files.

Frames, iframes, and Pop-ups

Files within frames, iframes, and pop-ups are indexed as separate files and can end up being orphaned and showup in search results without the web site’s header, navigation, or footer, which can cause confusion and result in missed opportunities and sales. It’s best to avoid frames, iframes, and pop-ups and exclude them from the pages you wish to have indexed by the search engines. You can do this by putting these types of files in a directory (such as “popups” and “iframes”), and then adding a rule to your robots.txt file (usually found in the root folder of your website) that tells to search engines to ignore these directories.

Keywords

When conducting keyword research, it is important to remember that the goal is to find keywords that are relevant to the site or product. To do this, it is important to understand what users are actually looking for when they search for information. This can be difficult to ascertain, but there are a few methods that can be used to get an idea.

One way to get an idea of user intent is to look at the keywords being used on Google Trends. This tool can show you how often particular keywords are being searched for across different time periods. It can also give you an idea of how popular a given keyword is and how competitive it is for ranking on Google.

Another way to get an idea of user intent is to look at the keywords that are being used in organic search results. This information can be found by using Google AdWords Keyword Planner. This tool allows you to input a list of keywords and see how many impressions they generate per day, as well as how much money they are worth. It can also show you the competition for each keyword and what terms people are bidding on.

Both of these methods provide useful information about what users are looking for when they search for information on a given topic. However, they are not always accurate because search engines change their algorithms frequently, which can impact how well any of these methods work in predicting user intent.

The best way to work on content while building your site is to think about how your reader might search for that specific page with search terms like:

tool to build a e-commerce website

what is robots.txt file?

how to make my website rank in the first page?

seo audit for saas website

how to apply for saas funding?

when is the ASTC deadline?

what is the difference between robots.txt and xml sitemap?

Multiple Keyword Phrases

It is very difficult for a webpage to achieve search engine rankings for multiple keyword phrases—unless those phrases are very similar. A single page may be able to rank for both "software development jobs" and "software development careers". Ranking for "Web development services" and "UI/UX services" or "Digital marketing techniques" and "Marketing analytics tools" with a single page is unlikely.

If you want to rank for multiple keywords phrases with your website, you will need to make a separate landing page for each keyword phrase you are targeting.

Placing Keywords

Once your keyword phrase is chosen for a given page, consider these questions:

Can I use part or all of the keyword phrase in the page URL (by using keywords in folders)?

Can I use part or all of the keyword phrase in the page title?

Can I use part or all of the keyword phrase in page headings and subheadings?

Answering yes to these questions can improve your search engine ranking. Be natural and user-friendly, though. For instance, you do not want the word "engineering" to show up three or more times in the URL or have the phrase Northern Lights repeated in the page title and also every heading. Readability and usability still trump search engine optimization.

Web Page URLs

One of the most important components of a web page, from a SEO perspective, is the page’s URL or file name. Putting keywords into the url’s name is a powerful way to give the search engines information about the subject of the page. Files can be placed in directories or folders that provide further context for the search engines.

An example of a good URL might be… https://bullet.so/templates/ A bad URL might look something like this… http://aspireid.com/pr/B938ske/ref=pr1?pr=2293&id=129383

The bad url contains database queries. Which url do you think is easier for the search engines to read? This is a missed opportunity to optimize your web page. If your site uses bad urls, have your developer create rewrites and rename the urls. Notice the hyphens between words in the url. These help to separate the keywords and Google prefers hyphens to underscores. A small fix of the url names on your web site can potentially have a large impact on your site’s ranking in the search engines.

Metadata

When designing your website, each page contains a space between the <head> tags to insert metadata, or information about the contents of your page. If you have a CMS site originally produced by the UMC web team will have pre-populated this data for you. However, it is important for you to review and update metadata as your site changes over time.

Title Metadata

Title metadata is responsible for the page titles displayed at the top of a browser window and as the headline within search engine results. It is the most important metadata on your page.

For those with a CMS website, the web team has developed an automated system for creating the meta title for each webpage based on your page title. This adds to the importance of using well-thought-out page titles rich with keyword phrases.

Description Metadata

Description metadata is the textual description that a browser may use in your page search return. Think of it as your site's window display—a concise and appealing description of what is contained within, with the goal of encouraging people to enter. A good meta description will typically contain two full sentences. Search engines may not always use your meta description, but it is important to give them the option.

Keyword Metadata

Keyword metadata is rarely if ever used to tabulate search engine rankings. However, you should already know your keyword phrases, so it doesn't hurt to add them into your keyword metadata. You'll want to include a variety of phrases. As a general rule, try to keep it to about 3-7 phrases with each phrase consisting of 1-4 words. A great example would be "computer science degree."

Domains and Sub-domains

Search engines look for keywords in domain names to further assist them in the process of determining relevancy for search queries. A simple, easy to remember domain is beneficial for branding purposes and sharing between prospective customers. Lengthy domains, or domains made up of abbreviations and acronyms can be difficult both from a usability perspective and are less desirable for SEO purposes. Search engines treat subdomains as separate sites. While it is possible to obtain additional rankings with optimized subdomains, the major search engines have said they will no longer allow subdomains to rank for an additional two search results. These changes have not yet been implemented, but should be taken into consideration when planning site structure and organization of your site’s content. It would be better to include keywords in main domain as well as sub-domains.

Navigation

Navigation is one of the most important aspects of SEO. Without it, an user’s journey through your site will be difficult, if not impossible. Poor navigation can lead to users becoming lost and frustrated, which will damage your rankings.

There are a few things you can do to improve your navigation. First, make sure all your pages are clean and easy to read. Use clear headline font sizes, legible text colors, and consistent spacing between paragraphs. Make sure all your links open in a new window so that users can easily explore the page’s content.

All site navigation elements in text format impact search engine visibility. Using keywords in the site’s navigation will provide another opportunity for keyword usage on a page and help search engines understand relationships between pages on your site. Utilizing breadcrumb navigation (i.e. Home > Products > Widget) can improve both usability and visibility. If using dynamic or image-based navigation, it’s essential to complement with HTML text navigation (usually in the footer).

Finally, make sure your site’s layout is optimized for mobile devices. This includes placing all important information directly on screen rather than forcing users to scroll down or click through menus. In addition, include easy-to-follow instructions for using your site on mobiles.

Off Page Factors

Inbound Links

Arguably one of the most critical factors surrounding search engine optimization is the inbound links that point to your site. Search engines not only find content by following links, they also determine relevancy and credibility by looking at the quantity, quality and topical relevance of sites that are linking to your content. Keep in mind that link quality, i.e., links from popular, highly trafficked sites, is always more important than link quantity. Indicators of a Quality Inbound Link: • Optimized anchor text (descriptive keywords within hyperlink) • Trusted and credible site (.org and .gov are typically “trusted” for example) • Industry/topical relevance (think of this as your link “neighborhood”)

No two search engines determine relevancy in precisely the same manner. Historically, Google has placed more emphasis on inbound link quality than search engines such as Yahoo and MSN. The result is that the indexes of competing search engines differ, meaning their search results may vary.

Clean Web Pages

Uncluttering your web page’s code makes it easier for search engines to read them and may help the search engines to index your pages more efficiently. A web page with uncluttered code also makes the pages load faster. One way to unclutter the code on a web page, is to separate the Javascript and CSS information from the web page and load these elements as external files.

Monitor Your Rank

Give the search engines a few days, then check to see how the page is ranking with its intended keywords

The ultimate SEO checklist

Now that we have a basic understanding of the core concepts of SEO, the following checklist may be used a starting point for optimization efforts.

Identify target audience

Identify the theme for each all site pages

Create a list of keyword phrases appropriate to your target audience

Position target keyword phrases near the top of each page to boost relevancy

Craft keyword-loaded META descriptions, title tags, etc

Address the frequency of use of keywords and phrases

Ensure that the position of graphics, images, and/or other design elements support, rather than hinder, the relevancy of each page.

Optimize internal your linking strategy

Execute dedicated link development efforts to increase quality links from other sites

Submit a keyword-loaded description of your site to industry relevant directories

Monitor search engine rankings for targeted keywords

Track search engine referral traffic

Submit an XML feed

Continue to develop unique, fresh content

Ensure that you are able to edit content and tags via a reliable CMS

Conclusion

SEO (Search Engine Optimization) is the process of improving the visibility and ranking of a website in search engine results pages (SERPs). By optimizing your website, you can increase your chances of appearing first in search engine results pages (SERPs). There are many factors to consider when optimizing your website for SEO, and a wise website builder like Bullet can take care of all the groundwork for you. By following a few simple guidelines and using our SEO checklist, you can ensure that your website is optimized and ready to rank #1 in search engine results pages.

Akil Natchimuthu