Table of Contents

1. Introduction
2. Landing Page
3. What are people really looking for?
4. Spider analysis
5. Internal site search tracking
6. Related resources
7. Unsubscribe

I. Introduction

Search engine play more and more important role in Internet. When a potential visitors want to find something on the web, he will look to a search engine such as Google, Yahoo and Bing. This makes it imperative that your site get ranked high enough for important keywords that visitors can find it. Knowing what keywords are important means knowing what visitors are looking for when they find your site.

Using Nihuo Web Log Analyzer, you can find out what engines, phrases and keywords visitors are using to reach your site and produce reports to help you improve your site content and search engines listings. Each search engine will offer links to your site for certain keywords or phrases. When a visitor types these phrases your link will be given, along with many others.

To view search engines report of your web site, go to reports under Search Engines.

II. Landing Page Landing Page is the page that appears when a potential visitor clicks on a search-engine result link. It could be your home page, or any other page in your site. The best use of a landing page is not what it is, but what is can do. Your landing pages should provide a customized sales pitch for the visitor. The best way to do this is consider where the person has come from, and who they are. By providing a good match, your chances of engaging the visitor goes up, as should your conversion rate.

To view landing page report of your web site, go to reports under Search Engines > Top Searched Files.

III. What are people really looking for?

Knowing what search phrases folks use to find your web site is one thing. Knowing what they are really looking for is quite something else. Visitors type keywords into search engines they think will find the page they want. The pages of results might or might not contain what they’re looking for. When a searcher immediately clicks the “back” button, it may suggest that the search phrases is not as relevant to your landing page as previously supposed. You need carefully research phrases which is with high bounce rate, try to find out what are they really looking for, and adjust landing page or create a new page to match the search phrase with lower bounce rate. To view bounce rate of search phrases of your web site, go to reports under Search Engines > Top Search Phrases.

IV. Spider analysis One part of log analysis that has remained surprisingly lacking, both in terms of content and accuracy, is spider analysis. Though traffic analysis programs may look at spider activity, the information often isn’t detailed enough or presented in a format to do you much good. Also, spider and robot analysis is acknowledged as being a main culprit for inaccurate log analysis measurements. Therefore, the need for detailed spider analysis has begun entering the minds of search engine marketers.

You know that when you submit a Web page to an engine for indexing, the engine sends a spider to your site to index the contents of the page. “Spider analysis” is simply analyzing the search engine spider visits to your site.

With powerful filter function of Nihuo Web Log Analyzer, you can learn the following information about your site in a concise, easy-to-read format through effective spider analysis.

• Has your site been spidered?
• If so, by which engines?
• When did the spiders visit?
• Which directories and pages did they visit?
• Are certain pages getting respidered more often, signaling their
  importance to the search engines?
• Are certain pages not getting spidered at all?
• Are the spiders indexing inappropriate content?
• Are the spiders getting everything they want and need, or are they receiving error messages?
• Was your site spidered within the specified time agreed upon in the
  pay inclusion programs you’re participating in?
• Is your site getting respidered on a regular basis, as agreed upon in
  your participating pay inclusion programs?

V. Internal site search tracking

You can track phrase which visitor search in your internal site search engine with Nihuo Web Log Analyzer. Please follow below guides step by step to add your site search engine url into search engine configuration list: Launch Nihuo Web Log Analyzer Click Option button on toolbar Select Search Engine page Click Add button Input “Internal Site Search Engine” into Name field Input your site search url idAssume your search url looks like this: http://search.yoursite.com/search.cgi?q=keyword Input “search.yoursite.com” as URL ID. Input “q=” as Phrase Header. Please select default code page of your search url. Click OK button

VI. Related resources

• Landing page optimization
• Google webmaster tools
• Yahoo site explorer
• Bing webmaster tools

Tags: Tutorials, web log analysis

Table of Contents

1. Broken Links
2. Stolen Object
3. Web Server Issues

I. Broken Links Broken links occur when web pages are deleted or moved but the link to the old address still exists. When a user clicks on the broken link, the page cannot be found, so a 404 error page is displayed. This looks bad and frustrates customers, so you want to fix them as soon as possible. To help you maintain your site, Nihuo Web Log Analyzer has the 404 Errors report that show which pages on your site failed and the referrer that was associated with it. You can use this report to quickly find and fix the links.

Two entries you will often see near the top of your 404 Errors report are /robots.txt and /favicon.ico.

The robots.txt file is something that well-behaved robots or spiders check to see what parts of your site they are allowed to index. You can find complete details on how to set up your site with robots.txt inhttp://www.robotstxt.org/robotstxt.html.

Microsoft Internet Explorer 5 introduced a new technique to allow you to customize the icon that your visitors see when they save links to your
site. IE 5 and later (and now Konqueror and perhaps other browsers) look for a file called favicon.ico on your web server when the user bookmarks a page on your site or saves a link to it on her computer. IE looks first at the root of your site (/favicon.ico) then in the directory
where the page is located. If you create an icon in the root of your site it will be applied to all pages. If you have one in a particular directory, then each directory can have its own icon. You can find complete details on how to set up your site with favicon.ico in http://en.wikipedia.org/wiki/Favicon.

404 Errors report is under Errors > 404 Errors.

II. Stolen Object Graphics and other non-page files that you have on your site are something you have put effort and time (and perhaps money) into developing to represent your business or service. They are indicative of your brand or service and are a valuable part of the consistent image your present to your customers. The way the Web works, however, it is not very hard for another site to simply include a reference to these files from your site in their pages. The HTML can tell a visitor’s browser to retrieve graphics or other files from anywhere else on the Web, not just the same site the page is on. When this happens, the visitors at the other site may have no idea that the files belong to you. In Nihuo Web Log Analyzer the files are referred to as ‘Stolen Object’ and Nihuo Web Log Analyzer has the Stolen Object report to help you diagnose this behavior.

If you have setup an agreement with partner web sites to allow them to include references to your graphics and other files in their site (for example, linking to your logo to promote your product or linking to your advertisements to host on their sites) Nihuo Web Log Analyzer will not
recognize this and these will show up in the Stolen Object report.

The Domains Stolen Object report only shows you when other sites include references to graphics or other non-page files on your site. If a malicious user really wants to steal your files without you knowing she can simply save the file to her computer and copy it to her web site.
Saving a file produces the same request that viewing it in the first place does (in fact web browsers just make a copy of the file they have
already cached, so you never get a request for the ‘save’ action if the file was already viewed.) This behavior is effectively impossible to detect.

Stolen Object report is under Resources Accessed > Stolen Object.

III. Web Server Issues

Many server issues are best diagnosed with network and process monitoring tools. There are a number of good ones available for all platforms, both commercial and free. However, some server diagostic information can be gathered from your web site logs using Nihuo Web Log Analyzer.

1. Bandwidth Usage

Your web site probably has a limited available bandwidth for connecting with visitors. There may be times, say during a promotion or a product release, when your bandwidth usage is well above normal. Most service providers will allow you to ‘burst’ well beyond your average usage to handle times such as these. However, if your monthly usage typically exceeds your contractual allowance, you could be paying more for connection than you need to be. The solution to this is to either increase your contract quota with your provider or determine where the bandwidth is being consumed and alter the site to reduce the amount of bandwidth required to present the information you need.

2. Server Attack

Server attacks are worm or virus attacks as well as unauthorized users trying to gain access to protected directories. A common server attack is a request to the “cmd.exe” file caused by the Nimda or the Code Red virus present on a Windows machine. If the machine is on a local network, you can find out which machine it is on and inform the network administrator. If this file was requested from the Internet, you can do a DNS lookup through the program or do a Whois lookup (using www.allwhois.com or www.arin.net ) to find out who this comes from. If you can find out where the request came from, you can inform the originator of the virus on their server. These requests usually cause many 404 (File Not Found) errors in your log files. If you find successful accesses to the file cmd.exe on your Windows Server, your server is probably infected.

Server Attack report is under Server Attack > Server Attack.

3. Loading Time

Everyone knows how annoying it can be having to deal with websites that take forever to load. There are recent researches, in fact, confirming that 75% of the Internet users do not return to sites that take longer than four seconds to load.

In order to minimize loading time for fast user experiences, you need find out which page waste most time of your server.

If your web server is IIS, please ensure export time-taken field in your
log files.

If your web server is apache, Here’s a simple trick to track time taken data:

1. Please add “%T” ( time-taken ) into your log format string.e.g.
   LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-agent}i\” %T” timetakenlog

   CustomLog log/access_log timetakenlog

2. Launch Nihuo Web Log Analyzer
3. Right click the project and select Edit from menu
4. Select Format page
5. Select Apache/NCSA custom log as Log file format
6. Click Custom button
7. Input %h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-agent}i\” %T
   
8. Click OK button
9. Analyze

Loading time report is under Resources Accessed > Time Taken.

Tags: Tutorials, web log analysis

Table of Contents

1. Introduction
2. How to exclude invalid activities?
3. Which data should you focus on?
4. Unsubscribe

I. Introduction

The internet is a fast-paced environment. People can come to your website at any hour from a wide range of locations, each of them with different intentions or needs. Unlike physical retail stores, you can’t see who is coming in and browsing around. You don’t know much about the
people reading you.

You already get a glimpse of visitors everyday when they interact with your website. Some may register for an account, leave a comment or send
you an email. But many are ‘invisible’. They get to your site, see what you put out, click on a outbound link and disappear.

What you currently know about these individuals comes from a combination of visible user actions (e.g comments/emails) and statistics (e.g visit frequency/visit length). Is this knowledge sufficient for most businesses? Yes. But I think it would be tremendously helpful to learn even more about your visitors.

It is helpful to analyze and construct a general profile of your visitors, however shifting it may be, because it provides you with information that will allow you to better improve your content scope, site usability, conversation rate or marketing campaign.

II. How to exclude invalid activities?

Before analyzing visitor behavior, activities caused by spiders, bots and stolen object need be excluded to avoid noise.

To exclude invalid activities:

1. 1Right click profile and select Edit from menu
2. Select Hit Filter page
3. Click Template button and select Exclude all invalid activity
4. Click OK button
5. Clearing database is required ( Right click profile and select Clear database from menu)
6. Re-analyze ( Right click profile and select Analyze from menu)

III. Which data should you focus on?

Nihuo Web Log Analyzer provide a lot of information on how visitors are using your website, where they come from and what they are looking at. There are obviously a lot of different metrics to look at but I’m listing what I think is more relevant to understanding visitors in general: 1. Bounce Rate Understanding bounce rate is an important aspect of analyzing your overall statistics, especially when it comes to determining the effectiveness of an individual page. The bounce rate measures the number of visitors to a website that leave before a specified amount of time has elapsed (this time period varies among analytics tools, but typically it is 30 minutes). This means that if a user accesses your site and leaves it within 30 minutes or leaves their browser idle for that time, they will be registered as a bounce. The bounce rate for an individual page of a website is etermined by the number of users that access a page and leave the site without clicking to another page within the specified time period.

It is really hard to get a bounce rate under 20%, anything over 35% is cause for concern. Generally a bounce rate less than 50% is considered
ok. This is based on our experience, but hopefully it gives you a feel for what you are shooting for.

One thing to keep in mind is that your expectation for meeting the standard on any given page of your site should also be measured against
the entrance sources for that page. Depending upon how a user is referred to your site, his or her understanding of the relevance of your
site’s content to their query will vary quite a bit.

To view the bounce rates for your website and the bound rate of each page on your site, go to the Bounce Rate report under Resources Accessed > Bounce Pages > Top Bounce Pages.

Below are other reports which are include bounce rate information.

• Bounce rate by visitor location -Visitor & Demographics > By Countries > Most Active CountriesVisitor & Demographics > By US States > Most Active US StatesVisitor & Demographics > By Cities > Most Active Cities
• Bounce rate by visitor ip classVisitor & Demographics > Top Class A IP > Top Class A IPVisitor & Demographics > Top Class B IP > Top Class B IPVisitor & Demographics > Top Class C IP > Top Class C IP
• Bounce rate by visitor referrerReferrers > Top Referring Sites > Top Referring SitesReferrers > Top Referring URLs > Top Referring URLs
• Bounce rate by search engineSearch Engines > Top Search Engines > Top Search EnginesSearch Engines > Top Search Phrases > Top Search Phrases

2. Visitor Location This allows you to make cultural and linguistic assumptions of your visitors. If you know you receive the most visitors from a few specific countries, you might want to create landing pages/offers or content with a geographic focus. If you geo-target your site, it makes your business look like it is there to serve the specific location of your visitor.

To view visitor location of your website, go to reports under Resources Accessed > Visitor & Demographics.

3. Search Phrase This includes both search engines and on-site search boxes. The clearest indicator of visitor interest, search terms tell you what they want to get from your site and it reveals information gaps you can fill up. This is where data collection gets specific. If you consistently get a lot of queries for a specific phrase, you can safely assume that there will be visitor interest in content or offers related to it. To view search phrases of your website, go to Top Search Phrases reports under Search Engines > Top Search Phrases > Top Search Phrases.

4. Traffic Source This includes search engines, referrer sites, type-in/bookmark traffic and ad campaigns. Pay attention to referrer sites: it reveals what visitors are reading or using. Traffic sources also tell you where to improve for greater visibility. It is not just about the amount of traffic but the quality of the traffic.

Let’s look at the ways you can determine the quality of traffic using your website analytics. Here are a couple of key metrics you need to pay
attention to:

• Visits
• Pages Per Visit
• Average Time on Site
• Bounce Rate

To view traffic sources of your website, go to reports under Referrers and Search Engines.

5. Visitor Path

Visitor Path is a process of determining a sequence of pages visited in a visitor session prior to some desired event, such as the visitor purchasing an item or requesting a newsletter. The precise order of pages visited may or may not be important and may or may not be specified. In practice, this analysis is done in aggregate, ranking the paths (sequences of pages) visited prior to the desired event, by descending frequency of use. The idea is to determine what features of the website encourage the desired result. “Fallout analysis,” a subset of path analysis, looks at “black holes” on the site, or paths that lead to a dead end most frequently, paths or features that confuse or lose potential customers. To view visitor path of your website, go to reports under Resources Accessed > Paths Through.

While this isn’t an exhaustive list, you will find other ways to get more visitor data. When combined with other visitor statistics, it’s easy to understand your visitors, allowing you to better accommodate their needs or interest.

Tags: Tutorials, web log analysis

Table of Contents

Introduction Why should I filter data? Hit Filter & Visit Filter Wildcards How to exclude spiders and bots data? How do I exclude my internal traffic from reports?

I. Introduction

Filters hold the key to unlocking the power of Nihuo Web Log Analyzer. Understanding how filters work will help you to get the most out of your
reports. Filters allow you to limit the scope of Nihuo Web Log Analyzer’s analysis to specific parts of your site, providing only the most important information in reports to make reports more readable or relevant.

II. Why should I filter data?

Filtering or excluding certain data from your reports is important to ensure your data reports are accurate. If you want to measure performance
of your promotion plan and you do not set up any filters, all those visits from your web team are going to have a negative effect on the reported data. Generally it is a good idea to filter as many known people and domains as possible to ensure the reported data is as accurate as possible.

You can create extra profiles of the same website and filter specific traffic to only appear in all reports of that profile.

III. Hit Filter & Visit Filter

Nihuo Web Log Analyzer provides two filter types: Hit Filter and Visit Filter.

Hit filters include or exclude raw data generated by individual actions on a web site.

Visit filters include or exclude all the data in a visitor session.

Multiple filters can be combined in a profile with boolean operator ( AND, OR , NOT ).

Here’s the correct list for all the currently available filters and how Nihuo categorizes them.

1. Hit Filter

• Advertising
• Agent
• Authenticate User
• Browser
• Client Host Country
• Client Host Domain
• Client Host IP
• Cookie
• Day Of Week
• File Type
• HTTP Method
• OS
• Referrer
• Requested File
• Return Code
• Spider
• Stolen Object
• Time
• URL Parameter
• Virtual Domain

2. Visit Filter

• Visitor with specified entry page
• Visitors with specified exit page
• Visitors who came from specify referrer
• Visitors who accessed specified file
• Visitors who accessed specified file type
• Visit Depth
• Visitors who came from specify search phrase

For step-by-step instructions on creating filters, please refer to http://loganalyzer.net/tutorial.html#filter.

IV. Wildcards

Nihuo Web Log Analyzer support using wildcards in hit parameters.

Here are the wildcards supported by Nihuo Web Log Analyzer:

1. An example using *

Let’s say you want exclude all files and subdirectory below /admin/ from reports.

Just create exclude Requested File hit filer and input below parameter:

2. An example using ?

Let’s say you have several files on your website:

Let’s say we use this wildcard file name:

The ? matches a single character. The wildcard file name above means

“match any filename which starts with file, is followed by a single character, and then the .htm extension follows”. This wildcard will select:

… but it will NOT select:

because it has two characters, instead of one, between file and .htm.

But if you specify the wildcard:

then only

will be selected, because the wildcard file name specifies that there must be 2 characters between file and .htm

For more detail information about wildcards, please refer to http://www.loganalyzer.net/log-analysis-tutorial/how-to-use-wildcards.html

V.How to exclude spiders and bots data?

If you want to exclude spiders and bots traffic from appearing in your reports, you can use spider hit filter to filter out visits from spiders and bots.

To exclude spiders and bots:

1. Right click profile and select Edit from menu
2. Select Hit Filter page
3. Click Template button and select Exclude all spiders from menu
4. Click OK button
5. Clearing database is required ( Right click profile and select Clear database from menu)
6. Re-analyze ( Right click profile and select Analyze from menu)

You may also use agent filter to filter out visits from particular spiders and bots which aren’t recognized by Nihuo Web Log Analyzer.

For example: exclude agent string “Microsoft Data Access Internet Publishing Provider DAV 1.1″

1. Right click profile and select Edit from menu
2. Select Hit Filter page
3. Click And button and select Agent from menu
4. Input “Microsoft Data Access Internet Publishing Provider DAV 1.1″ ( include double quotes ).
5. Click OK button
6. Right click the filter node and select Not from context menu
7. Click OK button
8. Clearing database
9. Re-analyze

VI.How do I exclude my internal traffic from reports?

If you want to exclude internal traffic from appearing in your reports, you can filter out a specific IP address or a range of IP addresses. You can also use cookies to filter out visits from particular users. We’ll explain how below.

To exclude by IP address:

1. Right click profile and select Edit from menu
2. Select Hit Filter page
3. Click And button and select Client Host IP from menu
4. Enter correct IP range value and click OK button
5. Right click the filter node and select Not from context menu
6. Click OK button
7. Clearing database
8. Re-analyze

To exclude traffic by Cookie Content:

To exclude traffic from dynamic IP addresses, you can use a JavaScript function to set a cookie on your internal computers. You’ll then be able to filter all visitors with this cookies from appearing on your Analytics reports. How to exclude traffic by cookie:

1. Create a new page on your domain, containing the following code:
   < script > function SetCookie(cookieName, cookieValue, nDays) {

   var today = new Date();

   var expire = new Date();

   if (nDays == null || nDays == 0)
   nDays = 1;

   expire.setTime(today.getTime() + 3600000 * 24 * nDays);

   document.cookie = cookieName + “=” + escape(cookieValue) +
   “; expires=” + expire.toGMTString();

   }

   </script>

   (Please note you must ensure cookie field had been exported in log files.)

2. In order to set the cookie, visit your newly created page from all computers that you would like to exclude from your reports.
3. Launch Nihuo Web Log Analyzer and right click profile and select Edit from menu
4. Select Hit Filter page
5. Click And button and select Cookie from menu
6. Enter “test_value” and click OK button
7. Right click the filter node and select Not from context menu
8. Click OK button
9. Clearing database
10. Re-analyze

Tags: Tutorials, web log analysis

Table of Contents

1. Hits and Visits
2. Page views
3. Bandwidth
4. Web Spider
5. Stolen object
6. Unique Visitors
7. Session
8. Referrer
9. Bounce rate

I. Hits and Visits A log entry will generate a “Hit” on the web server. This can include pages, images, animations, audio, video, downloads, PDF or Word documents or anything else that you allow visitors to access. When a web browser loads a page, it also loads all the components referenced by that page. For example, if a web page contains 5 images, a visit on that page will generate 6 “Hits” on the web server, one hit for the web page, 5 hits for the images. A unique visitor is determined by the IP address or cookie. By default, a visit session is terminated when a user falls on inactive state for more than 30 minutes. So a unique visitor may visit your web site twice and get reported as two visits.

If the visitor left the web site and came back 30 minutes later, Nihuo Web Log Analyzer will report 2 visits. If the visitor came back within 30 minutes, Nihuo Web Log Analyzer will still report 1 visit.

II. Page views

Page is any file or content delivered by a web server that would generally be considered a web document. This includes HTML pages (.html, .htm, .shtml), script-generated pages (.cgi, .asp, .cfm, etc.). Image files (.jpeg, .gif, .png), javascript (.js) and style sheets (.css) are generally not considered to be pages. A page view (PV) or page impression is a request to load a single page of an Internet site. On the World Wide Web a page request would result from a web surfer clicking on a link on another HTML page pointing to the page in question. This should be contrasted with a hit, which refers to a request for a file from a web server. There may therefore be many hits per page view since a page can be made up of multiple files.

III. Bandwidth Measure (in kilobytes of data transferred) of the traffic on a site. If you are billed for bandwidth usage on a monthly basis you can see an estimate of the amount of bandwidth your web site used in the General Statistics report.

IV. Web Spider Web spider is a program used by search engines, also known as a crawler or robot, searches the internet scanning web pages to include in the search engines index. All activities caused by web spiders will also be recorded into web log files.

V. Stolen object Stolen object report reveals cases in which your images and other non-page objects have been embedded in, or directly linked to by, pages on other web sites. This does NOT mean that the files have been stolen in any legal sense. It does, however, mean that your content is being displayed, heard or shown outside the context of your own web pages. For example, if an outside site places this code in a popular web page: <img src=”http://www.yoursite.com/yourpicture.jpg”> Then your image will be displayed thousands of times, possibly without any attribution or permission on your part. This report is extremely valuable in identifying such situations.

VI. Unique Visitors The number of individuals who visit a web site during a specific time. The same person visiting twice is only counted once.

VII. Session A period of interaction between a visitor’s browser and a particular web site, ending when the browser is closed or shut down, or when the user has been inactive on that site for a specified period of time. For the purpose of Nihuo Web Log Analyzer reports, a session is considered to have ended if the user has been inactive on the site for 30 minutes. You can update this setting in Option dialog.

VIII. Referrer

An http referrer or referrer is anything online that drives visits and visitors to your Web site. This can include: search engines blogs link lists banner ads email affiliate links links built into software

Technically, even offline sources like print ads or references in books or magazines are referrers, but these aren’t specifically captured in
the server referrer log. When a Web developer uses the term “referrer” she means those sites or services that are referenced in the Web server
logs.

IX. Bounce rate It essentially represents the percentage of initial visitors to a site who “bounce” away to a different site, rather than continue on to other pages within the same site. The formula used to calculate bounce rate is:

A bounce occurs when a web site visitor only views a single page on a website, that is, the visitor leaves a site without visiting any other pages before a specified session-timeout occurs. There is no industry-standard minimum or maximum time by which a visitor must leave in order for a bounce to occur. Rather, this is determined by the session timeout of the analytics tracking software.

Tags: Tutorials, web log analysis