As any other web site administrators, I care much about the site visit statistics and other marketing data from the web site visiting log. I used DeepMetrix LiveStats a few years ago, then upgraded to DeepMetrix .NET 8. But later the company DeepMetrix dropped the support on its products after Microsoft bought it. From there, I tried the online tools like Google Analytics including the Microsoft one based on DeepMetrix – Microsoft Ad-Central. They are good enough for my general use, but I still want a website log stats tool installed locally.
So I re-installed DeepMetrix older version of LiveStats 6.2 and newer one .NET 8. But the stupid DeepMetrix .NET only works for the Data Collection part, not the reporting part. And the LiveStats 6.2 was working fine before, and now it totally stopped working after the license domain name deepmetrix.com was gone.
Finally, I had a reason to stop trying, testing and wasting time on DeepMetrix. I moved on and decided to use AWStats which is a free open-source web log analyzer tool.
At first, I still had the concern about the disadvantage of parsing the log files vs. the site tracking scripts. But since I still can use the DeepMetrix Data Collection part, then I can use the AWStats tool to parse the pre-processed log files from DeepMetrix instead of the raw log files from IIS.
This sounds a plan. And I went ahead to download the latest version of AWStats 6.9 and of course the ActivePerl 5.10 which is the requirement to run AWStats.
Then I opened a log file from DeepMetrix Pre-processor pool folder, and found out it was not in the standare w3c extended log file format. So I had to use the personalized log format in the AWStats configure file as the following:
The log file format of DeepMetrix:
#Fields: date time c-ip cs-uri-stem cs-uri-query sc-status cs(User-Agent) cs(Referer) cs(cookie) cs(Visits) cs(ScreenSize) cs(TimeZone) cs(Language) cs(PLT) cs(Cart) cs(Now) cs(Last)
AWStats configure file parameters
LogFormat=”%time2 %host %url %query %code %ua %referer %other %other %other %other %other %other %other %other %other”
But when I saved the configure file and ran AWstats, it could not build the update log. And the error message is the following:
“Your personalized LogFormat does not include all fields required by AWStats (Add \%methodurl or \%method in your LogFormat string).”
I know it was caused by the DeepMetrix log file which does not have Method (GET, etc.) parameters inside at all.
It is not feasible for me to change the way DeepMetrix collects the data and pre-processes, what I can do is to make the AWStats not take the Method parameter as the required field.
I searched but could not find a way from the Internet, then I decide to roll out my own sheet to DIY. Since AWStats is a open-source tool, it is easier than other tools because you can get into the source code.
Here was what I did to resolve this problem and make AWStats work smoothly on DeepMetrix LiveStats .NET log files.
# if ( $pos_method < 0 ) {
# error( "Your personalized LogFormat does not include all fields required by AWStats (Add \%methodurl or \%method in your LogFormat string)."
);
In this way, the AWStats will not validate if your log file has the %Method or %methodurl parameters.
# $LogType eq ‘W’
# && (
# $field[$pos_method] eq ‘GET’.
…..
….
….
# else {
# $NbOfLinesDropped++;
# if ($ShowDropped) {
# print
#”Dropped record (method/protocol ‘$field[$pos_method]‘ not qualified when LogType=$LogType): $line\n”;
# }
# next;
# }
Have fun.
0 Responses to “How to re-use DeepMetrix Livestats NET pre-processed log files for AWStats”