487 lines
8.4 KiB
Markdown
487 lines
8.4 KiB
Markdown
iwla
|
|
----
|
|
|
|
Main class IWLA
|
|
Parse Log, compute them, call plugins and produce output
|
|
For now, only HTTP log are valid
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
analyzed_filename
|
|
domain_name
|
|
locales_path
|
|
compress_output_files*
|
|
|
|
Output files :
|
|
DB_ROOT/meta.db
|
|
DB_ROOT/year/month/iwla.db
|
|
OUTPUT_ROOT/index.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
|
|
meta :
|
|
last_time
|
|
start_analysis_time
|
|
stats =>
|
|
year =>
|
|
month =>
|
|
viewed_bandwidth
|
|
not_viewed_bandwidth
|
|
viewed_pages
|
|
viewed_hits
|
|
nb_visits
|
|
nb_visitors
|
|
|
|
month_stats :
|
|
viewed_bandwidth
|
|
not_viewed_bandwidth
|
|
viewed_pages
|
|
viewed_hits
|
|
nb_visits
|
|
|
|
days_stats :
|
|
day =>
|
|
viewed_bandwidth
|
|
not_viewed_bandwidth
|
|
viewed_pages
|
|
viewed_hits
|
|
nb_visits
|
|
nb_visitors
|
|
|
|
visits :
|
|
remote_addr =>
|
|
remote_addr
|
|
remote_ip
|
|
viewed_pages
|
|
viewed_hits
|
|
not_viewed_pages
|
|
not_viewed_hits
|
|
bandwidth
|
|
last_access
|
|
requests =>
|
|
[fields_from_format_log]
|
|
extract_request =>
|
|
extract_uri
|
|
extract_parameters*
|
|
extract_referer* =>
|
|
extract_uri
|
|
extract_parameters*
|
|
robot
|
|
hit_only
|
|
is_page
|
|
|
|
valid_visitors:
|
|
month_stats without robot and hit only visitors (if not conf.count_hit_only_visitors)
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.all_visits
|
|
--------------------------
|
|
|
|
Display hook
|
|
|
|
Create All visits page
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
display_visitor_ip*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/all_visits.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.referers
|
|
------------------------
|
|
|
|
Display hook
|
|
|
|
Create Referers page
|
|
|
|
Plugin requirements :
|
|
post_analysis/referers
|
|
|
|
Conf values needed :
|
|
max_referers_displayed*
|
|
create_all_referers_page*
|
|
max_key_phrases_displayed*
|
|
create_all_key_phrases_page*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/referers.html
|
|
OUTPUT_ROOT/year/month/key_phrases.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.top_downloads
|
|
-----------------------------
|
|
|
|
Display hook
|
|
|
|
Create TOP downloads page
|
|
|
|
Plugin requirements :
|
|
post_analysis/top_downloads
|
|
|
|
Conf values needed :
|
|
max_downloads_displayed*
|
|
create_all_downloads_page*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/top_downloads.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.top_hits
|
|
------------------------
|
|
|
|
Display hook
|
|
|
|
Create TOP hits page
|
|
|
|
Plugin requirements :
|
|
post_analysis/top_hits
|
|
|
|
Conf values needed :
|
|
max_hits_displayed*
|
|
create_all_hits_page*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/top_hits.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.top_pages
|
|
-------------------------
|
|
|
|
Display hook
|
|
|
|
Create TOP pages page
|
|
|
|
Plugin requirements :
|
|
post_analysis/top_pages
|
|
|
|
Conf values needed :
|
|
max_pages_displayed*
|
|
create_all_pages_page*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/top_pages.html
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.top_visitors
|
|
----------------------------
|
|
|
|
Display hook
|
|
|
|
Create TOP visitors block
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
display_visitor_ip*
|
|
|
|
Output files :
|
|
OUTPUT_ROOT/year/month/index.html
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.display.referers_diff
|
|
-----------------------------
|
|
|
|
Display hook
|
|
|
|
Enlight new and updated key phrases in in all_key_phrases.html
|
|
|
|
Plugin requirements :
|
|
display/referers
|
|
|
|
Conf values needed :
|
|
None
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
None
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.post_analysis.referers
|
|
------------------------------
|
|
|
|
Post analysis hook
|
|
|
|
Extract referers and key phrases from requests
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
domain_name
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
month_stats :
|
|
referers =>
|
|
pages
|
|
hits
|
|
robots_referers =>
|
|
pages
|
|
hits
|
|
search_engine_referers =>
|
|
pages
|
|
hits
|
|
key_phrases =>
|
|
phrase
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.post_analysis.reverse_dns
|
|
---------------------------------
|
|
|
|
Post analysis hook
|
|
|
|
Replace IP by reverse DNS names
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
reverse_dns_timeout*
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
valid_visitors:
|
|
remote_addr
|
|
dns_name_replaced
|
|
dns_analyzed
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.post_analysis.top_downloads
|
|
-----------------------------------
|
|
|
|
Post analysis hook
|
|
|
|
Count TOP downloads
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
None
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
month_stats:
|
|
top_downloads =>
|
|
uri
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.post_analysis.top_hits
|
|
------------------------------
|
|
|
|
Post analysis hook
|
|
|
|
Count TOP hits
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
None
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
month_stats:
|
|
top_hits =>
|
|
uri
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.post_analysis.top_pages
|
|
-------------------------------
|
|
|
|
Post analysis hook
|
|
|
|
Count TOP pages
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
None
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
month_stats:
|
|
top_pages =>
|
|
uri
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.pre_analysis.page_to_hit
|
|
--------------------------------
|
|
|
|
Pre analysis hook
|
|
Change page into hit and hit into page into statistics
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
page_to_hit_conf*
|
|
hit_to_page_conf*
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
visits :
|
|
remote_addr =>
|
|
is_page
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|
|
plugins.pre_analysis.robots
|
|
---------------------------
|
|
|
|
Pre analysis hook
|
|
|
|
Filter robots
|
|
|
|
Plugin requirements :
|
|
None
|
|
|
|
Conf values needed :
|
|
page_to_hit_conf*
|
|
hit_to_page_conf*
|
|
|
|
Output files :
|
|
None
|
|
|
|
Statistics creation :
|
|
None
|
|
|
|
Statistics update :
|
|
visits :
|
|
remote_addr =>
|
|
robot
|
|
|
|
Statistics deletion :
|
|
None
|
|
|
|
|