iwla/docs/modules.md

630 lines
11 KiB
Markdown
Raw Normal View History

2014-12-31 14:52:14 +01:00
* iwla.py
* plugins/display/top_downloads.py
2015-01-08 21:08:30 +01:00
* plugins/display/all_visits.py
2014-12-31 14:52:14 +01:00
* plugins/display/top_hits.py
2015-01-08 21:08:30 +01:00
* plugins/display/browsers.py
* plugins/display/referers.py
* plugins/display/operating_systems.py
2014-12-31 14:52:14 +01:00
* plugins/display/top_visitors.py
* plugins/display/referers_diff.py
2015-01-08 21:08:30 +01:00
* plugins/display/top_pages.py
2014-12-31 14:52:14 +01:00
* plugins/post_analysis/top_downloads.py
* plugins/post_analysis/top_hits.py
2015-01-08 21:08:30 +01:00
* plugins/post_analysis/browsers.py
* plugins/post_analysis/referers.py
* plugins/post_analysis/operating_systems.py
* plugins/post_analysis/reverse_dns.py
2014-12-31 14:52:14 +01:00
* plugins/post_analysis/top_pages.py
* plugins/pre_analysis/page_to_hit.py
* plugins/pre_analysis/robots.py
2014-12-23 07:48:25 +01:00
iwla
----
Main class IWLA
Parse Log, compute them, call plugins and produce output
For now, only HTTP log are valid
Plugin requirements :
None
Conf values needed :
analyzed_filename
domain_name
locales_path
compress_output_files*
Output files :
DB_ROOT/meta.db
DB_ROOT/year/month/iwla.db
OUTPUT_ROOT/index.html
2014-12-31 14:52:14 +01:00
OUTPUT_ROOT/year/_stats.html
2014-12-23 07:48:25 +01:00
OUTPUT_ROOT/year/month/index.html
Statistics creation :
meta :
last_time
start_analysis_time
stats =>
year =>
month =>
viewed_bandwidth
not_viewed_bandwidth
viewed_pages
viewed_hits
nb_visits
nb_visitors
month_stats :
viewed_bandwidth
not_viewed_bandwidth
viewed_pages
viewed_hits
nb_visits
days_stats :
day =>
viewed_bandwidth
not_viewed_bandwidth
viewed_pages
viewed_hits
nb_visits
nb_visitors
visits :
remote_addr =>
remote_addr
remote_ip
viewed_pages
viewed_hits
not_viewed_pages
not_viewed_hits
bandwidth
last_access
requests =>
[fields_from_format_log]
extract_request =>
extract_uri
extract_parameters*
extract_referer* =>
extract_uri
extract_parameters*
robot
hit_only
is_page
valid_visitors:
month_stats without robot and hit only visitors (if not conf.count_hit_only_visitors)
Statistics update :
None
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.top_downloads
-----------------------------
Display hook
Create TOP downloads page
Plugin requirements :
post_analysis/top_downloads
Conf values needed :
max_downloads_displayed*
create_all_downloads_page*
Output files :
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2014-12-23 09:18:30 +01:00
plugins.display.all_visits
--------------------------
2014-12-23 07:48:25 +01:00
Display hook
2014-12-23 09:18:30 +01:00
Create All visits page
2014-12-23 07:48:25 +01:00
Plugin requirements :
2014-12-23 09:18:30 +01:00
None
2014-12-23 07:48:25 +01:00
Conf values needed :
2014-12-23 09:18:30 +01:00
display_visitor_ip*
2014-12-23 07:48:25 +01:00
Output files :
2014-12-23 09:18:30 +01:00
OUTPUT_ROOT/year/month/all_visits.html
2014-12-23 07:48:25 +01:00
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.top_hits
2014-12-23 09:18:30 +01:00
------------------------
2014-12-23 07:48:25 +01:00
Display hook
2015-01-08 21:08:30 +01:00
Create TOP hits page
2014-12-23 07:48:25 +01:00
Plugin requirements :
2015-01-08 21:08:30 +01:00
post_analysis/top_hits
2014-12-23 07:48:25 +01:00
Conf values needed :
2015-01-08 21:08:30 +01:00
max_hits_displayed*
create_all_hits_page*
2014-12-23 07:48:25 +01:00
Output files :
2015-01-08 21:08:30 +01:00
OUTPUT_ROOT/year/month/top_hits.html
2014-12-23 07:48:25 +01:00
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.browsers
------------------------
2014-12-23 07:48:25 +01:00
Display hook
2015-01-08 21:08:30 +01:00
Create browsers page
2014-12-23 07:48:25 +01:00
Plugin requirements :
2015-01-08 21:08:30 +01:00
post_analysis/browsers
2014-12-23 07:48:25 +01:00
Conf values needed :
2015-01-08 21:08:30 +01:00
max_browsers_displayed*
create_browsers_page*
2014-12-23 07:48:25 +01:00
Output files :
2015-01-08 21:08:30 +01:00
OUTPUT_ROOT/year/month/browsers.html
2014-12-23 07:48:25 +01:00
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.referers
2014-12-23 09:18:30 +01:00
------------------------
2014-12-23 07:48:25 +01:00
Display hook
2015-01-08 21:08:30 +01:00
Create Referers page
2014-12-23 07:48:25 +01:00
Plugin requirements :
2015-01-08 21:08:30 +01:00
post_analysis/referers
2014-12-23 07:48:25 +01:00
Conf values needed :
2015-01-08 21:08:30 +01:00
max_referers_displayed*
create_all_referers_page*
max_key_phrases_displayed*
create_all_key_phrases_page*
2014-12-23 07:48:25 +01:00
Output files :
2015-01-08 21:08:30 +01:00
OUTPUT_ROOT/year/month/referers.html
OUTPUT_ROOT/year/month/key_phrases.html
2014-12-23 07:48:25 +01:00
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.operating_systems
---------------------------------
2014-12-23 07:48:25 +01:00
Display hook
2015-01-08 21:08:30 +01:00
Add operating systems statistics
2014-12-23 07:48:25 +01:00
Plugin requirements :
2015-01-08 21:08:30 +01:00
post_analysis/operating_systems
2014-12-23 07:48:25 +01:00
Conf values needed :
2015-01-08 21:08:30 +01:00
create_families_page*
2014-12-23 07:48:25 +01:00
Output files :
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
2014-12-31 14:52:14 +01:00
plugins.display.top_visitors
----------------------------
2014-12-23 07:48:25 +01:00
2014-12-31 14:52:14 +01:00
Display hook
Create TOP visitors block
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2014-12-31 14:52:14 +01:00
display_visitor_ip*
2014-12-23 07:48:25 +01:00
Output files :
2014-12-31 14:52:14 +01:00
OUTPUT_ROOT/year/month/index.html
2014-12-23 07:48:25 +01:00
Statistics creation :
None
Statistics update :
2014-12-31 14:52:14 +01:00
None
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2014-12-31 14:52:14 +01:00
plugins.display.referers_diff
-----------------------------
2014-12-23 07:48:25 +01:00
2014-12-31 14:52:14 +01:00
Display hook
2014-12-23 07:48:25 +01:00
2014-12-31 14:52:14 +01:00
Enlight new and updated key phrases in in all_key_phrases.html
2014-12-23 07:48:25 +01:00
Plugin requirements :
2014-12-31 14:52:14 +01:00
display/referers
2014-12-23 07:48:25 +01:00
Conf values needed :
2014-12-31 14:52:14 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
Statistics creation :
None
Statistics update :
2014-12-31 14:52:14 +01:00
None
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.display.top_pages
-------------------------
Display hook
Create TOP pages page
Plugin requirements :
post_analysis/top_pages
Conf values needed :
max_pages_displayed*
create_all_pages_page*
Output files :
OUTPUT_ROOT/year/month/top_pages.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
plugins.post_analysis.top_downloads
-----------------------------------
2014-12-23 07:48:25 +01:00
Post analysis hook
2015-01-08 21:08:30 +01:00
Count TOP downloads
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2015-01-08 21:08:30 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
Statistics creation :
None
Statistics update :
2015-01-08 21:08:30 +01:00
month_stats:
top_downloads =>
uri => count
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.post_analysis.top_hits
------------------------------
2014-12-23 07:48:25 +01:00
Post analysis hook
2015-01-08 21:08:30 +01:00
Count TOP hits
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2015-01-08 21:08:30 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
Statistics creation :
None
Statistics update :
2015-01-08 21:08:30 +01:00
month_stats:
top_hits =>
uri => count
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.post_analysis.browsers
------------------------------
2014-12-23 07:48:25 +01:00
Post analysis hook
2015-01-08 21:08:30 +01:00
Detect browser information from requests
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2014-12-31 14:52:14 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
Statistics creation :
2015-01-08 21:08:30 +01:00
visits :
remote_addr =>
browser
month_stats :
browsers =>
browser => count
2014-12-23 07:48:25 +01:00
Statistics update :
2015-01-08 21:08:30 +01:00
None
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2015-01-08 21:08:30 +01:00
plugins.post_analysis.referers
2014-12-23 09:18:30 +01:00
------------------------------
2014-12-23 07:48:25 +01:00
2014-12-23 09:18:30 +01:00
Post analysis hook
2015-01-08 21:08:30 +01:00
Extract referers and key phrases from requests
Plugin requirements :
None
Conf values needed :
domain_name
Output files :
None
Statistics creation :
None
Statistics update :
month_stats :
referers =>
pages => count
hits => count
robots_referers =>
pages => count
hits => count
search_engine_referers =>
pages => count
hits => count
key_phrases =>
phrase => count
Statistics deletion :
None
plugins.post_analysis.operating_systems
---------------------------------------
Post analysis hook
Detect operating systems from requests
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2014-12-23 09:18:30 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
2015-01-08 21:08:30 +01:00
Statistics creation :
visits :
remote_addr =>
operating_system
month_stats :
operating_systems =>
operating_system => count
os_families =>
family => count
Statistics update :
None
Statistics deletion :
None
plugins.post_analysis.reverse_dns
---------------------------------
Post analysis hook
Replace IP by reverse DNS names
Plugin requirements :
None
Conf values needed :
reverse_dns_timeout*
Output files :
None
2014-12-23 07:48:25 +01:00
Statistics creation :
None
Statistics update :
2015-01-08 21:08:30 +01:00
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2014-12-31 14:52:14 +01:00
plugins.post_analysis.top_pages
-------------------------------
2014-12-23 07:48:25 +01:00
2014-12-23 09:18:30 +01:00
Post analysis hook
2014-12-23 07:48:25 +01:00
2014-12-31 14:52:14 +01:00
Count TOP pages
2014-12-23 07:48:25 +01:00
Plugin requirements :
None
Conf values needed :
2014-12-23 09:18:30 +01:00
None
2014-12-23 07:48:25 +01:00
Output files :
None
Statistics creation :
None
Statistics update :
2014-12-23 09:18:30 +01:00
month_stats:
2014-12-31 14:52:14 +01:00
top_pages =>
2015-01-08 21:08:30 +01:00
uri => count
2014-12-23 07:48:25 +01:00
Statistics deletion :
None
2014-12-31 14:52:14 +01:00
plugins.pre_analysis.page_to_hit
--------------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
is_page
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
Pre analysis hook
Filter robots
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
robot
Statistics deletion :
None