NGINX Log Analytics Dashboard

GitHub ↗
Node.jsPostgreSQLNext.jsReact

A complete JavaScript solution for processing NGINX logs and visualizing the data in a modern dashboard.

Features

  • Real-time log processing using Node.js
  • PostgreSQL database for efficient storage and querying
  • Interactive dashboard built with Next.js and React
  • Line-by-line log file processing for handling large log files
  • Support for compressed logs (.gz files)
  • Incremental log processing (only processes new log entries)
  • Bot detection based on user agent patterns
  • Extensive log statistics and visualizations
  • Filtering and search capabilities

System Architecture

The system consists of three main components:

  1. Log Processor (real_log_processor.js): Processes NGINX log files line by line and stores the data in PostgreSQL.
  2. Dashboard API (nginx-dashboard/app/api): Retrieves data from PostgreSQL and serves it to the dashboard.
  3. Dashboard UI (nginx-dashboard/app/dashboard): Visualizes the log data with interactive charts and tables.

Dashboard Features

  • Summary statistics
  • Status code distribution
  • Request timeline
  • Traffic analysis
  • Top IPs and endpoints
  • Bot vs. user traffic
  • Error analysis
  • Raw log search and filtering

Run

npm install
npm run setup:db
npm start

Dashboard available at http://localhost:3000.