Production System • Live Since Nov 2024

Support Dashboard

Unified ticket visibility platform consolidating 3 separate systems, aggregating provider status, and powering 24/7 operational coverage for a 6-person engineering team.

Python PHP-FPM Nginx SQLite API Integration OpenStack

The Problem

The team relied on three separate ticketing systems: ICS Ubersmith, VPS.net Ubersmith, and Zoho Desk. Without a unified view, every shift required:

  • Logging into three separate dashboards to see all queues
  • Manual context-switching between systems when triaging tickets
  • No centralised view of incident status across external providers
  • Handover notes scattered across Slack, email, and individual system notes
  • Manual identification of critical (VIP) clients requiring special handling

The legacy infrastructure was outdated—a CentOS 7 instance running on cPanel, no longer maintained and approaching end-of-life. Operations depended on a fragile, unmaintainable system.

The Solution

Built a modern, production-grade support operations platform from scratch:

Infrastructure

  • Provisioned hardened Rocky 9 instance on OpenStack with Nginx/PHP-FPM
  • Multi-layer security: IP whitelisting (VPN/office-only access), session authentication, CSRF protection
  • Let's Encrypt SSL, comprehensive audit logging for all actions
  • Scalable architecture designed for future cross-region deployment

Core Features

  • Unified Ticket View: Single queue aggregating ICS Ubersmith, VPS.net Ubersmith, and Zoho Desk with full search functionality
  • Provider Status Aggregation: Custom scrapers pulling live incident data from 5+ external providers and internal status pages
  • Integrated Tooling: IP lookup utility, escalation contacts for all internal teams and external providers, KB links
  • Shift Handover: Persistent notes system enabling smooth team transitions
  • Role-Based Access Control: SQLite-backed RBAC with viewer, tech, and admin roles

Performance & Reliability

  • Response time improvement: 2-5 seconds → <50ms (97%+ cache hit rate)
  • APCu caching with background warming ensures data freshness without latency
  • 25+ RESTful API endpoints with token-based authentication
  • Security group rules at OpenStack level for defence in depth

Out-of-Hours Coverage

  • Real-time VictorOps webhook integration for automatic escalation
  • Auto-detection of VIP/Priority clients with visual highlighting
  • De-duplication logic prevents alert fatigue on high-volume tickets
  • Comprehensive OOH analysis and reporting dashboard for management visibility

Key Achievements

Metric Result
Response Time 40x improvement (2-5s → <50ms)
Cache Hit Rate 97%+
API Endpoints 25+ RESTful endpoints
External Integrations 5 API sources + custom scrapers
Active Users 6 engineers + manager, daily use
Status Production, actively iterated

Technical Architecture

Note: Full codebase available on request. Detailed code samples and architecture diagrams to be added.

Stack

  • Frontend: PHP-FPM with server-side rendering, integrated IP lookup widget
  • Caching: APCu with intelligent background warming
  • Database: SQLite for RBAC, audit logging, configuration
  • Web Server: Nginx with Let's Encrypt SSL
  • Infrastructure: OpenStack with security groups for network isolation
  • External APIs: Ubersmith, Zoho, VictorOps, custom provider scrapers

Security Model

  • Multi-layer IP whitelisting (office + VPN networks only)
  • Session-based authentication with role separation
  • CSRF token protection on all state-changing operations
  • Token-based API authentication for programmatic access
  • Comprehensive audit logging of all user actions
  • OpenStack security groups enforce network-level restrictions

Performance Optimisation

  • APCu caching layer with configurable TTLs
  • Background cache warming prevents cold reads
  • Database query optimisation with proper indexing
  • API response caching for frequently accessed data

Business Impact

  • Operational Efficiency: Single pane of glass eliminates context switching between systems. Team can triage tickets without constant tab-switching.
  • Real-Time Visibility: 40x performance improvement enables live queue monitoring and instant provider status awareness.
  • 24/7 Coverage: Automated VIP escalation ensures no critical tickets are missed outside business hours. 5-10 OOH callouts per week shows system is catching real incidents.
  • Infrastructure Maturity: Production-ready infrastructure on corporate cloud platform with audit logging, RBAC, and scalable architecture for future growth.
  • Knowledge Preservation: Shift handover notes and audit trail create institutional memory reducing knowledge loss.
  • Team Autonomy: RBAC system enables graduated access control as team grows.

Ongoing Development

The platform is actively iterated and improved almost daily based on team feedback and operational needs. Recent improvements include enhanced VIP detection logic, additional provider integrations, and refined alert de-duplication.

Lessons Learned

  • API Integration Complexity: Each ticketing system has different rate limits, authentication schemes, and data models. Abstractions matter.
  • Cache Invalidation: Properly invalidating caches across multiple sources is harder than the initial caching strategy.
  • Real-Time vs Eventual Consistency: Users want live data, but hitting 5 APIs on every request kills performance. Trade-offs required.
  • Audit Logging ROI: Initially seemed like overhead, but has been invaluable for debugging escalation issues and compliance.
← Back to Projects Interested in similar work?