Content

Unified Grafana-Based Dashboards Scale Black Friday Delivery for Global Fashion Brand

Written by Shadow-Soft Team | Dec 19, 2024 2:51:29 PM

Client Overview

The client is a leading global brand specializing in premium shapewear and activewear, renowned for its innovative approach to design and empowering mission. 

With a strong e-commerce presence and retail partnerships, the company serves millions of customers worldwide, delivering high-quality products that combine comfort and performance.

The Objective: Improve Data Flow and Dashboards Using Grafana

At the time of the project, the client faced significant challenges managing and interpreting critical operational data during Black Friday. 

They monitored several key systems but lacked a central dashboard for actively tracking orders at peak demand. If a system went down, it was possible that it would go unnoticed. And orders would get lost or delayed, leading to lost client loyalty and revenue.

From the previous year, they knew which systems they needed to monitor (Orders, Shipping, and Inventory). However, they did not have a clear dashboard for accurately and easily observing all these channels. And they lacked an in-depth knowledge of Granfana to build one.

Building a system for collecting critical data and a series of dashboards for displaying that data would empower them to quickly address system failures, ensuring product delivery and customer satisfaction.

By stabilizing their data infrastructure and implementing a more intuitive and structured dashboard system, the client could unlock reliable, real-time insights. As a result, their team could make faster, data-driven decisions and improve overall productivity. 

The Solution: Restructuring Data and Display from the Bottom

Using Grafana, our team took a bottom-up approach to evaluating the data infrastructure and building intuitive dashboards tailored to the client’s operational needs. By collecting data from our user interviews, we identified key metrics to track within the dashboards.

The solution involved multiple structured phases, culminating in a user-friendly, hierarchical dashboard system that provided real-time insights and robust monitoring.

Our Process

The implementation process began with establishing prerequisites for data collection and monitoring. We did this by interviewing several members of the administrative team to determine the workflow and best source for data because it was buried in numerous separate databases.

After speaking with the team, we diagrammed their responses and verified them while prototyping a solution. Once we had a level of agreement, we went into implementation.

From there, we focused on infrastructure and application monitoring first. Then, we transitioned to data visualization and alerting. Our goal was to build a series of dashboards that reflected the operational structure of the organization, making them easy to use.

The Step-by-Step Process

Phase 0: Prerequisites and Preparation

  • Identify individuals and groups for application and infrastructure alerts
  • Identify alerting mechanism (email default)
  • Collect credentials for Azure, SQL servers, and SNMP Gets

Phase 1: Create Monitoring Infrastructure

  • Configure alert delivery mechanism (email)

Phase 2: Implement Application Monitoring

  • Create data sources in Grafana to query Azure metrics
  • Create data sources in Grafana to query MS-SQL databases
  • Prototype recorded queries to pull time series data:
    • Azure service busses
    • Azure function calls
    • SQL queries
    • Azure resource graph results
  • Establish alerting criteria for application monitoring
  • Create specific queries to pull time series data for application metrics used in dashboarding and alerting 
  • Create application monitoring dashboards in a 3-tiered hierarchy (executive, summary, detail)
  • Implement alerting for application monitoring

Phase 3: Turnover

  • Application Implementation Training

Key Features

By collaborating with the client to identify pain points, priorities, and key data, we created accessible dashboards that allow users to click on elements within the dashboard to get a granular view of the data.

The custom monitoring and dashboard solution tracks and displays data across multiple departments in real-time for accurate order monitoring and decision-making.

  • Structured Hierarchical Dashboards: 3-tiered dashboard hierarchy built in Granfana that shared relevant data to users based on their needs:
  • Executive Dashboards: High-level, real-time insights for decision-makers, focusing on critical KPIs such as orders received and shipments completed.
  • Summary Dashboards: Consolidates data from multiple sources, offering mid-level users actionable insights into operations without overwhelming them.
  • Detail Dashboards: Delivers a deep dive into metrics for technical teams, enabling troubleshooting and root-cause analysis with granular data views.
    • Workflow Diagram: Built directly within the dashboard, users can drill down to specific areas of the workflow and see metrics in real-time to identify short-term trends.
    • Real-Time Monitoring and Alerts: The system captures live data from Azure and SQL databases, enabling real-time updates and alerts.
  • Streamlined Data Navigation: Intuitive navigation and hierarchical model within the Grafana dashboard allowing users to move between data layers easily.
  • Optimized Data Presentation: Displays concise, relevant, and actionable data based on the client’s goals. Data tracks over time (For example, queries track and graph orders from order placement to shipping, showing outstanding orders).
  • Scalability and Flexibility: Seamlessly integrates new data sources, metrics, and alerting criteria, ensuring the system evolves based on requirements.

As requested by the client, the dashboards were not restricted by roles, making it easier for anyone in the company to observe workflows in real time.

Roadblocks

There were several challenges the team had to resolve before we could deploy hierarchical dashboards that provided key metrics for every layer of the organization. Specifically, there were gaps in the data and issues with the database infrastructure.

Inconsistent Data and Database Systems

The SQL servers often failed to respond to queries, providing neither data nor error messages. This unpredictability made it challenging to establish consistent data pipelines.

Our team addressed this issue by implementing robust error-handling mechanisms and designing queries that managed interruptions. This allowed the system to provide accurate data despite missing key metrics.

We also worked closely with the client’s IT team to identify and mitigate underlying server performance issues, ensuring a more stable environment for monitoring.

Data Accessibility and Permissions

Gathering the necessary credentials and permissions to access Azure resources, SQL servers, and SNMP devices proved to be a time-intensive task. Multiple stakeholders were involved, introducing delays in securing access to critical systems.

We created a detailed checklist to track all required credentials and permissions. Regular communication with the client ensured progress and avoided bottlenecks.

Tight Deadline

The client reached out five weeks before their immovable deadline—Black Friday. They needed the dashboards up and running to prevent what had happened the previous year.

The tight deadline limited our ability to build the most robust solution with comprehensive alerts and infrastructure tracking. However, we created and deployed a solution on time that helped their team avoid the issues from the previous year.

Tools

We shared our extensive experience with Grafana to navigate the product features, helping our client optimize their dashboards and reporting.

Results: Smoother Black Friday with Cleaner Data

By the time the client’s critical sales period arrived, the new monitoring system provided a single, coherent view of operations. 

Instead of scrambling across multiple disconnected tools and numerous log-ins, the client’s teams accessed user-friendly dashboards and reliable, real-time insights. They quickly identified trends, spotted bottlenecks, and maintained uninterrupted service, resulting in a significantly smoother, more controlled peak season experience than the previous year.

The client’s internal teams reported substantial improvements in decision-making speed and confidence, with less time spent chasing missing data and more time focused on improving the customer experience.

What’s Next: Improving 

We will work with the client to improve their infrastructure monitoring, specifically traffic through switches and how customers use cloud services. 

The client has also upgraded their accounting system (new software and hardware) and will be looking to us for help mapping that system to the reporting built out in Grafana.