The client is a provider of eLearning, training, and performance support solutions aimed at improving safety, compliance, and operational efficiency across various industries.
Their platform offers a comprehensive suite of online courses, training management tools, and performance support resources, helping organizations in fields such as education, industrial manufacturing, and public safety to enhance workforce development, ensure compliance, and mitigate risks.
The client used a tool to collect data from the environment and manage vulnerabilities. The tool also monitored malicious behavior, investigated and shut down attacks, and automated operations. However, there were still gaps in these practices.
The ETL queries had not been written effectively, which resulted in the need to re-run all ETL jobs (full historical load) every quarter. This incurred additional costs and time.
The client’s BI tool experienced issues with unrefreshed report filters, which cached selected values in report filters even after removal from the dataset. The GUI also offered limited options.
We developed AWS data engineering solutions that comprised an API endpoint with a Lambda function to authenticate GitHub webhook push events.
We provided a robust solution for tracking changes with AWS DMS to streamline the cloud data migration process, making it effective and reliable.
Our team developed a custom implementation for seamless data handling, as we knew that MySQL does not support materialized views.
We applied a medallion architecture comprising six logical layers around a data lake, promoting separation of concerns and component-oriented design.
We created a robust application data transformation by shifting to a canonical data format across applications for seamless integration into data warehouses. This resulted in better analytics and reporting.
By implementing AWS DMS and Lake Formation, we helped streamline data ingestion from various sources.
We enhanced observability across the entire data flow by centralizing pipeline management within AWS Lake Formation.
By implementing the refined architecture, we integrated new data sources quickly and supported diverse analytics methods.
The tiered approach of our refined architecture facilitated incremental data refinement and enhanced data quality as it progressed through each layer. A layered, component-oriented architecture promoted separation of concerns, decoupling of tasks, and flexibility.
Insights from the generated reports resulted in $2 million in PPP (Preferred Pricing Program) within six months of the launch, solely from one product.
