Deep Dive into Creating Reporting Framework on AWS

The goal of this project is to create a custom reporting solution/framework using AWS services for fulfilling the requirement of creating Reporting Dashboard displaying multiple analytics reports representing data from different data sources. We have recently finished developing this solution while working in raksan consulting.

The reporting solution/framework architecture is based on ReportingDatabase design pattern recommended by Martin Fowler. As per the recommended design pattern, given the fact that transactional/operational database and reporting database fulfill different data requirements/needs, it would be good to have different databases for managing transactions and reporting data.

Taking above into consideration, following represents key building blocks of application architecture of reporting solution/framework:

Following represents technology architecture of the reporting framework:

Technology Architecture - Reporting Framework

Following represents key building blocks of technology architecture, as shown in the above diagram, which is dealt in detail in later sections:

Spring Batch/Spring Data and Docker for Containerized ETL Jobs

Following technologies are used to create containerized ETL jobs:

AWS Services for Reporting Database

Following could be different AWS services which can be used for storing reporting data:

We shall take into consideration both of the above AWS storage service for our reporting database. Details can be found on this page: AWS Services for Reporting Database

AWS Services for Scheduling/Running Reporting ETL Jobs

Following could be some of the different solution approaches which can be used to create reporting solution/framework using different AWS services:

Details can be found on this page: AWS Services for Reporting ETL Jobs

Reporting Application using Springboot, AWS Services

Following is one of the solution approach for using creating and deploying reporting application on AWS:

Details can be found on this page: Reporting Apps using Springboot, AWS Services

Continuous Delivery of Reporting Apps to AWS

Following is done to continuously deliver reporting app and containerized ETL jobs to AWS:

Details can be found on this page: Continuous Delivery of Reporting Apps to AWS