Design steps of building analytical data platform in the clouds

Imagine that a company needs to build an analytical platform that includes collecting data from services, delivering to storage layers (Data Lake, detailed data, display), calculations (batching, streaming), and integration with BI tools (reporting, ML).

For a year and a half ManyChat has solved this problem entirely in the clouds (AWS), choosing the latest tools for maximum convenience and minimizing the cost of ownership. The winning configuration is based on Snowflake, ElastiCashe Redis, RDS PostgreSQL, Tableau integrated with Python.

During this session, Nikolay plans to describe the selection process at each step of building the platform, possible risks, and the final experience. The final platform contains over 100 TB of data, receives data from dozens of services, and supports hundreds of analytics users.