Senior data engineer with over 5 years of experience in creating data warehouses (DWH) and data lakes. Worked on various projects related to consulting, banking, telecommunications and food technology. Created ETL/ELT processes on Apache Spark on Python/Scala and with DBT in the Kubernetes environment. Developed DQ systems, which helped prevent loading data with incorrect business information. Created a local testing system for Spark applications, which helped speed up development and reduce the load on the dev cluster. Configured CI/CD for testing code, building a DBT project and deploying it to Kubernetes. Created monitoring and alerting systems for quick response to incidents. Mentoring for junior specialists.
Experience: 5 years
Yearly salary: $40,000
Hourly rate: $20
Nationality: 🇰🇿 Kazakhstan
Residency: 🇦🇪 United Arab Emirates
Experience
Senior Data Engineer
Axenix (ex. Accenture) 2023 - 2024
Axenix (ex. Accenture) company provides consulting services in the markets of Russia, Saudi Arabia, and United Arab Emirates. Participated in architecture design, decided to use Data Vault 2.0. Configured interaction between dbt + Airflow + k8s for easy scalability of data models. Optimized Snowflake operation and its interaction with dbt. loading data to snowflake. Created and configured a monitoring and alerting system, enabling easier resource planning and quicker incident response. Participated in interviews, worked on developing data engineers at middle, junior, and intern levels. Optimized loading in ElasticSearch, allowing stakeholders to easily obtain final data.
Data Engineer
Beeline 2022 - 2023
Company Beeline is one of the leaders in the telecommunications sector in Russia. Worked on a project for creating reporting and on a project for sales survival forecasting. Participated in creating ETL/ELT processes using Spark on Scala and PySpark. Developed a monitoring system that allowed for optimizing cluster resource usage. Implemented a system for local testing of Spark applications and integrated these checks into the CI/CD process, preventing errors in production environment. Created a monitoring system for sending alerts to a specific on-call engineer.
Data Engineer
Digital Economy League 2021 - 2022
The Digital Economy League provides consulting services to major Russian companies. Developed ELT processes on Spark on Scala based on Hadoop. Orchestrated dashboard builds in Apache Airflow. Created a custom Airflow operator allowing alerts to be sent to corporate email.
Data Engineer
Mars 2020 - 2021
One of the largest retail companies. Was involved in creating dbt models, also was involved in creating custom incremental strategies and changing materializations. Administered the Clickhouse service. Created dashboards in Tableau.
Skills
aws
blockchain
gcp
hadoop
python
scala
web3
data-science
