| Job Position and Company | Posted | Location | Salary | Tags |
|---|---|---|---|---|
| $98k - $103k | ||||
| $80k - $100k | ||||
| $130k - $140k | ||||
| $153k - $225k | ||||
ISO 9001 Certified | 400+ students | Learn more | by Metana | ||
| $84k - $104k | ||||
| $98k - $109k | ||||
| $94k - $150k | ||||
| $269k - $336k | ||||
| $72k - $75k | ||||
| $90k - $150k | ||||
| $85k - $115k | ||||
| $90k - $106k | ||||
| $112k - $115k | ||||
| $126k - $200k | ||||
| $75k - $100k |
Senior Data Engineer - Trading Analytics
Key Responsibilities:
- Design, implement, and manage scalable ClickHouse infrastructure for large datasets, ensuring optimal performance, availability, and maintainability.
- Develop robust, scalable ETL pipelines using Python to handle data ingestion, transformation, and storage.
- Collaborate with infrastructure teams to ensure proper AWS setup, covering everything from EC2, S3, RDS, Lambda, and more.
- Build and maintain data orchestration workflows using Airflow and integrate Kafka for real-time data streaming and processing.
- Work closely with cross-functional teams, including trading, risk management, and middle office, to ensure the data infrastructure supports all business needs.
- Optimize performance and scalability of data pipelines, ensuring data integrity and accuracy.
- Monitor and troubleshoot data infrastructure, implementing best practices for data governance and security.
- Mentor junior engineers and provide technical leadership within the team.
Qualifications:
- 8+ years of experience in data engineering, with a focus on building and managing large-scale data systems.
- Strong hands-on experience with ClickHouse, both in infrastructure setup and operational use.
- Proficiency in Python, especially for building ETL pipelines and data workflows.
- Deep understanding of AWS infrastructure, including networking, storage, compute, and security.
- Experience with data orchestration and workflow automation using Airflow.
- Familiarity with Kafka for building real-time data pipelines.
- Exposure to trading, crypto, middle office, or trade capture/trade matching systems is a significant plus.
- Experience or knowledge in machine learning and AI is a bonus.
- Strong problem-solving skills, attention to detail, and the ability to work under pressure in a fast-paced environment.
Preferred Skills:
- Experience in financial services, particularly in trading or crypto environments.
- Ability to work with diverse stakeholders to gather requirements and deliver solutions that meet business needs.
- Familiarity with database optimization, query performance tuning, and monitoring tools.
What does a data scientist in web3 do?
A data scientist in web3 is a type of data scientist who focuses on working with data related to the development of web-based technologies and applications that are part of the larger web3 ecosystem
This can include working with data from decentralized applications (DApps), blockchain networks, and other types of distributed and decentralized systems
In general, a data scientist in web3 is responsible for using data analysis and machine learning techniques to help organizations and individuals understand, interpret, and make decisions based on the data generated by these systems
Some specific tasks that a data scientist in web3 might be involved in include developing predictive models, conducting research, and creating data visualizations.