Sr. Data Engineer
Index Analytics is seeking a Sr. Data Engineer to support Government clients to design, build, and optimize scalable data pipelines and cloud-based solutions. The Sr. Data Engineer plays a key role in modernizing the organization’s data ecosystem by helping transition legacy solutions to a contemporary infrastructure.
As part of a cross-functional team including Data Engineers, health policy researchers, Analysts, the engineer will support efforts to design and implement a robust environment capable of ingesting diverse data sources to support advanced analytics and reporting needs. Core responsibilities include defining structural, interface, and business requirements for data solutions; designing relational and non-relational databases and their associated integration components; and implementing Python based automated data pipelines
This role blends advanced data engineering with hands‑on cloud solutions engineering, leveraging AWS, Snowflake, and modern DevOps practices. The ideal candidate has experience delivering high‑quality solutions in an Agile environment.
Responsibilities
- Collaborate closely with stakeholders, cross‑functional and internal technical teams to gather requirements, document business rules and develop thorough understanding of the business context and objectives
- Collaborate to design secure, scalable, and cost‑optimized data solutions
- Design, build, and maintain scalable, reliable ETL/ELT data pipelines using AWS and Snowflake and Databricks
- Develop and optimize data models, both conceptual and physical, to support analytics, reporting, and operational consumption
- Implement data quality, validation, and monitoring frameworks to ensure accuracy and reliability
- Ensure data workflows are modular, testable, and properly version‑controlled
- Operationalize pipelines with monitoring, alerting, and automated recovery mechanisms
- Conduct advanced data analysis using languages such as Python and SQL.
- Develop documentation to include data models, data dictionaries, and data usage guides
- Improve end-to-end performance of data workflows
- Build and maintain CI/CD pipelines using Jenkins to support automated testing, deployments, and continuous integration
- Meet schedule deadlines and commitments with a high-level of quality of deliverables. Collaborate with a team of cross-functional resources in an Agile delivery environment to deliver iterative value