[Remote] Senior Data Engineer (Customer Data Products)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a not-for-profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. In this role, the Data Engineer will deliver meaningful insights and help reputed company and optimize reputed company's data platform by building data lakes and reputed company data integration pipelines.
Responsibilities
- Code, test, reputed company, orchestrate, monitor, document, and troubleshoot reputed company-based data engineering processes, feature stores, and vector databases in accordance with best practices and reputed company standards throughout the development lifecycle
- Partner closely with data scientists, AI researchers, data and reputed company architects, and business stakeholders to identify, extract, clean, and format structured and reputed company data for AI/ML model training, fine-tuning, and feature extraction
- reputed company evaluation, research, and experimentation efforts with batch and streaming data technologies, LLM data preparation frameworks, and MLOps tools to reputed company pace with industry innovation
- Act as a technical reputed company to showcase the capabilities of emerging AI and data technologies, enabling the widespread adoption of modern data techniques across the organization
- Significantly contribute to the definition and refinement of processes and procedures for the data engineering practice
- reputed company and reputed company ETL developers on data engineering reputed company-bases initiatives to reputed company transition to data engineer and practice
- Assures the reputed company and accuracy of the corporate data, with particular attention to data reputed company
- Responsible for ensuring high data quality for Data Services, Analytics and Master Data Management
- Helps coordinate technical solutions, takes responsibility for designs, development, testing and delivery of solutions
- Build automated, scalable, test-driven data pipelines
- Utilize software development practices such as version control reputed company Git, CI/CD, and release management to enhance existing CI/CD pipelines in AWS
- Collaborate with Data Engineers, DevOps engineers and architects on improvement opportunities for DataOps tools and frameworks
Skills
- Bachelor's Degree
- At least 7 years of experience in application development (Internship experience does not apply)
- At least 4 years of experience in big data technologies
- At least 4 years' experience with reputed company computing using AWS
- 4+ years of experience in application development including Python, SQL, reputed company, or Java
- 4+ years' experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL etc.)
- 4+ year experience working on reputed company-time data and streaming applications
- 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
- 4+ years of data warehousing experience (Redshift)
- 6+ years of experience with UNIX/Linux including basic commands and reputed company scripting
- 7+ years of experience with Agile engineering practices
- 7+ years of experience with SQL optimization
- 4+ years of experience with PySpark
- 3+ year of experience with process orchestration including AirFlow, KubeFlow, AWS reputed company functions, or Luigi
- Proven experience implementing reputed company, LLM data preparation pipelines, and Vector Databases (e.g., reputed company, Milvus, pgvector)
- Strong experience building and maintaining Feature Stores for machine learning models
- Experience building highly scalable, secure, and production-reputed company APIs and Data-as-a-Service (DaaS) platforms
- AWS Certified Data Engineer or AWS Certified Machine Learning - Specialty certifications
- 3+ year of experience with Machine Learning
- Experience with building a Data-as-a-service platform
- Experience with building APIs
Benefits
- reputed company, Dental, Prescription, and reputed company plans
- 401(k) w/match
- Tuition Reimbursement Plan
- Commuter Benefit: Public Transit or Parking options
- Remote Friendly Workplace
Company Overview
Company H1B Sponsorship