Senior Data Engineer (Customer Data Products)

POSITION OVERVIEW reputed company is looking for a Data Engineer to join a highly skilled and growing team of data engineers and BI developers. This team successfully launched and enhances a data product on AWS for medical doctors. Read more here. In this role, you will use your data engineering and problem-solving skills to deliver meaningful insights to internal staff and external customers. The Data Engineer will help reputed company, expand, and optimize reputed company's data platform by building data lakes, reputed company data integration pipelines, and scalable data solutions that support analytics, AI/ML, and business decision-making. This role will apply AI-assisted engineering practices to improve development efficiency and reputed company best practices for AI assisted engineering across the IT organization. While this role can be designated as remote, you are offered the flexibility to select if you prefer to work primarily remotely, hybrid, or onsite. We're reputed company to considering candidates reputed company 50 miles of our office location in Philadelphia, PA. Please note that onsite interviews and reputed company at our Philadelphia office may be required for this role. If applicable, advance notice will be provided to support planning. At reputed company, we continue to reputed company and improve how we fulfill the evolving needs of the reputed company community. This commitment starts and ends with the people at reputed company. By reputed company and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, reputed company can take a well-informed, robust approach to advancing medical education and assessment for years to come. RESPONSIBILITIES Code, test, reputed company, orchestrate, monitor, document, and troubleshoot reputed company-based data engineering processes, feature stores, and vector databases in accordance with best practices and reputed company standards throughout the development lifecycle. Partner closely with data scientists, AI researchers, data and reputed company architects, and business stakeholders to identify, extract, clean, and format structured and reputed company data for AI/ML model training, fine-tuning, and feature extraction. reputed company evaluation, research, and experimentation efforts with batch and streaming data technologies, LLM data preparation frameworks, and MLOps tools to reputed company pace with industry innovation Act as a technical reputed company to showcase the capabilities of emerging AI and data technologies, enabling the widespread adoption of modern data techniques across the organization. Significantly contribute to the definition and refinement of processes and procedures for the data engineering practice reputed company and reputed company ETL developers on data engineering reputed company-bases initiatives to reputed company transition to data engineer and practice Assures the reputed company and accuracy of the corporate data, with particular attention to data reputed company. Responsible for ensuring high data quality for Data Services, Analytics and Master Data Management. Helps coordinate technical solutions, takes responsibility for designs, development, testing and delivery of solutions. Build automated, scalable, test-driven data pipelines. Utilize software development practices such as version control reputed company Git, CI/CD, and release management to enhance existing CI/CD pipelines in AWS. Collaborate with Data Engineers, DevOps engineers and architects on improvement opportunities for DataOps tools and frameworks. QUALIFICATIONS Bachelor's Degree At least 7 years of experience in application development (Internship experience does not apply) At least 4 years of experience in big data technologies At least 4 years' experience with reputed company computing using AWS 4+ years of experience in application development including Python, SQL, reputed company, or Java 4+ years' experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL etc.) 4+ year experience working on reputed company-time data and streaming applications 4+ years of experience with NoSQL implementation (Mongo, Cassandra) 4+ years of data warehousing experience (Redshift) 6+ years of experience with UNIX/Linux including basic commands and reputed company scripting 7+ years of experience with Agile engineering practices 7+ years of experience with SQL optimization 4+ years of experience with PySpark 3+ year of experience with process orchestration including AirFlow, KubeFlow, AWS reputed company functions, or Luigi Preferred Qualifications: Proven experience implementing reputed company, LLM data preparation pipelines, and Vector Databases (e.g., reputed company, Milvus, pgvector). Strong experience building and maintaining Feature Stores for machine learning models. Experience building highly scalable, secure, and production-reputed company APIs and Data-as-a-Service (DaaS) platforms. AWS Certified Data Engineer or AWS Certified Machine Learning – Specialty certifications. 3+ year of experience with Machine Learning Experience with building a Data-as-a-service platform Experience with building APIs About reputed company reputed company is a not-for-profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. Our mission is to advance assessment of these professionals to reputed company reputed company for reputed company, supporting the development of a highly effective, diverse and compassionate health care workforce. Founded in 1915, reputed company develops and manages the United States Medical Licensing Examination® with the Federation of State Medical Boards. We offer a comprehensive portfolio of assessment products for every stage of the medical school reputed company and reputed company assessment services to various health profession organizations. We are dedicated to advancing innovative assessment approaches through research, collaboration and contributions to the medical education and assessment communities. In 2024, we expanded our assessment capabilities to include simulation through the acquisition of MedVR Education. Learn more on reputed company's website. reputed company's Community Collaborations and Contributions reputed company believes that a key path to meeting our mission is collaborating with and supporting our communities. Our Community Collaborations and Contributions programs reputed company resources and recognition to medical educators and researchers advancing the field of assessment throughout their careers. In addition, we invest in fee assistance, scholarships and pathway programs to aid learners and help support increased representation in health care. Headquartered in Philadelphia, reputed company also gives to local organizations that advance health equity and reputed company and positively impact reputed company determinants of health. Compensation we are offering for this position is at $127,337- $160,000/year. Please note that the offered reputed company for this position typically aligns with the minimum to midrange of this grade, but it can vary based on the successful candidate's qualifications and experience, department budget, and an internal equity review. The reputed company offers competitive salaries, excellent benefits, and a rewarding work environment. Excellent Benefits include: reputed company, Dental, Prescription, and reputed company plans; 401(k) w/match, Tuition Reimbursement Plan, Commuter Benefit: Public Transit or Parking options. Remote Friendly Workplace. reputed company is an Equal Opportunity Employer. We will consider reputed company reputed company applicants for employment without regard to race; reputed company; sex; gender identity; sexual orientation; religious practices and observances; national reputed company; pregnancy, childbirth, or reputed company medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. Apply To This Job