Senior Data Engineer II

Company: Biological Dynamics
Location: San Diego
Posted on: November 19, 2023

Job Description:

Biological Dynamics, Inc. is a biotechnology company committed to improving global health outcomes by detecting disease in its earliest stages. Our proprietary Verita--- platform simplifies access to biomarkers, nanoparticles, and nucleic acids, enabling differentiated multiomics applications. We are applying our proprietary methods with machine learning to detect early-stage cancer and other diseases. For more information, please visit www.biologicaldynamics.com.

Our talented staff thrive in an open working environment where curiosity, insights and vision are part of the equation. With an inclusive culture and comprehensive benefits package, we offer a terrific opportunity to discover, learn and grow. If you are ready to be inspired, challenged, and want to join a dynamic team where you can contribute to developing a revolutionary new way to understand diseases and save lives, you have come to the right place.

Job Summary

Biological Dynamics is seeking a Senior Data Engineer II to work hybrid in San Diego, Ca. The Senior Data Engineer will play a crucial role in designing, building, and maintaining data infrastructure and pipelines that support our vision statement - "A world where illness is never diagnosed too late." The Senior Data Engineer will take responsibility for Biological Dynamic's Extract Transform Load (ETL) process, transitioning raw measurement results to scalable pipelines, datasets, and cloud infrastructure. As the primary interface for Biological Dynamics' datasets, this position will collaborate with cross-functional teams to ensure data availability, reliability, and scalability, enabling data-driven decision-making and innovation within the organization.

• Design, develop, and maintain robust and scalable data pipelines that collect, process, and store data from various sources, including medical devices and diagnostics tools in a regulated environment.
• Integrate disparate data sources, both structured and unstructured, to create a unified and comprehensive data repository for analysis and reporting.
• Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data.
• Optimize data pipelines and infrastructure for maximum performance and efficiency, ensuring minimal latency in data processing.
• Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide data engineering support for analytical and machine learning projects.
• Proactively monitor data pipelines and infrastructure for issues, troubleshoot and resolve data-related problems, and perform routine maintenance tasks.
• Create and maintain documentation for data pipelines, infrastructure, and processes to facilitate knowledge sharing and onboarding of new team members.
• Keep up to date with industry best practices, emerging technologies, and trends in data engineering and healthcare informatics.
• Design and implement data pipelines using Azure Cloud Services and Python. Build scalable data warehouses using Azure Cloud Services.
• Design and build robust data models for various cases such as reporting, API, or web applications.
• Maintain and improve existing data pipeline processes, services, and web applications (Django).
• Proactively contribute to a high level of understanding of product requirements, industry needs, and patent applications.
  Experience/Education/Skills
  
  • Typically requires a minimum of 8 years of related experience and a Bachelor's and/or at least 6 years and a related Master's degree in a scientific/engineering discipline, or a PhD and at least 4 years, equivalent combination of education and experience.
  • 5+ years of experience developing API services using Python/Azure Cloud Services.
  • 5+ years of experience utilizing Apache Spark (Pyspark) and Databricks.
  • Must be knowledgeable in Python and Python packages - pandas, AzureML.core, MLflow.
  • Must be proficient with Python and SQL (can solve easy and medium difficulty Leetcode problems).
  • Strong experience with data warehousing, ETL processes, and data modeling.
  • Familiarity with industry compliance regulations, including HIPAA and FDA, is highly preferred.
  • Must be well versed in container services such as Azure Container Apps.
  • Must be knowledgeable in other technology stocks such as web framework (Django).
    The estimated base salary range for this role is: $140,000 - $160,000 annually. Compensation decisions are dependent on several factors including, but not limited to, an individual's qualifications, location where the role is to be performed, internal equity, and alignment with market data.
    
    Biological Dynamics is an Equal Opportunity Employer that does not discriminate on the basis of actual or perceived race, color, religious creed, national origin, ancestry, citizenship status, age, sex or gender (including pregnancy, childbirth and related medical conditions), gender identity or expression (including transgender status), sexual orientation, marital status, military service and veteran status, physical or mental disability, protected medical condition as defined by applicable state or local law (such as cancer), genetic information, or any other characteristic protected by applicable federal, state or local laws and ordinances.

Keywords: Biological Dynamics, San Diego , Senior Data Engineer II, Engineering , San Diego, California