Software Engineer

Job Summary

As a Software EngineerforDataPelagodata processingcustomer success, you willdevelop workload assets and automation tools for productvalidation andproductsupportwhich will helpensurecustomers deploy and operateDataPelagodata processing products successfully in production.Youwill apply AI effectively in combination with the knowledge from both product validation and product support to proactively avoid bugs and issues and to drastically reduce time to diagnose, mitigate, and resolve bugs and issues.

Job Requirements

What You'll Do:

• Develop synthetic workload queries and datasets for testing DPA-S, which reflect the essential environment and workload characteristics represented in the field.
• Develop analysis and diagnostic tools that analyze various logs from DPA-S, Apache Spark, and their deployment environment to aid in pinpointing the root cause of functional and performance anomalies and potential mitigations and resolutions.
• Field and address or escalate customer support issues as part of the product support rotation.
• Conduct end-to-end product validation, collaborating with others to define, review, and execute test plans.
• Analyze artifacts from product validation testing and issues raised by customers to identify potential configuration tuning opportunities, product usage recommendations, and product bugs.
• Leverage automation and AI techniques effectively in the entire process of product validation and product support.
• Create bug reports, RCA documents, configuration guides, and best practice documents.

What You'll Bring:

• 5+ years ofexperience related todevelopment,validationand supportof enterprise data processing software products.
• 2+ years of experienceproviding direct support to external customers.
• 2+ years of experience supporting Apache Spark pipelines for dataengineering,analytics.Proficient in running Spark jobs from various clients, Spark resource provisioning, Spark parameter configuration and tuning, using Spark History Server and Spark UI,analyzing Spark event logs and other Spark logs.
• Solid knowledge of SQL and Python.
• Proficient intesting data products and testautomation.
• Good understanding of the deployment, and operations of data processing products and pipelines in the cloud - in SaaS, VM-based, container-based, and serverless scenarios. Experience with GCP, AWS, and their Apache Spark related offerings preferred.
• Good customer engagement and communication skills.

Education

A Minimum of 5 years of experience is required. 5 to 7 years of experience is preferred.