Summary The Backend Software Engineer will join the Company&rsquos Data Engineering Platform team and play a key role in building and evolving large-scale backend data systems, real-time and batch processing pipelines, and AI-enabled services. This position focuses on backend engineering using Scala, Python, and Java, distributed data platforms including Cloudera and Spark, and cloud-based architectures. The role also contributes to AI-powered automation initiatives that support analytics, decision-making, and operational efficiency across the organization. The ideal candidate will have experience working with large-scale datasets, modern data platforms, and cloud technologies while collaborating with cross-functional teams to deliver reliable and scalable backend solutions.
Responsibilities
Design, develop, and maintain backend services and data pipelines using Scala, Python, and Java.
Build, optimize, and support batch and streaming workloads on Cloudera Data Platform (CDP) using Spark.
Configure, monitor, troubleshoot, and improve platform performance using Cloudera Manager.
Design and implement high-quality DataMart&rsquos and curated datasets with a focus on data integrity, reliability, and performance.
Develop and integrate AI-powered agents that support intelligent automation, operational efficiency, and scalable workflows.
Contribute to anomaly detection, operational intelligence, and workflow automation initiatives.
Support cloud-based data and compute workloads within AWS environments.
Utilize Databricks for large-scale data processing and advanced analytics workloads.
Contribute to cloud-native and hybrid architectures that integrate on-premises and cloud-based platforms.
Enable downstream consumers, including analytics, reporting, and visualization tools, through reliable and scalable backend data interfaces.
Collaborate with analytics, business, and technical teams to understand requirements and deliver backend solutions that support evolving business needs.
Create and maintain technical documentation, including architecture diagrams, design documents, and data flow specifications.
Participate in Agile and Scrum ceremonies, design reviews, and cross-functional planning activities.
Support knowledge sharing and mentor team members in backend engineering, big data technologies, and AI-related concepts.
Continuously evaluate opportunities to improve platform scalability, automation, stability, and efficiency.
Education Requirements
Bachelor&rsquos degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent professional experience.
Experience Requirements
Strong hands-on experience developing backend or data-intensive systems using Scala, Python, and/or Java.
Experience working with Cloudera Data Platform (CDP) and Spark.
Experience supporting large-scale distributed data processing environments.
Experience working in Agile or Scrum-based development teams.
Required Skills
Proficiency in Scala, Python, and Java for backend development.
Experience with Cloudera Data Platform (CDP).
Strong knowledge of Apache Spark.
Familiarity with Cloudera Manager for cluster administration, monitoring, and troubleshooting.
Strong understanding of distributed systems and large-scale data processing.
Strong data modeling and data engineering skills.
Experience designing, developing, and supporting data pipelines.
Excellent analytical and problem-solving abilities.
Ability to work independently in complex technical environments.
Strong verbal and written communication skills.
Experience creating technical documentation and design specifications.
Ability to collaborate effectively with cross-functional stakeholders.
Preferred Skills
Experience building, integrating, or supporting AI-driven agents and intelligent automation solutions.
Experience with Databricks for data engineering or machine learning workloads.
Experience working within AWS environments, including services such as S3, EC2, EMR, Glue, Lambda, IAM, or similar technologies.
Knowledge of streaming and big data technologies, including Kafka.
Knowledge of Hadoop ecosystem technologies.
Experience with Hive and Impala.
Exposure to model monitoring or AI platform enablement activities.
Experience with ETL tools such as Informatica.
Experience working within large enterprise environments.
Familiarity with modern cloud and hybrid architecture patterns.
#AditiConsulting # 26-05061
More roles at Aditi Tech Consulting Private Limited