Required Skills
About the Job
Societe Generale Global Solution Centre is seeking a seasoned Lead Big Data Engineer with extensive experience in Spark, Scala, and Java for our Bengaluru/Bangalore office. This full-time Technical Lead role requires a strong hands-on engineer with mature leadership capabilities, architectural judgment, and a commitment to software craftsmanship. You will be instrumental in developing high-quality, maintainable code using principles like TDD and CI/CD, and will lead the development of large-scale ETL/ELT pipelines on Apache Spark within Azure-based ecosystems.
Your responsibilities will include overseeing data processing frameworks in Hadoop and Hive, optimizing Spark/Hive workloads, and contributing to the architecture and operation of distributed applications on Kubernetes. You will also drive the adoption of containerization best practices, build and maintain Azure infrastructure using Terraform, and ensure end-to-end observability using tools like Prometheus, ELK, and Grafana. A key aspect of this role involves defining and owning the platform's CI/CD strategy using Jenkins and GitHub Actions, and automating critical processes. You will provide leadership on PostgreSQL cluster management, including HA setup and performance tuning. Furthermore, you will ensure service reliability and availability, implement comprehensive monitoring and alerting, and guide L2/L3 support for production stability.
As a leader, you will mentor a cross-functional team, translate business requirements into technical designs, and collaborate effectively with distributed teams and stakeholders. We are looking for a candidate with 10+ years of experience in software or data engineering, strong expertise in Apache Spark (Java/Scala), Hadoop, Hive on Azure, Kubernetes, Docker, Azure Kubernetes Service (AKS), PostgreSQL (HA + automation), and CI/CD tools. A good understanding of networking fundamentals and production support experience is essential. Experience with Azure Data Lake, Databricks, or Synapse, and microservices architecture is a plus. Join us to make a positive impact and grow your expertise in a stimulating and caring environment.
