Required Skills
About the Job
Wells Fargo is seeking a Lead Software Engineer to join our team. In this role, you will design and execute comprehensive data migration strategies to Google Cloud Platform (GCP), moving data from on-premises and legacy systems to GCP storage solutions like BigQuery and Bigtable. You will build, optimize, and maintain scalable data pipelines using Python, SQL, and distributed processing frameworks like Apache Spark on Dataproc. Implement batch and streaming migration patterns with Apache Airflow (Cloud Composer) and Kafka. Your responsibilities will include schema mapping, data transformation, reconciliation, and validation to ensure data accuracy and consistency. You will leverage Generative AI (GenAI) and automation techniques to accelerate data discovery, schema understanding, mapping recommendations, and migration assessments, including applying GenAI for data transformation and code generation. You will also enable metadata management, lineage, and governance using Dataplex and Data Catalog, and handle large-volume data transfers using industry best practices. We are looking for candidates with 5+ years of Software Engineering experience, including 3-4 years of project leadership, with a strong background in big data implementation on GCP and GenAI automation. Desired qualifications include hands-on experience with GCP services (BigQuery, Dataproc, Bigtable, Dataplex, Data Catalog), Python for ETL/ELT, SQL, various databases (MySQL, Hive, MongoDB), Apache Airflow, Kafka, Google Pub/Sub, and experience with GenAI concepts and their integration with data platforms.
