JOB RESPONSIBILITIES
- Manage and resolve issues in Hadoop environments.
- Perform changes on production systems and escalate unresolved issues.
- Ensure data trustworthiness, security, and quality.
- Fulfill service requests within defined SLAs.
- Provide daily operational oversight of data assets/products.
- Architect, deploy, and manage Hadoop clusters using Cloudera or Hortonworks.
- Optimize HDFS, YARN, Hive, HBase, Spark, and Kafka for performance.
- Use Cloudera Manager or Ambari for monitoring and troubleshooting.
- Implement data replication and streaming using Oracle Goldengate, Qlik Replicate, or Kafka.
- Design and integrate scalable Lakehouse architectures.
- Perform capacity planning and performance tuning.
- Automate administrative tasks using Python, Shell scripting, or Terraform.
- Enforce data governance and security protocols (Kerberos, Ranger, Sentry).
- Collaborate with IT, business stakeholders, and data scientists.
- Lead innovation by staying ahead of emerging big data technologies.
- Set performance metrics, provide feedback, and foster a culture of innovation.
JOB REQUIREMENTS
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- Minimum 4 years of experience in Hadoop administration or big data management.
- Hands-on experience with Cloudera Data Platform (CDP) or Hortonworks Data Platform (HDP).
- Proficiency in Hadoop ecosystem tools (HDFS, YARN, Hive, HBase, Spark, Nifi, Kafka).
- Experience with Lakehouse technologies (Apache Iceberg, Hudi, Dremio).
- Familiarity with monitoring tools (Grafana, Prometheus, Cloudera Manager, Ambari).
- Strong understanding of data governance, compliance, and security best practices.
- Excellent communication and interpersonal skills.
- Ability to manage complex projects and lead technical teams.
- High level of confidentiality, integrity, and business ethics.
- Strong organizational and time management skills.
- Ability to work under pressure and meet tight deadlines.
- Willingness to support a 24/7 environment including on-call duties.
- Knowledge of industry-specific data practices and company culture.
- Ability to coach, mentor, and guide team members effectively.
- Detail-oriented with the ability to manage multiple assignments.