HNM Systems
HNM Systems is the leading tech-enabled professional services provider, shaping the workforce across the telecom, energy, and technology sectors. We connect specialized talent with direct hire, contract-to-hire, and consulting opportunities that support fiber expansion, digital infrastructure, wireless networks, and smart grid energy solutions. Our innovative approach delivers high-impact outcomes, with a 94% success rate in talent placement and project execution. Relentlessly people-driven, HNM Systems sets the industry standard through exceptional employee experience and long-term client value.
Why This Opportunity Stands Out
This is an opportunity to work on large-scale, cloud-native data platforms that support critical business operations and high-volume data processing. You will contribute to the design and optimization of modern Big Data solutions using leading technologies across AWS, Apache Spark, and distributed computing frameworks. The role offers exposure to enterprise-scale data engineering challenges, cloud automation initiatives, and advanced microservices architectures while collaborating with highly skilled engineering teams.
Overview
We are seeking an experienced Senior Big Data Engineer to design, develop, and support large-scale Big Data applications and data processing solutions on AWS Cloud. The ideal candidate will possess extensive experience building scalable distributed systems, developing both API-based and batch-processing applications, and implementing cloud-native solutions that drive business-critical data initiatives.
Key Responsibilities
- Design, develop, implement, maintain, and support scalable Big Data applications and data processing pipelines on AWS Cloud.
- Build and support API-driven microservices and batch-processing solutions using Java, Scala, Kotlin, and Python.
- Develop, optimize, and maintain distributed data processing applications using Apache Spark, Hadoop, HDFS, MapReduce, and Cassandra.
- Design and maintain RESTful APIs and SOAP Web Services to support enterprise integrations.
- Develop cloud-native solutions leveraging AWS services including EMR, EC2, ECS, S3, AWS API Gateway, Step Functions, and Airflow.
- Perform application performance tuning and optimization across Spark, Hadoop, EMR, Java, and Python workloads.
- Develop Linux-based automation solutions using Shell Scripting and Python.
- Collaborate with Data Engineering, Architecture, DevOps, Product, and cross-functional stakeholders.
- Build and maintain CI/CD pipelines that automate application delivery and infrastructure provisioning.
- Participate in code reviews, production support, troubleshooting, and implementation of engineering best practices.
- Ensure solution scalability, reliability, maintainability, and operational efficiency.
- All other duties as assigned.
Required Qualifications
- 7+ years of experience in software development, data engineering, and Big Data application development.
- Proven experience designing and supporting scalable data platforms and distributed processing systems on AWS Cloud.
- Strong hands-on experience with:
- Scala
- PySpark
- Java
- Kotlin
- Python
- Deep experience with:
- Apache Spark
- Hadoop Ecosystem
- HDFS
- MapReduce
- Cassandra
- Experience developing:
- REST APIs
- SOAP Web Services
- Microservices-based applications
- Strong AWS expertise including:
- Amazon EMR
- EC2
- ECS
- S3
- AWS API Gateway
- AWS Step Functions
- Apache Airflow
- Experience administering and developing solutions in Linux environments.
- Advanced Shell Scripting and Python automation skills.
- Experience building and maintaining CI/CD pipelines using:
- Maven
- Gradle
- Jenkins
- Artifactory
- AWS CodeCommit
- AWS CloudFormation
- Strong experience with performance tuning and optimization of large-scale batch-processing applications.
Preferred Qualifications
- Experience with Snowflake data warehousing and analytics.
- Experience building automated data ingestion and transformation pipelines using Spark and Snowflake.
- Familiarity with:
- Git
- GitHub
- GitLab
- Bitbucket
- IntelliJ IDEA
- PyCharm
- Experience working within Agile/Scrum environments.
- Exposure to enterprise DevOps practices and automation frameworks.
EEO / Disclaimer (India)
HNM Systems is committed to providing equal employment opportunity and fair treatment in accordance with applicable Indian law. All qualified applicants will be considered based on merit, qualifications, business needs, and the requirements of the role. India’s legal framework includes protections around equal remuneration and specific protections related to disability and workplace harassment.
HNM Systems does not tolerate unlawful discrimination or harassment in recruitment or employment and is committed to maintaining a respectful workplace for all employees and applicants. HNM Systems will administer its employment practices in accordance with applicable laws and workplace policies in the relevant jurisdiction.
Where required by applicable law, HNM Systems will provide reasonable workplace support or accommodation and will consider qualified applicants in a manner consistent with applicable legal requirements. India’s Rights of Persons with Disabilities Act includes non-discrimination in employment and equal opportunity policy requirements for covered establishments.