Job description
Our client, one of the largest telecommunication companies in Australia, is requiring the skills of a Big Data Engineer. You will primarily be responsible for leading, architecture, design and implementation of data pipelines on Networks Big Data platforms (on premise and on cloud). This is a migration; moving from On-Prem to GCP Cloud.
Essential Requirements:
- Understand business requirement and develop solution design
- Fine-tuning of new and existing data pipelines
- Schedule and maintain data pipelines
- Drive optimization, testing and tooling to improve data quality & data governance
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes and optimizing data delivery
- Architect, design & build robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external users
- Troubleshoot Big Data platforms and software, including performance-tunes of BI applications
- Budgeting, tracking and forecasting capital & operational expenditure
- Interface with product vendors to keep abreast of new technologies, pricing and customer applicability. Participate in vendor evaluations
- Manage resource capacity and staffing
- Identify capability for the team and provide guidance and training.
Technical Stack:
- GCP (Big Query, Dataflow and ETL orchastration)
- Hadoop
- Extensive ETL experience with an understanding of Big Data Architecture
- Airflow
- Kafkan migration (Dataflow to process data in real time).
Flexible Work from Home Arrangements
To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Jennifer Bohm on 02 9464 5518 or email: [email protected] and quote the above job reference number.
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.
