308 Big Data Hadoop jobs in Malaysia

Company Description

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail view and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population. For more information, visit NIQ.com.

Job Description

This Data Processing Specialist is responsible for data input validation and data cleaning (PH market), focusing on specific industry/category or set of categories (e.g., Beverages, Food, Health and Beauty Products). As a Data Processing Specialist in Input Validation at NielsenIQ, you support creating and delivering client outputs accurately and on time.

• Be the contact for commercial queries, leading resolution in a client-focused and timely manner in cooperation with Client Liaison and Commercial teams. Identify, investigate, and coordinate the resolution of data, process, or product-related queries.
• Plan and execute a complex daily workload to meet departmental and company schedules.
• Analyze and identify gaps and areas for improvement in coding, data input validation, and client instructions environments (CIP - Consumer Information Platform), driving process and tool convergence.
• Provide accurate and timely feedback to country teams, driving SOPs and best practices.
• Work across relevant Operations/Enablement teams to ensure delivery meets client expectations and satisfaction.
• Operate in a virtual/multi-cultural environment, liaising with stakeholders.

Qualifications

• Bachelor’s degree in Computer Science, Information Systems, IT, Statistics, Mathematics, Economics, Business, or equivalent.
• Fresh graduate or with up to 1 year of operational experience.
• Excellent organizational skills, team-oriented, and deadline-driven.
• High accuracy, proactivity, and attention to detail.
• Good analytical skills and aptitude for data and operational processes.
• Project management skills (critical path, task sequencing, problem-solving).
• Excellent written and verbal communication skills in English.
• Proficiency in Excel, including vlook-up, charts, spreadsheets, and tables.
• Knowledge of Python or R is an advantage.

Additional Information

• Flexible working environment
• Volunteer time off
• LinkedIn Learning access
• Employee Assistance Program (EAP)

Our Benefits

• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee Assistance Program (EAP)

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering comprehensive insights into consumer behavior and growth opportunities. In 2023, NIQ merged with GfK, expanding its global reach. With advanced analytics and state-of-the-art platforms, NIQ provides the Full View of consumer markets. NIQ operates in over 100 markets, covering more than 90% of the global population.

For more information, visit NIQ.com

Follow us on: LinkedIn | Instagram | Twitter | Facebook

Our commitment to Diversity, Equity, and Inclusion

NIQ is dedicated to reflecting the diversity of the clients and communities we serve. We aim to embed inclusion and diversity into all aspects of our work and are proud to be an Equal Opportunity Employer. Learn more at: NIQ Diversity & Inclusion

Job Description

• Acts as point-of contact for Data Validation processes (CPS)
• Act as secondary point of contact in the absence of team lead in managing E2E operational queries and deliverables
• Act as the SPOC for all the queries/data investigations and managing the internal stakeholders

Responsibilities

• 1st function IV - Input Validation level, Production flow, what kind of validation, overall flow of the production, how functions coordinate each other
• Database management system - understanding; can see a broader picture of the function
• Get data from different sources, data handling, data evaluation, data analytics
• Data Production process, reports
• what kind of output that needs to be impacted from the input approach
• 60%/ 70% system knowledge; 40%/ 30% analytics
• Focus on IV - should see a broader level, what can foresee/ view on data
• In-house system/ execution system
• Multiple projects (CPS Consumer Panel), research - getting input from consumers or different sources, data validation
• team size: 1 person to cover (back-up) 90% 1 man army (individual contributor)

• Bachelor of Science/ Computer Science/ Business Information Systems or IT equivalent
• Minimum experience of fresh to ~1 years
• Excellent organization skills, aptitude to work in team and meet deadlines
• A high degree of accuracy, proactivity and attention to detail
• Good analytics skills and aptitude for data and operational processes
• Project management aptitude (critical path, task sequencing, problem solving, etc.)
• Common business language knowledge (English): speaking and writing with fluency

Remote-hybrid based role, must be physically located within Malaysia and candidate must be a Country/Region Resident

Our Benefits

• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee-Assistance-Program (EAP)

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on:LinkedIn |Instagram |Twitter |Facebook

Our commitment to Diversity, Equity, and Inclusion

NIQ is committed to reflecting the diversity of the clients, communities, and markets we measure within our own workforce. We exist to count everyone and are on a mission to systematically embed inclusion and diversity into all aspects of our workforce, measurement, and products. We enthusiastically invite candidates who share that mission to join us. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class. Our global non-discrimination policy covers these protected classes in every market in which we do business worldwide. Learn more about how we are driving diversity and inclusion in everything we do by visiting the NIQ News Center:

Job Description

• Acts as point-of contact for Statistical operations processes (CPS)
• Act as secondary point of contact in the absence of team lead in managing E2E operational queries and deliverables
• Act as the SPOC for all the queries/data investigations and managing the internal stakeholders

Responsibilities

• 1st function SO level, Production flow, what kind of validation, overall flow of the production, how functions coordinate each other
• Database management system - understanding; can see a broader picture of the function
• Get data from different sources, data handling, data evaluation, data analytics
• Data Production process, reports
• what kind of output that needs to be impacted from the input approach
• 60%/ 70% system knowledge; 40%/ 30% analytics
• Focus on SO - should see a broader level, what can foresee/ view on data
• In-house system/ execution system
• Multiple projects (CPS Consumer Panel), research - getting input from consumers or different sources, data validation
• team size: 1 person to cover (back-up) 90% 1 man army (individual contributor)

• Bachelor of Science/ Computer Science/ Business Information Systems or IT equivalent
• Minimum experience of 2 years
• Excellent organization skills, aptitude to work in team and meet deadlines
• A high degree of accuracy, proactivity and attention to detail
• Good analytics skills and aptitude for data and operational processes
• Project management aptitude (critical path, task sequencing, problem solving, etc.)
• Common business language knowledge (English): speaking and writing with fluency

Remote-hybrid based role, must be physically located within Malaysia and candidate must be a Country/Region Resident

Our Benefits

• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee-Assistance-Program (EAP)

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on:LinkedIn |Instagram |Twitter |Facebook

Our commitment to Diversity, Equity, and Inclusion

NIQ is committed to reflecting the diversity of the clients, communities, and markets we measure within our own workforce. We exist to count everyone and are on a mission to systematically embed inclusion and diversity into all aspects of our workforce, measurement, and products. We enthusiastically invite candidates who share that mission to join us. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class. Our global non-discrimination policy covers these protected classes in every market in which we do business worldwide. Learn more about how we are driving diversity and inclusion in everything we do by visiting the NIQ News Center:

Position Summary

We are looking for a full-time Big Data Engineer II to join our team at Shirlyn Technology in Petaling Jaya, Malaysia and Palo Alto, California. The Big Data team is at the heart of our operations, playing a critical role in scaling and optimizing our data infrastructure. As we continue to grow, we are seeking an experienced and driven Big Data Engineer to help us tackle complex data challenges, enhance data solutions, and ensure the security and quality of our data systems. You will be instrumental in building and deploying scalable data pipelines to ensure seamless data flow across systems, driving business success through reliable data solutions.

Job Responsibilities

• Collaborate with global teams in data, security, infrastructure, and business functions to understand data requirements and provide scalable solutions.
• Design, implement, and maintain efficient ETL (Extract, Transform, Load) pipelines to enable smooth data flow between systems.
• Conduct regular data quality checks to ensure accuracy, consistency, and completeness within our data pipelines.
• Continuously improve data pipeline performance and reliability, identifying and addressing any inefficiencies or bottlenecks.
• Ensure data integrity, security, privacy, and availability across all systems.
• Proactively monitor data pipelines, quickly diagnosing issues and taking action to resolve them and maintain system uptime.
• Conduct root cause analysis of data-related issues and work on long-term solutions to prevent recurring problems.
• Document pipeline designs, data processes, and troubleshooting procedures, keeping stakeholders informed with clear communication of updates.
• Provide on-call support for critical data operations, ensuring systems are running 24/7, with rotational responsibilities.

Job Requirements

• Bachelor’s degree in Computer Science, Information Systems, or a related technical field.
• 5+ years of hands-on experience building and optimizing data pipelines using big data technologies (Hive, Presto, Spark, Flink).
• Expertise in writing complex SQL queries and proficiency in Python programming, focusing on maintainable and clean code.
• Solid knowledge of scripting languages (e.g., Shell, Perl).
• Experience working in Unix/Linux environments.
• Familiarity with cloud platforms such as Azure, AWS, or GCP.
• High personal integrity and professionalism in handling confidential information, exercising sound judgment.
• Ability to remain calm and resolve issues swiftly during high-pressure situations, adhering to SLAs.
• Strong leadership skills, including the ability to guide junior team members and lead projects across the team.
• Excellent verbal and written communication skills, with the ability to collaborate effectively with remote teams globally.

Senior Java Engineer (Base Kuala Lumpur)

Job Advantages

Cutting-edge technology (web3 and blockchain), globalized company

Job Responsibilities

1. Responsible for the architectural design and development of the company's data center.
2. Continuously deliver data value to the company's businesses with high quality, low cost and high efficiency.
3. Provide solutions for platform architecture and performance optimization, and lead the rapid iteration of big data platform.
4. Enabling the rapid development of the company's business through data-driven approaches, continuously improving the company's core competitiveness.

Job Requirements

1. Bachelor's degree or above in computer science or related majors, with more than 3 years of experience in Java development, proficient in Java development, and an in-depth understanding of JVM principles.
2. Familiar with commonly used frameworks (Spring, Spring MVC, gRPC, Mybatis) and able to master their principles and mechanisms.
3. Solid computer fundamentals, mastery of computer operating systems and network architecture, familiar with commonly used algorithms, data structures, and design patterns.
4. Familiar with distributed systems, caching, messaging, and other mechanisms; experience with Redis, Kafka, Spark, Flink, Zookeeper, and TiDB is preferred.
5. Experience in data center or microservice design and development, with a focus on high availability and scalability of architecture is preferred.
6. Experience in tagging systems or algorithmic recommendation systems is preferred.
7. Full of enthusiasm and expectation for the blockchain industry, committed to working hard for the development of the industry.

Company : Web3 & Blockchain-focused Company

Job Advantages

• Cutting-edge technology (Web3 and blockchain)
• Globalized company

Job Responsibilities

As a Senior Java Engineer , you will play a pivotal role in the development and optimization of the company’s data infrastructure, ensuring it supports the evolving needs of our blockchain-focused business. Your primary responsibilities will include:

• Architectural Design & Development : Lead the design and development of the company’s data center, ensuring robust performance and scalability.
• Data Platform Optimization : Provide ongoing solutions for platform architecture and performance optimization to support the company’s rapid growth and business needs.
• Business Enablement : Drive data-driven solutions that accelerate business development and continuously enhance the company's core competitiveness.
• Big Data Iteration : Lead the rapid iteration of big data platforms, ensuring efficiency, cost-effectiveness, and high-quality output.

Job Requirements

We are seeking candidates who are passionate about blockchain technology and possess a strong technical foundation. Ideal candidates will meet the following criteria:

• Education : Bachelor’s degree or above in Computer Science or related majors.
• Experience : More than 3 years of experience in Java development, with a deep understanding of JVM principles and Java development best practices.
• Frameworks : Proficient in frameworks such as Spring, Spring MVC, gRPC, Mybatis , and an ability to understand their underlying principles and mechanisms.
• Core Computer Fundamentals : Strong knowledge of computer operating systems, network architecture, and proficiency in commonly used algorithms, data structures, and design patterns.
• Distributed Systems : Familiarity with distributed systems, caching mechanisms (Redis), messaging systems (Kafka), and big data processing frameworks like Spark, Flink , and Zookeeper . Experience with TiDB is a plus.
• Experience in Microservices : Experience in the design and development of data centers or microservices, with an emphasis on high availability and scalability.
• Tagging Systems/Recommendation Systems : Experience with tagging systems or algorithmic recommendation systems is highly desirable.
• Passion for Blockchain : A strong enthusiasm for the blockchain industry and a commitment to contributing to its growth and development.

Why Join Us?

This role offers the opportunity to work at the forefront of blockchain and Web3 technologies within a global company. You will have the chance to develop and optimize critical infrastructure that powers innovative and scalable solutions in the blockchain space. If you’re ready to work in a fast-paced and cutting-edge environment, this role could be the perfect fit for you.

Apply now to be part of an exciting journey in revolutionizing the Web3 ecosystem!