Search

Sunday 22 January 2023

How do you increase mappers in Hadoop

 To increase the number of mappers in Hadoop, you can increase the number of input splits by setting the mapreduce.input.fileinputformat.split.maxsize configuration parameter. Increasing this value can increase the number of mappers used by the application, but it can also lead to decreased performance if the splits are too large. The default value for mapreduce.input.fileinputformat.split.maxsize is 128 MB.

You can also tweak the size of each split and memory allocations for the application to optimize its performance.


Deprecated property: mapred.min.split.size