Use Pluggable Sorting Algorithms
Starting in version 3.0.2 of the MapR software, you can specify the DMExpress custom sorting algorithm from SyncSort for your MapReduce job. The JAR file dmxhadoop_mrv1_mapr.jar is installed along with DMExpress in the /lib subdirectory of your DMExpress installation directory.
NOTE:
Prerequisite
Before you use the DMExpress custom sorting algorithm, add the libMapRClient.so and libjvm.so to the LD_LIBRARY_PATH environment variable on each TaskTracker node and then restart Warden on each TaskTracker node.
- To update the LD_LIBRARY_PATH, add the following text to
opt/mapr/conf/env.sh file:
export LD_LIBRARY_PATH=<full path to the directory libMapRClient.so>:<full path to the directory containing libjvm.so>:$LD_LIBRARY_PATH
- To restart Warden, run the following commands:
sudo service mapr-warden stop
sudo service mapr-warden start