Upgrading Drill

About this task

As of Drill 1.8, Drill can run under the MapR Warden service or under YARN. You can upgrade Drill 1.8 to Drill 1.9 under Warden or YARN. For example, if you are currently running Drill 1.8 under Warden or YARN, you can upgrade to Drill 1.9 and continue running Drill under the same management service. If you are running Drill under Warden, and you want to run Drill under YARN, see Migrate Drill to Run Under YARN.

When you upgrade, Drill should preserve storage plugin configurations for Drill under Warden or YARN. However, if Drill is managed by Warden, you must backup your configuration files and UDF JAR files and restore them after the upgrade. If Drill is managed by YARN, you do not have to backup and restore the configuration files because the site directory stores them independent of other Drill files.

Complete the following steps on Drill servers before you upgrade Drill:

Optionally, back up storage plugin configurations:
1. Open the Drill Web Console. The Drill node that you use to access the Web Console is a node that is currently running the Drillbit process.
2. Click the Storage tab.
3. Click Update next to a storage plugin.
4. Copy the configuration to a text file, and save the file.
5. Repeat steps c and d for each storage plugin configuration that you want to save.
Stop the Drillbit service on all nodes.
- For Drill under Warden, issue the following command:
```
maprcli node services -name drill-bits -action stop -nodes <node host names separated by a space>
```
- For Drill under YARN, complete the following steps:
  1. Issue the following command to remove all of the drillbits from the YARN cluster:
```
$DRILL_HOME/bin/drill-on-yarn.sh --site <path/to/the/site/directory/for/Drill/cluster> resize 0
```
    NOTE: Site is the site directory where the site specific files are stored. For example, opt/mapr/drill/site. If you have multiple Drill clusters running under YARN, repeat steps a and b for each cluster.
  2. Issue the following command to shutdown the Drill Application Master:
```
$DRILL_HOME/bin/drill-on-yarn.sh stop --site <path/to/the/site/directory/for/Drill/cluster>
```
To upgrade Drill, issue the command appropriate for your system on a Drill node as root or using sudo:
- For Drill under Warden, issue the command appropriate for your system:
  - RedHat/CentOS
```
yum update mapr-drill
```
  - Ubuntu
```
sudo apt-get install mapr-drill
```
- For Drill under YARN, issue the command appropriate for your system:
  - RedHat/CentOS
```
yum update mapr-drill-yarn
```
  - Ubuntu
```
sudo apt-get install mapr-drill-yarn
```
Complete the following post-upgrade steps for Drill under Warden or YARN.
- For Drill under Warden, complete the following steps:
  1. If you made any changes to configuration files in the previous version, compare and restore your previous configurations in the /opt/mapr/drill/drill-<version>/conf directory. Configuration files from the previous installation now reside in /opt/mapr/drill/OLD_DRILL_VERSIONS. Also, copy over any UDF or custom storage or format plugin JAR files that you added to the previous Drill directory.
    NOTE: The drill-override.conf file contains your ZooKeeper configuration and any other options specified in the file. The drill-env.sh file contains any options that you modified, such as Drill memory allocation. The logback.xml file contains changes you may have made to use Lilith.
  2. Run configure.sh to refresh the node configuration.
```
$ /opt/mapr/server/configure.sh -R
```
- For Drill under YARN, complete the following steps:
  1. Issue the following command to start the Application Master:
```
$DRILL_HOME/bin/drill-on-yarn.sh start --site <path/to/the/site/directory/for/Drill/cluster>
```
    NOTE: Note the provided URL at start-up.
  2. Optionally, issue the following command to add or remove drillbits to the cluster:
```
$DRILL_HOME/bin/drill-on-yarn.sh resize <value>
```
    NOTE: Value is the number of drillbits that you want started in the cluster. The default value is 1. You can also configure the resize value of the drill cluster in the Application Master Web UI. You can change the number of default drillbits in the drill-on-yarn.conf file.
  3. If you have multiple Drill clusters running under YARN, repeat steps 2 and 3 for each cluster.
Optionally, verify that the Drillbit service is running on the node(s):
- For Drill under Warden or YARN, issue the following command to verify the status of the Drillbit service from the command line:
```
jps
```
- For Drill under Warden, log in to the MCS at https://<host name>:8443 to verify the status of the Drillbit service.
- For Drill under YARN, access the Application Master Web UI, using the URL provided at start-up, to see the active drillbits.
If your storage plugin configurations were not preserved, restore them using the backup taken before the upgrade:
- For Drill under Warden or YARN, enter the following URL in a web browser to access the Drill Web Console for an individual drillbit:
```
http://<IP address or host name of drillbit>:8047/storage
```
- For Drill under YARN, access the Application Master Web UI, using the URL provided at start-up, to see the active drillbits.
Optionally, review the Drill log files for issues:
- - For Drill under Warden or YARN, you can access the Drill log files in the Drill Web Console.
  - For Drill under YARN, you can access the log files in the Drill container.
NOTE: If needed, you can issue the following commands to start and stop drillbits:
- For Drill under Warden, you can start, stop, or restart the Drillbit service on one or more nodes using the MCS or the following command:
```
$ maprcli node services -name drill-bits -action start|restart|stop -nodes <node host names separated by a space>
```
  Use the host name if possible. Using host names instead of IP addresses is a best practice.
- For Drill under YARN, you can start or stop the Application Master using the following command:
```
DRILL_HOME/bin/drill-on-yarn.sh stop|start --site <path/to/the/site/directory/for/Drill/cluster>
```
  NOTE: Site is the site directory where the site specific files are stored. For example, opt/mapr/drill/site.