Mask Sensitive Data in Query Logs and Profiles

Drill includes the following Logback encoder and layout classes that enable you to configure Drill logs such that data in the final message is masked:

• org.apache.drill.logback.MaskingPatternEncoder
• org.apache.drill.logback.MaskingPatternLayout

The Drill encoder and layout provide the same functions as the following encoder and layout:

• ch.qos.logback.classic.encoder.PatternLayoutEncoder
• ch.qos.logback.classic.PatternLayout

The following examples demonstrate how to configure the Drill masking pattern encoder and layout in the /opt/mapr/drill/drill-<version>/conf/logback.xml file:

Masking Pattern Encoder Example

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="org.apache.drill.logback.MaskingPatternEncoder">
      <rulesConfig>${pathToJsonConfig}</rulesConfig>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <root>
    <level value="error" />
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

Masking Pattern Layout Example

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
      <layout class="org.apache.drill.logback.MaskingPatternLayout">
        <rulesConfig>${pathToJsonConfig}</rulesConfig>
        <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
      </layout>
    </encoder>
  </appender>

  <root>
    <level value="error" />
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

The JSON file that defines the masking rules must include an array of objects with the following fields:

• search
• replace
• description

Use these fields to define the rules, as shown in the following examples:

[
  {
    "search": "([\\w\\d]+\\.)+(com)",
    "replace": "secret.domain.com",
    "description": "Mask domain names"
  },
  {
    "search": "MagicCompany",
    "replace": "TopSecretCompany",
    "description": "Mask company name"
  }
]

The following table describes each of the fields that define the masking rules: