Configuring LDAP in AWS Edition Enterprise

note

• This requires deleting of all existing Dremio-created users. Some user-specific data will be lost in this procedure.

It is best to choose either Dremio-managed user authentication or LDAP authentication at the start of your Project. Switching from Dremio-managed to LDAP requires you to delete all existing internal users. Any user-specific data will be lost in this process.

As Dremio requires at least one user to create access credentials upon initial provisioning of Dremio, in all cases there will be at least one internal user. This will require an initial setup of deleting all existing users.

Important information when switching to LDAP-managed user control

• delete-all-users is a dremio-admin CLI command that allows the admin to delete all internal users, roles, and home spaces associated with the users. This command runs successfully when the Dremio server is not running as it accesses the KV store.
• When you restart Dremio in non-ldap mode, you will be directed to the first-user creation page. If you restart in ldap mode, you will be directed to the login page.
• When the delete-all-users command is executed, all internal users, roles, and home spaces (i.e any data residing within the home spaces) will be lost.

To switch from Dremio-managed user authentication to LDAP-managed user authentication

1. Enter your license activation key to enable Enterprise features:

   • Go to Admin > Engine > Activation.
   • Enter or paste your license key in the text field, then click Activate.

2. SSH into the coordinator node

3. Stop Dremio (sudo service dremio stop)

4. Cd into /opt/dremio/bin

5. Delete all existing internal users, roles, and home spaces (sudo -u dremio ./dremio-admin delete-all-users)

6. Create ad.json in /opt/dremio/conf

7. Add config to /opt/dremio/conf/dremio.conf

   • services.coordinator.web.auth.type=ldap

   • services.coordinator.web.auth.config=ad.json

   note

   The services.coordinator.web.auth.config configuration property replaces services.coordinator.web.auth.ldap_config, which is deprecated.

8. Start dremio (sudo service dremio start)

9. Log in with LDAP credentials

Configuring Dremio for LDAP

note

Ad.json location: The location of the ad.json file is relative to the conf directory. In addition, the path to the file can be an absolute path; the file can live anywhere in the system.

Configuring LDAP

To configure LDAP, edit the ad.json file and add your properties.

This file located under the Dremio configuration directory (same as dremio.conf path) and is specified in the dremio.conf file as a coordinator service.

Properties

| Property | Required | Description | | - | - - | | bindMethod | No | Authentication method: UNAUTHENTICATED/ANONYMOUS/SIMPLE_BIND.
If this property is not specified, authentication defaults to SIMPLE_BIND mode.
Default: SIMPLE_BIND.
SIMPLE_BIND -- Connect and authenticate to LDAP server using bindDN and bindPassword.
ANONYMOUS -- Connect anonymously to the LDAP server. Note: When authenticating to Dremio, empty passwords for users are not allowed.
UNAUTHENTICATED -- Connect to LDAP server using an unauthenticated bind. bindDN is required. | | bindDN | No | Credentials for the user who connects from the Dremio LDAP client to the LDAP server.
If you are using ANONYMOUS mode, this property is not required. In particular, the following property must not be present:
"CN=admin,DC=drem,DC=io" | | bindPassword | No | Password credential for the user who connects from the Dremio LDAP client to the LDAP server.

NOTE: In Dremio 24+, bindPassword can be encrypted using the dremio-admin encrypt CLI command.
If you are using UNAUTHENTICATED or ANONYMOUS mode, this property must not be present. | | baseDN | Yes | The root path for all searches.
If userAttributes.baseDNs or groupAttributes.baseDNs are specified, they override baseDN for search purposes. | | searchScope | No | Scope of user searches:
SUB_TREE -- Searches subtrees below the specified baseDNs (default).
ONE -- Searches immediate children below the specified baseDNs.
BASE -- Match the exact entry. | | userAttributes | No | A mapping of LDAP user attributes to Dremio user attributes. This should include firstname, lastname and email | | userDNs | No | List of templates for user DNs. | | id | No | If used with the userAttributes property, id is the attribute for the login name. Defaults to sAMAccountName
If used with the groupAttibutes property, id is the attribute for the group name (default: CN). | | firstname | No | Attribute for the first name. | | lastname | No | Attribute for the last name. | | email | No | Attribute for the email address. | | userFilter | Yes | LDAP filter for validating users.
Only users who fit the specific criteria are allowed to authenticate. | | groupDNs | No | List of templates for group DNs. | | groupAttributes | No | A mapping of LDAP group attributes to Dremio group attributes. The baseDNs, searchScope, id properties are used. | | groupFilter | Yes | LDAP filter for groups. | | userGroupRelationship | No | Determines whether you are implementing lists based on users or groups.
USER_ENTRY_LISTS_GROUPS (Default) -- Specifies whether the user entry in LDAP lists the groups that they belong to. The group attribute in LDAP is configured by the groupMembership property.
GROUP_ENTRY_LISTS_USERS -- Specifies whether the group entry in LDAP lists the users that belong to it. | | groupMembership | No | memberOf value. This attribute specifies the groups that a user or a group belongs to. | | groupRecursive | No | Attribute of a user or a group that lists transitive group membership. |

Defining Users

The following examples show how to define users.

Example: Using userDNs

In the following example, a list of templates for user DNs are specified. The placeholder {0} is replaced with the username entered by the user and that DN is used during LDAP bind. Dremio attempts binding to the provided userDNs in the order they are specified.

userDNs example

"userDNs": ["cn={0},dc=staticsecurity,dc=dremio,dc=com"],
"userAttributes": {
	"firstname": "givenName",
	"lastname": "sn",
	"email": "mail"
}

Example: Using userAttributes

In this example, you are mapping LDAP user attributes to Dremio user attributes.

userAttributes example

"userAttributes": {
	"baseDNs": [
		"OU=test,OU=ad,DC=drem,DC=io"
	],
	"searchScope": "SUB_TREE",
	"id": "sAMAccountName",
	"firstname": "givenName",
	"lastname": "sn",
	"email": "mail"
}

note

In the attribute-based approach, userDNs field must not be specified.

note

In the DN-based approach, the baseDNs, searchScope, id properties cannot be specified under userAttributes.

caution

Do not change the value of id in the ad.conf file after you start Dremio. Changing the value can result in lost user privileges.

Example: Using userFilter

The following example using the userFilter property to limit access to members of the engineering group.

userFilter example

"userFilter": "&(objectClass=user)(memberOf=cn=engineering,OU=Groups,OU=ad,DC=drem,DC=io)",

Defining Groups

The following examples show how to define groups.

Example: Using groupDNs

In the following example, a list of templates for group DNs are specified. The placeholder {0} is replaced with the group name entered by the user. Dremio attempts searching given groupDNs in the order they are specified.

groupDNs example

"groupDNs": ["cn={0},OU=engg,OU=test,OU=ad,DC=drem,DC=io"]

note

In the DN-based approach, the groupAttributes property must not be specified.

Example: Using groupAttributes

In this example, the groupAttributes property is used to specify a list of baseDNs and group name IDs. These properties map LDAP group attributes to Dremio group attributes. The baseDNs, searchScope, id properties are required.

groupAttributes example

"groupAttributes": {
    "baseDNs": ["dc=roles,dc=dremio,dc=com"],
    "searchScope": "SUB_TREE",
    "id": "CN"
 }

note

In the attribute-based approach, groupDNs field must not be specified.

Defining User-Group Relationships

The relationship between users and groups can be defined with one of the following methods:

• Group membership
• Group lists

Group Membership

This method implements user entries in LDAP that list their group membership. (The user entries in LDAP are configured to list their group membership via the internal field memberOf.) See Sample Microsoft AD Configuration for a full example with the properties in context.

To establish this user-group relationship:

• Specify groupMembership property.
• Specify (if applicable) the groupRecursive property.

For example, if Dan is part of the BI group and the BI group is part of the engineering group, groupMembership property will contain only the BI group but the groupRecursive property will contain the engineering group.

Settings for groupMembership and groupRecursive properties

"groupMembership": "memberOf",
"groupRecursive": "transitive-memberOf",

note

If you include the groupRecursive key, make sure the value is the correct property for recursive lookups for your LDAP implementation. If you do not specify the correct property, Dremio skips recursive lookup and finds only the group membership. If you omit the groupRecursive key-value pair from your configuration, Dremio defaults to recursive lookup.

In addition, you can specify the "userGroupRelationship": "USER_ENTRY_LISTS_GROUPS" property-value, however, this property is optional since it is the default.

Group Lists

This method implements group entries in LDAP that lists users(by using the IDs of the users) that belong to that group . In the following sample, uid is the ID attribute that is used for the user entry and memberUid is the ID attribute that is used for the group entry.

Group list settings

"userGroupRelationship": "GROUP_ENTRY_LISTS_USERS",
    "groupEntryListsUsers": {
        "userEntryUserIdAttribute: "uid",
        "groupEntryUserIdAttribute": "memberUid"
},

For example, Dan's ID is 1234 and is represented by the attribute uid in Dan's LDAP records. Dan is part of the BI group and the LDAP entry for group BI lists memberUid = 1234. This indicates that Dan is a valid member of the group.

Securing the Connection

The LDAP connection can be secured by specifying either ANY_SSL or TRUSTED_SSL. The default, PLAIN, is an unencrypted connection.

To set up TRUSTED_SSL mode, do the following:

• Update ad.json with the connectionMode set to TRUSTED_SSL.
• Update dremio.conf with javax.net.ssl settings for trustStore and trustStorePassword.

Secure connection example

// ad.json entry for TRUSTED_SSL mode
"connectionMode": "TRUSTED_SSL",


// dremio.conf entry for TRUSTED_SSL mode with a self-signed certificate**
javax.net.ssl {
    trustStore: "<path/to/truststore/jks/file>",
    trustStorePassword: "trustStorePassword"
    }

note

Unencrypted connection: If you want an unencrypted connection, the connectionMode property is set to PLAIN: "connectionMode": "PLAIN",

Sample Group List Configuration

In the following example, the group list method is being implemented. You are defining the user-group relationship where the group entry lists the users that belong to that group).

To establish this user-group relationship:

• Set userGroupRelationship to GROUP_ENTRY_LISTS_USERS.
• Specify the groupEntryListsUsers property and it's sub-properties, userEntryUserIdAttribute and groupEntryUserIdAttribute.

In the following example, uid is the ID attribute that is used for the user entry and memberUid is the ID attribute that is used for the group entry.

Sample group list configuration

{
    "connectionMode": "PLAIN",
    "servers": [
        {
            "hostname": "host_ip",
            "port": 389
        }
    ],
    "names": {
        "bindDN": "CN=admin,DC=drem,DC=io",
        "bindMethod": "UNAUTHENTICATED",
        "bindPassword": "admin",
        "baseDN": "dc=drem,dc=io",
        "userFilter": "&(objectClass=posixAccount)",
        "userAttributes": {
            "baseDNs": [
                "OU=Users,OU=ldaptest,DC=drem,DC=io",
            ],
            "id": "uid",
            "firstname": "givenName",
            "lastname": "sn",
            "email": "mail"
        },
        "userGroupRelationship": "GROUP_ENTRY_LISTS_USERS",
        "groupEntryListsUsers": {
            "userEntryUserIdAttribute": "uid",
            "groupEntryUserIdAttribute": "memberUid"
        },
        "groupDNs": ["cn={0},OU=test,OU=ldaptest,DC=drem,DC=io",
                     "cn={0},OU=dev,OU=ldaptest,DC=drem,DC=io"],
        "groupFilter": "|(objectClass=posixGroup)(objectClass=sub)",
        "autoAdminFirstUser": false
    }
}

Sample Group Membership Configuration

The following sample is a Microsoft AD configuration that illustrates the group membership method (the user entry lists the groups that the user is a member of).

Sample group membership configuration

{
	"connectionMode": "PLAIN",
	"servers": [
		{
			"hostname": "<LDAP_HOST>",
			"port": 389
		}
	],
	"names": {
		"bindDN": "CN=Admin,OU=Users,OU=ad,DC=drem,DC=io",
		"bindPassword": "password",
		"baseDN": "dc=dremio,dc=io",
		"userFilter": "&(objectClass=user)(|(memberOf=CN=QA,OU=temps,OU=test,OU=ad,DC=drem,DC=io)(memberOf=CN=qa,OU=engg,OU=test,OU=ad,DC=drem,DC=io))",
		"userAttributes": {
			"baseDNs": [
				"OU=test,OU=ad,DC=drem,DC=io"
			],
			"searchScope": "SUB_TREE",
			"id": "sAMAccountName",
			"firstname": "givenName",
			"lastname": "sn",
			"email": "mail"
		},
		"groupMembership": "memberOf",
		"groupRecursive": "transitive-memberOf",
		"groupDNs": ["cn={0},OU=engg,OU=test,OU=ad,DC=drem,DC=io"],
		"groupFilter": "(objectClass=group)",
		"autoAdminFirstUser": true
	}
}

Granting Admin Privileges at Configuration Time

By default, the first valid LDAP user to log in to Dremio is given the Admin role. This is done via the autoAdminFirstUser : true property in the ad.json file. The assignment of the first valid user to Admin occurs during initial login.

Alternatively, you can specify a list of users and/or groups to be given the Admin role. This is valid only during initial login; it is used for boot-strapping only.

caution

When you set autoAdmiFirstUser to false, then you must specify users/groups in a bootstrap-admin-users.json file. Otherwise, an administrator won't be a specified.

To specify users/groups as administrators up-front, during initial login:

1. In the ad.json file, set autoAdminFirstUser to false.

   autoAdminFirstUser setting

   autoAdminFirstUser : false

2. Create a file called bootstrap-admin-users.json under the Dremio configuration directory.

3. In bootstrap-admin-users.json, add the users and groups arrays and specify the names of the users and groups that should belong to the ADMIN role. Use the Common Name (CN) for each user and group that you list in the arrays.

   Example users and groups properties

   {
     users: ["joe", "bob"],
     groups: ["marketers", "sales wizards"]
   }

note

The users/groups specified in the bootstrap-admin-users.json file are used only during initial login and only when autoAdminFirstUser is set to false.
To add other Users/Groups to the Admin role after the initial login, use the Dremio UI.

Deleting Users

note

Important: When deleting users from LDAP, ensure that all Personal Access Tokens (PATs) are also deleted.