Dremio Cloud
Dremio Software
 
On this page

    Clean Metadata

    This topic provides usage information for the dremio-admin clean CLI command.

    Requirements

    • Perform a backup before running the command (see Backup for more information).

    • Shut down all cluster nodes completely before running the command (see Startup/Shutdown for more information).

    Syntax

    Clean command syntax 
    dremio-admin clean [args...]

    Options

    Clean command options 
        -c, --compact
      compact kvstore
      Default: false
    -o, --delete-orphans
      delete orphans records in kvstore (e.g., old splits)
      Default: false
    -h, --help
      show usage
    -j, --max-job-days
      delete jobs, profiles, and temporary dataset versions older than provided number of days
      Default: 2147483647
    -i, --reindex-data
      reindex data
      Default: false
    -p, --delete-orphan-profiles
      remove orphaned jobs
      Default: false
    -d, --delete-orphan-datasetversions
      delete dataset versions older than the provided number of days
      Default: 2147483647

    Examples

    This section provides examples for cleaning up different types of data.

    Compact Metadata

    Compacts metadata store entries.

    Compact metadata store entries 
    dremio-admin clean -c
dremio-admin clean --compact

    Delete Orphaned Entries

    Deletes orphaned metadata store entries.

    Delete orphaned metadata store entries 
    dremio-admin clean -o
dremio-admin clean --delete-orphans

    Delete Jobs

    Deletes jobs, profiles, and temporary dataset versions older than the specified threshold days. If no threshold is specified, items older than the default number of days (2147483647) will be deleted. Using the default threshold will effectively not delete anything. It is recommended that you use a reasonable value (e.g., 7, 14, 30, etc.).

    Delete jobs 
    dremio-admin clean -j=7
dremio-admin clean --max-job-days=7

    Re-index Data

    Re-index data.

    Re-index data 
    dremio-admin clean -i
dremio-admin clean --reindex-data

    Delete Orphaned Profiles

    Delete orphaned Dremio job profiles.

    Delete orphaned job profiles 
    dremio-admin clean -p
dremio-admin clean --delete-orphan-profiles

    Delete Orphaned Dataset Versions

    note:

    This command is available in Dremio 19.6.3+, 19.8.0+, 20.4.0+, 21.2.0+, and 22.0.0+.

    Deletes dataset versions that Dremio is not using that are older than the specified threshold days. If no threshold is specified, dataset versions older than the default number of days (2147483647) will be deleted. Using the default threshold will effectively not delete anything. It is recommended that you use a reasonable value (e.g., 7, 14, 30, etc.).

    Delete orphaned dataset versions 
    dremio-admin clean -d=7
dremio-admin clean --delete-orphan-datasetversions=7