Skip to main content
Version: current [24.2.x]

Clean Metadata

This topic provides usage information for the dremio-admin clean CLI command.

Requirements​

  • Perform a backup before running the command (see Backup for more information).
  • Shut down all cluster nodes completely before running the command (see Startup/Shutdown for more information).

Syntax​

Clean command syntax
dremio-admin clean [args...]

Options​

Clean command options
    -c, --compact
      compact kvstore
      Default: false
    -o, --delete-orphans
      delete orphans records in kvstore (e.g., old splits)
      Default: false
    -h, --help
      show usage
    -j, --max-job-days
      delete jobs, profiles, and temporary dataset versions older than provided number of days
      Default: 2147483647
    -i, --reindex-data
      reindex data
      Default: false
    -p, --delete-orphan-profiles
      remove orphaned jobs
      Default: false
    -d, --delete-orphan-datasetversions
      delete dataset versions older than the provided number of days
      Default: 2147483647

Examples​

This section provides examples for cleaning up different types of data.

Compact Metadata​

Compacts metadata store entries.

Compact metadata store entries
dremio-admin clean -c
dremio-admin clean --compact

Delete Orphaned Entries​

Deletes orphaned metadata store entries.

Delete orphaned metadata store entries
dremio-admin clean -o
dremio-admin clean --delete-orphans

Delete Jobs​

Deletes jobs, profiles, and temporary dataset versions older than the specified threshold days. If no threshold is specified, items older than the default number of days (2147483647) will be deleted. Using the default threshold will effectively not delete anything. It is recommended that you use a reasonable value (e.g., 7, 14, 30, etc.).

Delete jobs
dremio-admin clean -j=7
dremio-admin clean --max-job-days=7

Re-index Data​

Re-index data.

Re-index data
dremio-admin clean -i
dremio-admin clean --reindex-data

Delete Orphaned Profiles​

Delete orphaned Dremio job profiles.

Delete orphaned job profiles
dremio-admin clean -p
dremio-admin clean --delete-orphan-profiles

Delete Orphaned Dataset Versions​

note

This command is available in Dremio 19.6.3+, 19.8.0+, 20.4.0+, 21.2.0+, and 22.0.0+.

Deletes dataset versions that Dremio is not using that are older than the specified threshold days. If no threshold is specified, dataset versions older than the default number of days (2147483647) will be deleted. Using the default threshold will effectively not delete anything. It is recommended that you use a reasonable value (e.g., 7, 14, 30, etc.).

Delete orphaned dataset versions
dremio-admin clean -d=7
dremio-admin clean --delete-orphan-datasetversions=7