Skip to main content
Version: current [24.2.x]

Example: Number of Rows

This example determines the number of rows that were projected by each operator.

To find out how many rows were projected by each operator at the major fragment level:

  1. Upload your profile_attempt_0.json or you can use the sample profile_attempt_0.json file. See Uploading Profile Data for upload instructions.
  2. Create a file called operator_type.json and add the sample field values in Example: Operator Type Mapping to the file.
  3. Upload the operator type JSON file.
  4. Using a SQL query, obtain the number of rows projected. Group by majorFragmentId, operatorId, and order by majorFragmentId, operatorId.

The following is a sample SQL query snippet for obtaining the number of rows projected by the operator. The results are grouped/ordered by MAJOR FRAGMENT, OPERATOR_ID.
This information is very useful when trying to troubleshoot incorrect results.

Get number of rows projected by operator
SELECT majorFragmentId, operatorId,, sum(records) records
-- minorFragmentId, nested_4.minorFragmentProfile.maxMemoryUsed AS maxMemoryUsed, nested_4.minorFragmentProfile.memoryUsed AS memoryUsed, nested_4.endpoint.address AS address, nested_4.endpoint.maxDirectMemory AS maxDirectMemory
SELECT minorFragmentProfile, nested_3.minorFragmentProfile.endpoint AS endpoint, minorFragmentId, operatorType, operatorId, nested_3.inputProfile.records AS records, majorFragmentId
SELECT minorFragmentProfile, nested_2.minorFragmentProfile.minorFragmentId AS minorFragmentId, operatorProfile, nested_2.operatorProfile.operatorType AS operatorType, nested_2.operatorProfile.operatorId AS operatorId, flatten(nested_2.operatorProfile.inputProfile) AS inputProfile, majorFragmentId
SELECT fragmentProfile, minorFragmentProfile, flatten(nested_1.minorFragmentProfile.operatorProfile) AS operatorProfile, majorFragmentId
SELECT fragmentProfile, flatten(nested_0.fragmentProfile.minorFragmentProfile) AS minorFragmentProfile, nested_0.fragmentProfile.majorFragmentId AS majorFragmentId
SELECT flatten(fragmentProfile) AS fragmentProfile
FROM "@dremio"."profile_attempt_0" AS "profile_attempt_0"
) nested_0
) nested_1
) nested_2
) nested_3
) nested_4, "@dremio".operator_type op_type
where nested_4.operatorType =
group by majorFragmentId, operatorId,
order by majorFragmentId, operatorId